proposal: spec: permit non-interface types that support == to satisfy the comparable constraint

Background
In Go 1.18 we introduced a type constraint comparable.  The type constraint is satisfied by any type that supports == and for which == will not panic.  For example, it is satisfied by int and string and by composite types like struct { f int } or [10]string.  On the other hand, it is not satisfied by types like any or interface { String() } or struct { f any } or [10]interface{ String() }.
This decision has led to considerable discussion; for example, #50646, #51257, #51338.
When considering whether a type argument satisfies the comparable constraint, there are two cases to consider.
For an interface type, the rule in the spec is simple: an interface type T implements an interface I if "the type set of T is a subset of the type set of I."  By this definition the type any does not implement comparable: there are many elements in the type set of any that are not in the type set of comparable.  More generally, no basic interface implements comparable.  Some general interfaces, such as interface { ~int }, implement comparable; it is not possible today to use this kind of general interface as an ordinary type, but a type parameter constrained by such a general interface will implement comparable.
For a non-interface type, the rule is different.  A non-interface type T implements an interface I if it "is an element of the type set of I."  For a language-defined type like comparable, the language defines that type set.  The current definition says that comparable "denotes the set of all non-interface types that are comparable."
However, the current implementation is slightly different.  In the current implementation, a composite type that includes an element of interface type does not implement comparable, although such a composite type is "comparable" according to the definition in the spec.  The implementation was written that way based on the belief that comparable should only be implemented by types that will not panic when not used in a comparison.  This is a valuable property, but it leads to some complications.
For example (this is based on a comment by @Merovius), consider a package "annotated" that implements an annotated value type:
type Value struct {
	Annotation string
	Val        any
}
// Various methods on Value.

Now consider a package "p" that uses that type, and suppose that package p ensures that it only stores values with comparable types in the Val field.  It's fine for package p to use the type map[annotated.Value]bool.  That works because the type annotated.Value is comparable according to the language definition.  However, annotated.Value does not implement comparable, because the type of the element Val is an interface type that does not implement comparable.  That means that code like this does not work:
type Set[Elem comparable] map[Elem]bool
var ValSet Set[annotated.Value]

Since annotated.Value does not implement comparable, the compiler will reject this code.
There is no good workaround for package p in this scenario.  There is no way for p to say that it wants a version of the type annotated.Value that restricts Val to comparable types.  It wouldn't be appropriate to change the annotated package, since that package can also be used by other packages that don't want to restrict Val to comparable types.
Proposal
Therefore, we propose that we change the implementation.  Arguably the implementation is not quite following the spec here, so there may be no need to change the spec.
The new rule is:

the type set of comparable is all non-interface types that are comparable per the language spec

As before:

an interface type implements comparable if the type set of the interface type is a subset of the type set of comparable
a non-interface type implements comparable if it is a member of the type set of comparable

For example, by this definition, annotated.Value implements comparable, and the problem outlined above goes away.
Note on interface types
This change means that there is little reason to seek the comparable version of a non-interface type T, as such types are now always comparable or never comparable.  However, people may still want to get the comparable version of an interface type.  For example, code might want the fmt.Stringer type but only permitting comparable types.  If we adopt #51338, then people can get this type by writing interface { comparable; fmt.Stringer }.  However, that is not part of this proposal.
Timing
Because of the confusion in this area, and the apparent discrepancy between the spec and the implementation, it may be worth implementing this in the 1.19 release, even though that is soon.

So, to clarify, this would allow the following?
func Equal[T comparable](t1, t2 T) bool {
  return t1 == t2
}

type Example struct {
  Val any
}

func main() {
  Equal(Example{""}, Example{""}) // true
  Equal(Example{[]int(nil)}, Example{[]int(nil)}) // panic
}

However, Equal(any(0), any(0)) would still not be allowed, correct? And in the panicking case where it is allowed, it would just be up to the user to make sure that nothing that isn't actually comparable is used with a comparable constraint?

Practically, this seems no different to me from just removing the special case that interface types are comparable types that are not in the type set of comparable. If I have a generic wrapper for sync.Map and I want the type behavior that sync.Map currently provides, I write var m Map[[1]any]any instead of var m Map[any]any, along with extra syntax around uses.

The issue is that an interface can reside at an arbitrary deep level within a struct. (struct of struct of struct...)
Hence, it may not be obvious that a comparison may panic.
I'm not too convinced and would rather keep the current, stricter, behavior.
It seems to me that this example exposes a rather rare pathological case where the comparison intent is being guaranteed by convention and not code.
On the other hand, I see the other tenant of the issue: it forces the creators of packages to find out in advance whether an interface field should require comparable or not.
That definitely deserves more thinking.
Would be great to get such examples from the corpus of Go code that exists at large.
I don't have such examples myself.

I think ultimately the least confusing thing to allow interface values to match the comparable constraint. That doesn't stop you from allowing values of the comparable interface later. It is admittedly somewhat confusing but this is carpet stuck under furniture that's too heavy to move so it's going to stick up somewhere.

I think ultimately the least confusing thing to allow interface values to match the comparable constraint. That doesn't stop you from allowing values of the comparable interface later. It is admittedly somewhat confusing but this is carpet stuck under furniture that's too heavy to move so it's going to stick up somewhere.

Unfortunately, I don't think that it would mix well with allowing constraint interfaces (such as comparable) as regular interfaces.
That would make any interface embedding comparable only usable as a constraint. A "color" that could be more annoying to deal with.

You can't put interface values in other interfaces so you could still allow values of the comparable interface.

You can't put interface values in other interfaces so you could still allow values of the comparable interface.

I meant by way of embedding.

@DeedleFake I agree with what you wrote above.
@DeedleFake @zephyrtronium Yes, it is easy to use a struct or array to permit using an arbitrary interface type to satisfy a comparable constraint.  I think that's OK.  As noted, I think that #51338 will provide a cleaner way to do the same thing while avoiding the potential panic.
@jimmyfrasche We could do that.  In fact we could do that even after adopting this proposal if we decide to.  I'm not personally sure that it's the least confusing approach; see the lengthy discussion in #50646 when we started down that path before changing our minds.  I think it's become clear that any approach in this area is confusing.

@atdiar

Unfortunately, I don't think that it would mix well with allowing constraint interfaces

I think it is clear that we should either allow comparable as a normal interface type or make interface-types implement comparable, but not both. They are complementary, equally simple solutions to the issues.

@Merovius that's not at all clear to me. Doing both seems like a perfectly fine thing to do me—and the best thing to do overall. It's possible I'm missing something, though.

@atdiar

Unfortunately, I don't think that it would mix well with allowing constraint interfaces

I think it is clear that we should either allow comparable as a normal interface type or make interface-types implement comparable, but not both. They are complementary, equally simple solutions to the issues.

Well, it also depends on whether there are plans to make all constraint interfaces usable as regular interface types (which was in consideration, I recall).
The present decisions could preclude some future plans to do just that.

@jimmyfrasche If all interface types implement comparable and comparable is a type, then
var a any = func() {}
var b comparable = a

should be allowed. In that case, I don't see the point of making comparable a type, as it does not seem to provide better guarantees than any.
To me, making comparable a type is a specific solution to guarantee that func Eq[T comparable](a, b T) bool { return a == b } never panics. To me, that goal isn't achieved if you can then instantiate Eq[any].
i.e. to me, they are both different solutions to the same problem, attacking it from opposite angles.

You would have to write
var a any = func() {}
var b comparable = a.(comparable)

replace comparable with any other nonempty interface and your example fails at compile time and my rewrite fails at runtime so I would assume that the same would hold true for comparable.

@ianlancetaylor

As noted, I think that #51338 will provide a cleaner way to do the same thing while avoiding the potential panic.

Personally, I think that if we make comparable a type and only comparable types assignable to that, it seems non-committal not to go all the way with that. If comparable is added, it should actually provide a guarantee that the comparison never panics. Otherwise, why even introduce it? That is, if we accept #51338, I'd be in favor of biting the bullet on making comparable viral and reject this proposal.

@jimmyfrasche

replace comparable with any other nonempty interface and your example fails at compile time

Right, but not all interface types implement any other nonempty interface. But all interface types would implement comparable.

If any as a type/interface satisfies comparable as a metatype/constraint does not imply that the dynamic type of an interface value is then necessarily comparable

@Merovius How do you think we should address the problem with annotated.Value (which was based on one of your comments)?
The comparable constraint and interface types are a bad mix no matter what we do.  I think that comparable still serves a purpose even if we adopt this proposal, though: it rejects types that are clearly not comparable, and it rejects attempts to instantiate with an interface type that may not be comparable.  As @jimmyfrasche said above the carpet is going to stick up somewhere, and this proposal is suggesting a specific spot where it will stick up: composite types that contain interface types.

There is also the option of not using a generic Set but rather an interface-based set (i.e. based on map[interface{}]bool)  when one wants to be able to force comparison of potentially non-comparable values.
If applicable, then there might not be a need to special-case composite types that contain interface types.
But really it depends on the cases and how widespread there are. Would be great to have real code examples.

There is also the option of not using a generic Set but rather an interface-based set (i.e. based on map[interface{}]bool) when one wants to be able to force comparison of potentially non-comparable values.

The problem isn't really with "potentially non-comparable values." There are interface types other than any which I might want to use as map keys, where what I want is to use several different concrete types in one map. reflect.Type has been a common example. ast.Node – for several different packages named ast – have been another. Since interfaces don't implement comparable, there is no way to write type-generic code for this case.
I have to duplicate the code, or at least a type-safe wrapper, for every interface type I want to use as a map key. This is exactly the problem that generics should be solving. Out of caution, for situations where comparable types are useful, it was chosen to address code duplication for all comparable types except interface types by introducing a special case for comparable. In my view, this proposal attempts to make that special case even more special and adds extra steps for actually writing the code I want to write, instead of simply making comparable consistent with the type system I've been using for eleven years.

Another tact might be to allow the use of (possibly panicking) == with [T any]. Then you would only use the comparable constraint when you want assurances that no incomparables can be passed in.

The correct approach is redesign the type-system, not patch it, it would make type-system not self-consistent.
Comparable should be ordinary interface type, and primitive type, struct of comparable type, array of comparable type, pointer etc implement Comparable default.

@ianlancetaylor

How do you think we should address the problem with annotated.Value (which was based on one of your comments)?

Require to duplicate the type and write custom conversions. That's a hassle, of course. But to me, that's an argument against #51338, not in favor of patching something worse over it. In my opinion, if we think we need #51338, we ought to accept this problem and see how much it comes up - at least at first.
There are two other ways (from this proposal) how I think we could address it, if need be:

Make type-assertion on comparable "recursively" assert comparability. That is, var v annotated.Value; v.(comparable) succeeds, if the dynamic value of the Val field is comparable. This might feel a bit strange, but it makes sense in spirit of a type-assertion and I think it would keep the guarantee of comparable.
Make struct types convertible, if interface fields only differ in comparabaleness, similar to how we do it with tags. That is, make annotated.Value convertible to struct{ Annotation string; Val comparable }. That would still require to duplicate the type, but at least a) you don't need a manual conversion and b) the compiler will check if the struct-definitions drift. It would still mean that the guarantee of comparable is violated, but not worse than this proposal and it would require an explicit conversion to do so.

@carlmjohnson

Another tact might be to allow the use of (possibly panicking) == with [T any]. Then you would only use the comparable constraint when you want assurances that no incomparables can be passed in.

We forbade interface-types from implementing comparable because it was deemed unacceptable for == to panic, when using it. I don't think extending this problem to all types makes sense, then. i.e. this seems strictly worse than the original situation of having interface-types implement comparable - it extends the downsides to more situations, while also making comparable less useful.

Just a clarifying question: why it was deemed unacceptable for == to panic?

This proposal is quite confusing because I think it breaks the decision we made in https://github.com/golang/go/issues/50646.
If https://github.com/golang/go/issues/51338 is get adopted, maybe we can consider the following workgroud for aboved example:
type Set[Elem comparable] map[Elem]bool
set := make(Set[comparable])
v := annotated.Value{}
set[v.(comparable)] = true

@atdiar We have that differentiation today. It's the difference between an interface having only methods and one which also has type elements (or is/embeds comparable).

From my perspective, the current system doesn't let me write code that I want to use in practice. I have a helper Equal[T comparable](t *testing.T, want, got T). I can't do Equal(t, nil, err) because error is not comparable. I'd be happy if I could change Equal[T comparable] to Equal[T any] and just pay the price in panics if someone passes in a slice or map by mistake.
Another situation that I haven't run into yet, but I'm sure exists is maps.Keys(m) where m has an interface key type.
It's not clear to me why I should prefer not being able to use Equal and Keys, particularly as I can write the same code without generics: func EqualErr(t *testing.T, want, got error) { if want != got { ....

@carlmjohnson

I'd be happy if I could change Equal[T comparable] to Equal[T any] and just pay the price in panics if someone passes in a slice or map by mistake.

You can:
func Equal[T any](t *testing.T, want, got T) {
    if any(want) != any(got) {
        t.Errorf("%v != %v", want, got)
    }
}

@changkun I think it is because == is used for nul checks which everyone uses to prevent panics. If a == nil can panick, safe nil checks become impossible.

@atdiar We have that differentiation today. It's the difference between an interface having only methods and one which also has type elements (or is/embeds comparable).

Yes. Only difference is that those constraints are still typeset based and could be reified later. Currently, constraining interface types happens only indirectly through typesets.
Only, the typeset definition of constraint is incongruent with the fact that interface types have the comparison operators in spite of the elements of the typeset.
It might well be the only issue were interfaces types require an extended definition of a type constraint that applies directly to them.

@beoran I don't think that makes sense. You can't, in a function constraint on comparable, compare a type-parameter typed variable to nil - for that, you need to actually know that the underlying type can be nil. Also, comparison to nil never panics.

@ianlancetaylor

As @jimmyfrasche said above the carpet is going to stick up somewhere, and this proposal is suggesting a specific spot where it will stick up: composite types that contain interface types.

I understand that this is a tradeoff, but I think that's a confusing and fairly subtle place to draw the line. It would mean that struct { any } is comparable where just any is not, and when folks run into problems the incentive will be to wrap problematic interfaces in otherwise-useless struct wrapper types. That will add type-declaration noise to the code, for no clear benefit.
I can understand the desire to provide static type safety and prevent avoidable run-time errors, but in most other similar tradeoffs Go biases toward conciseness over type-safety. It seems awkward to bias differently here.
In other cases we have suggested static analysis to detect failure modes that aren't covered in the formal type system. Would that be a viable alternative here? (What if we allowed all interface types to match comparable, but emitted a vet warning when passing a known-incomparable value as a comparable interface?)

@beoran in general, composite types whose components include interface types may panic when being compared.
The current implementation of the comparable constraint does not allow for such panics to occur which is valuable imho.
This proposal would relax this rule.
Still would not solve the issue that people want to use interface types in generic code. At least, it wouldn't be straightforward.

Maybe there is nothing needed to do here.
We could just interpret any as MayBeCompared.
Set[K any, V any] and Set[K comprable, V any] bot have their use scenarios.

The issue with any as a type constraint is that it allows the coder to pass functions, slices and maps.
There is probably a need for something in between to accomodate for interface types.
But that constraint cannot define a typeset at the same time as well.

any(want) != any(got) is a very neat solution to the comparability problem for Equal, but it doesn't work for set.Set[iface] or maps.Keys(ifaceMap). I think it's good to keep use cases in mind here.
Part of the problem is that the theory is misaligned with practice. Even though the spec only recognized comparable and incomparable, Go has always had three levels of comparability: definitely incomparable (slices, maps), definitely comparable (primitives, compounds of primitives), possibly comparable (interfaces, compounds containing interfaces). Any mapping of three values onto two names is going to create mismatches. ISTM, this debate is partly about which ox to gore in the aligning. This issue proposes to treat compounds containing interfaces as comparable but not interfaces themselves, which seems like a very odd place to draw the line, since it moves some of the possibly comparable into comparable set but not all of them. Why not add a third name instead? You could loosen comparable and add strictlycomparable or keep comparable strict and add equatable or some other, better name.
[There's a separate problem that the definitely incomparable group actually are comparable to their zero values, but there's no way to express that comparison with generics. See #35966.]

There's a separate problem that the definitely incomparable group actually are comparable to their zero values

That's not really true. They are comparable to the pre-declared identifier nil, not their actual zero-value.

Since code out there already exists with [K comparable, V any, M ~map[K]V] and that code is intended to work with any valid map, my tendency is to think that comparable should be broadened to include all possibly comparable types (including interfaces), and a new name made up for "strictly comparable" types for cases where you want to guarantee non-panicking. I can be argued into other ideas though.

Why not add a third name instead?

This was proposed before in https://github.com/golang/go/issues/49587, although I found it difficult to parse and follow why the decision was converged to decline. To me, it seems just a biased decision.

Before generics there were three kinds of comparability in Go: (copied from #26842)

incomparable, there is no == operator defined for a type (structs and arrays with incomparable fields).
0-comparable, there is an == operator but it can only test against the zero value (funcs, maps, and slices can be compared against nil). The spec treats these as incomparable types and notes the special case.
comparable, there is an == operator and it works for all values

There were three subsets of comparable types

nice
might panic (interfaces)
irreflexive (floats)

Generics added a third kind of comparable to the language: recursively panic-free.
This proposal is to amend that to not directly panic free. As @bcmills notes that's effectively the same as using the old definition of comparable but with an extra step sometimes.
The choices here are ultimately to simplify what comparability means or to add more kinds of comparability.
I'd rather simplify by:

introducing a universal zero value that any value can be compared against
letting all interface types satisfy the comparable type constraint
allowing values of the comparable interface type

That way there is just comparable and incomparable.
The comparable constraint may not be as strict as one would like but the only subtlety would be between comparable as a type and comparable as a constraint.
@carlmjohnson would need a second helper IsZero[T any](t *testing.T, v T) and while you could write Set[any] you could also write Set[comparable].

@carlmjohnson
I like the overall idea but it's going to be my turn to be nitpicky 😂.
I think that a constraint that admits interface types should not be an interface. Because interfaces defines typesets.
So I would rather keep comparable as it is but create a looser constraint for map keys.
This new constraint would be a standalone, not embeddable in interfaces whether constraint interfaces or regular interfaces. (no real need for it anyway, if I'm not mistaken)
The universe of type constraints would include the (reifiable) constraint interfaces. Probably with a cardinality just augmented by that one new constraint.
But yes, I would rather be in favor of such an idea. Unless anyone sees a problem.

@jimmyfrasche

letting all interface types satisfy the comparable type constraint
allowing values of the comparable interface type

I still can't really get behind having both of these. Here's another question: Should struct{ any } be assignable to comparable? If any is comparable, then all fields of that struct are comparable, so it probably should be comparable. Meanwhile, that's exactly the situation we are getting in with this proposal, which so many people find strange, where any is not assignable to comparable (at least you've argued above it shouldn't), yet struct{ any } is.

@atdiar I'm still confused what you are trying to say. comparable already does exactly what you describe - it's a constraint, which can't be used as a type and it means (among other things) "can be used as a map key". I don't understand why you are trying to introduce a third kind of type constraint, if it does exactly the same as one of the ones we already have.

I still can't really get behind having both of these. Here's another question: Should struct{ any } be assignable to comparable? If any is comparable, then all fields of that struct are comparable, so it probably should be comparable. Meanwhile, that's exactly the situation we are getting in with this proposal, which so many people find strange, where any is not assignable to comparable (at least you've argued above it shouldn't), yet struct{ any } is.

I read Jimmy as saying struct{ someInterface } would be comparable.
Not speaking for him, but I take it that the point of having var x comparable is that then you can have map[comparable]bool etc. and for that, you want strict comparability (otherwise, it's not much better than map[any]bool). OTOH, for type constraints, it seems preferable to have loose comparability. Perhaps when you write var x comparable = y, it looks at the concrete type (not interface type) of y, so the fact that y was any type is lost in the conversion. But that doesn't do much for the struct{ any } or *any case. So maybe the bite the bullet option is to say comparable is loose as a type parameter/constraint, but strict as an interface value type. I'm not sure I like that though. Then again maybe it will just be one of those little inconsistencies you never run into in practice so you never notice it.
It would be nice to be able to consolidate down to a single comparable/incomparable binary (ignoring the zero value comparisons thing for the moment), but every time someone pushes down the rug in one spot, it seems to bump up elsewhere. :-) So I'm still leaning towards a third name as the best solution so far.

@Meroviues

Here's another question: Should struct{ any } be assignable to comparable? If any is comparable, then all fields of that struct are comparable, so it probably should be comparable.

struct { any } is a comparable type so it satisfies the comparable interface. That's not great but you could also write struct { comparable } if you want better.

Meanwhile, that's exactly the situation we are getting in with this proposal, which so many people find strange, where any is not assignable to comparable (at least you've argued above it shouldn't), yet struct{ any } is.

I don't really understand why you think that a comparable interface would behave so differently from other interfaces. comparable is a subset of any so you can't assign a value from an any to a comparable without a type assertion to check that the dynamic type is comparable. The fact that the any type statically satisfies the comparable type constraint is irrelevant.
fmt.Stringer statically satisfies the any type constraint but that doesn't mean you can write:
var a any = func() {}
var b fmt.Stringer = a

@jimmyfrasche I was, in that particular comment, just observing that many people have above commented that it's strange for any to not be comparable, but struct{ any } to be and that that's what they didn't like about this proposal. And that your suggestion would have the same flaw, to those people. I agree that comparable should probably behave that way, under your suggestion.

If generics were left as they are and the comparable interface only accepted dynamic types who were recursively panic-free wouldn't that be another solution?

@Merovius

I was, in that particular comment, just observing that many people have above commented that it's strange for any to not be comparable, but struct{ any } to be and that that's what they didn't like about this proposal. And that your suggestion would have the same flaw, to those people.

I see them as different circumstances.
If we suppose #51338 is accepted and interface values are included in the type set of comparable, then the fact that any is not assignable to comparable is a consequence of the definition of assignability that we've had since the beginning, and the fact that struct{any} would be assignable to comparable is a consequence of the definition of comparability that we've had since the beginning.
If this proposal is accepted and the definition of comparable does not change, then the fact that any is not in the type set of comparable is a consequence of a special-case rule that interfaces are not in the type set of comparable despite being comparable, and the fact that struct{any} is therein is a consequence of a special-case rule that comparable doesn't care about the recursive structure of types like comparability does.

@jimmyfrasche If I understand you correctly, that's exactly what #51388 proposes - making comparable a type, which only (recursively) comparable types can be assigned to.

@zephyrtronium

If we suppose #51338 is accepted and interface values are included in the type set of comparable, then the fact that any is not assignable to comparable is a consequence of the definition of assignability that we've had since the beginning

I think that can be argued, but it can also be argued against. The definition of assignability for interface types today is

T is an interface type, but not a type parameter, and x implements T.

The suggestion is for interface types to implement comparable, therefore if x's type is any, it implements comparable and should be assignable to a variable of type comparable.
I don't think we can argue with current semantics here. The point is exactly to introduce a completely new type comparable and make that type more restrictive than other interfaces. What those restrictions are remains to be sussed out, but they certainly should be as useful and consistent as possible.

Personally, I'd prefer if we tried to largely stick with jargon which is already established.

My argument is that this discussion is particularly thorny because the current jargon is missing a necessary term. It makes everything harder to think about when you can't use a proper noun to stick something in your mind.
Looking at Jimmy's 3 kinds + 3 sub-kinds, do we need to distinguish between truly incomparable and 0-comparable for this discussion? No, they can both be lumped under "incomparable" for our purposes here. But failure to pick apart the three sub-kinds of comparability has caused all of the other problems.

I think phrases like "comparable, but might panic at runtime" is within currently established jargon and, while a bit less wieldy, not that hard to use. But FWIW arguing about this is off-topic here, I was just trying to point out that I find it confusing to introduce new jargon, if we have not agreed on it. At least what's in the spec leaves a pretty good definition of what's "agreed upon jargon".

@atdiar I'm still confused what you are trying to say. comparable already does exactly what you describe - it's a constraint, which can't be used as a type and it means (among other things) "can be used as a map key". I don't understand why you are trying to introduce a third kind of type constraint, if it does exactly the same as one of the ones we already have.

@Merovius
It's mainly because the current implementation enables us to answer https://github.com/golang/go/issues/49587 (and more)
and that might be something we want to keep since it provides compile-time guarantees.
There is also the fact that typesets are currently transitive sets and I think it plays well with the concept of interface embedding.
I have a hunch that it simplifies the computation of operation sets derived from type sets.
IF interfaces are deemed comparable, we lose the transitivity. Which is why I argue that in that case, comparable should not be an interface (i.e. it would not define a typeset)
Of course, we could change comparable, but one may still want a way to get the current semantics back in some shape or form. In the end, that would still equate to introducing something.

I must confess that I'm getting a bit lost in the different permutations being discussed in this now-long thread, and so I'm sorry if I'm repeating something that was already said in different words above.

My existing intuition is that whenever I use a value of an interface type I'm opting in to various things being checked at runtime rather than compile time, because the purpose of an interface type is to allow dynamically choosing a concrete type dynamically at runtime and so runtime checks are a necessary cost in return for that flexibility. With runtime checks comes the possibility of panics if my code is found incorrect at runtime, which is the risk I take in return for the benefit of choosing types dynamically.
With the introduction of type parameters, I extended that intuition to say that if I use a type parameter then I am expecting to fix a specific type at compile time. In return, the compiler should be able to guarantee that my chosen type is suitable and thus ensure that anything type-related which would have been a runtime panic will be a compile-time error instead.
My intuitions above make me feel like currently the type parameter handling is overreaching into world of interface types, with this rule aiming to make a certain subset of runtime panics impossible even though I've opted in to that risk by using interface types. It isn't a responsibility of the compiler to prevent me from using interface types in a way which might panic, only to prevent me from using them in a way that would certainly panic.
I would therefore be in favor of a design like what is proposed here, where I am allowed to use a value of any interface type (or an otherwise-comparable type containing one) to satisfy the comparable type constraint.
I do see the argument above that it would therefore be no different than any. I suspect that is be true from the perspective of what the compiler can guarantee, but it still carries an additional meaning that I can take into account as a developer: if a type parameter is constrained as comparable and I'm intending to pass an interface type into it, I know I must take care to only pass comparable types through that interface value in order to avoid a runtime panic. Conversely, if I want the compiler to guarantee at compile time that the given type is valid, I (as the caller) can be sure to only use non-interface types.
If map types didn't already allow the calling program to take this risk when desired then perhaps my existing intuitions here would have developed differently, but with the language as it existed before type parameters the current restriction seems inconsistent and it subverts my ability to opt in to dynamic type checking and the possibility of runtime panics by using interface types.
(I am considering the current prose in the spec about what is allowed as a map key to imply that the generic signature of a map type would be map[K comparable]V any, and I would prefer the implementation to behave that way, by changing comparable to match the map behavior rather than the other way around.)

There are three major definitions of comparable relevant to this thread:

comparable/S - the current spec definition of comparable
comparable/RPF - the recursively panic-free definition of comparable generics use now
comparable/1PF - the 1-layer of panic free comparable being proposed (which is any non-interface comparable type)

Regardless of definition, there are two uses of comparable:

C(comparable) - as a constraint
T(comparable) - as an interface value [should that be allowed in the future]

Note that as interface vaules don't nest T(comparable/1PF) and T(comparable/S) are identical.
If we continue to use C(comparable/RPF) and permit T(comparable/RPF) [this is #51338]

== never panics
comparable/RPF needs to be enshrined in the spec
T(comparable/RPF) is viral and lots of interfaces will have to be marked comparable for it to be useful

If we permit C(comparable/1PF) [this proposal]

== may panic
comparable/1PF needs to be enshrined in the spec
T(comparable/1PF) could be permited later
this can be later relaxed to C(comparable/S) without any operational changes to T(comparable)

If we permit C(comparable/S)

== may panic
T(comparable/S) could be permitted later
no additional notions of comparable needed

Did I get anything wrong?

@atdiar I'm still confused what you are trying to say. comparable already does exactly what you describe - it's a constraint, which can't be used as a type and it means (among other things) "can be used as a map key". I don't understand why you are trying to introduce a third kind of type constraint, if it does exactly the same as one of the ones we already have.

@Merovius It's mainly because the current implementation enables us to answer #49587 (and more) and that might be something we want to keep since it provides compile-time guarantees.
There is also the fact that typesets are currently transitive sets and I think it plays well with the concept of interface embedding. I have a hunch that it simplifies the computation of operation sets derived from type sets.
IF interfaces are deemed comparable where comparable is an interface type, we lose the transitivity. Which is why I argue that in that case, comparable should not be an interface (constraint or regular) (i.e. it would not define a typeset). I can expand on this if needed.
So, of course, we could change comparable, but one may still want a way to get the current semantics back in some shape or form. In the end, that would still equate to introducing something.

I am exceedingly confused by who is arguing for what behavior here, and must apologize for increasing the amount of confusion in the world by offering an opinion.
The Go spec has a definition of comparable, which clearly states that "interface values are comparable". I find it very surprising that the comparable constraint excludes interface values, given that these values are comparable under the spec's definition.
I'm further surprised that the compiler believes struct { V any } doesn't satisfy the comparable constraint, since the spec states that "a type T implements comparable if T is not an interface type and T supports the operations == and !=." struct { V any } is not an interface type, and it supports == and != (although these operators may cause a run-time panic, of course).
I think the simplest available option is to state that all comparable types (under the spec's definition) implement the comparable constraint, so that there's one consistent definition of "comparable".
This means that it is impossible to write a type constraint for values which support == and != and also guarantee that these operators will not cause a run-time panic. That's entirely consistent with the treatment of comparable interface values in non-generic code, which seems fine to me.

@carlmjohnson
I think you have it right. The core problem is that the concept of comparability is not clean cut, so with a single comparable type, we cannot cover all cases. The simplest solution would be to add additional predefined interfaces such as hashable, strictlycomparable , etc. Would that work?

@jimmyfrasche

Did I get anything wrong?

No, I believe everything you say is correct. With the caveat that "could" and "should"/"it would be a good idea" are different.
@beoran

The core problem is that the concept of comparability is not clean cut, so with a single comparable type, we cannot cover all cases. The simplest solution would be to add additional predefined interfaces such as hashable, strictlycomparable , etc. Would that work?

Lots of things might work, but I don't think this is a good idea at all. We don't really want to expose more pre-declared identifiers unless we absolutely have to. I also don't really think it does solve the problem. The real question is how interface types should behave. If we have an answer to that, we don't need to actually introduce new identifiers for those behaviors, we can just enshrine the behavior we want.

I opened #52509 to hopefully keep the "comparable and comparable could be the same" comments focused on a proposal about doing that, rather than filling the comments of this one.

I am a little confused as to how the proposal solves the problem stated.

In Go 1.18 we introduced a type constraint comparable. The type constraint is satisfied by any type that supports == and for which == will not panic. For example, it is satisfied by int and string and by composite types like struct { f int } or [10]string. On the other hand, it is not satisfied by types like any or interface { String() } or struct { f any } or [10]interface{ String() }.

For background, my interest is mainly in making generic versions of (e.g.) sync.Map.
As I understand it you can have a map of any interface type, because 'Interface values are comparable. Two interface values are equal if they have identical dynamic types and equal dynamic values or if both have value nil.' (source). However, under this proposal despite interface values being comparable, the interface types would not in general still implement comparable. This is not only confusing but makes it difficult to write code like the following which works for all types eligible to be keys for maps:
func Make[K comparable](k K) map[K]bool {
	m := make(map[K]bool)
	m[k] = true
	return m
}

This is despite the fact that comparison of interface values can never panic (as far as I know).
I wonder whether the issue (specifically with generic versions of maps, sets etc) is that we're trying to use comparable as the constraint, which ... overconstrains things. The notion of 'comparable' (the language definition, not the current interface / keyword) corresponds well to what is currently accepted as a map key: 'The comparison operators == and != must be fully defined for operands of the key type; thus the key type must not be a function, map, or slice. If the key type is an interface type, these comparison operators must be defined for the dynamic key values; failure will cause a run-time panic.'.
If we can't make the definition of comparable the same as what the language spec treats as 'comparable' (which is as far as I can see 'everything you can use as a map key'), then could we at least have some form of constraint mapkey which includes only those types that are used as map keys - I think this is a superset of comparable.

@abligh

This is despite the fact that comparison of interface values can never panic (as far as I know).

It can

could we at least have some form of constraint mapkey which includes only those types that are used as map keys - I think this is a superset of comparable

It's not a superset. The types usable as map keys are exactly the same as the types which can be compared using ==. And storing an interface value in a map panics exactly if comparing that value panics.

@Merovius

could we at least have some form of constraint mapkey which includes only those types that are used as map keys - I think this is a superset of comparable

It's not a superset. The types usable as map keys are exactly the same as the types which can be compared using ==. And storing an interface value in a map panics exactly if comparing that value panics.

Sorry, I missed backticks, accidentally illustrating the confusion :-). I meant "could we at least have some form of constraint mapkey which includes only those types that are used as map keys - I think this is a superset of comparable" - i.e. the set of things that can be used as keys of maps is a superset of the things which implement comparable either under this proposal or Go 1.18. I know the set of things that can be used as keys of maps is exactly equal to the things that are comparable (per the language spec) (i.e. can be compared using ==), though as you point out some comparisons may panic at runtime.

@abligh My point was kind of that, if we think the behavior of mapkey is better, then comparable should just behave that way. There is no difference between "useful as a map key" and "useful for comparisons", both would panic or not panic in the same situations and allow or disallow the same operations to happen. I don't think we should have two things in the language to mean the same thing, where the only difference is that one behaves worse than the other (or, let's say "some people think one of them is worse, others think the other is worse, but neither is actually better or worse").
If we think mapkey is the better behavior, we should adopt #52509.

Meta. I think regardless of the outcome of these various proposals what comparable means needs to match what the spec defines as comparable, even if that means coming up with a new term for types that permit == but don't match this hypothetical new definition of comparable.

Created a sort of counter-proposal https://github.com/golang/go/issues/52531
I think it might be a way to address the issue about comparability in its entirety.
(it's also my first proposal and issue ever lol)

Will the following code panic if this proposal is accepted?
package main

type s struct {
	f any
}

func eq[T comparable](x, y T) bool { return x == y }

func main() {
	_ = eq(s{func() {}}, s{func() {}})
}

If yes, how does it differ from the following case?
package main

func eq[T comparable](x, y T) bool { return x == y }

func main() {
	_ = eq(any(func() {}), any(func() {}))
}

@changkun

Will the following code panic if this proposal is accepted?

Yes.

If yes, how does it differ from the following case?

That case is not valid, currently or under this proposal. If you mean "it doesn't seem very different, so if we should allow one, why not allow the other?", I agree.

Yes.

That's great. So we can't prevent panic anyway. I think this might clarify https://github.com/golang/go/issues/52474#issuecomment-1106059058 and left for the decision.

FWIW I think there is an argument to be made that preventing panics is important (not in favor of this proposal, because it doesn't, but e.g. for #51338), even if we didn't do it so far, which goes like this:

We maybe want, at some point, want to use type element interfaces to add variant types to Go
If we do that, we probably want to be able to do it for comparable as well, because it would be strange to have an exception to this
If comparable becomes a full type, it should probably be distinguished by preventing panics on comparison statically, because that would be the only thing differentiating it from any, in terms of real benefit in usage.
For consistency, comparable as a constraint should then be fulfilled by the same set of types which are assignable to comparable.
Therefore, we should maintain the meaning of comparable as a constraint to "types on which equality is statically prevented from panicing"

This chain would then argue against this proposal as well as #52509, as they don't maintain that meaning. So, if we want to use type element interfaces as variant types at some point, rejecting this proposal and #52509 and accepting #51338 will ultimately lead to the most consistent and simple language.
Personally, I don't like that this argument depends on wanting to do something which we haven't yet decided to do. I don't know how likely it is that we will and at what point that would happen. And we might want to solve the "instantiating comparable constrained generic types with interface types" before that. But it is a fair argument.
(I also agree with @jimmyfrasche here, that we should then make sure comparable reflects what the spec calls comparable)

I went back to the discussion in #51338 to evaluate how this proposal compares.
As far as I can tell, it does have one advantage — namely, it allows type-assertions to comparable to succeed for all values that are actually comparable. (Compare https://github.com/golang/go/issues/51338#issuecomment-1058223193.)
However, that comes with an equivalent disadvantage: a successfull type-assertion to comparable would not actually ensure that the value can be compared without panicking. So it would allow run-time weakening of static checks, but would not succeed in providing an equivalent dynamic check that programs could use to avoid the panic.

Thanks for the good discussion.  I'm persuaded that this proposal makes the language harder to understand without providing a commensurate improvement.  I'm withdrawing it.

See #52614 for another take on this problem.


Allocations should inherit the OwnerID of the parent object before insert

If none is set, inherit the OwnerId of the parent opportunity/recurring donation/campaign before insert to allow simple inserts via the API without explicitly setting the field value. Issue described in the hub.

**lurch: add

Tracking W-017616

Salesforce has determined that this issue will not be explored further.


Add CategoricalMADE

What does this implement/fix? Explain your changes
This implements a CategoricalMADE to generelize MNLE to multiple discrete dimensions addressing  #1112.
Essentially adapts nflows's MixtureofGaussiansMADE to autoregressively model categorical distributions.
Does this close any currently open issues?
Fixes #1112
Comments
I have already discussed this with @michaeldeistler.
Checklist
Put an x in the boxes that apply. You can also fill these out after creating
the PR. If you're unsure about any of them, don't hesitate to ask. We're here to
help! This is simply a reminder of what we are going to look for before merging
your code.

[x] I have read and understood the contribution
guidelines
[x] I agree with re-licensing my contribution from AGPLv3 to Apache-2.0.
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have reported how long the new tests run and potentially marked them
with pytest.mark.slow.
[ ] New and existing unit tests pass locally with my changes
[ ] I performed linting and formatting as described in the contribution
guidelines
[ ] I rebased on main (or there are no conflicts with main)
[ ] For reviewer: The continuous deployment (CD) workflow are passing.

Hey @janfb,
would very much appreciate your input at this stage:
Currently the PR adds the CategoricalMADE and builder build_autoregressive_categoricalestimator + some minor modifications to build_mnle and MixedDensityEstimator. This enables multiple discrete variables with different numbers of classes via trainer = MNLE(density_estimator=lambda x,y: build_mnle(y,x,categorical_model="made")) Note that for some reason x and y have to be flipped for mnle.
As far as I can tell all functionalities of CategoricalMADE work for both 1D and ND inputs and running the Example_01_DecisionMakingModel.ipynb with the CatMADE matches the ground truth

The question now is: How should I verify this works? / Which tests should I add/modify? Do you have an idea for a good toy example with several discrete variables that I could use?
I have cooked up a toy simulator, for which I am getting good posteriors using SNPE, but for some reason MNLE raises a RuntimeError: probability tensor contains either 'inf', 'nan' or element < 0 Even for the unmodified MNLE. Any ideas why this could be?
This is the simulator
def toy_simulator(theta: torch.Tensor, centers: list[torch.Tensor]) -> torch.Tensor:
    batch_size, n_dimensions = theta.shape
    assert len(centers) == n_dimensions, "Number of center sets must match theta dimensions"

    # Calculate discrete classes by assiging to the closest center
    x_disc = torch.stack([
        torch.argmin(torch.abs(centers[i].unsqueeze(1) - theta[:, i].unsqueeze(0)), dim=0)
        for i in range(n_dimensions)
    ], dim=1)

    closest_centers = torch.stack([centers[i][x_disc[:, i]] for i in range(n_dimensions)], dim=1)
    # Add Gaussian noise to assigned class centers
    std = 0.4
    x_cont = closest_centers + std * torch.randn_like(closest_centers)

    return torch.cat([x_cont, x_disc], dim=1)

The setup:
torch.random.manual_seed(0)
centers = [
    torch.tensor([-0.5, 0.5]),
    # torch.tensor([-1.0, 0.0, 1.0]),
]

prior = BoxUniform(low=torch.tensor([-2.0]*len(centers)), high=torch.tensor([2.0]*len(centers)))
theta = prior.sample((20000,))
x = toy_simulator(theta, centers)

theta_o = prior.sample((1,))
x_o = toy_simulator(theta_o, centers)

NPE:
trainer = SNPE()
estimator = trainer.append_simulations(theta=theta, x=x).train(training_batch_size=1000)

snpe_posterior = trainer.build_posterior(prior=prior)
posterior_samples = snpe_posterior.sample((2000,), x=x_o)
pairplot(posterior_samples, limits=[[-2, 2], [-2, 2]], figsize=(5, 5), points=theta_o)

and the equivalent MNLE:
trainer = MNLE()
estimator = trainer.append_simulations(theta=theta, x=x).train(training_batch_size=1000)

mnle_posterior = trainer.build_posterior(prior=prior)
mnle_samples = mnle_posterior.sample((10000,), x=x_o)
pairplot(mnle_samples, limits=[[-2, 2], [-2, 2]], figsize=(5, 5), points=theta_o)

Thanks for your feedback.

Hey @janfb,
you might have missed this, but I would be happy for feedback :)

Cool, thanks for all the feedback! A quick call would be great, also to discuss suitable tests for this. Will reach out via email and tackle the straight forward things in the meantime.

After discussion with  @janfb I will:

adapt the simulator of Example_01_DecisionMakingModel.ipynb to multiple discrete variables.
Get this to run for 1D and ND
fix remaining comments/issues
possibly refactor (fold 1D into ND code).

@janfb could you still check tho what is up with the simulator above? Do you have a hunch why the SNPE and MNLE posteriors different?


ERROR: Top-level await is not available in the configured target environment

Which @angular/* package(s) are the source of the bug?
Don't known / other
Is this a regression?
Yes
Description
17349 │ /******/ __webpack_exports__ = globalThis.pdfjsLib = await (globalThis.pdfjsLibPromise = __webpack_exports__);

01:55:45 [vite] error while updating dependencies:
Error: Build failed with 1 error:
node_modules/pdfjs-dist/build/pdf.mjs:17349:53: ERROR: Top-level await is not available in the configured target environment ("chrome118.0", "edge118.0", "firefox115.0", "
ios16.0", "safari16.0" + 7 overrides)

Please provide a link to a minimal reproduction of the bug
No response
Please provide the exception or error you saw
No response
Please provide the environment you discovered this bug in (run ng version)
@angular-devkit/architect       0.1602.10
@angular-devkit/build-angular   16.2.10
@angular-devkit/core            16.2.10
@angular-devkit/schematics      16.2.10
@angular/cli                    16.2.10
@schematics/angular             16.2.10
rxjs                            7.8.1
typescript                      4.9.5
zone.js                         0.13.3

Anything else?
No response

From the logs, it appears that the project is using the new build system with Angular v16.  For v16, the build system was considered a developer preview and may not operate as expected in all cases.  Please consider updating to Angular v17 where the new build system is now stable and fully supported.
If you are still encountering issues after using v17, please provide an updated version output as well as a minimal reproduction demonstrating the problem so that we can investigate further.

@clydin Thank you for your reply, here is a demo:
https://stackblitz.com/edit/npfsyd

This is expected as top level awaits are not supported.
Angular uses Zone.js which in requires always async/await to be downlevelled, in Esbuild this is done by enabling This is because we enable async/await downlevelling in esbuild which disables top level await usage.
Let's keep tracking this in https://github.com/angular/angular-cli/issues/22191

In this case adding the plugin in vite will only fix it it when running ng serve, however the application will not successfully build when running ng build.

@alan-agius4 Thank you for your reply, as far as I know this vite plugin vite-plugin-top-level-await can handle this situation, where should I add this plugin?

You can see how to config the plugin here https://www.npmjs.com/package/vite-plugin-top-level-await

For those who experience the same issue in the both dev and build commands, add the following

// vite.config.js
optimizeDeps: {
  esbuildOptions: {
    target: 'esnext'
  }
},
build: {
  target: 'esnext'
}

@madurapa can you please share the complete config file. I'm not sure where to add optimizeDeps and build.

The vote config is internal to the CLI and cannot be modified.
As stated before top level await is not supported.

@alan-agius4 topLevelAwait is enabled by default in Webpack but disabled in angular. Is there a chance you could enable it so we won't need custom webpack builder?

@benbro, no, as that would cause the application not to be buildable if you decide to migrate to the esbuild based builders.
See https://github.com/angular/angular-cli/issues/26507#issuecomment-1829331289

@madurapa can you please share the complete config file. I'm not sure where to add optimizeDeps and build.

export default defineConfig({
    optimizeDeps: {
        esbuildOptions: {
            target: 'esnext'
        }
    },
    build: {
        target: 'esnext'
    },
});

Put your config if you find it difficult to edit, I can include those values and past here


Add an example for creating channel neighborhoods with triangulation?

Creating channel neighborhoods (for e.g. cluster-based permutation tests) is non-trivial for the math impaired. I was just asked by another researcher if this was possible in MNE. Maybe we could have something like this gist as an example?
https://nbviewer.jupyter.org/gist/jona-sassenhagen/4d5db1a30ef47f26de468aeec4e04dd6

Yes please add an example if you have time. Although you can save a number of lines by using spatial_tris_connectivity once you have the triangulation

... and we can't use _find_topomap_coords so we'll have to find a better way to do that


Backend server

Add a backend to the project

Sqlite3 db running, but data shape is different than supplied. Replies are no longer nested under parents but have a replyTo column referencing which post they are replying to. A new pull request that refactors how data is handled will be created.


line up/down left/right key too sensitive

In older versions of this really nice software there were big arrows to hit to move up/down and left/right line by line or character by character respectively. the new single button that does all moves way too fast to just pick the last line and edit a character or two.
*** Never mind, now it is working properly, but a few days ago (maybe before the 14.4 update) it would scroll rapidly through lines.
Informations

Device : iPhone 11/iPad Air 2
OS : iOS 14.4
WebSSH Pro version : latest

Hello @wpballa Bill 👋
You can revert to the old behavior, here is how to :
https://webssh.net/documentation/help/howtos/SSH/use-old-special-keys/
Does this fix your trouble?
Have a nice day ☀

Yes, that solved my problem nicely, thanks.


Config is returning an empty object {} despite correct setup in Android

Description:
I am encountering an issue where Config from react-native-config is returning an empty object {}, even though the .env file and build configurations are properly set up. I have followed the documentation and verified all steps, but the issue persists.

Steps to Reproduce:

Installed react-native-config in my React Native project.
Created a .env file in the project root with the following content:APP_NAME=SampleApp
PRODUCT_BUNDLE_IDENTIFIER=com.example.app
PRODUCT_NAME=Sample Product
APP_FOLDER=sample-folder

Added the following configuration in android/app/build.gradle:project.ext.envConfigFiles = [
    debugFlavor: ".env.sample-debug",
    releaseFlavor: ".env.sample-release"
]

apply from: project(':react-native-config').projectDir.getPath() + "/dotenv.gradle"

Built the app using:./gradlew assembleDebugFlavor

Checked the Config object in JavaScript:import Config from 'react-native-config';
console.log('Config:', Config); // Output: {}

Expected Behavior:
The Config object should contain the variables from the .env.sample-debug file, e.g.:
{
   APP_NAME: "SampleApp",
   PRODUCT_BUNDLE_IDENTIFIER: "com.example.app",
   PRODUCT_NAME: "Sample Product",
   APP_FOLDER: "sample-folder"
}

Actual Behavior:
The Config object is returning an empty object:
{}

Environment:

react-native-config version: 1.5.3
React Native version: 0.72.4
Node version: 16
Gradle version: 8.1.1
Android Studio version: Ladybug
Package Manager<EMAIL_ADDRESS>

Troubleshooting Steps Taken:

Verified that .env.sample-debug exists in the project root and has no formatting issues (e.g., no spaces around =).
Confirmed dotenv.gradle is being applied by adding a debug log:println("Using ENV_FILE: " + project.envConfigFiles[project.getProperty('ENVFILE')])

Output showed the correct .env file is being used.
Checked BuildConfig in the decompiled APK. The variables from .env were not embedded in BuildConfig.
Restarted Metro with yarn start --reset-cache.
Updated to the latest compatible version of react-native-config.
Tried a minimal .env file (e.g., TEST_KEY=hello-world), but Config.TEST_KEY was still undefined.

Additional Notes:

This issue occurs consistently for both debug and release builds.
Using ./gradlew clean and rebuilding did not resolve the issue.
I am unable to find the variables in BuildConfig after decompiling the APK.

I am experiencing this exact same issue!  Works fine in iOS, but the Config object is undefined in Android.

@DhanushKG-25, I tried this solution, and it worked for me:  https://github.com/lugg/react-native-config/issues/223#issuecomment-441662643.

I've similar issue I'm using react-native cli not expo. I was upgrading my app with the help of react-native-upgrade helper but whenever i switch the version from 0.73.11 to 0.74.0 the npx react-native config return "project": {"ios": null}" and it does not let me install pods saying undefined method for nil class in

The suggestions in this thread didn't resolve it for me (RN 0.76.5)
Here's my hack:
    : Platform.select({
      ios: Config,
      android: {
        TELEMETRY_DECK_APP_ID: '04256010-DEE0-4BA9-B27C-15C3BB8A76BA',
        BUNDLE_STORYBOOK: 'true',
      },
      default: Config,
    });

God speed developers!


Implement dispatch event to pass gain level values to Meter component

Fixes Meter in GainPlugin example.

Read gainPeakValue in processBlock.

Initialise 100ms (arbitrary number) Timer to call dispatchEvent passing gain values to Meter component

Add public getReactAppRoot  getter to GenericEditor in order to call dispatchEvent from appRoot instance created by it as suggested by Nick.

@nick-thompson , happy for this to be merged now?

@nick-thompson , happy for this to be merged now?

Yep this is excellent! Thanks very much @ilionic 🙏 , I'm going to squash these commits into one on the way in

Yep this is excellent! Thanks very much @ilionic 🙏 , I'm going to squash these commits into one on the way in

Noticed build failed due to Juce 6 namespacing left overs, sorry about that :)

Noticed build failed due to Juce 6 namespacing left overs, sorry about that :)

No worries! I'm surprised I didn't notice it before hitting the merge button 🤷♂ . Anyways, all is good now

No worries! I'm surprised I didn't notice it before hitting the merge button 🤷♂ . Anyways, all is good now

Actually might be good idea to run checks on PR's as well, not only upon master merge?

Actually might be good idea to run checks on PR's as well, not only upon master merge?

Definitely!

Definitely!


Revert "Add ValueProvider class for FileBasedSource I/O Transforms"

This reverts commit 1e2168a127fb3047fb15d231a001bbf951892e11.
Break post commit tests, it is a clean rollback.
R: @robertwb
cc: @mariapython

LGTM

Coverage remained the same at 70.001% when pulling c018b13e4b0151ec601d1136b630a17405eab50e on aaltay:rollb into 8e5cfdea9d331b2bfac0dd054bafc7f3f5295b48 on apache:master.


run rush change -b if targetBranch equal imodel02

Add logic to catch  edge case where target branch and current branch both equal refs/heads/master
Issue was caused by a previous change that changed logic when current branch == target branch
Previous change: https://github.com/iTwin/itwinjs-core/pull/4819
The root cause of these changes seem to be running rush change on imodel02 branch instead of master
This fix may be a band aid because it affects our nightly runs of the iTwin.js pipeline. Reconsidering the purpose of this script might be necessary to improve its logic.

@mergifyio backport imodel02


Worker daemons produce a misleading error message when starting

Expected Behavior
Worker daemons should not produce misleading (and unimportant) error messages.
Current Behavior
Worker daemons produce an error on startup that "This JVM does not support getting OS memory, so no OS memory status updates will be broadcast", which is misleading (the jvm does support getting OS memory) and unimportant (this doesn't matter to worker daemons, anyways).
Context
Worker daemons use the same mechanism to produce and broadcast memory information as the build daemon (DefaultMemoryManager).  However worker daemons get injected with a disabled OS memory info instance.  This is because they only produce jvm information that is sent back to the build daemon - they do not need os memory for anything.  However, because they are using the the same infrastructure, a warning is produced about the fact that OS memory cannot be queried.

Hi, I saw the very same message and thought there is some access issue until I saw the post. Can we reopen the issue and remove this misleading message ?


FileNotFoundError: [Errno 2] No such file or directory: 'config.yaml'

Good afternoon. I seem to be doing everything as always. But the program does not see the configuration file in the directory. What am I doing wrong?

You're doing nothing wrong - the config path wasn't specified correctly in the Dockerfile. I've updated the repo which should now fix the problem.

I updated the docker image, but the problem with the configuration file remained.

Thank you. Everything is working now.
I have two more questions about work

Are the covers for the collection not loading? or am I having problems with pictures for collections?

Apparently there is some kind of problem with adding TV series. I have 1300 TV series in my library and not one of them was not included in the collections created by your plugins.

Thank you. Everything is working now.
I have two more questions about work
1. Are the covers for the collection not loading? or am I having problems with pictures for collections?

This script doesn't currently add covers to the collections it makes (See #33 ). You can of course add your own manually in jellyfin for now.

2. Apparently there is some kind of problem with adding TV series. I have 1300 TV series in my library and not one of them was not included in the collections created by your plugins.

Can you open a new issue with the specific lists you're using?


feat: change default cairo ver

Partial fix for #1271
Introduced changes
Set the default cairo_version to 1 since Cairo 1 contracts are now the default. Otherwise, initializing a Cairo 1 contract without setting cairo_version=1 throws a validation error.

[x] This PR contains breaking changes

Cairo version now defaults to 1.

@tserg
Feel free to explore other issues, you're welcome to tackle any that you find interesting. Appreciate your work!


Fix wording of "proper nouns followed by a verb" example

Thanks for the course!
Going through, noticed that example where we're finding PROPN followed by a VERB prints out the wrong direction of that relationship. :)

Ahhh, thanks! Looks like I messed this up when I adjusted that exercise to fix #6 🤦♀


IIR filter math wrong?

https://github.com/voidqk/sndfilter/blob/master/src/reverb.c#L128
static inline float iir1_step(sf_rv_iir1_st *iir1, float v){
	float out = v * iir1->b1 + iir1->y1;
	iir1->y1 = out * iir1->a2 + v * iir1->b2;
	return out;
}

I think that should be:
	iir1->y1 = out * iir1->a2 - v * iir1->b2;

need to confirm

nvm I'm wrong


adds dependabot configuration

@adhiambovivian i don't have merge permissions on this repo, you'd have to merge for me please.


Matrix entry with custom marking

On creating a matrix entry part to a question, if one tries to extend the base marking algorithm an error shows above the unit tests saying "Error marking: There was an error in this part's marking algorithm. Please report this. Numbas.jme.evaluate must be given a Scope". As a result, the answers are no longer marked as correct but no error (of the form above) is shown. I haven't been able to find a way to make this work even if I replace, rather than extend, the base marking algorithm.
MWE (obviously not the question I've got planned!) here: https://numbas.mathcentre.ac.uk/question/share/view/79393b08-8efa-43b2-99ad-3defa86be96f
Removing the custom marking makes the question behave as expected.

You didn't even need to make a custom marking algorithm. This bug was introduced when I added a 'scope' parameter to the part marking function. Thanks for reporting it!
Fixed by numbas/editor@2a1d25be3bd601ed2cdfe9f66d41edccb705a903


SP-3151- Backport of PDI-15591 - Fuzzy Match step throws NullPointerException on null data fields

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/steps/fuzzymatch/FuzzyMatch.java

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory

Branch Change:  + 7.14%
Complexity Change:  + 7.69%

org.pentaho.di.job.entries.getpop.MailConnectionMeta

Branch Change:  + 1.00%
Complexity Change:  + 1.41%
Instruction Change:  + .30%

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change:  + 3.28%
Line Change:  + 3.87%

org.pentaho.di.trans.step.StepMeta

Instruction Change: -.19%:small_red_triangle_down:
Line Change: -.27%:small_red_triangle_down:

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change: -.04%:small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change:  + .66%
Complexity Change:  + .41%
Instruction Change:  + .50%
Line Change:  + .24%

org.pentaho.di.trans.steps.fuzzymatch.FuzzyMatch

Branch Change:  + .27%
Complexity Change: -.19%:small_red_triangle_down:
Instruction Change:  + .18%
Line Change:  + .06%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -4.17%:small_red_triangle_down:
Complexity Change: -4.21%:small_red_triangle_down:

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -2.35%:small_red_triangle_down:
Complexity Change: -2.70%:small_red_triangle_down:
Instruction Change: -.18%:small_red_triangle_down:
Line Change: -.48%:small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change:  + .07%

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change:  + .53%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.www.TransformationMap

Branch Change: -1.14%:small_red_triangle_down:
Complexity Change: -1.54%:small_red_triangle_down:

Integration test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.trans.step.BaseStep

Complexity Change: -.18%:small_red_triangle_down:
Instruction Change: -.20%:small_red_triangle_down:
Line Change: -.38%:small_red_triangle_down:
Method Change: -.49%:small_red_triangle_down:

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/steps/fuzzymatch/FuzzyMatch.java

Still Broken Tests:
org.pentaho.di.trans.TransTest.transWithNoStepsIsNotEndless:
:red_circle: java.lang.Exception
java.lang.Exception: test timed out after 1000 milliseconds
	at org.mockito.cglib.core.CollectionUtils.bucket(CollectionUtils.java:35)
	at org.mockito.cglib.core.EmitUtils.string_switch_hash(EmitUtils.java:260)
	at org.mockito.cglib.core.EmitUtils.string_switch(EmitUtils.java:171)
	at org.mockito.cglib.reflect.FastClassEmitter.signatureSwitchHelper(FastClassEmitter.java:160)
	at org.mockito.cglib.reflect.FastClassEmitter.emitIndexBySignature(FastClassEmitter.java:122)
	at org.mockito.cglib.refle

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.injection.bean.BeanInjector

Branch Change: -1.27%:small_red_triangle_down:
Complexity Change: -2.00%:small_red_triangle_down:

org.pentaho.di.job.entries.ftpsget.JobEntryFTPSGet

Branch Change: -.53%:small_red_triangle_down:
Complexity Change: -.56%:small_red_triangle_down:
Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change: -.66%:small_red_triangle_down:
Complexity Change: -.93%:small_red_triangle_down:
Instruction Change: -.13%:small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Instruction Change:  + .19%
Line Change:  + .27%

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change: -.11%:small_red_triangle_down:

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change:  + .10%

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Instruction Change: -.09%:small_red_triangle_down:

org.pentaho.di.trans.steps.fuzzymatch.FuzzyMatch

Branch Change:  + .27%
Complexity Change: -.19%:small_red_triangle_down:
Instruction Change:  + .18%
Line Change:  + .06%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 1.04%
Complexity Change:  + 1.05%

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change: -.05%:small_red_triangle_down:

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change:  + .06%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -3.53%:small_red_triangle_down:
Complexity Change: -4.05%:small_red_triangle_down:
Instruction Change: -.35%:small_red_triangle_down:
Line Change: -.97%:small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change:  + .14%

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.systemdata.SystemDataMeta

Branch Change: -2.04%:small_red_triangle_down:
Complexity Change: -2.04%:small_red_triangle_down:
Instruction Change: -.32%:small_red_triangle_down:
Line Change: -.81%:small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change: -.91%:small_red_triangle_down:
Complexity Change: -.82%:small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.www.TransformationMap

Branch Change:  + 3.41%
Complexity Change:  + 3.08%
Instruction Change:  + .94%
Line Change:  + 1.54%

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  engine/src/org/pentaho/di/trans/steps/fuzzymatch/FuzzyMatch.java
  engine/test-src/org/pentaho/di/trans/steps/fuzzymatch/FuzzyMatchTest.java

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory

Branch Change: -7.14%:small_red_triangle_down:
Complexity Change: -7.69%:small_red_triangle_down:

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change:  + 1.01%
Complexity Change:  + 1.06%
Instruction Change:  + .17%

org.pentaho.di.trans.step.StepMeta

Instruction Change:  + .19%
Line Change:  + .27%

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change:  + .11%

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change: -.10%:small_red_triangle_down:

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta

Branch Change:  + .66%
Complexity Change:  + .41%
Instruction Change:  + .50%
Line Change:  + .24%

org.pentaho.di.trans.steps.fuzzymatch.FuzzyMatch

Branch Change:  + 1.42%
Complexity Change:  + 1.71%
Instruction Change:  + .44%
Line Change:  + .40%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -3.12%:small_red_triangle_down:
Complexity Change: -3.16%:small_red_triangle_down:
Instruction Change: -.14%:small_red_triangle_down:
Line Change: -.37%:small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change: -2.00%:small_red_triangle_down:
Complexity Change: -1.89%:small_red_triangle_down:
Instruction Change: -.17%:small_red_triangle_down:

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change:  + .06%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change:  + 3.53%
Complexity Change:  + 4.05%
Instruction Change:  + .18%
Line Change:  + .48%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change:  + .07%

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta

Instruction Change: -.14%:small_red_triangle_down:

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change:  + .18%

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.systemdata.SystemDataMeta

Branch Change: -2.04%:small_red_triangle_down:
Complexity Change: -2.04%:small_red_triangle_down:
Instruction Change: -.32%:small_red_triangle_down:
Line Change: -.81%:small_red_triangle_down:

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta

Branch Change:  + .91%
Complexity Change:  + .82%

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change: -.53%:small_red_triangle_down:


Porting all v1 api to v2

I'm HAPPY to say that most of the core function in v1, has been port to v2 version!
cheers!


Integrate SAM predictor and improve mouse event handling

This pull request integrates the SAM predictor into the main application, allowing users to generate segmentation masks and bounding boxes with a single click. It also improves mouse event handling for a more seamless user experience.
Main changes include:
Integration of the SAM predictor:
Added a new function to load images into the predictor and obtain segmentation masks and bounding boxes.
Processed the masks and added them to the LabelImg canvas, allowing users to view and edit the generated annotations.
Improved mouse event handling:
Fixed issues related to handling mousePressEvent and mouseReleaseEvent.
Streamlined code to handle different mouse button events and their corresponding actions.
Enhanced user experience by providing appropriate cursors and smoother interactions.
These changes will significantly improve the annotation process for users, making it more efficient and intuitive. We believe this integration will be a valuable addition to the project and will greatly benefit the user community.
Call the setup_sam method when the user provides the SAM checkpoint path. You might want to add a menu item or a button in the GUI to let the user choose the checkpoint file.

Related to #89


doc: Change old Lean 3 commands to Lean 4 in implementation notes

I changed Lean's 3 old "variables" command to Lean's 4 command "variable" in some implementation notes.
I might have missed some

Taking into account Damiano's comment LGTM, thanks!
bors d+

bors r+

It seems like a random failure, let's try again.
bors merge


feat: Add support to Kui proxy for pre-shared-key authorization

This PR also bumps the npm-shrinklock deps in packages/proxy/app
Fixes #7772

Description of what you did:

My PR is a:

[ ] 💥 Breaking change
[ ] 🐛 Bug fix
[ ] 💅 Enhancement
[x] 🚀 New feature

Please confirm that your PR fulfills these requirements

[x] Multiple commits are squashed into one commit.
[x] The commit message follows Conventional Commits, which allows us to autogenerate release notes; e.g. fix(plugins/plugin-k8s): fixed annoying bugs
[x] All npm dependencies are pinned.

/lgtm


Out of memory

The build doesn't seem to exist anymore - but here is the error log.
\x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: SIL.Machine.AspNetCore.Services.ClearMLMonitorService[0]
      Build faulted (65419fb23627520aa5115888). Error: OutOfMemoryError : CUDA out of memory.
      Tried to allocate 16.00 MiB (GPU 0; 23.68 GiB total capacity; 22.84 GiB already allocated; 47.50 MiB free; 23.24 GiB reserved in total by PyTorch)
      If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.
      See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Should we use PYTORCH_CUDA_ALLOC_CONF?  Or, should we just use a smaller batch size when on a 3090 rather than the A100?  If so, let's have a recommendation to give out so that SF, etc. can reliably run jobs.

https://stackoverflow.com/questions/73747731/runtimeerror-cuda-out-of-memory-how-setting-max-split-size-mb
Alternatively, we may be able to catch out-of-memory errors and just torch.cuda.empty_cache()  - which should clear up the memory.  It's "expensive", but should be acceptable rather than bombing the entire pretranslation job.  I don't know the best implementation for this, possibly this pseudocode:
def pretranslate_all():
    for batch in pretranslations:
        try:
            pretranslate()
        except OutOfMemoryError as e:
            # See if clearing the cache helps
            torch.cuda.empty_cache()
            pretranslate()

while(config["batch_size"] > 8):
    try:
        pretranslate_all()
        break
    except OutOfMemoryError as e:
        # let's reduce the batch size and try to pretranslate again
        torch.cuda.empty_cache()
        config["batch_size"] = config["batch_size"] // 2

After being able to successfully train the model, we should not be failing the job from OOM errors during pretranslate.  Through releasing cached memory and incrementally reducing the batch size, we should be able to avoid these errors.

In my experience I've usually needed to reduce the batch size when running jobs on the 3090, since it would fail during the training stage for me. However, I think the idea of clearing the cache and reducing the batch size automatically when encountering memory issues is a good one.

I believe we should investigate setting the batch size for inferencing to 1. This would remove the chance for OOM errors. Since we are aiming to support any language, we don't have a good handle on how variable verse lengths will be. If verse lengths are not regular, batching can have a negative impact on inferencing speed. Here are guidelines for batching from the Huggingface transformers documentation:

For users, a rule of thumb is:

Measure performance on your load, with your hardware. Measure, measure, and keep measuring. Real numbers are the only way to go.
If you are latency constrained (live product doing inference), don’t batch
If you are using CPU, don’t batch.
If you are using throughput (you want to run your model on a bunch of static data), on GPU, then:

If you have no clue about the size of the sequence_length (“natural” data), by default don’t batch, measure and try tentatively to add it, add OOM checks to recover when it will fail (and it will at some point if you don’t control the sequence_length.)
If your sequence_length is super regular, then batching is more likely to be VERY interesting, measure and push it until you get OOMs.
The larger the GPU the more likely batching is going to be more interesting

As soon as you enable batching, make sure you can handle OOMs nicely.

Biggest thing from the advice As soon as you enable batching, make sure you can handle OOMs nicely.  I will be ok with any solution that stops the OOMs.

Another relevant quote for our case is

If you have no clue about the size of the sequence_length (“natural” data), by default don’t batch

More errors:
2023-11-02 12:39:57
ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start
2023-11-02 12:41:27
Your input_length: 200 is bigger than 0.9 * max_length: 200. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)
2023-11-02 12:48:17
Your input_length: 188 is bigger than 0.9 * max_length: 200. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)
2023-11-02 12:54:27
Your input_length: 297 is bigger than 0.9 * max_length: 200. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)
2023-11-02 12:57:46
/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py:1101: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
  warnings.warn(
2023-11-02 12:59:04
Your input_length: 218 is bigger than 0.9 * max_length: 200. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)
2023-11-02 13:06:05
Your input_length: 233 is bigger than 0.9 * max_length: 200. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)
2023-11-02 13:07:13
Your input_length: 596 is bigger than 0.9 * max_length: 200. You might consider increasing your max_length manually, e.g. translator('...', max_length=400)
rror: OutOfMemoryError : CUDA out of memory. Tried to allocate 24.00 MiB (GPU 0; 39.39 GiB total capacity; 35.84 GiB already allocated; 13.94 MiB free; 38.88 GiB reserved in total by PyTorch)

@ddaspit - did we remove truncation?  It looks like a nearly 600 token segment caused it to bomb.  This may still be there even if we don't batch.  Someone could put in a 2000 token segment and still cause the issue.

Truncation hasn't been removed.

If that is so, why did the A100 get a OOM?

it appears that as the warnings about long segments get larger, more GPU memory is used.  If it is truncating, why would that be an issue?

It looks like the truncation strategy would be passed down - but has it been verified?

Yes, I verified it.

We are already running very close to the memory limit. If a batch contains a lot of 200 length verses, then it could push the memory usage over the limit. Dropping the batch size down to 1 should fix this issue.

I am reducing the batch size to 1 and then just waiting for the issue to happen again (if it will).

Timing for different batch sizes:

For translating GEN, EXO, RUT, EST, and JON from Hindi to Maithili:
The batch_size: 16 version finished in 14:50 (min:sec).
The batch_size: 1 version finished in 53:49.
Configs here if you're interested: S:\MT\experiments\FT-India\

Based upon it taking nearly 4 times as long without batching, let's batch, catch any OOM's and then reduce the batch size by half until we get to 1.

This has been resolved by auto-stepping down the batch size.


DeprecationWarning

What version of this package are you using?
16.0.3
What operating system, Node.js, and npm version?
Mac + Node v16.5.0
What happened?
DeprecationWarning
(node:97401) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./2020/" in the "exports" field module resolution of the package at /usr/local/lib/node_modules/standard/node_modules/es-abstract/package.json.
Update this package.json to use a subpath pattern like "./2020/*".
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:97401) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./helpers/" in the "exports" field module resolution of the package at /usr/local/lib/node_modules/standard/node_modules/es-abstract/package.json.
Update this package.json to use a subpath pattern like "./helpers/*".

What did you expect to happen?
not to see any warning
Are you willing to submit a pull request to fix this bug?
Nope

Indeed, just tested it, on my local machine with Node.js v16.6.0.
Good catch, thanks! :+1:

Related issue: https://github.com/ljharb/es-abstract/issues/133

Closing as the issue has been fixed.
To make sure you have the fix, please delete package-lock.json and node_modules, and run npm install, that way you'll get automatically the latest updates of es-abstract, so no DeprecationWarning.

To make sure you have the fix, please delete package-lock.json and node_modules, and run npm install, that way you'll get automatically the latest updates of es-abstract, so no DeprecationWarning.

An npm update or yarn upgrade should also do. "Lockfile maintenance" :)


HTML template: Facilitate Specification Extensions

Fixes https://github.com/asyncapi/generator/issues/136

@fmvilas It is finished. Ready to be merged.


Send the progress report to the logger

Expected Behaviour
Expected all the messages to be written to the logger.
tickerProgress directly writes to stdout and assumes its an interactive terminal that is being consumed by an human, which is something that I would expect to not be applicable to a Tinkerbell action, which is executed by a machine.
Current Behaviour
While trying to send all logs to a central place, this is what happens:

Possible Solution
Use the logger to output the progress report as its already being used at:
https://github.com/tinkerbell/hub/blob/9dbfd747f8754314e03441bd91aa8093dcc851f4/actions/image2disk/v1/pkg/image/image.go#L88
Also make the progress report less verbose (e.g. every 5 seconds and only if there is something new to report; the current 500ms is too much verbose).
Steps to Reproduce (for bugs)
Observe the logs in a file instead of a terminal.

Looks reasonable, thanks for flagging, added to backlog

Obsolete


Parsing error for getAvailableGifts method from 8.0 Telegram Bot API

Error: Parser: Failed to extract type from description: "Returns the list of gifts that can be sent by the bot to users. Requires no parameters. Returns a Gifts object."

Caused by:
    Failed to extract type from description: "Returns the list of gifts that can be sent by the bot to users. Requires no parameters. Returns a Gifts object."

I guess this error happends because telegram mention which returns twice in the description.
Any workaround?
This error in CICD logs

https://github.com/ark0f/tg-bot-api/pull/22 fixes this

closed because https://github.com/ark0f/tg-bot-api/pull/22 merged


job should run every day at 18:00. if run before loop, next run would be 18:00 tomorrow.

Hi,
If the job need to be run at 18:00 everyday.
I run the job before the loop and the time is 10:00. after job run, next_run would be 18:00 tomorrow. not today.
Is the next_run should be 1800 today if the run time is early then 1800 ?
Code:
def job():
    logger.info('Job running...')

dailyJob = schedule.every(1).days.at('18:00').do(job)
dailyJob.run()
logger.info(dailyJob)

Logs:
2020/03/25 10:26:28	[INFO]	Running job Every 1 day at 18:00:00 do job() (last run: [never], next run: 2020-03-25 18:00:00)
2020/03/25 10:26:28	[INFO]	Job running...
2020/03/25 10:26:28	[INFO]	Every 1 day at 18:00:00 do job() (last run: 2020-03-25 10:26:28, next run: 2020-03-26 18:00:00)

Thanks for bringing this up! This has been solved in #337 and released as part of release 1.0.0.


Lazyloaded Swift_Message fails to initialize when using memory queue.

Since https://github.com/symfony/MonologBundle/pull/109 the Swift_Message used for email logging is lazy-loaded. This seems to interact badly with the memory queue strategy, because it clones messages.
It seems only the initial message is properly (lazily) initialized, but the clone is not, so, this might be an error in the lazy load library as well, but I figure I mention it here since this is where I first ran into this effect.
The specific error you'll get is: Fatal error: Uncaught exception 'Swift_TransportException' with message 'Cannot send message without a sender address'

Actually, upon further investigation this must be more related to the ProxyManager code, so I'll close this issue.


Update Start Page

[x] Text is outdatd (we support more languages)
[ ] The style of the page may be prettified (optional)
[x] The stlye buttons should be in a drop-down menu at the top (like in editor page)

Not important


refactor: Extract common login code to script

Summary
Solves: #2974
Testing Process

[ ] Add how to test.

Checklist

[x] Review the contributing guidelines
[x] Add a reference to related issues in the PR description.
[ ] Update documentation if applicable.
[ ] Add unit tests if applicable.
[ ] Add changes to the CHANGELOG if applicable.

I'm not exactly sure how to test this.
I suspect I should use this e2e script but I don't know how to point to my changes.
Any help is appreciated.