unique: new package with unique.Handle

Proposal: unique package
Updated: 4 October 2023
Oct 4th EDIT: Changed package name from "intern" to "unique" and renamed "Symbol" to "Handle" based on feedback.
CC @golang/runtime
Motivation
The lack of runtime support for interning in Go represents a gap with other languages. Demand in the Go community for a weak map and/or string interning has shown up in a few GitHub issues over the years. Although the section of the community requesting these features is quite small, we see the consequences of not having built-in support for this functionality in the form of the go4.org/intern package.
go4.org/intern is a package that implements a global intern pool. Its goals are two-fold: intern data to deduplicate copies, and provide fast comparisons for such interned data. Functionally, it returns a unique global identity for every value placed into it. That value is weakly held by the pool's internals, to avoid letting the pool accumulate in an unbounded manner. This implementation follows in the footsteps of Lisp: interning a value produces a "symbol" that may be used for a fast direct comparison and provides a method through which the interned value may be recovered.
Although it has only 4 direct importers, it is transitively used quite widely (estimated 0.1% of modules) thanks to use by inet.af/netaddr (reverse module deps), which uses it to deduplicate strings and reduce the cost of their comparisons. This has caused friction within the ecosystem because go4.org/intern makes assumptions about the implementation of Go. In particular, it imports the package go4.org/unsafe/assume-no-moving-gc to validate those assumptions. That package, by design, needs to be updated with every release of Go.
inet.af/netaddr's functionality has also recently been merged into the standard library as the net/netip package, bringing along with it a copy of go4.org/intern as the internal/intern package. Furthermore, go4.org/unsafe/assume-no-moving-gc now references an internal Go runtime variable to avoid having to update every release.
Although the immediate issue has been mitigated, the gap in functionality remains, and we now have a copy of go4.org/intern in the standard library anyway.
Goals
The goal of this proposal is to tidy up internal/intern's interface with generics, integrate it more tightly into the runtime (for efficiency and future-proofing), and expose it as a standard library package.
This proposal will also motivate why we should move forward with an API that's similar go4.org/intern and not some other direct or indirect ways of enabling efficient value interning.
Design
We more-or-less propose to move the go4.org/intern package API into the standard library, but with a few tweaks. The biggest difference is that go4.org/intern uses an interface to represent the "key" part of the mapping, with a special case for strings (GetByString) to avoid an additional allocation for the string header. Using generics, we can avoid these additional allocations while also improving type-safety and ergonomics.
Here's a sketch of the revised API, which will live in a new package called "unique":
package unique

// Make returns a globally unique handle for a value of type T. Handles
// are equal if and only if the values used to produce them are equal.
func Make[T comparable](value T) Handle[T] {
    ...
}

// Handle is a globally unique identity for some value of type T.
//
// It is trivially and efficiently comparable with other Handle[T] for the same T.
//
// If the value used to create the symbol is the same, the symbols are guaranteed
// to be equal.
type Handle[T comparable] struct {
    value *T
}

// Value returns a shallow copy of the T value that produced the Handle.
func (h Handle[T]) Value() T {
    return *h.value
}

The unique package must maintain an internal mapping of values to globally unique symbols, which risks growing without bound. However, once no copies of symbols produced by Make are reachable, the program loses access to that globally unique identity. The intern package is then free to produce a new one without the program noticing. If it can produce a new one, it can just as well delete the old one during that time period in which no copy of a symbol exists for a given value.
In practice, integration with the garbage collector is necessary to achieve this, introducing the risk of leaking details about the garbage collector's implementation.
A key property to consider with functionality that relies on specific behavior from the garbage collector is whether it preserves the illusion that the executing program has infinite memory (with a GC there's a malloc but no free). The utility of such an illusion is that it makes programs significantly simpler to reason about. When programs are able to observe the fact that the garbage collector reclaimed memory, it is possible for programs to rely on non-deterministic properties of that reclamation. Finalizers are a good example of a feature that breaks the illusion and are difficult to use correctly.
Luckily, it's not possible for programs to notice when New returns a different Handle[T] precisely because it can produce new symbols without the program noticing. Therefore, it's not possible for a program to observe when memory is reclaimed. (Someone could use the unsafe package to write down the Handle[T]'s internal value as a uintptr, but that specifically requires the use of unsafe. One also can't do much with that uintptr; casting that back to a pointer violates the unsafe.Pointer rules.)
Implementation
The core data structure is approximately a map[any]*T, where the *T is a weak reference (effectively a uintptr). The runtime fully controls the *T created here, so it would attach a special to it that references the map entry. Once the *T is no longer referenced, the GC will clear the pointer in the map entry. Later, a background goroutine will clean up map entries with nil values.
 It's possible that in between the garbage collector noticing that there are no more references to the *T and the value being collected, the program may read the value out and produce a new strong pointer to the *T. To alleviate this, the reader needs to leave behind a marker in the map entry indicating that the *T was resurrected. If the garbage collector notices that the *T was resurrected when it tries to sweep it, it must re-mark the value as live and unset the resurrection bit. All of this also implies that accesses to the *T in the map entry must be atomic with this marker. This is easy to do: we can steal a bit in the pointer for this state (i.e. pointer tagging) and access the *T atomically. To avoid frequent writes to the map value, the reader can safely CAS the bit only if it sees the bit is unset after the initial load. If it loses the race with the GC, it takes the slow path and creates a new identity value.
This is very similar to the implementation of go4.org/intern, but because we have access to runtime internals, we can release the memory in at most 2 GC cycles instead of 3: one to let the resurrection bit drop, another to finally collect the value.
Next is the map implementation itself. Because the map will be global, the underlying data structure will need to be thread-safe. Unfortunately there are far too many choices for a concurrent map data structure, so we need to cut them down by setting some goals and requirements. Here's a list:

Reads are expected to be much more popular than writes.

Serializing writers (though allowing them to be concurrent with readers) is likely acceptable.

Reads from the map should be allowed to execute concurrently with one another.
Deletions from the map can happen relatively rarely (once per GC), in the background, and in bulk.
Iteration is not strictly necessary since a candidate list of removals may be maintained by the GC.

If it turns out to be desirable, there could only ever be a single iterator at a time for deletion.

To eliminate an indirection, being able to directly mutate map values would be nice.
The map needs to be able to grow and shrink efficiently.

A traditional bucketed hash map is not terribly difficult to make concurrently under these circumstances, but has the downside that incremental growth and shrinking of such a map is quite complicated. Trying to extend the existing map implementation with concurrency features is also likely to be complicated.
Another option is to use something like an adaptive radix tree over 8-byte hash values. The growth and shrinking of such a tree comes naturally, and making them concurrent within the bounds of our requirements isn't too complicated. The downside is poor locality because of the tree structure. I suspect that at least to begin with, something like this will provide good enough performance.
As an additional note on performance, calls to Make should always avoid allocating in the case where the provided value is already in the map. Meanwhile they should explicitly clone the value when adding it to the map. (This means if T is a struct with pointers in it, the values those pointers point to are almost always going to be forced to escape to the heap, like with regular Go maps.)
Risks
API design impact
The fact that interned values are represented by a type Handle[T] means that details around interning may encourage two versions of APIs, one supporting interned values and the other not. Contrast this with an "opaque" alternative (see "Opaque string interning" in the alternatives section) that makes no distinction between interned and non-interned values in the type system.
I believe this is a legitimate concern, but suspect that it will mostly be mitigated in practice. Interning is a somewhat niche technique that helps performance dramatically in certain cases, but only in those certain cases that clearly benefit from data deduplication and/or fast comparisons of data. Elsewhere it's more clearly just cumbersome and simple microbenchmarks should reveal the slowdown.
Thus, I think "polluting" APIs in the cases where it's useful is likely worth the tradeoff, and it's sufficiently cumbersome to use that where it's not necessary it will simply be ignored. I believe the fact that go4.org/intern has relatively few direct importers supports this hypothesis.
Poor performance
One situation we absolutely want to avoid is performance issues like with Java's String.intern. My best understanding of the situation (as of when the linked blog post was written) is that:

The global intern map is not resizable, so once the map fills up, lookups become expensive.
The global intern map leaks memory (map entries are not bound to the lifetime of their values).
The global intern map is part of the GC root set, and it's not well accounted for by the GC (it's just scanned during a STW), leading to high worst-case pause times when the map has many entries.

I believe we avoid all three of these issues in the implementation described above. The third one is less obvious if you're not familiar with the existing Go GC implementation. Briefly, the global intern map would only be a GC root insofar as any global variable is a GC root (i.e. the entire map would not be part of the root set, just the reference to it). And even if it was fully part of the root set, the Go GC already shards and scans the root set concurrently with the mutator executing. Hence, no pause-time impact.
Disclaimer: I don't know if this is still an issue in the Java world; I didn't look into this too deeply. If it's not, then that's great to hear. Nevertheless, it's still worthwhile to learn from past attempts at interning.
Alternatives considered
Opaque string interning
One alternative is interning just for strings, which is a common case. For instance:
package strings

// Intern takes a string and returns a string with equivalent contents,
// though possibly one that shares memory with other strings. This can
// be used to save memory in applications with many duplicate string
// values. If a string is successfully interned, a copy is always made.
func Intern(s string) string

Although such an API is temptingly simple, it's not really useful for any other kind of data structure. Only strings are properly immutable in the language.
Furthermore, because the interning is opaque, we don't get the full benefit of cheap comparisons out-of-the-box. The string comparison fast path would be taken more frequently, but when there's no pointer equality the string data would still need to be compared. To get the full benefit, there would need to be some additional runtime and/or compiler support to identify when two interned strings are being compared, which may end up slowing down non-interned string operations as well.
Background string deduplication
Another alternative is to just have the runtime deduplicate strings in the background. For instance, the JVM G1 garbage collector has a flag for such a feature (off by default). The advantage of this approach is the programmer has to set one flag and they get to save on memory costs.
However, like the previous alternative, this only really applies to strings again, because only strings are properly immutable at the language level. The other problem is that this feature would need to be opt-in, requiring a new top-level runtime knob. (Presumably this feature isn't on by default because it's not always worth the CPU cost in the garbage collector.) It's also substantially more complex to implement this in the Go garbage collector, because it doesn't currently know the type of anything in the heap.
Weak references
An even more general alternative to the proposed API is to just add support for weak references to the standard library and/or language. After all, the proposed implementation conceptually just uses a weak reference in its implementation anyway.
The main issue with weak references is that they're very hard to use in a system with tracing garbage collection, since they can turn out to be nil at very surprising times (or possibly never). Fundamentally, they break the aforementioned infinite memory illusion, because they reveal when memory is reclaimed.
The bar is extremely high for adding anything this difficult to use, and I believe we should prefer easier-to-use abstractions as much as possible.

@seebs We went deep down the weak-keyed maps (specifically, ephemerons, which preserve the GC abstraction) rabbit hole as a solution for interning. I think we concluded that it's insufficient for interning, though they are useful for other purposes (as you note, map[*T]metadata). I actually have a rough draft of a proposal for that somewhere, but we focused on this because it seems more immediately important.
RE: Weak-value maps, I believe I came to the conclusion that those are basically weak references, and thus suffer from the same issues.

String interning with the proposed API would be equivalent to this right?
var b []byte = ...
s := intern.New(string(b)).Value()

where the string(b) conversion does not allocate, but intern.New may allocate if the string is not found.

IIUC, this doesn't solve the global type cache problem because you can't attach a value to the key.
Quite a few packages have a global map that caches a complex data structure for a Go type:
https://github.com/golang/go/blob/e844d72421fb34b57eddf2653b33ed5ebf146b64/src/encoding/json/encode.go#L336
Today, there's a memory leak in "json" where dynamically constructed types with reflect.New can never be reclaimed because json.encoderCache is never cleared.

String interning with the proposed API would be equivalent to this right?

That's basically right, but in this design you're encouraged to keep the Symbol[string] instead of immediately getting the string header back out. As soon as symbols disappear, the internal map entry for them becomes eligible for cleanup. Of course, it won't get cleaned up until at least the next GC cycle, so subsequent lookups will still probably be a hit in the map. But you'll likely get better behavior out of keeping the symbol. Plus, you get a cheap comparison.

you're encouraged to keep the Symbol[string] instead of immediately getting the string header back out

That seems a bit unfortunate. The "json" package when unmarshaling would be storing a string into the user type. There's no place to hold onto a Symbol[string] for long, storing it somewhere leads us back to the same problem this is trying to solve. That said, there is still benefit since we will share string values within a GC cycle.

EDIT: I have to look back in my notes and chat history to recall why weak-keyed maps were insufficient for interning.

I think the reason weak-keyed maps weren't a great solution because if you allow anything comparable except a pointer, you end up being able to break the GC abstraction. Heap pointers are special because they are an unforgeable identity. So type WeakMap[K any, PK interface{ *K }, V any] struct { ... } is fine, but allowing all comparable types as keys is not because you can construct a value such that you observe when a map entry is reclaimed.
Notably, for the pointers-only case, it's also surprisingly easy to implement, because the Go runtime already has a way to associate arbitrary data with a pointer. But, you probably also want that weak map to be cyclic. That is WeakMap[*T, *T] should still be allowed to collect the map entry where the same *T exists as a value. That's more complicated but is certainly possible to do. It's just going to impact the mark path, so it needs to be designed carefully to be fast.

IIUC, this doesn't solve the global type cache problem because you can't attach a value to the key.

It doesn't. A WeakMap would, but you'd have to unwrap the reflect.Type into the underlying *internal/abi.Type, which is at least fine for std. This particular case might be common enough to warrant a TypeMap or something.

That seems a bit unfortunate. The "json" package when unmarshaling would be storing a string into the user type. There's no place to hold onto a Symbol[string] for long, storing it somewhere leads us back to the same problem this is trying to solve. That said, there is still benefit since we will share string values within a GC cycle.

I believe the only thing that would fully satisfy this particular use-case is the "opaque string interning" alternative. It may be worth considering in the future as it's somewhat orthogonal to this.
I'm pessimistic that there's a single API that will solve all these problems, but I think this package is a decent starting point.

@mknyszek was it your intention to remove the proposal label, or was it a 'fat-finger' moment?

you're encouraged to keep the Symbol[string] instead of immediately getting the string header back out

That seems a bit unfortunate. The "json" package when unmarshaling would be storing a string into the user type. There's no place to hold onto a Symbol[string] for long, storing it somewhere leads us back to the same problem this is trying to solve. That said, there is still benefit since we will share string values within a GC cycle.

FWIW there is also @josharian's string interning library, which works somewhat as you've described by keeping a sync.Pool of map[string]string around such that strings are mostly interned until the pool gets rid of its entries: https://pkg.go.dev/github.com/josharian/intern
It seems like it'd be possible to create a sync.Pool of map[Symbol[string]]string instead such that the Symbol[string]s get kept around a little longer... but I guess maybe this isn't an improvement because someone's still going to have to hash the string and this just shifts the work elsewhere?
(I do agree that it's unfortunate that the Symbol has to be kept around to ensure that it gets reused.)

(I do agree that it's unfortunate that the Symbol has to be kept around to ensure that it gets reused.)

I suppose we could make an exception for strings and have the Symbol refer directly to the backing store for the string, though now we need to carry around a length in each Symbol as well. (It would just be zero for all other types.) So, the cheap comparison becomes slightly less cheap. Maybe that's fine?
FWIW, I think strings are a reasonable exception given that they're the only type that is defined as a reference yet behaves like a value type.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

I suppose we could make an exception for strings and have the Symbol refer directly to the backing store for the string, though now we need to carry around a length in each Symbol as well. (It would just be zero for all other types.) So, the cheap comparison becomes slightly less cheap. Maybe that's fine?

It occurs to me that if we're going to force a copy and a new allocation for an insertion, then strings are entirely uniquely identifiable via their data pointer (regardless of length). The Value method is slightly more complicated since we'd have to store the length next to the data, it should actually be doable to properly intern strings and also enable using Symbol[string] for cheap (single pointer) comparisons. The downside of this approach is it prevents optimizations like deduplicating strings in the map by prefix. But maybe that's fine.

It took me three-four reads to understand, but it sounds like a useful addition to the standard library.
What I'm wondering is if there's an automatic way in the profiling tooling to recommend "hey, you should intern this value" as part of the output. Otherwise I can't really see the usage of intern becoming more ubiquitous, and then adding the value of adding it to the stdlib is diminished (integrating it more tightly to the runtime is not without it's costs!)

Hi all, I'm still learning about Golang (I'm the group manager for Java and now Golang @ MSFT), so take all of this with that in mind :-).
String Dedup - I wanted to add a quick note that in Java, String Deduplication is accessible to all of the GCs but it is still a runtime flag to enable. https://malloc.se/blog/zgc-jdk18 for the extra detail there. As you can imagine, since it's not on by default it's usage is pretty low, not sure 'we' got the user experience right there.
String intern perf - Although I know you have a different design in mind, I (or more likely one of our real engineers ;-)) will take a look at Java's intern performance and see if the concerns raised above still hold true or if that was patched or a new design put in place. All good things to learn either way I guess. I'll post back here if there's anything interesting.
public + internal? - Is there a use case where keeping an internal version of this for the runtime that can evolve more rapidly for internal only needs as well as an external public facing package make sense? I know if Java they started doing this a bunch with certain com.sun packages within the new module system.
weak references - Your note about using weak references resonates! Speaking from Java-land, its memory management challenges are usually confined to the realms of issues with strongly referenced memory leaks or whatever, but on the odd occasion when WeakReferences go wrong then its an order of magnitude harder to debug with tooling etc. That might just be more a reflection on their design or gaps in o11y tooling but for me its always a case of this is where some Dragons live.

'intern' may not be the best name for this package. It requires knowledge of other systems (particularly Lisp) that not everyone has. Also intern here is not the same meaning as intern string in Java. The package is about creating unique identities for values, so what about package unique? Then the bikeshed becomes what the name of the type is.
'Symbol' has the right meaning for people with a Lisp or compiler background, but not to others. Symbol has many other meanings in many other parts of CS.
unique.Handle seems like it strikes the right notes: the thing is a Handle, and it aligns with cgo.Handle too (although this one uses generics but cgo predated them).
package unique

// New returns a globally unique symbol for a value of type T. Symbols
// are equal if and only if the values used to produce them are equal.
func New[T comparable](value T) Handle[T] {
    ...
}

// Symbol is a globally unique identity for some value of type T.
//
// It is trivially and efficiently comparable with other Symbol[T] for the same T.
//
// If the value used to create the symbol is the same, the symbols are guaranteed
// to be equal.
type Handle[T comparable] struct {
    value *T
}

// Value returns a shallow copy of the T value that produced the Symbol.
func (h Handle[T]) Value() T {
    return *h.value
}

I don't feel strongly about the name. I also think both unique and Handle are reasonable. Happy to update the proposal if that's the consensus.

While we are talking names: I dislike New. 1. New generally is connoted with returning pointers, 2. unique.New reads strangely and  3. it doesn't actually return a new Handle/Symbol, (hopefully) most of the time. I think Make is slightly (though maybe imperceptibly) better. I can't really come up with a significantly better one.

Some JavaScript-inspired bikeshedding:
package symbol

func For[T comparable](v T) Of[T]

If we want more raw name fodder, I believe that in Elixir (something like) this is called an atom.

It might be worth using the term "identity" somewhere here, since that's really what's being created. Putting that together with the verb "make" and the package name "unique", you get:
package unique

func Make[T comparable](value T) Identity[T]

This roughly corresponds to "make a unique identity for this value." In usage:
id := unique.Make(v)

🤷

To contribute something not naming-related as well: Do we have examples of use-cases for this besides strings?
The API constrains on comparable. But, thinking about some comparable types, we have primitives like integer/floating point types, for which interning doesn't make sense. We have pointers (and channels), which already represent a unique identity (that is, two pointers compare as equal only if they are the same allocation - pointers are not compared by pointee). Functions, slices and maps are not comparable.
ISTM that what makes strings meaningful here, is that they are implemented as references - thus representing non-trivial/dynamic amounts of memory - but compared by the referred value. So the only other kind of type I can think of where this would be useful are linked lists based on interfaces, which is esoteric, to say the least. And of course structs and arrays with string fields/elements. Do we have examples of that? And wouldn't they also replace the string-fields with unique.Handle fields?
I'm asking because if we really ~only expect this to be used with strings, that would change the calculus of a string-only API for me, at least. And it might also suggest something about the naming.

Also, if it’s only strings, maybe it would belong in the strings package

My thought is that structs and arrays built using primitives (though not necessarily with string fields) are also useful to intern. One could imagine wanting a cheap comparison (and hash) of a [8]int or something. However, I confess that I don't have a great example off the top of my head.
Focusing on strings is an option, but a solution that doesn't make interned values explicit probably means some kind of overhead when not interning at all. You need a bit of information somewhere indicating that the string has been interned so that the comparison slow path can be skipped and gain the full benefit. That bit could be propagated in the string header, but then that means a mask on access to the length or data pointer. We could also allocate an extra byte for each string to contain this information, but that's just memory overhead on all strings. (If someone has a clever trick up their sleeve that I haven't thought of, I'd love to hear it.)
Thus, my logic is:

A plain string-to-string API will incur some overhead to support, and it would be nice to avoid that.
If we have a solution that doesn't hide the fact that something is interned, it seems strange to limit it to only strings.

And if some form of tuples is added, I could see direct struct comparisons becoming more common, too.

One possibility is to allocate string backing store in a separate region of memory.
Then when comparing strings, if both string pointers are in that region, just do == of those pointers instead of comparing contents.
One useful optimization is that we only need to check the containing memory region if the string size is large. We can, for example, check the contents unconditionally if the length is <= 16. Only for longer strings is it worth looking up the region of the backing store pointers.
Probably "region" is just a special mark on the containing span?

Ah, good point. I didn't consider just having a differently-classified memory region.
The straightforward implementation of that has some negative effects like increasing the size of mcaches since we'd have to duplicate our size classes to get good memory overheads. We could have a new type of memory layout for these spans, but that's also gonna be somewhat complex (Immix style? Something else?). At least it's noscan so that limits the complexity a bit (no need to scan, but you still need to be able to mark and release the relevant memory). It's also a little unfortunate that we can't just have every interned string be a cheap pointer compare in this design. I do like that this doesn't have any overheads if you're not interning at all.

Or maybe...this should be a part of runtime package?
String interning happens only during runtime, runtime.Intern(s string) string (or similar) sounds very simple.

@cristaloleg The runtime package is a bit of a kitchen sink of random things that don't all make sense together. I'd prefer not to keep filling it. 😅 If we go the string-only interning route, though, I would argue strongly for strings.Intern.
FWIW, I still prefer something along the lines of the original proposal. Having an explicit "interned-ness" of a value is easier to reason about in terms of performance in my opinion. It's easier to tell at-a-glance whether a comparison will be cheap or not. Also, as mentioned above, we can still have full interning strings for strings without holding onto the Symbol[T] by stuffing the string's data pointer directly into the Symbol[T] value.

I agree with @mknyszek.  My opinion is that string interning is subtle, whereas the proposed package (under whatever name) is a bit complicated but is clear.  Also I find it easy to imagine wanting to intern struct values.  I prefer the proposed package.

Is the lifetime of the deduplication tied to the lifetime of the Handle, or the lifetime of the allocation? That is, would I have to keep a Handle around to benefit from interning, or would it be enough to keep the interned string value around?
Because if it is the former, it seems you have to expose it in your API and the prototypical use-cases of encoding/json unmarshalling into a map[string]any, or encoding/xml filling up xml.Name with the same namespace/tag names would benefit less. Or at least I don't think I understand the mechanic of how repeated Unmarshal calls would share interned strings.
If it is the latter, I think I'm sufficiently satisfied that interning can be an implementation detail like sync.Pool, instead of leaking into APIs like arenas would.

Regarding limiting the package to strings, that seems unnecessarily restrictive. @bradfitz points out that a program might be juggling many SHA256 hashes or some other comparable type, and deduplicating those down to handles can be a significant memory and CPU savings. It's not just strings.
We're avoiding atom and symbol because they are overloaded terms. Package unique seems to have good support.
The only question left seems to be the name of the constructor. It's true that New doesn't always create a new handle, nor does it return a pointer. (Also cgo.NewHandle does create a new handle on each call.)
Make is perhaps better because it does not imply a pointer is returned, although some people might still think it "makes" a new thing each time. Documentation will have to be the answer there.
package unique

// Make returns a globally unique symbol for a value of type T. Symbols
// are equal if and only if the values used to produce them are equal.
func Make[T comparable](value T) Handle[T] {
    ...
}

// Symbol is a globally unique identity for some value of type T.
//
// It is trivially and efficiently comparable with other Symbol[T] for the same T.
//
// If the value used to create the symbol is the same, the symbols are guaranteed
// to be equal.
type Handle[T comparable] struct {
    value *T
}

// Value returns a shallow copy of the T value that produced the Symbol.
func (h Handle[T]) Value() T {
    return *h.value
}

The only question left seems to be the name of the constructor.

LookUp? Fetch? Get? Value?

For?

@Merovius

Is the lifetime of the deduplication tied to the lifetime of the Handle, or the lifetime of the allocation?

The former (at least in my understanding of the package).  So, yes, if you want to guarantee deduplication, you have to expose Handle in the API.  That said, the most likely implementation is still a pool.  If all the handles to a given object go away, and then before a GC runs you ask for another handle to an identical object, you'll get the one in the pool (and, of course, we won't necessarily discard all unreferenced objects in the pool at every GC any more than we do for sync.Pool, I don't know what the right policy should be).  I don't currently see a reason why we would expose Handle in the API of packages like encoding/json or encoding/xml.

The former (at least in my understanding of the package).

We can actually have the latter behavior (i.e. tied to the lifetime of the allocation) for just strings. Because strings are reference types that behave like value types, I don't think it would be unreasonable to stuff the string data pointer into Handle. If we do that, then the interning really is bound to the lifetime of the string itself, and callers could just do:
s = unique.Make(s).Value()

as a replacement for the alternative:
s = strings.Intern(s)

As I mentioned above, there are some subtleties with reconstructing the string value, but there are also reasonable workarounds to that. This doesn't really make sense for other comparable types because they really are just plain values and thus a copy must be made out of the Handle.
Though, maybe this will be a sufficiently surprising special case for strings that we shouldn't do this. I don't have a good grasp on what could go wrong.
As an aside, we could also extend the encoding packages to understand unique.Handles. For example:
type myJSON struct {
    Name unique.Handle[string] `json:"name"`
}
...
var msg myJSON
json.Unmarshal(&msg, data)

This would unmarshal the string and then call unique.Make on it for you.

I think unique.Handle(s).Value will be flagged in most code reviews and require discussion to understand. It seems better to have a complementary api implemented that way e.g unique.String so that it’s clear to everyone this is a blessed use.

I think adding something best-effort for just strings (which could just be return unique.Handle(s).Value) is reasonable, but it won't have the O(1) guaranteed comparison like a Handle. Hence, I think it makes sense to have both levels, especially if one can be trivially implemented in terms of the other.

It still seems like unique.Handle, Handle.Value, and unique.Make are the best names we have.
Have all remaining concerns about this proposal been addressed?

unique.MakeHandle leaves room for other Make* functions if needed later and, at least in my opinion, is clearer at the call site about what it makes.

How best effort would string -> string interning using unique.Make(s).Value() be? For example, if the caller uniques s in GC cycle 1, and s survives into cycle 2, but no Handle for s survives into cycle 2 (because we're immediately throwing away the Handle), and then in cycle 2, the caller uniques a string == to s again, do they get s or a new copy of s? Either way, the strings would still be ==, but if the point of doing this is to save space, it doesn't save as much as it could.
The weak map[string]*string approach suggests you would get a new string in cycle 2. While the backing store of s is still live in cycle 2, the weakness is attached to some copy of the string header, which will get thrown out with the Handle.

I feel like we shouldn't mix up this idea with string interning.  Personally I think this idea is solid and useful by itself.  String interning is really a different idea that solves a different problem.

I agree that we should not mix this up with string deduplication.
The weak map[any]*any must be weak on its values (meaning if the value is GC'ed, the map entry is deleted) and not weak on its keys, because in general the keys may not be GC-able at all. For example we've discussed using unique.Make on [32]byte values to reduce SHA256 hashes to unique Handles for easier comparison. For the entries to be cleaned up, they have to be cleaned once the *[32]bytes contained in handles are all GC'ed. It doesn't make sense to GC the key. Same for the less useful case of unique.Make on int values.
If the app keeps the Handles around, then that does have the effect of deduplicating memory, and that's fine. But if you want that, you need to keep the Handles, so s = unique.Make(s).Value() is in more of a no-op than a useful optimization.
All that said, we could also provide unique.String(s) returning string, for which the string itself is both the value and the "handle" for GC purposes. If we did this, that would be a separate value-weak map[string]string map, not a map[string]*string as in the general case. However, in that case the strings in the map really need to be separate, GC'able objects - in general strings might have been constructed using unsafe from mmap'ed memory or other sources. That somewhat implies that unique.String(s) needs to make a copy of s the first time it is called with a given s, which is fine in apps where there are many copies already and use of unique.String is letting us GC them. But it might still surprise people.

@ChrisHines, I see your comments about MakeHandle, but the intention of this package is that there is nothing more than unique.Handle here. unique.MakeHandle ends up redundant, and unique.Make is fine.
Even if we added unique.String later, it would still be very clear which was which, since the signature of Make is:
func Make[T comparable](val T) Handle[T]

What about unique.Obtain(v)?
Also, docstrings in examples above still talk about symbols, should those be updated to handles as well? (E.g., "Make returns a globally unique symbol for a value of type T.")

But the first feels like an incomplete sentence. What are we making?

FWIW, I read unique.Make(v) as "make v unique". In that sense it's not an incomplete sentence, just a sentence with out-of-order grammar (from an English speaker's perspective). One could imagine Make being called MakeUnique but it seems redundant with the package name.

Also, docstrings in examples above still talk about symbols, should those be updated to handles as well? (E.g., "Make returns a globally unique symbol for a value of type T.")

Yes, thank you. Fixed.

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group
The proposal details are as follows.
New package with this API:
// TODO
package unique

// Make returns a globally unique symbol for a value of type T. Symbols
// are equal if and only if the values used to produce them are equal.
func Make[T comparable](value T) Handle[T] {
    ...
}

// Symbol is a globally unique identity for some value of type T.
//
// It is trivially and efficiently comparable with other Symbol[T] for the same T.
//
// If the value used to create the symbol is the same, the symbols are guaranteed
// to be equal.
type Handle[T comparable] struct {
    value *T
}

// Value returns a shallow copy of the T value that produced the Symbol.
func (h Handle[T]) Value() T {
    return *h.value
}

If the value used to create the symbol is the same, the symbols are guaranteed
to be equal.

Should this be iff?

Apologies for a late comment - I checked and didn't see this discussed above.
Do the handles need to be global? Most applications I know process the data in some context so holding to on an object wouldn't be too hard. Without understanding the runtime - could users splitting up a-priori known disjoint types into explicit (smaller) namespaces potentially help runtime in some way?
A bit stretched argument could be that namespaces could be more forward compatible and allow for different configuration - for example, if my application already protects for concurrent access in some way, maybe I could configure some lighter concurrency checks. Or, to configure the aggressiveness of GC removing the elements (?) in the json example above. Or, to explicitly clear a namespace (? not sure how useful practically this could be)
Or is the goal to allow for global object sharing from the start?

(Potentially bikeshedding.)
What about unique.Obtain(v) instead of unique.Make(v)?

Obtain is a long way to spell Get :)

unique.Get sounds good, too.

The doc comments in https://github.com/golang/go/issues/62483#issuecomment-1782053117 still says Symbol. They probably should be changed to Handle?

Do the handles need to be global? Most applications I know process the data in some context so holding to on an object wouldn't be too hard. Without understanding the runtime - could users splitting up a-priori known disjoint types into explicit (smaller) namespaces potentially help runtime in some way?

They don't need to be, but the ability to explicitly create separate local namespaces creates the potential for hard-to-debug issues. I would not be surprised to see bugs filed where two equal values are placed into different namespaces, but end up getting compared and compare false as a result. To catch that at runtime (with say, a panic) would require also carrying around the namespace the handle came from, which would be substantially less efficient (2x larger memory footprint for handles, comparisons aren't quite as cheap anymore and need to be a method on the handle). With a single global registry, this can never happen. If the handles compare equal for some type, they are always going to be equal.
It also doesn't help the runtime much to make this part of the API. If we care about keeping separate types in separate internal maps for efficiency, that's an internal implementation detail and we could certainly do that.

ISTM that it's trivial to have separate "namespaces" if you care about it. You can just use a different type, no? any(unique.Make(int64(0))) != any(unique.Make(time.Duration(0))), even if they have the same representation?

Do the handles need to be global? Most applications I know process the data in some context so holding to on an object wouldn't be too hard. Without understanding the runtime - could users splitting up a-priori known disjoint types into explicit (smaller) namespaces potentially help runtime in some way?

They don't need to be, but the ability to explicitly create separate local namespaces creates the potential for hard-to-debug issues. I would not be surprised to see bugs filed where two equal values are placed into different namespaces, but end up getting compared and compare false as a result. To catch that at runtime (with say, a panic) would require also carrying around the namespace the handle came from, which would be substantially less efficient (2x larger memory footprint for handles, comparisons aren't quite as cheap anymore and need to be a method on the handle). With a single global registry, this can never happen. If the handles compare equal for some type, they are always going to be equal.

All true. Enforcing comparison via the namespace object could (?) slightly reduce the risk of this happening but not make the recovery behavior easier. Automatically namespacing per package could help too, but defeat some potential wins.
The global approach can have unexpected side-effects too, right? Imagine an app that uses this package to dedup a small practically frozen set of strings (countries? timezones?) from a single goroutine. I then import another component (or multiple ones) that also dedup strings, but something of a much higher cardinality and much more dynamic (hostports? email addresses?) - from multiple gorutines. The imports start affecting each other, even if the user knows that the categories of strings can never overlap.
Would the solution be to type an alias for each of the categories? Like
type email string
var b []byte = ...
s := unique.Handle(email(b)).Value()

would this also not allocate?

@rabbit I'm really not sure what you think the negative side-effects would be. Note that the fundamental guarantee of the package - that a Handle[T] compares equal to another Handle[T] if and only if they represent the same T - remains true, no matter what. I fail to see how anything would break when using a Hande[T] as proposed, that wouldn't also break if those packages used a T directly. Like, in your example, if those packages used string for "countries? timezones?" and "hostports? email addresses?" (no snark intended, just trying to reflect the example) respectively, exactly the same things would compare as equal as before.
On the contrary, semantically it seems extremely brittle to have them use different Handle[T]s, because now if they would move from string to Handle[string], things would change how they compare to each other. I actually think not having that happen and continuing the semantic guarantee that (unique.Make(a) == unique.Make(b)) == (a == b) is an extremely important property for correctness.
And yes, as you note, if you want to explicitly have different "namespaces", you can declare new defined types (nit: not "aliases"), which then compare as unequal (when stored in interfaces) even if they represent the same value. That is a) the same as it is with current types and b) already something we are somewhat used to, with context.Context keys.
For performance it might have subtle implications, but a) I find it very hard to make any kind of educated guess in what direction that performance difference would swing and b) if it does, we can always decide to use different stores for different types after the fact, without changing everything about the API - the package has access to T, as the types and functions are parametric.

@Merovius I wasn't talking about comparing the handles, which I understand is cheap, but rather generating them. I understand we would need some kind of global map (mentioned in the proposal, assumed to be read heavy) - thus one misbehaving package can others, right?
I understand that some shared global state is unavoidable and desirable. I was just trying to convey that some applications might not need the global state here, which might allow for increased isolation.
You don't see the global state as a concern - I understand this now.

understand some kind of global map would be needed (mentioned in the proposal, assumed to be read heavy) - thus one misbehaving package can others, right?

Not really - the semantics are such that no one can tell what anyone else is doing as far as handles, and yes one package could make the map big, but big maps are still fast.

thus one misbehaving package can [affect] others, right?

I don't see how this is much different than the fact a misbehaving package today and OOM the entire Go process.
Also, it seems that the implementation of unique could shard the global map based on the Go type passed to Make, thus providing functional isolation if necessary. What's nice about this is that it's entirely an implementation detail that's not exposed in the API.

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group
New package with this API:
// Package unique provides a way to construct globally unique identities
// for comparable values, to make storage and comparison of those
// identities more efficient.
package unique

// Make returns a globally unique handle for a value of type T. Handles
// are equal if and only if the values used to produce them are equal.
func Make[T comparable](value T) Handle[T] {
    ...
}

// A Handle is a globally unique identity for some value of type T.
// It is trivially and efficiently comparable with other Handle[T] for the same T.
// Handles are equal if and only if the values used to produce them are equal.
type Handle[T comparable] struct {
    value *T
}

// Value returns a shallow copy of the T value that produced the Handle.
func (h Handle[T]) Value() T {
    return *h.value
}

Not trying to argue or to change anyone's mind, but just to close the discussion (thank you for your comments @rsc and @dsnet):

and yes one package could make the map big, but big maps are still fast.

Rather than size (large size implies it's a valid use-case for handles), I was more worried about handle generation. A package could ("incorrectly"), create A LOT of short-lived handles that get created and removed all the time.

Also, it seems that the implementation of unique could shard the global map based on the Go type passed to Make

I was talking about different disjoint "categories" of the same (Go) type. A string could be an email, IP address, or HTTP tag - but not simultaneously. The application knows this and can shard, thus reducing the need for runtime sharding. Like above, the primary worry here was part of an application that (unnecessarily) creates a lot of short-lived handles.
Perhaps I'm being too pessimistic about the potential misuse of the package.

Random tangentially-related thought ahead:
Can we put all the runtime._types in one of these weak maps? So when no more pointers to that runtime._type exist, the entry itself can be garbage collected.
This is an issue with any of the reflect type-building functions, e.g. reflect.StructOf. Once built, those types live forever. It would be great if we could GC those types from the "uniquification map" as soon as no one references them.
This is very close to the problem that prevents us from GCing plugin invocations. If we could determine that no types from a plugin are still accessible from anywhere, we could unload the plugin (and all of its code). A long outstanding issue #20461. (There are more difficulties than just this one, but it is a big one.)
Perhaps we could start with using this implementation for runtime._types (aka internal/abi.Types). It would require no new ABI so we could use it as a "first customer" experiment that involves no new API.

looks this is a similar implementation to unique value + pool.
so why not directly put it into package sync？
Not a native english speaker，but to me Make is worse name to Intern.
I believe sync.Unique is a nice name.

@rsc is the unique.Handle[T] a kind of weak reference or not?
Looks like Handle holds pointer to value and Handle.value cannot be collected by GC.

@hauntedness The proposal is accepted, so barring any serious concerns about the name, I doubt we'll be changing the name at this point.
@snglagolev unique.Handle[T] is not a weak reference. For the proposal to work, it must explicitly not be weak reference, since the whole point is to keep the canonical value being referenced live. The internal map for deduplicating values of type T will retain a pointer to the canonical value via something like a weak reference, but this is not exposed in the interface. (The deduplicating behavior and the fact that stale references get cleaned up in the map is guaranteed though, since that's necessary for the API to be useful.)

Please read the proposal in the original post; weak references in the standard library and/or language are discussed briefly as an alternative.

@snglagolev Yes, I definitely would expect that. Handle.Value returns a copy of the contained value. In this case, the contained value is a [1024]byte. So that gets copied into two new variables v1 and v2 and they obviously have different addresses. Note that for arrays, &x[0] is the same as &x. That is different from strings or slices, in which the value contains a reference.
It should still be noted that you end up with three copies - one in v1, one in v2 and one on the heap (that is in the uniqe storage). And in particular, if v1 and v2 don't escape, while you do incur a copy, you don't incur additional allocations - there will only be one copy on the heap, even though you obtain with h1 and h2 two handles from two arrays.

I know this has been closed and already accepted, and I'm not opposed to having small packages or anything, but I'm a bit surprised that this didn't end up in the cmp package. It seems like a natural place for it.

Apologies if my question does not make sense, but I am still a bit confused as to where benefits can be expected (or not) with this feature.
Would there be any runtime performance benefits in using a handle on a hardcoded string? For example:
var data = map[unique.Handle[string]]int{
	unique.Make("the_first_key"):  1,
	unique.Make("a_second_key"):   2,
	unique.Make("some_third_key"): 3,
}

//go:noinline
func getUsingUserInput(key string) int {
	return data[unique.Make(key)]
}

//go:noinline
func getSecond() int {
	return data[unique.Make("a_second_key")] // Hardcoded string
}

func main() {
	userInput := "some_third_key" // Assume it's not hardcoded
	fmt.Println(getUsingUserInput(userInput))
	fmt.Println(getSecond())
}

In the first case (getUsingUserInput), my understanding is that since the string has been created at runtime, and thus gets an arbitrary internal memory address, the Handle would have to search for this string into it's internal table anyway, so there should be no theoretical improvement (maybe even slower than a standard map[string]). Is that correct?
In the second (getSecond) case, where the string is hardcoded, what should I expect? Is there some compiler optimization that would pre-resolve the handle, giving me full performance benefits, or would the string be looked-up in it's internal table every time the function is called?

@Seb-C In both of your cases, there would be an added cost with little benefit, yes. But the reason has little to do with whether or not the strings are constant and everything to do with the fact that you don't keep them around and only create the handle once and only do so for lookup.
The benefit comes if you don't pass string but Handle[string] around as values/fields in your code, because then you might save some allocations and if you do many comparisons, they'll be cheaper.
The prototypical example is if you have a parser and return tokens from that. If you use Handle[string] in the token instead of storing them as a plain string, you only need to keep the memory for each identifier or keyword around once, instead of allocating them for every token using them.

+1 to @Merovius' comment. For another example, see the net/netip package's implementation, which uses the unique package internally for IPv6 zone names.


[C-2166] Inconsistent spacing in error messages for create activity fields

Currently, the display of spacing appears rather disjointed. It could be improved with better alignment of the error messages to ensure a cleaner look and improved readability.

From SyncLinear.com | C-2166

Can I take up this issue?

@Nike682631 Thanks for your interest in contributing! assigned the issue to you 🙂

Yes I do need help. I was finding some issues with docker images trying to run the application. I'll post them soon here for your reference or in your discord server.

@Nike682631 I'd suggest you use Orbstack as a docker replacement but I'm not sure if that might help in your situation. Regardless, the installation process definitely needs a lot of improvement

picking this up :)


Deserialization should take discriminators into account

For types that have a discriminator, we should hide the discriminator property itself by default (serves no purpose as a public API once deserialized into an appropriate type).

It shouldn't be exposed anywhere. @odata.type should be used only as a polymorphic discriminator during deserialization (and emitted during serialization).
Originally posted by @brjohnstmsft in https://github.com/Azure/azure-sdk-for-net/pull/10667

Updated the description to reflect DTs are being deserialized, but we still need to hide the ODataType property by default.

Fixed


ISPN-16214 Time quantity units

https://issues.redhat.com/browse/ISPN-16214
We should implement something similar for JGroups

Thanks, @tristantarrant!


[Feature Request]: Add support for BigQuery Storge Write API in Python

What would you like to happen?
Much like #20687 to read to BigQuery, The Java SDK contains an implementation which uses the BigQuery Storage API to write to BigQuery. We should implement the same for Python.
In our case, we would like to use it instead of using the FILE_LOADS ingestion method in streaming mode, where errors during the loading process are not propagated to the pipeline. We are currently using it as we do not need STREAMING_INSERT features, and hence want to save on cost.
Issue Priority
Priority: 3
Issue Component
Component: io-py-gcp

I took a look at this, and it seems like the bigquery storage api has a straightforward jsonstreamwriter but for java.  for python, it only exists as protobufs. Hence why the bigquery storage write api for dataflow in python is missing, but exists in java. That api will likely need to be made before the dataflow functions can be made by the core contributors group. Sad that this blog post made it seem like it was in GA for everything.

@chamikaramj FYI for if/when this can be done via xlang


[Bug] Connection failure on new config

Describe the bug
After the configuration of the instance and connection to Discord with the login prompt, the instance displays Connection Failure
To Reproduce
Steps to reproduce the behaviour:

From a fresh install, create a discord instance
Enter credentials
Save and enable
Accept connection in the Discord app
Connection Failure

I made a new app on Discord Dev Portal to get new credentials as the console says "Login err" but it is not the solution
System (please complete the following information):

Companion ver. 3.0.0 (3.0.0+6014-beta-c62d68b4)
Discord module v 1.1.2

Console log
system: ** Starting Connection from "C:\Program Files\Companion\resources\bundled-modules\discord-api\main.js" **
system: ** Connection started **
Starting up module class: c
Sentry disabled
Module-host accepted registration
debug: Initializing Discord client
warn: Login err: {}
error: [object Object]`

Is the user account that created the Discord app the same user account that's logged in to Discord?

Yes, same account.

The module works perfectly fine when ran directly as a dev module, but Companions packaging process seems to be breaking the module when packaged so I'm looking in to why that's happening.

This should be fixed in the latest beta builds.


[Skia / CanvasKit] Browser zoom is buggy at < 50%

Steps to Reproduce

Copy this code:

import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Demo',
      home: Container(
        width: double.infinity,
        height: double.infinity,
        color: Colors.green,
        child: Center(
          child: Text(
            "Center Text",
            style: TextStyle(fontSize: 128),
          ),
        ),
      ),
    );
  }
}

Run flutter web with skia (--dart-define=FLUTTER_WEB_USE_SKIA=true)
Set browser zoom to < 50%

Expected results:
The zoom behavior should be like the behavior, when you zoom + 50%.
Actual results:

Flutter doctor
[✓] Flutter (Channel master, 1.21.0-10.0.pre.55, on Mac OS X 10.15.5 19F101,
    locale en-DE)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
[✓] Xcode - develop for iOS and macOS (Xcode 11.6)
[✓] Chrome - develop for the web
[✓] Android Studio (version 3.6)
[✓] VS Code (version 1.49.0)
[✓] Connected device (3 available)
    ! Error: iPhone von Roland is not connected. Xcode will continue when iPhone
      von Roland is connected. (code -13)

• No issues found!

Hi @AndroidNils
Unable to reproduce this issue using release mode or debug mode
flutter run --debug --dart-define=FLUTTER_WEB_USE_SKIA=true -d chrome
flutter run --release --dart-define=FLUTTER_WEB_USE_SKIA=true -d chrome
Tried on the latest Channel master, 1.22.0-10.0.pre.252, channel, looks you're on an older Channel master, 1.21.0-10.0.pre.55 channel.

code sample
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Demo',
      home: Container(
        width: double.infinity,
        height: double.infinity,
        color: Colors.green,
        child: Center(
          child: Text(
            "Center Text",
            style: TextStyle(fontSize: 128),
          ),
        ),
      ),
    );
  }
}

flutter doctor -v
[✓] Flutter (Channel master, 1.22.0-10.0.pre.252, on Mac OS X 10.15.6 19G2021
    x86_64, locale en-GB)
    • Flutter version 1.22.0-10.0.pre.252 at
      /Users/tahatesser/Code/flutter_master
    • Framework revision 78929661fb (5 minutes ago), 2020-09-18 08:08:30 -0700
    • Engine revision 2abe69c608
    • Dart version 2.10.0 (build 2.10.0-136.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/tahatesser/Code/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/tahatesser/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0, Build version 12A7209
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 49.0.2
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.49.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.14.1

[✓] Connected device (4 available)
    • Taha’s iPhone (mobile) • 00008020-001059882212002E • ios            • iOS
      14.0
    • macOS (desktop)        • macos                     • darwin-x64     • Mac
      OS X 10.15.6 19G2021 x86_64
    • Web Server (web)       • web-server                • web-javascript •
      Flutter Tools
    • Chrome (web)           • chrome                    • web-javascript •
      Google Chrome 85.0.4183.102

• No issues found!

I cannot reproduce this issue on the latest master channel.
Can you please upgrade to the latest  master and try again
flutter channel dev
flutter upgrade --force
flutter doctor -v
Thank you

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.

I can't reproduce it 👍  So I think it's fixed :)


The subscription of the peer network needs to register Microsoft.ContainerService

If peering with a network outside the subscription where ARO is deployed then that subscription needs to:
az provider register -n Microsoft.ContainerService --wait

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 93aa980e-1970-235c-228a-7b19880171b8
Version Independent ID: 6f7937a4-12e0-d61c-3bd8-c6347ec48385
Content: Tutorial - Create an Azure Red Hat OpenShift cluster
Content Source: articles/openshift/tutorial-create-cluster.md
Service: openshift
GitHub Login: @jimzim
Microsoft Alias: jzim

Thanks for the feedback! We are currently investigating and will update you shortly.

@clauslund thanks for this feedback. I have updated the doc to include this information. Once the PR merges, the changes will go live after a few hours.


Less eval

Instead of
let onTheBuild = import <nixpkgs> {} ; # this is accessible from that latter
    onTheHost = import <nixpkgs> {
      crossSystem = rec {
        system = "mips-linux-gnu";
        openssl.system = "linux-generic32";
        withTLS = true;
        inherit (platform) gcc; # this is no longer needed
        inherit platform;
      };
};

try
let onTheBuild = onTheHost.buildPackages;
    onTheHost = import <nixpkgs> {
      crossSystem = rec {
        system = "mips-linux-gnu";
        openssl.system = "linux-generic32";
        withTLS = true;
        inherit platform;
      };
};

Thanks, will apply


fixing isObject() method for old IE browsers

Object.prototype.toString.call(undefined) returns "[object Object]" in IE8. It means what converting 'x' to string is not the best way for this function. On the other side, we can use the code from underscore library, which works even in IE8

Fixed isObject() with 'obj' instead 'x'.
Thank you!


doc: Bluetooth: Rework Controller doc

Codecov Report

Merging #6803 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6803      +/-   ##
==========================================
+ Coverage   54.64%   54.64%   +<.01%
==========================================
  Files         447      447
  Lines       42598    42598
  Branches     8044     8044
==========================================
+ Hits        23278    23279       +1
+ Misses      16086    16085       -1
  Partials     3234     3234

Impacted Files
Coverage Δ

kernel/posix/pthread.c
70% <0%> (+0.5%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 185ff16...31152dd. Read the comment docs.

@dbkinder @jhedberg comments addressed and expanded a bit the hci_uart sample doc. Thanks!

@dbkinder fixed, thanks!


Svelte support in StrykerJS

Working:

Able to mutate Svelte files (including template expressions)
Svelte integration tested with Jest test runner (e2e test)

Not done:

Typescript support (Typescript Svelte files are unable to be parsed)
Syntax highlighting in Svelte files HTML reporter

It would be nice to merge this and provide some example repo. @nosovk
I've recently have come back from a break and will be working on this in the near term to get it merged. An example repo would be a good idea, thanks for the suggestion.

I've been working a bit on this. Almost finished.

[ ] Add svelte minimal version e2e
[ ] Support <script lang="ts">
[ ] Docs: add svelte guide

Thanks @TeunLamers, for updating it with master by the way, made the work a lot easier 😅

Still something wrong with the offset when reporting mutants:


Make Bn_382 tests compile conditionally

Currently build fails locally for me without this, since the crate::curves::tests crate is only exposed for #[cfg(test)].

Thanks for fixing this!


Add "Problem Details for HTTP APIs" semantics standard tags

Background
The semantics related to errors seem too technical and related on the language.
Problem
In a polyglot microservice environment errors/warning/exceptions/problems could be logged in a custom way by every component. "Errors may be described by OpenTracing in different ways, largely depending on the language.". The "error.kind" and "error.object" don't capture all the useful semantics related to an error. Error investigation in this context looks more like exception logging - which is helpful mainly for developers and not other users that might make use of the instrumentation/tracing. Especially related to the one of the most important errors/problems in our context - errors returned by HTTP services.
Proposal
More microservices are adopting "Problem Details for HTTP APIs" (https://tools.ietf.org/html/rfc7807) in order to communicate problems in a structured way.  Adopting as standard tags the fields described in this RFC in case of an error would give the tools to enhance the process related to problem investigation, and give the ability for the tracers to offer more functionalities in their UI.
So something like adding tags like "problem-details.type"  "problem-details.title", "problem-details.description", "problem-details.instance", which reflect the problem details RFC.

More microservices are adopting "Problem Details for HTTP APIs"

Is there data to back up this claim?  I'm curious because we're internally in a heated discussion about how microservices should model errors (in the gRPC world, but the principle is the same). E.g. gRPC has ErrorDetails protos, but those are passed back via headers, not as the response payload.

Not real numbers, but the fact that Microsoft ASP.NET Core team added this standard recently in asp.net core 2.1 brought me to this conclusion: https://blogs.msdn.microsoft.com/webdev/2018/02/27/asp-net-core-2-1-web-apis/

Given that there's no guarantee that OpenTracing will be used strictly in the context of HTTP APIs, I'm not sure if this is a great fit for mainline spec. I do wonder if it would be something that could extend error.object in a conventional way via a contrib project?

The mainline spec contains already stuff specific to HTTP, like http.method, http.status_code, http.url, so why not extend it? It doesn't have to be a complete error specification for all protocols, it could be only about http ones. Maybe it's clearer this way, since there are different standards for errors depending on the protocol.

Sure, but it also contains stuff specific to databases, message busses, etc. I get your point though.

I may have missed it in the RFC, but what is the exact list of fields you are proposing to add? We already have protocol-agnostic equivalents to many that you mentioned, eg error.type, error.message, error.object.

From the RFC:
3.1.  Members of a Problem Details Object
A problem details object can have the following members:

"type" (string) - A URI reference that identifies the problem type.  This specification encourages that, when dereferenced, it provide human-readable documentation for the problem type...
"title" (string) - A short, human-readable summary of the problem type.  It SHOULD NOT change from occurrence to occurrence of the problem, except for purposes of localization...
"status" (number) - The HTTP status code generated by the origin server for this occurrence of the problem.
"detail" (string) - A human-readable explanation specific to this occurrence of the problem.
"instance" (string) - A URI reference that identifies the specific occurrence of the problem.  It may or may not yield further information if dereferenced.

Also, as presented in 3.2.  Extension Members, problem type definitions may extend the problem details object with additional members.
Ex: "invalid-params" parameter:
{
"type": "https://example.net/validation-error",
"title": "Your request parameters didn't validate.",
"invalid-params": [ {
"name": "age",
"reason": "must be a positive integer"
},
{
"name": "color",
"reason": "must be 'green', 'red' or 'blue'"}
]
}
Existing fields:
type = already there: error.kind
status = already there: http.status_code
detail = already there:  error.object
Missing fields:

title = custom attribute, like error.title. This could provide a human readable description of the "error.kind", so for example a search UI for different problem types would be more user friendly.
instance = custom attribute. like error.ref
extension parameters = custom attribute, like error.params. This would allow a user to construct a complex query like give me the traces that had problems with the invalid age parameter for example, if he knows the problem details of the whole system.

My plan is to add error.params, and error.title tags, in our problem details middleware that is used in several services, and enhance our custom tracer UI to reflect this. It would have been nice to know if other systems that want to use problem details would do the same thing, and maybe if other tracers would enhance their UI to use these extra semantics, that's why I created this issue/proposal.

@mciureanu would you be interested in writing an RFC for this proposal and submitting a PR?


Generate Subsets

Subset generation is a common problem in combinatorics and is often used in algorithms related to searching, optimization, and backtracking problems. Using recursion might be an easy way to generate subsets.

Add readme file also and create a separate folder for it.


Adding Payee/Description field to QIF export.

Adding Payee/Description field to QIF export and moving the Memo to Memo QIF field.
Implements #343 as discussed there.

Merged manually onto hotfix/patches (26277c6). Thanks!


Native Apple Pay

This fixes a bug that was reported by a customer. The bug hides the Apple Pay button when there is no card available in the user's wallet or no supported cards available.
This bug sometimes causes a rejection from Apple Store's review process.
For now, the implemented native button should only be used when the strange bug in the 'pay ' lib happens.
Some enhancements before a full transition to the native button approach:

Handling cancelled behavior by the user
UI styling like the supported in the 'pay' lib
Loading indicator

Excellent work! 👏 @AbdulazizAlrabiah
I tested this new change on a physical iPhone and both scenarios (card already added VS no card added) are functioning as expected. 👍
I'm gonna merge this PR and add a subsequent commit for the release related chores.


Boot to PXE only works after issuing both boot to PXE and then boot to BIOS

Describe the bug 🪲
Issuing a power/bootoptions POST call with action 400 (Reboot to PXE), reboots the devices but does not cause the device to boot to the PXE boot options.  However, if you then send a power/bootopions POST call with action 101 (Reboot to BIOS) the device will reboot and go to the PXE boot option.  The device will not boot to BIOS.
To Reproduce 🪜
Steps to reproduce the behavior:

Send device power/bootoptions POST call with action set to 400
Wait for device to complete reboot
Send device power/bootoptions POST call with action set to 101
Device will reboot to PXE boot option

Expected behavior
Sending power/bootoption POST call with action set to 400 will cause the device to reboot to PXE boot option.  Subsequent reboots (without setting action to 400) will not cause it to boot to PXE boot option.
Sending power/bootoption POST call with action set to 101 after the same call with action set to 400 will cause the device to reboot to BIOS, not PXE boot option
Screenshots 🖼
If applicable, add screenshots to help explain your problem.
AMT Device (please complete the following information): 🖥

OS: Windows
AMT Version: 12.0.85
AMT Configuration Mode: Admin Control Mode
Network Configuration DHCP

Service Deployment (please complete the following information): ⛈

Deployment Type: Docker
Node Version: LTS 16
Component & Version: MPS 2.7 (release candidate)

Additional context
Add any other context about the problem here.

We are currently implementing Boot control in our application, through the toolkit, and noticed the behavior exactly as described in the issue, while testing with version 2.2.
Also, since we noticed version 2.7 stated as a release candidate to receive the fix, even though the issue is yet open, we checked it anyway and it ended up presenting an even worse bug scenario. Both "Power up" and "Reset" commands act on the device as they should, but neither of them lead to PXE nor BIOS screens by using each of their specific commands (400, 401, 100 and 101).
Is there any new estimate for this fix? We would appreciate it if we have an estimate for the next LTS version as well.

The linked PR fixes the coding issues we were having when executing boot to PXE calls.  That being said, actually getting a system to boot to PXE is very dependent on the particular BIOS of the system and even the BIOS version.  We had mixed results actually seeing PXE boot in our testing across different test machines.  That being said, all of our calls to AMT to set the boot to PXE option are succeeding and we are able to get several different versions of AMT to reliably boot to PXE.  If there is additional feedback on this issue, please feel free to re-open this issue and provide additional information.


Update Posts “test2”

Automatically generated by Netlify CMS

👷 Deploy Preview for incomparable-pegasus-6408a6 processing.

Name
Link

🔨 Latest commit
19b9fde1f2202ff0590ec2294a2815839865f00d

🔍 Latest deploy log
https://app.netlify.com/sites/incomparable-pegasus-6408a6/deploys/62419bfcb96074000846a842


Implementation of a basic authentication scheme using standard .htpasswd

This carries #409.

@NathanMcCauley PTAL

@noxiouz I've refactored the PR to take into account your feedback.
@dmcgowan @NathanMcCauley PTAL.

@NathanMcCauley @noxiouz @dmcgowan I've locked this down a lot more than before. PTAL.

@stevvooe added follow-up ticket for doc on this.

LGTM

LGTM

@stevvooe I tried curl -G http://<IP_ADDRESS>:1000/v2/ it returns {"errors":[{"code":"UNAUTHORIZED","message":"access to the requested resource is not authorized","detail":null}]}. I wonder if it's expected that one can't even ping the v2 registry.
Either way, I then tried with username:$apr1$TRMs2Cbs$WLPy8RLnDhZBPFqvAyCuW/ in htpasswd which should be just username:password, and it failed with the same error message.
Here's the log: https://gist.github.com/tiborvass/c905df6099b41200cac9

@tiborvass
+The _htpasswd_ authentication backed allows one to configure basic auth using an
+[Apache HTPasswd File](https://httpd.apache.org/docs/2.4/programs/htpasswd.html).
+Only [`bcrypt`](http://en.wikipedia.org/wiki/Bcrypt) format passwords are
+supported. Entries with other hash types will be ignored. The htpasswd file is
+loaded once, at startup. If the file is invalid, the registry will display and
+error and will not start.

You must generate the htpasswd files using bcrypt. Otherwise, the entry will not authenticate.

@stevvooe sorry :( thanks. Yeah it worked with bcrypt :)
However is it expected to not be able to ping without being authed ?

@tiborvass For V2, auth is required for ping.

To add a new user we have to restart the registry. Seems It's not convenient, but it can be improved in the future if someone needs this.

LGTM

@noxiouz That behavior is intentional, after some discussion. This ensures the registry cannot start without a valid htpasswd file if htpasswd auth is configured.

@stevvooe  from that point of view it makes sense. Thanks!

LGTM


Extended the README

As this will probably be one of the earlier examples, I thought it would be a good idea to provide some more information on how to get the dependencies, and how to build.
Most of the stuff I wrote will probably be available in lessons, but I think it is a good idea to provide it here as well, as a general resource.
Someone with more experience should probably read through and correct before it is merged though. I have little experience with anything outside cmake -> make..

Thank you! I did some changes to the README.md though, and made the instructions more direct and hopefully easier to follow.


Resteasy 1325

RETEASY-1325: Netty server should close idle connections.
Even NettyJaxrsServer has SO_KEEPALIVE socket option, I would like to add idle timeout option to cleanup idle TCP clients.
By defaul, this feature doesn't turn on, so it's identical to previous versions.
If call to setIdleTimeout with number of seconds, Http client connection will be closed after specific idle time.
Description of this JIRA from https://issues.jboss.org/browse/RESTEASY-1325
"
Netty server adapter should handle client idle when there is no traffic within a certain amount of time.
Currently, if client open connection without sending any message. The connection will keep open.
If there is no Connection: close header, the connection will keep open.
Unless there is "Connection: close" header, they always require client to close.
For Jetty, the default idleTimeOut for Connector is 30 seconds.
Even I can add an IdleHandler to NettyJaxrsServer.addChannelHandlers, it's better to build in NettyJaxrsServer and add setter and attribute for idleTimeOut.
"

Added comment to https://issues.jboss.org/browse/RESTEASY-1325.

Thank you for the unit test, Ruiqiang. This PR is ready to merge.

This is cool! Thanks :-)


Renaming datasets

@charyeezy I don't think it's necessary for us to force breaking changes here.
I think it's better to have multiple assignments.
For example:
get_ecommerce_1_dataset = get_ecommerce_dataset_vectors = get_ecommerce_dataset


Alembic Support

Overrides two methods from the HiveDialect:

has_table: Databricks raises a different exception if a table doesn't exist
visit_primary_key_constraint: Databricks doesn't support PK when creating tables

@crflynn is there anything else I should add to get this merged in?

Hey I haven't had the chance to review / test yet. I will get to it hopefully by next week.


QuSpin Installation on Macbook M1 Chip

There have been instances of users unable to install QuSpin on Macbook's M1 chip, as reported in the following GitHub issues: #499 and #624.
To overcome this problem, here is a step-by-step guide to installing QuSpin:
Step 1: Install mini-forge
First, install mini-forge using the following command:
brew install miniforge
conda init zsh

Step 2: Set Up a Conda osx-64 Environment on ARM Mac
Next, set up a conda osx-64 environment on your ARM Mac with the commands:
CONDA_SUBDIR=osx-64 conda create -n spin python=3.7
conda activate spin
conda config --env --set subdir osx-64

Step 3: Install QuSpin
Finally, install QuSpin using the command:
conda install -c weinbe58 quspin

After following these steps, QuSpin should be successfully installed on your Macbook with the M1 chip. Enjoy using QuSpin!

thanks, @JiahaoYao
Do you know if this installation makes use of the M1 chip features, or it simply allows you to run quspin on M1 devices?

Good question!
As far as I know, the installation process does not specifically exploit the features of the M1 chip. Instead, it sets up a conda environment in a manner similar to how it would function on an Intel chip, thus enabling the operation of QuSpin on M1 devices.

Somehow this didn't work for me, it coulnd't solve the environment. Compiling it from scratch still worked well for me though if assigning the correct cython version:
conda create -n MY_ENV python pybind11 cython=0.29.37 scipy numpy "blas=*=*accelerate*" joblib six dill gmpy2 numexpr boost llvm-openmp numba openmpi-mpicc
By using "blas=*=accelerate" when installing numpy, the accelerate library is used which speeds up numpy operations by a factor of 2-4.
After this, install QuSpin by
Git clone https://github.com/weinbe58/QuSpin.git cd Quspin python setup.py install --omp --record install_file.txt

this should be fixed with quspin 1.0.0 (see doc on how to install)


Value for Property schemaName

In section "HIPAA schemas and message types", step 2: does this mean that I just add a "_P" suffix to whatever I named my schema? For example, if the name I gave the schema I loaded to the integration account is "schema2024", do I need to edit the schemaReferences section and change the value of property schemaName to "schema2024_P?
Thanks!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: f056523f-e6db-b732-b4db-2a8b2f713d3c
Version Independent ID: a894d878-53f5-f032-0e0c-54f3c0245d20
Content: X12 message settings - Azure Logic Apps
Content Source: articles/logic-apps/logic-apps-enterprise-integration-x12-message-settings.md
Service: logic-apps
GitHub Login: @divyaswarnkar
Microsoft Alias: divswa

@asocorro
Thanks for your feedback! We will investigate and update as appropriate.

asocorro If you have named your schema “schema2024” and you want to use the variant message type for the document version number 005010X222A1 with the 837 message type, you would need to add another entry to the schemaReferences section with the following properties and values:
{
"messageId": "837_P",
"schemaVersion": "00501",
"schemaName": "schema2024_P"
}
In this case, you would need to add a “_P” suffix to the name of the schema you uploaded to the integration account. This indicates the variant message type for the document version number you’re using.
This GitHub forum is more inclined towards the documentation feedback, so that we can improve our documentation. If its technical issue, kindly post on Q & A Forum.
We will close this out, but if you feel you need more information, please just let us know.


Add filters in hardware details

[x] #617
[x] implement config filter
[x] implement arch filter
[x] implement compiller filter
[x] build duration
[x] boot duration
[x] test duration

Part of #447

No test or build duration for filter tests.


Symlinks missing for OSX packages

When packing for OSX, the symlinks are missing, leading to an app that will not load.
Take the Electron framework as an example.
The app contents of a electron-packager output will have:

Frameworks/Electron Framework.framework/Libraries -> versions/A/Libraries
Frameworks/Electron Framework.framework/Resources -> versions/A/Resources
Frameworks/Electron Framework.framework/Electron Framework -> versions/A/Electron Framwork
Frameworks/Electron Framework.framework/versions/Current -> A
Frameworks/Electron Framework.framework/versions/A/Electron Framework
Frameworks/Electron Framework.framework/versions/A/Libraries
Frameworks/Electron Framework.framework/versions/A/Resources

However, in the electron-zip-packager output, only the Frameworks/Electron Framework.framework/versions/A/ structure is there, although the app expects to find everything at the symlinks.

I will take a look at this today (hopefully!)


Effect of toggle button

When using head-tracking in larger airliners with complex displays it is useful for toggle-tracking to freeze the view in the current head direction when turned off, and to resume to the new head position when turned back on. Could you add such an option?

Hello Amy,
Yesterday was working fine, today i'm receiving a "Error: No such file or directory".
Help please,
Thank you.
Carlos


node doesn't work with umlaut/mutation

SUMMARY
Hey,
I cannot use the german umlaut/mutation in the name of the echo device. For example I want to use "Verstärker" as echo device. When I search for new devices, Amazon only finds this  "VerstÃÂ¤rker".
I thought it will work with "Verstaerker". Amazon notice then a "Verstaerker" device. But when I say "Turn Verstärker on" the echo devices says that there is no device called "Verstärker".
ECHO DEVICES
Echo Plus 2nd generation
MODULE VERSION
0.1.9
CONFIGURATION
[
    {
        "id": "73e65d70.ac0974",
        "type": "amazon-echo-device",
        "z": "987089f6.de86e8",
        "name": "Licht",
        "topic": "",
        "x": 390,
        "y": 80,
        "wires": [
            [
                "78d6b8b5.b465c8"
            ]
        ]
    },
    {
        "id": "9eae9c48.189ba8",
        "type": "ioBroker out",
        "z": "987089f6.de86e8",
        "name": "Licht_Schlafzimmer",
        "topic": "shelly.0.SHSW-1#944265#1.Relay0.Switch",
        "ack": "false",
        "autoCreate": "false",
        "x": 990,
        "y": 80,
        "wires": []
    },
    {
        "id": "78d6b8b5.b465c8",
        "type": "switch",
        "z": "987089f6.de86e8",
        "name": "switch on/off",
        "property": "payload",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "on",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "off",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 610,
        "y": 80,
        "wires": [
            [
                "af48725.fa9fa1"
            ],
            [
                "1a419b7d.b7cec5"
            ]
        ]
    },
    {
        "id": "af48725.fa9fa1",
        "type": "change",
        "z": "987089f6.de86e8",
        "name": "true",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "true",
                "tot": "bool"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 790,
        "y": 40,
        "wires": [
            [
                "9eae9c48.189ba8"
            ]
        ]
    },
    {
        "id": "1a419b7d.b7cec5",
        "type": "change",
        "z": "987089f6.de86e8",
        "name": "false",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "false",
                "tot": "bool"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 790,
        "y": 120,
        "wires": [
            [
                "9eae9c48.189ba8"
            ]
        ]
    },
    {
        "id": "6dc61b56.72f5a4",
        "type": "link in",
        "z": "987089f6.de86e8",
        "name": "",
        "links": [
            "ee8bef37.935848"
        ],
        "x": 215,
        "y": 80,
        "wires": [
            [
                "73e65d70.ac0974",
                "934394ad.663d28",
                "fcca1a9c.b2ab5",
                "24439b2c.39891c"
            ]
        ]
    },
    {
        "id": "934394ad.663d28",
        "type": "amazon-echo-device",
        "z": "987089f6.de86e8",
        "name": "Fernseher",
        "topic": "",
        "x": 410,
        "y": 280,
        "wires": [
            [
                "570d2594.f894e4"
            ]
        ]
    },
    {
        "id": "56b72247.f69d7c",
        "type": "ioBroker out",
        "z": "987089f6.de86e8",
        "name": "Fernseher_An_Aus",
        "topic": "broadlink2.0.RM:192-168-178-21.L.Fernseher_An_Aus",
        "ack": "false",
        "autoCreate": "false",
        "x": 990,
        "y": 240,
        "wires": []
    },
    {
        "id": "19ceb37.ef2b94d",
        "type": "ioBroker out",
        "z": "987089f6.de86e8",
        "name": "Fernseher_Aus",
        "topic": "samsung.0.Power.off",
        "ack": "false",
        "autoCreate": "false",
        "x": 980,
        "y": 320,
        "wires": []
    },
    {
        "id": "570d2594.f894e4",
        "type": "switch",
        "z": "987089f6.de86e8",
        "name": "switch on/off",
        "property": "payload",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "on",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "off",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 610,
        "y": 280,
        "wires": [
            [
                "8051ff37.4814d"
            ],
            [
                "5cff8e6a.dc4fb"
            ]
        ]
    },
    {
        "id": "8051ff37.4814d",
        "type": "change",
        "z": "987089f6.de86e8",
        "name": "true",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "true",
                "tot": "bool"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 790,
        "y": 240,
        "wires": [
            [
                "56b72247.f69d7c"
            ]
        ]
    },
    {
        "id": "5cff8e6a.dc4fb",
        "type": "change",
        "z": "987089f6.de86e8",
        "name": "false",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "false",
                "tot": "bool"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 790,
        "y": 320,
        "wires": [
            [
                "19ceb37.ef2b94d"
            ]
        ]
    },
    {
        "id": "fcca1a9c.b2ab5",
        "type": "amazon-echo-device",
        "z": "987089f6.de86e8",
        "name": "Verstärker",
        "topic": "",
        "x": 410,
        "y": 480,
        "wires": [
            [
                "104778fb.4e9737"
            ]
        ]
    },
    {
        "id": "24439b2c.39891c",
        "type": "amazon-echo-device",
        "z": "987089f6.de86e8",
        "name": "Wandlampe",
        "topic": "",
        "x": 410,
        "y": 640,
        "wires": [
            []
        ]
    },
    {
        "id": "104778fb.4e9737",
        "type": "debug",
        "z": "987089f6.de86e8",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": true,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "x": 700,
        "y": 480,
        "wires": []
    }
]

Thanks for your help. Best regards! :)

+1, same here

Same problem here. Is there just something like charset="utf-8 missing?
Seems to be a typical enconding problem.

Just tested on my setup and it is still working:

I will need more info about Node.js and Node-RED versions.
Here is what I have:
Node-RED version: v0.20.8
Node.js  version: v11.15.0
Echo Dot version: 3

Same trouble in french but the fix work ;) thanks

Same here.
nodejs: v10.16.2
Echo Dot Gen2 and Gen3
node red: v0.19.5
node-red-contrib-amazon-echo: V0.1.9
In node-red-contrib-amazon-echo: V0.1.8 Umlauts worked. Nothing else changed in my setup.

My version:
nodejs: v8.16.2
iobroker node red: 1.12 (before v0.20.8)

Could be an Alexa Issue as well. Looks like the name is published correctly by the node:

Yes, looks like that. I have the same issue with another alexa-node, too.

I renamed the devices in the Alexa app manually. This workaround works

For now I can do it like that, great.

In node-red-contrib-amazon-echo: V0.1.8 Umlauts worked. Nothing else changed in my setup.

Also got the problem on v.0.1.8. Then updated to 0.1.9. still the problem.
I'm running on iobroker:
Node-RED version: v0.20.8
Node.js  version: v10.16.3
Echo Dot version: 2

I've managed to reproduce it with Echo Dot 2.
I've just asked Alexa to update the firmware, and now I'm experienced the same behavior.
I'm considering this as an Amazon issue, and I'm not sure if it can be fixed in the module, but I will give a try.

Same here, starts while using 0.1.8 and stays on 0.1.9 Think it's a Amazon problem.
Does Alexa understand Kueche as Küche? Maybe we can use this as workaround.

No, Kueche doesn‘t work. You have to rename the devices in the App as stated here https://github.com/datech/node-red-contrib-amazon-echo/issues/74#issuecomment-542823290 For a  working workaround.

I can't find any workaround. It looks it has to be fixed by Amazon. It is Dot 2 issue, as it is working fine on Dot 3

I can confirm, that this is a Dot 2 issue. After I plugged it off and re-searched for devices, they've been added correctly.

I only have Dot 2.

After I plugged it off and re-searched for devices, they've been added correctly.
Maybe because you also have an Dot 3?

Does it make sense to write amazon an E-Mail?

Bought an Echo Dot 3, problem solved. :D