Move the serializing component (TypeAdapter and hive_generator) to its own package?

There are several cases where you need to serialize data. For example, there's storage (what hive does), but also communication over a network or between Isolates where the serialization of data makes sense. My point is, serialization is not a problem specific to Hive.
I like Hive's protobuf-inspired TypeAdapter concept with hive_generator. It's lightweight, forward-compatible and produces a small output (in contrast to, for example, the json_serialize package).
I'm wondering whether it's possible to release a package separate from Hive that only exposes this serializer? Then, Hive and other packages could depend on it, making improvements in the serializer available to everyone.

Hive and other packages could depend on it, making improvements in the serializer available to everyone.

I like the idea of decoupling the BinaryReader and BinaryWriter.

I'm wondering whether it's possible to release a package separate from Hive that only exposes this serializer?

It is possible but currently, both BinaryReader and BinaryWriter have Hive specific logic (like writeHiveList()).
I'm not sure whether to include the Frame class and the corresponding logic like write().

Of course, naming is always a difficult topic. Maybe just something like BinarySerializer? Or some completely custom name?
But I didn't know the serialization part and the saving things-part are that tightly coupled. I assumed objects would be converted to an intermediate format, like a Uint8List, and then that would get written into storage.

Okay, I did some thinking and I possibly found a solution.
The BinaryReader and BinaryWriter are currently monolithic classes with lots of methods, some of which are directly tied to Hive. I'll describe my proposal for the BinaryReader, the BinaryWriter works analogous.
The BinaryReader's readHiveList method uses only the readUint32, readByte, readAsciiString and readKey methods instead of acting on the class's data directly. That means, the readHiveList method is at another layer of abstraction. That holds true for many of the methods – readList, readMap etc. all use simpler, more fundamental methods.
If we were to have different implementations of the BinaryReader (the Hive implementation, but also other ones for networking etc.), then it would make no sense for them to have to implement the readHiveList and readList, readMap etc. methods – rather they could simply implement the more fundamental methods and the more abstract ones live on top in another layer of abstraction.
If we would split the BinaryReader into a base class with only fundamental methods, like readByte, and readBytes plus seek and skip, the other methods could be converted into TypeAdapters. For example, the readHiveList method could easily be implemented as a TypeAdapter<HiveList> and then get automatically registered during startup. The same could even hold true for TypeAdapter<int>, TypeAdapter<String> etc.
Different encoding variants of the same type like ascii strings and several types of ints could be added as extension methods on the original writer or put directly into the abstract class.
No change would be necessary for the code generator, because it can still use the BinaryWriter and BinaryReader as usual. Instead of using readHiveList though, it would simply use read and then cast the result to a HiveList – just like with any other custom type.
If implemented, here's how the code would be separated:
The new serializing package would contain the abstract BinaryReader and BinaryWriter including already implemented methods for reading/writing basic types like Uint32, Maps, Lists, Sets etc. Everything from the hive_generator package would also move to a serializing generator package.
The hive package contains the BinaryReaderImpl and BinaryWriterImpl to actually read and write bytes to disk. Also, it implements and registers type adapters for hive-specific types (aka the HiveList).
Other packages could implement their own BinaryReaderImpl and BinaryWriterImpl to read/write bytes to the network or whatever.
How should we proceed? What do you think about this approach? And knowing that this has not the highest priority for you right now, should I start implementing it? It's currently a road blocker for a package I'm developing during my free time, so I'd be welcome to help.

Happy new year @marcelgarus
I had a similar thought. The Hive specific logic (readKey(), readFrame() and readHiveList()) could be implemented using extension methods for BinaryReader and BinaryWriter.
But what about readList(), readMap() and read()?

the readHiveList method could easily be implemented as a TypeAdapter and then get automatically registered during startup. The same could even hold true for TypeAdapter, TypeAdapter etc.

Yes, but it would impact performance. The TypeAdapter lookup is O(n) with n being the number of registered adapters.
Would you also put the TypeAdapter interface, the TypeRegistry and the hive_generator into the new package? The TypeAdapter is kind of Hive specific with the recent change (#163).
And the hive_generator depends heavily on Hive. Honestly, I have no idea how to create a generic build runner that can be extended by a package.

How should we proceed? should I start implementing it?

Let's work out the details first and then we can figure out the implementation.

Yes, but it would impact performance. The TypeAdapter lookup is O(n) with n being the number of registered adapters.

Architecture-wise, it seems more elegant to register TypeAdapters for List and Map, but for performance reasons, of course, it is reasonable to add readList() and readMap() as extension methods or implemented methods in the abstract class.
read() itself could also be an extension method or an implemented method in the abstract class. It depends on the TypeRegistry and TypeAdapters, but these really should be part of the serializer, see below.

Would you also put the TypeAdapter interface, the TypeRegistry and the hive_generator into the new package?

I think, TypeAdapters and the TypeRegistry as well as the hive_generator should also be part of the serializing package for several reasons.
They belong to the serializer semantically, because they care about serializing custom types.
Also, from the user's perspective, it would be much easier to just annotate the classes once to support binary serialization and it just works in all libraries that depend on it, rather than having to annotate each class and field with multiple annotations for the different libraries.
Not putting these classes into the serializing package, I feel like the benefit of a common serializing package gets greatly reduced.

The TypeAdapter is kind of Hive specific with the recent change (#163).

I don't see how adding the typeId to to TypeAdapter makes the package any more Hive-specific. Could you elaborate on that?

And the hive_generator depends heavily on Hive.

To be honest, I don't see that either. The @HiveType and @HiveField annotations would also be part of the serializing package (perhaps renamed?) and the HiveList can be encoded using a TypeAdapter. I do admit, this impacts performance a bit.

By the way, I do realize this is a very long term and breaking change, so if you are hesitant or want to wait for other features to be adopted first, I get that.
In the meantime, I'll try to implement my own serializer and also play with some optimizations and features that I think will be interesting. For example, by determining the adapter during code generation rather than at runtime for statically typed fields, encoding of both subclasses and superclasses, as well as generic types, could be supported.

Architecture-wise, it seems more elegant to register TypeAdapters for List and Map.

I agree.

I don't see how adding the typeId to to TypeAdapter makes the package any more Hive-specific.

Nevermind, I was confused for a second ^^

the HiveList can be encoded using a TypeAdapter

Unfortunately, I don't think using TypeAdapters works. The BinaryReader.read() method returns dynamic due to Dart limitations. So the hive_generator needs to cast any list or map (including HiveLists). In theory, HiveList could override the cast() method from List but I could not make it work. The problem is that HiveList requires its elements to extend HiveObject.
@override
HiveList<T> cast<T extends HiveObject>() { // -> invalid override
  return HiveListImpl(box, objects: _delegate.cast());
}

Due to this limitation, HiveList needs a function called castHiveList() and the hive_generator calls this method.

@HiveType and @HiveField annotations would also be part of the serializing package (perhaps renamed?)

Maybe @BinaryType and @BinaryField. Or we can find a way to make hive_generator (or whatever it is called then) customizable.

By the way, I do realize this is a very long term and breaking change, so if you are hesitant or want to wait for other features to be adopted first, I get that.

I think "Hive 2.0" is not too far away and we need some time for a future proof implementation anyway. I have no problem making breaking changes if it improves the package.
TLDR: I agree with everything you said but we need to figure out a way to let packages extend the hive_generator.

Hmm okay, I do see the problem with the HiveList. I'm not sure how to make the current hive_generator extensible either – I'm not sure it's even possible at all because all the code that runs during code generation is pre-defined.
Determining the adapter during code generation would solve the problem, so here are my thoughts on the topic so far:
Determining the adapter during code generation
If we would serialize objects this way, we retain type safety in statically typed contexts. We could rely on Adapter classes having a special name of the form AdapterForType<GenericType>. Because generated adapters live in the same library as the original class (the .g.dart file), classes that are in the namespace automatically have an adapter that's also in the namespace.
The syntax could look somewhat like this:
class AdapterForList<T> implements TypeAdapter<List<T>> {
  AdapterForList({@required this.adapterForT});

  final TypeAdapter<T> adapterForT;

  @override
  int get typeId => 4;

  void write(BinaryWriter writer, List<T> list) {
    writer.write(list.length, adapter: AdapterForCollectionLength());
    for (final item in list) {
      writer.write(item, adapter: adapterForT);
    }
  }

  List<T> read(BinaryReader reader) {
    final length = reader.read(adapter: AdapterForCollectionLength());
    return <T>[
      for (var i = 0; i < length; i++) reader.read(adapter: adapterForT),
    ];
  }
}

// A very simple generic class.
class Id<T> {
  Id(this.id);

  final int id;
}

// This adapter could omit the adapterForT by recognizing there's no field with the type.
class AdapterForId<T> implements TypeAdapter<Id<T>> {
  @override
  int get typeId => 10;

  @override
  Id<T> read(BinaryReader reader) {
    return Id<T>(reader.read(adapter: AdapterForInt()));
  }

  @override
  void write(BinaryWriter writer, Id<T> obj) {
    writer.write(obj.id);
  }
}

class MyClass<T> {
  MyClass({@required this.id, @required this.someNumbers});

  final Id<MyClass> id;
  final List<int> someNumbers;
}

class AdapterForMyClass<T> implements TypeAdapter<MyClass<T>> {
  AdapterForMyClass({@required this.adapterForT});

  final TypeAdapter<T> adapterForT;

  @override
  int get typeId => 11;

  @override
  MyClass<T> read(BinaryReader reader) {
    final id = reader.read(adapter: AdapterForId<MyClass>());
    final someNumbers = reader.read(
      adapter: AdapterForList<int>(
        adapterForT: AdapterForInt(),
      ),
    );
    return MyClass<T>(
      id: id,
      someNumbers: someNumbers,
    );
  }

  @override
  void write(BinaryWriter writer, MyClass<T> obj) {
    writer
      ..write(obj.id, adapter: AdapterForId())
      ..write(
        obj.someNumbers,
        adapter: AdapterForList<int>(adapterForT: AdapterForInt()),
      );
  }
}

Note that there are several great features that improve upon the current design:

There is no adapter lookup of O(n) anymore.
The typeId doesn't have to be saved explicitly anymore, because we know that the reader knows by context which type to expect. That also means types have no storage overhead by default. Currently, common cases like List<String> and List<int> have special encodings for more efficiency, this wouldn't be necessary anymore.
Encoding both a supertype and a subtype would work as well. Currently, when encoding the subtype, two type adapters match and it's impossible for the serializer to choose.
Generics work.

What still needs to be worked out is that this only works in statically typed contexts. Anywhere where a dynamic value is expected, we need to encode the type as well. That also applies to collections, because they may contain subclasses or implicit generic dynamics.
In the example, a List<Id> could be constructed, containing [Id<int>(0), Id<String>(3), Id<MyClass>(7)]. With some tweaking, retaining types in such a situation is possible, I think.
But there are limits to type safety: If we have a List<dynamic> with [3, Id<T>(8)], to my understanding there is currently no way in Dart to find out what T is.
Also, forward compatibility is still an issue, because in the code above, we don't save field ids anymore. There's probably a simple solution to that, like first saving all the field ids along with an offset for each and then running the appropriate adapters at the offsets.
About annotations

Maybe @BinaryType and @BinaryField.

I like @BinaryType and @BinaryField. If we find a name other than "Binary" for the serializer, prefixing with that would also be fine with me.

Or we can find a way to make hive_generator (or whatever it is called then) customizable.

I'd strongly advise against that, because then for multiple libraries that use the binary serialization, you'd need multiple annotations on every field again.
About keeping track of typeIds and fieldIds automatically
For typeIds and fieldIds , dynamically generating them is obviously not a good option, as you pointed out at #163. The key part is that these IDs are not generated but explicitly mapped in the code.
So for fieldIds, an extension would perhaps be better suited for the job. For example, in VS Code, there could be a command for making the current class serializable, aka automatically annotating it with @BinaryType and enumerating the fields with @BinaryField annotations as well as adding the part '...g.dart' directive.
There could also be a remove command when the cursor is on a field, automatically inserting a comment // don't use the following field ids for legacy reasons: 3 at the top of the class or appending to that list.
Every time, the first command is run, it could annotate fields that don't have an annotation yet, while respecting the comment at the top of the class.
For typeIds, a global map as proposed in #163 would probably be an option. It could be placed in a special binary.dart file or something similar and work just like the fieldId but on a class level.
Automatically registering types?
While we're at it and have a global map of types to their IDs, it's no harm to automatically register all these types, right? It would certainly reduce friction if you could just trigger a VS Code command to make a class serializable, run a command and it would just work out of the box without having to register any types.
Breaking changes

I have no problem making breaking changes if it improves the package.

I agree with you here on a code level, but if we were to change the binary representation itself, we would not only bother developers but also break apps on the user's side. So, we would certainly need a transition path for that.

I like your idea but I have still some questions:
How will objects without generated adapter be saved?
The adapterForT parameter is a good idea but Hive also supports storing objects without generated adapter. How would storing a List<SomeObject> for example work? The user would need to register AdapterForList<SomeObject> manually.
Information of subtypes is lost
I can't think of a solution for the following:
class Pet { }

class Dog extends Pet {
  String name;
}

class MyClass {
  Pet pet;

  MyClass(this.pet);
}

If the user wants to save MyClass(Dog()), all the fields of Dog will be lost because the AdapterForMyClass expects a Pet.
Forward compatibility

There's probably a simple solution to that, like first saving all the field ids along with an offset for each and then running the appropriate adapters at the offsets.

If I understand you correctly, you want to keep track of the position of the fields in a file or something similar. I don't think it is that easy: If the user changes the class multiple times, I don't see how we could track the field.

For example, in VS Code, there could be a command for making the current class serializable, aka automatically annotating it with @BinaryType and enumerating the fields with @BinaryField annotations as well as adding the part '...g.dart' directive.

While we're at it and have a global map of types to their IDs, it's no harm to automatically register all these types, right?

Sounds great :+1:

After several prototyping attempts, I just realized the gravity of the fact that if we have a List<T> given to a function that accepts a dynamic, there is no way in Dart to find out what T is. This is really mind-boggling to me because it prevents us from writing a type-safe serializer (which you can give any object, like List<T> and it returns a representation, that if interpreted will return a List<T> again).
Grudgingly accepting that, I'm now more comfortable with dropping type information.
Regarding generics, I think my initial aspirations with statically resolved adapters are not feasible – as you said if you have a List<Object> that saves several numbers and strings, we don't need or want the calling side to be forced to specify an adapter for Object. I agree that the approach with adapterForT doesn't work in that case, so let's abandon that idea.
However, I'd still use generic types in the AdapterFor… type itself:
class AdapterForList<T> implements TypeAdapter<List<T>> {
  @override
  int get typeId => 4;

  void write(BinaryWriter writer, List<T> list) {
    writer.write(list.length, adapter: AdapterForCollectionLength());
    list.forEach(writer.write);
  }

  List<T> read(BinaryReader reader) {
    final length = reader.read(adapter: AdapterForCollectionLength());
    return <T>[
      for (var i = 0; i < length; i++) reader.read(),
    ];
  }
}

When a typed List is saved inside an object, it is read using reader.read(adapter: AdapterForList<int>()), providing a List<int>. In dynamic contexts just reader.read() is called and returns a List<dynamic>.
A class adapter would look like the following:
@BinaryType(11)
class MyClass<T> {
  MyClass({@required this.id, @required this.someNumbers});

  @BinaryField(0)
  final Id<MyClass> id;

  @BinaryField(1, defaultValue: <int>[])
  final List<int> someNumbers;
}

class AdapterForMyClass<T> implements TypeAdapter<MyClass<T>> {
  AdapterForMyClass();

  @override
  int get typeId => 11;

  @override
  MyClass<T> read(BinaryReader reader) {
    var numOfFields = reader.read(adapter: AdapterForNumberOfFields());
    var fields = <int, dynamic>{
      for (var i = 0; i < numOfFields; i++) reader.read(adapter: AdapterForFieldId()): reader.read(),
    };
    return MyClass(
      id: fields[0],
      someNumbers: fields.containsKey(1) ? fields[1] : <int>[],
    );
  }

  @override
  void write(BinaryWriter writer, MyClass<T> obj) {
    writer
      ..write(2)
      ..write(0)
      ..write(obj.id)
      ..write(1)
      ..write(obj.someNumbers, adapter: AdapterForList<int>());
  }
}

Regarding your questions:

How will objects without generated adapter be saved?

Now, there is no more adapterForT, so that works.

Information of subtypes is lost

With the current implementation, that also happens. More specifically, it only happens if both the superclass and subclass have adapters:
@BinaryType
class Pet {}

@BinaryType
class Dog extends Pet {
  String name;
}

That doesn't work with the current version either:
_ResolvedAdapter findAdapterForValue(dynamic value) {
  for (var adapter in _typeAdapters.values) {
    if (adapter.matches(value)) return adapter;
  }
  return null;
}

If the PetAdapter comes before the DogAdapter in _typeAdapters.values, the PetAdapter gets chosen instead of the DogAdapter.
Is there any disadvantage of directly instantiating an adapter?
Then we could register the adapters in a hierarchy.
For example, each adapter could have a collection of subAdapters:
class AdapterForPet extends TypeAdapter<Pet> {
  Set<TypeAdapter<Pet>> subAdapters;

  void registerSubAdapter(TypeAdapter<Pet> adapter) => subAdapters.add(adapter);

  // ...
}

const adapterForPet = AdapterForPet();

I already mentioned automatically registering adapters, so we could automatically generate code like the following:
// dog_adapter.g.dart

// ... the DogAdapter

const adapterForDog = AdapterForDog();

// The adapterForPet is in scope, because the Dog extends Pet, so Pet is in scope.
adapterForPet.registerSubAdapter(DogAdapter());

Then, the findAdapterForValue method could be implemented like this:
_ResolvedAdapter findAdapterForValue(dynamic value) {
  final TypeAdapter matchingAdapter;
  while (true) {
    for (var adapter in matchingAdapter?.subAdapters ?? _typeAdapters.values) {
      if (adapter.matches(value)) {
        matchingAdapter = adapter;
        break;
      }
    }
    // No (sub)adapter matched.
  }
  return matchingAdapter;
}

Note that because of the hierarchical structure, performance is somewhere between O(log n) and O(n) (depends on the class hierarchy), as opposed to always O(n) currently.

I wasn't sure if the hierarchical type adapter structure would work, so I implemented a proof of concept.
Here's an example class along with the serializer that could be generated:
class MyClass<T> {
  MyClass({@required this.id, @required this.someNumbers});

  final String id;
  final List<T> someNumbers;
}

class AdapterForMyClass<T> extends TypeAdapter<MyClass<T>> {
  const AdapterForMyClass();

  @override
  void write(BinaryWriter writer, MyClass<T> obj) {
    writer
      ..writeNumberOfFields(2)
      ..writeFieldId(0)
      ..write(obj.id)
      ..writeFieldId(1)
      ..write(obj.someNumbers);
  }

  @override
  MyClass<T> read(BinaryReader reader) {
    final numberOfFields = reader.readNumberOfFields();
    final fields = <int, dynamic>{
      for (var i = 0; i < numberOfFields; i++)
        reader.readFieldId(): reader.read(),
    };

    return MyClass<T>(
      id: fields[0],
      someNumbers: fields[1],
    );
  }
}

Regarding registering adapters and ownership of type ids, I currently don't have the id saved in the adapter. There are arguments for both sides, but eventually, type ids have to be managed across multiple data types and with generic types working, the same adapter can be registered using multiple ids.
Also, all the ids are offset when being encoded, so that built-in ids are now negative and users can use ids starting at 0.
When trying to serialize the type from above, an exception gets thrown:
No adapter for the type MyClass<int> found. Consider adding an adapter for that type by calling AdapterForMyClass<int>().registerWithId(0).

Doing that, another exception occurs and a warning occurs in the console:
No adapter for the exact type List<int> found, so we're encoding it as a List<dynamic>. For better performance and truly type-safe serializing, consider adding an adapter for that type by calling AdapterForList<int>().registerWithId(1).

Because the static type of a List<int> is not equal to the runtime type of List<int> (that means, <int>[1,2,3].runtimeType != List<int>), sadly another warning appears in the console, but the serializing works fine:
No adapter for the exact type List<int> found, so we're encoding it as a List<int>. For better performance and truly type-safe serializing, consider adding an adapter for that type by calling AdapterForList<int>().registerWithId(3).
Serialized to [8, 1, 0, 2, 0, 0, 7, 249, 0, 0, 0, 7, 115, 111, 109, 101, 45, 105, 100, 0, 1, 8, 2, 0, 0, 0, 3, 7, 250, 0, 0, 0, 0, 0, 0, 240, 63, 7, 250, 0, 0, 0, 0, 0, 0, 0, 64, 7, 250, 0, 0, 0, 0, 0, 0, 8, 64]
MyClass(some-id, [1, 2, 3])

void main() {
  AdapterForMyClass<int>().registerWithId(0);
  AdapterForList<int>().registerWithId(1);

  final data = serialize(MyClass(id: 'some-id', someNumbers: [1, 2, 3]));
  print('Serialized to $data');
  print(deserialize(data));
}

I just realized the gravity of the fact that if we have a List given to a function that accepts a dynamic, there is no way in Dart to find out what T is

It is super annoying and was a major bummer when I implemented Hive. Unfortunately, we have to live with that.

If the PetAdapter comes before the DogAdapter in _typeAdapters.values, the PetAdapter gets chosen instead of the DogAdapter.

Yes, that's correct and I seemingly forgot to document it. I'm thinking about using a Map<Type, TypeAdapter> to allow O(1) access if an adapter has the exact same type as a value. If there is no such adapter, the TypeRegistry will use the same approach as before to find the correct adapter.
I think the subAdapters hierarchy is a great idea to improve performance.

Also, all the ids are offset when being encoded, so that built-in ids are now negative and users can use ids starting at 0.

Hive does that as well.
I'm still not sure whether I like the idea to create adapters for primitives and Lists. The reason is that the BinaryReader and BinaryWriter methods can access the byte buffer directly which improves performance. I don't see why anyone would want to override the default behavior for storing primitives.
In some cases it might be useful (for example in the Hive Sketchpad demo) but then implementing a custom TypeAdapter for the whole object is a better idea.
I've seen your implementation for portals. It looks very promising. A small note on using ByteData and Uint8List. Creating a new instance of either of them is rather expensive (even Uint8List.view()). You should consider replacing for example
void writeByte(int value) => writeByteData(ByteData(1)..setUint8(0, value));

with something that writes to the underlying buffer directly. I spent countless hours running microbenchmarks xD (the docs on improving performance of Dart programs is virtually non-existen)

I'm thinking about using a Map<Type, TypeAdapter> to allow O(1) access if an adapter has the exact same type as a value. If there is no such adapter, the TypeRegistry will use the same approach as before to find the correct adapter.

That's the exact approach I'm taking too. 😊 Constant lookup works fine with user-defined types, but types like List are very thoroughly engrained in the VM, so it doesn't work for them. Also, if the type is just an interface and the implementation is private, that doesn't work either (for example, <int>{} creates a _CompactLinkedHashSet<int>).
The fallback strategy of searching for the type in the type tree can be made even more efficient by registering common dart:core types first (like Iterable, which in turn contains List and Set first).

I'm still not sure whether I like the idea to create adapters for primitives and Lists. The reason is that the BinaryReader and BinaryWriter methods can access the byte buffer directly which improves performance.

Personally, I really like the cleanliness of the architecture where every type has a corresponding adapter instead of some types having hardcoded behavior.
Though I do get that performance is a top priority for you.
But I'm not sure I completely get your point here – adapters have no overhead whatsoever.
In fact, with the current constant type lookup before the tree traversal, looking up the AdapterForList and AdapterForSet is more efficient than before.
I'm not sure about the performance as I'm not well-versed with Dart performance – as you said, the docs on that are virtually non-existent. :trollface:
But if the current BinaryWriter and BinaryReader interfaces don't allow the most efficient operating on data, maybe the API could be changed so it's possible to delegate operations directly to a ByteBuffer? So, instead of using Uint8List as the exchange format, they might have some operations like writeUint8(value), which in turn just calls the corresponding call at the correct offset on an internal ByteBuffer.

I don't see why anyone would want to override the default behavior for storing primitives.

That's not what I meant, sorry if I didn't make myself clear.
I was talking about how it would make it easier for us as framework implementers to register these special types.

A small note on using ByteData and Uint8List. Creating a new instance of either of them is rather expensive (even Uint8List.view()).

I do like the simplicity of an API that just implements writeBytes(List<int> bytes), but I'll experiment with broadening the API surface to a few functions to yield better performance by directly acting on the ByteBuffer.

But I'm not sure I completely get your point here – adapters have no overhead whatsoever.

I suspect Dart can optimize the if (value is SomeType) else if ... code. But I have not tested it yet and need to run some microbenchmarks. I fully agree that the adapter approach is cleaner.

I do like the simplicity of an API that just implements writeBytes(List<int> bytes)

Yes I agree but in that case I don't think it is worth it.

I'll experiment with broadening the API surface to a few functions to yield better performance by directly acting on the ByteBuffer

I've used "package private" extensions to hide internal methods for the HiveObject and it worked quite well.
I will also create a prototype of this idea, then we can compare our solutions and maybe settle with the optimal compromise. Currently I'm working on a new project so I will probably need until late february to implement it.

🎉 I'm pretty much done with my serializer prototype.
You can now register several adapters for a single type. That allows for some nifty optimizations like encoding some ints as uint8 and others as int16 based on their value.
Admittedly, I went a little overboard with the optimization of the binary format. For example, the list [1, 6, 10000] gets encoded using the AdapterForPrimitiveList.short(AdapterForUint16()), which is really optimized for this kind of list – because the list is short and doesn't contain nulls, only one byte saves the size and the rest of the bytes save raw uint16s. Some of the currently over 100 adapters should probably be removed in favor of a less efficient but faster serialization – I guess that's the typical trade-off between time and space efficiency.
Also, I have to say I really like the two bytes for type ids. They give us a lot of room for external packages registering adapters. That means developers could mix and match packages if they all use non-overlapping type ids. A table of reserved type ids could ensure that's the case.

I really like the current implementation of binary as it provides easy-to-use type-safe serializing/deserializing. I'll probably make it a little more time-efficient.
I suppose in the new architecture, the communication between the backend and frontend probably happens over some standardized protocol like JSON or ProtoBuf. But at some point, a serialization of Dart objects still needs to happen. So, how should we proceed with this issue?

I suppose in the new architecture, the communication between the backend and frontend probably happens over some standardized protocol like JSON or ProtoBuf.

I think the BinaryReader and BinaryWriter are one of the few pieces that will actually remain in the new version.
Since Dart does not allow sending or receiving objects to C or Rust, they will need to be serialized. I have come up with a format that allows random accessing specific fields without reading the whole object. This should make queries super fast.
I did not have time to benchmark the binary package yet so I don't know if it is a suitable replacement for BinaryReader / BinaryWriter. I would leave this issue open until then.

I ended up inventing a whole new serialization format that is self-describing like JSON. That means it supports inspecting objects created with adapters unknown to us as well as parsing objects in other languages like Rust. These primitive blocks are also designed to be extensible over time.
Here's the repository: https://github.com/marcelgarus/tape
And here's the kind-of specification/thoughts about the basic building blocks: https://github.com/marcelgarus/tape/issues/27
The other layers (adapters, registry, etc) still need to be fleshed out.
I also plan to create more tooling and infrastructure that are needed for a great serializer, like drop-in-support for types of other packages and tools helping with annotating objects. Finally, because it's based on annotating existing classes (like Hive is currently), it also supports freezed (at least theoretically, I still need to implement that part).


Reddit page is not working

It shows empty page

Can you provide us with your selected tags?

Now It is working fine

Great, we've also added a language filter to the Reddit card.


'Nullable'  not supported by current database provider

Hello,
I've encountered a breaking/change bug when trying to update to .NET Core 2.1.1/ EF Core 2.1/ Npgsql 4.0.2.
The similar case was described here -  I tried to reproduce that issue but I cannot.
The issue I have is with a column type being set explicitly to numeric NULL.
In that case, while trying to create migrations or run tests with EnsureCreated() the following exception is being thrown:
System.InvalidOperationException: 'The property 'Entity.NullableValue' is of type 'Nullable<decimal>' which is not supported by current database provider. Either change the property CLR type or ignore the property using the '[NotMapped]' attribute or by using 'EntityTypeBuilder.Ignore' in 'OnModelCreating'.'

It works correctly without a call to method .HasColumnType("numeric NULL") and creates a numeric type column with NOT NULL not being set.
In the previous version of Npgsql that code was creating column with exactly the same type.
Steps to reproduce:
.csproj
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Npgsql" Version="4.0.2" />
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="2.1.1" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="2.1.1" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="2.1.1" />
    <PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="2.1.0" />
  </ItemGroup>

</Project>

Program.cs
using System;
using Microsoft.EntityFrameworkCore;

namespace NpgsqlRepro
{
    class Program
    {
        public const string ConnectionString = "xxxxx";

        static void Main(string[] args)
        {

            var options = new DbContextOptionsBuilder<TestContext>().UseNpgsql(ConnectionString).Options;
            var ctx = new TestContext(options);

            ctx.Database.EnsureDeleted();
            ctx.Database.EnsureCreated();

            Console.ReadLine();

            ctx.Database.EnsureDeleted();
        }
    }

    public class TestContext : DbContext
    {
        public const string Schema = "Test";

        public TestContext(DbContextOptions options) : base(options)
        {
        }

        public DbSet<Entity> Entities { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.HasDefaultSchema(Schema);
            base.OnModelCreating(modelBuilder);

            var entityBuilder = modelBuilder.Entity<Entity>();
            entityBuilder.HasKey(x => x.Id);
            entityBuilder.Property(x => x.NullableValue).HasColumnType("numeric NULL");
        }
    }

    public class Entity
    {
        public Entity()
        {
        }

        public long Id { get; set; }

        public decimal? NullableValue { get; set; }
    }
}

Details:
.NET Core SDK (reflecting any global.json):
 Version:   2.1.301
 Commit:    59524873d6

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17134
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.301\

Host (useful for support):
  Version: 2.1.1
  Commit:  6985b9f684

That was fast! I've just realized that I was not on the latest version of Npgsql.EntityFrameworkCore.PostgreSQL, I tested it and it works. Though, that bugfix was not listed on 2.1.1, at least not explicitly ;)


Allow for local Reveal.js installation

Allow the user to provide a path to a local Reveal.js installation.

Implemented in #5


Don't test for publish-able in PRs

The test step in the travis.yml checked whether the PR had changed the version numbers of all packages. This PR removes that step because it's cumbersome to merge chore PRs.

Codecov Report

Merging #81 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master    #81   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           3      3
  Lines         419    419
  Branches       60     60
=====================================
  Hits          419    419

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32f3b0a...e3aaeac. Read the comment docs.


Compiled storybook shows nested iframes

What version of vite are you using?
3.0.0
System info and storybook versions
Environment Info:
System:
OS: Windows 10 10.0.19044
CPU: (8) x64 Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
Binaries:
Node: 14.19.1 - C:\Program Files\nodejs\node.EXE
Yarn: 1.22.0 - ~\scoop\apps\yarn\current\Yarn\bin\yarn.CMD
npm: 6.11.3 - C:\Program Files\nodejs\npm.CMD
Browsers:
Chrome: 103.0.5060.114
Edge: Spartan (44.19041.1266.0), Chromium (103.0.1264.49)
npmPackages:
@storybook/addon-actions: ^6.5.9 => 6.5.9
@storybook/addon-essentials: ^6.5.9 => 6.5.9
@storybook/addon-interactions: ^6.5.9 => 6.5.9
@storybook/addon-links: ^6.5.9 => 6.5.9
@storybook/builder-vite: ^0.2.0 => 0.2.0
@storybook/react: ^6.5.9 => 6.5.9
@storybook/testing-library: ^0.0.13 => 0.0.13
Describe the Bug
Running story locally work fine.
However, once I try to build the static files using yarn build-storybook, this is what it shows:

Upon checking the logs, this is what it shows

Here what my vite.config.ts:
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import svgr from 'vite-plugin-svgr';
import tsconfigPaths from 'vite-tsconfig-paths';

import optimizationPersist from 'vite-plugin-optimize-persist';
import pkgConfig from 'vite-plugin-package-config';
import graphql from '@rollup/plugin-graphql';
import eslintPlugin from '@nabla/vite-plugin-eslint';

// https://vitejs.dev/config/
export default defineConfig({
  // This changes the out put dir from dist to build
  // comment this out if that isn't relevant for your project
  resolve: {
    alias: {
      // https://github.com/aws-amplify/amplify-ui/issues/268#issuecomment-953375909
      './runtimeConfig': './runtimeConfig.browser',
    },
  },
  build: {
    outDir: 'build',
  },
  // @ts-ignore
  test: {
    globals: true,
    testTimeout: 20000,
    environment: 'jsdom',
    include: ['src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
  },
  server: {
    host: true,
    port: 3000,
  },
  plugins: [
    react(),
    svgr(),
    tsconfigPaths(),
    pkgConfig({
      field: 'vite',
      packageJsonPath: 'vite-deps.json',
    }),
    optimizationPersist(),
    graphql(),
    eslintPlugin({
      eslintOptions: {
        overrideConfigFile: './.eslintrc-ide.json',
      },
    }),
  ],
});

Link to Minimal Reproducible Example
https://github.com/pussyslain/storybook-vite-build-issue-repro
Participation

[X] I am willing to submit a pull request for this issue.

Thanks for the reproduction.  I cloned, installed, built storybook, and launched it using npx http-server ./storybook-static, and did not see any problems:

What are you using to preview the built storybook?

@IanVS  I see. I am using serve -s storybook-static for preview. Probably need additional configuration to make it work.
http-server works for me as well. Thank you

:+1:


How do I run the tests?

I'd like to make a couple of tiny improvements, but I can't run the tests. If I run npm run test it says No gulpfile found.
I'm not a node developer, so I don't know if I'm missing something. Is there a way to run the tests at the moment?

You have to run the mocha command.  I see have a pr that is fixing this, I will merge and make a new release. Thanks for your contribution!


cannot reproduce the same accuracy in the paper

Thanks for your great work!
But I trained resnet18 with the default config, and only got the top1 acc 52, top5 acc 75

Ahh, Thanks you for using it.
I tested ResNet18 twice, it can achieve the same accuracy.
I am busy on other affairs recently, and I will dive into this issue after a few days (~10days).

The authors quantized the first & last layers to 8b integers, while I left them floating-pointed. I guess that's why I got a slightly better accuracy than the authors.
I will do more experiments in a few days, and release will-trained models. I have only two gaming GPUs, so it won't be very soon

The authors quantized the first & last layers to 8b integers, while I left them floating-pointed. I guess that's why I got a slightly better accuracy than the authors.
I will do more experiments in a few days, and release will-trained models. I have only two gaming GPUs, so it won't be very soon

Can I have your wechat, I have GPUs, can help you

Sorry for that I cannot find your email address on your GitHub page.
haozhe_zhu @ foxmail dot com, this is my email. I will reply you with my WeChat qrcode.

Sorry for that I cannot find your email address on your GitHub page.
haozhe_zhu @ foxmail dot com, this is my email. I will reply you with my WeChat qrcode.

邮件发了

@zhutmost Can you share your experimental results on ImageNet? I wonder the bit-width and its accuracy.

@cometonf, ResNet18 with act 2b and weight 3b, the top5 is about 86~87%. If approved, I can share a trained model with its config yaml.
I am still working to improve the accuracy but I think there is no difference between my code and the original paper's algorithm (correct me if any). And I found a slight difference in hyper-parameters can cause significant changes in results.

@zhutmost Thank you! I'm also trying to improve the LSQ (using Tensorflow), since I got 70.7% top-1 accuracy for the W4A4 case. If you figure out some detailed hyper-parameter effects, please share that point.

@cometonf, I have released a quantized model as well as its corresponding configuration YAML file. You can find it in the README. Its quantized bit width is a2/w3, and the acc is top1 66.9% and top5 87.2%.


PI-340 Run end-to-end tests on MOJ Cloud Platform

See the test report in the pipeline summary, here: https://github.com/ministryofjustice/hmpps-probation-integration-services/actions/runs/2739294700

See the test report in the pipeline summary, here: https://github.com/ministryofjustice/hmpps-probation-integration-services/actions/runs/2739294700#summary-7519707167


Implementation of Fixed-Size APC

This branch implements a "fixed active space size" APC following the approach used in a recently submitted preprint: https://chemrxiv.org/engage/chemrxiv/article-details/6570cf90cf8b3c3cd7ee2117 . It is now accessible through adding the "fixed=True" keyword to the APC call. Tests and examples have been added for this feature.
Additionally, a sanity check has been added to raise an error if the "n" argument exceeds the number of virtual orbitals in the molecule.

I found it a bit challenging to decide whether to include new methods like this within the core module. I suggest placing the code in the repository at https://github.com/pyscf/pyscf-forge first. This repository is designed for publishing preliminary functionalities and would be suitable for the initial implementation.

Sure, I can make a PR there first. What is the standard for having the method pulled into PySCF? Does it need to be published/used in X number of papers?

Sure, I can make a PR there first. What is the standard for having the method pulled into PySCF? Does it need to be published/used in X number of papers?
Normally I would not PR brand new methods but this was a fairly minor change functionally to the method already pulled into PySCF

The standards for pyscf-forge are not as strict as those for the PySCF main repo. Please see https://github.com/pyscf/pyscf-forge/blob/master/README.md . pyscf-forge encourages contributions of new methods and features. To help users understand the functionality, it’s recommended to include examples and tests, and a link to the related publications in the code.


Abstract migration: Content block - simple (Simple long-form)

user story
As a QA tester using a Windows PC, I need a way to access the design specs from my Windows PC so that I can conduct QA tests.
Additional Information

Prod QA Issue (#1293)

Acceptance criteria

[x] Change all instance of the name Simple long-form to Content block — simple
[x] Upload the design spec file(s) to the Layout patterns Abstract project
[x] A collection in Abstract is created and the link to it is available
[ ] Associated Git stories have been updated with the new name and link
[ ] Add or update README Box note with a link pointing to the hosted file
[ ] A comment is posted in the Prod QA issue, tagging Praveen and Chetan, when Abstract work is finished

Content block – simple is now updated and available on Abstract: https://share.goabstract.com/06a41475-72e4-45d0-9bef-3fb0158bcdca?


tools: Remove the special handle for Ubuntu under Windows 10

Summary
since the Native Linux support could be reused for this case:
https://lists.apache.org/thread.html/r315682ed20bbeb2f1403cf592f892ef009274423189ffc5b3841a6a9%40<dev.nuttx.apache.org>
Impact
User should use -l instead -u for WSL.
Testing

I agree, especially since WSL2 IS Linux there is really no reason to have the extra logic here.


Hide labels for hidden inputType

Currently fields that have an inputType: "hidden" still have labels show and must manually be hidden via CSS.
I am confused by this decision, perhaps you can explain to me why it is this way or whether I am approaching this situation in the wrong manner?

Sorry, this was my fault!


Track Email Conversation Threads with owasync.py

Hello,
When creating incidents via email messages, it would seem advantageous to only create an incident once per a conversation thread. Users will add the "On call" email address to a thread that requires emergency attention, which triggers the incident, but every successive message in that thread will continue to create new Iris incidents each time there is a reply.
I notice that there is a check in ResponseEmail.on_post for the 'In-Reply-To' header, but at least in my organization's Exchange implementation I don't see that header existing. However, Outlook/Exchange does have a well-documented 'Thread-Index' header that tracks the uniqueness of a conversation thread:
https://msdn.microsoft.com/en-us/library/ee202481(v=exchg.80)?ranMID=24542&ranEAID=TnL5HPStwNw&ranSiteID=TnL5HPStwNw-go3ivEugoQ2BnSvLr55H8A&tduid=(753a0626561d1d154f69a0a528d10da2)(256380)(2459594)(TnL5HPStwNw-go3ivEugoQ2BnSvLr55H8A)()
Using the Thread-Index, I have some code that will store the unique Thread-Index ID in the incident context. When an incoming email message arrives that would normally create an incident, ResponseEmail.on_post will first check incidents created in the past X number of hours (configurable), and if the context already contains the same Thread-Index ID, skip creation of a new incident and log a warning.
Let me know what you think about the feature and I can send a PR if you like it. Thanks!

Hey,
Thanks for looking into this. While we could store some metadata in the incident context and then iterate backward parsing the ID out of the json looking for it, this is likely to be inefficient and slow when you have lots of incidents.
We were hoping to find headers that only exist on email replies to a thread. Do you see any other headers aside from Thread-Index that we could merely check the presence of?

Also, quick question: how are you pulling your emails from exchange? Using our OWA sync script or something else?

I agree that looking backwards in all incident context carries the risk of being very slow with a lot of incidents. For this reason, I'd restrict it to incidents within the last 24 hours by default (with a configurable lookback time). As long as the incident.created is indexed (and it is), I don't foresee this being too much of a drag on performance.
Can I show you what I have in mind? Here's a diff of the proposed changes:
https://github.com/linkedin/iris/compare/master...p6knewman:issue_274_track_thread_index

Hey,
This might cause problems for us and other very high volume iris instances as we have around 2000 incidents per day.
It would be cleanest for this process to not have state. Would it be possible for you to instead of CC'ing, BCC iris when you need to loop it into a thread and that way subsequent replies won't create incidents?
Regards

Not sure I understand, or maybe we're not on the same page. The query of recent incidents is this line:
https://github.com/linkedin/iris/compare/master...p6knewman:issue_274_track_thread_index#diff-ddac0526a1fb66b4b38d3eaa27bce478R3157
If the worst-case scenario for incident volume is on the order of 2000 per/day, the WHERE clause on the default configuration setting would only produce 2000 rows returned. That doesn't seem too onerous to work with, IMO. If you have a high volume DB, would you be willing to pull down my branch and test performance with a couple test incidents created via email? I could also work on creating a ton of junk data on my dev instance to see what performance is like.
As for BCC vs. CC, that unfortunately wouldn't solve my use case. The users that may/will be sending incidents in via email may are unlikely to remember to go through such specific steps when logging an incident.

I went ahead and generated some junk data on my dev instance. I generated 2000 incidents per day going back for the last 365 days (730k total). I timed the performance of both the MySQL query and the combined time it takes to query the DB + ujson.loads the context and iterate over it looking for the same thread_id.
The performance numbers were quite acceptable even up to a retrieval of ~28,000 incidents (~336hrs at 2000 incidents/day), which really seems like a silly long period to be looking back for duplicate incidents. Long before that far back, I'd be considering this a new incident and it's time to page someone out again. At a more reasonable period of 24 hours, the performance impact is roughly 90-100ms.
with 730k incidents of random context data, 2000 incidents per a day, incident_lookback_time 48 hours:
web.1    | [2017-10-05 09:05:38 +0000] [98023] [WARNING] iris.api Fetched 4003 incidents from DB in 0.09 seconds
web.1    | [2017-10-05 09:05:38 +0000] [98023] [WARNING] iris.api Total time to fetch from DB + parse 4003 recent incidents: 0.1 seconds
web.1    | <IP_ADDRESS> - - [05/Oct/2017:09:05:38 -0500] "POST /v0/response/email HTTP/1.1" 204 - "-" "python-requests/2.9.1"

incident_lookback_time 72 hours:
web.1    | [2017-10-05 09:06:38 +0000] [98742] [WARNING] iris.api Fetched 6003 incidents from DB in 0.12 seconds
web.1    | [2017-10-05 09:06:38 +0000] [98742] [WARNING] iris.api Total time to fetch from DB + parse 6003 recent incidents: 0.13 seconds
web.1    | <IP_ADDRESS> - - [05/Oct/2017:09:06:38 -0500] "POST /v0/response/email HTTP/1.1" 204 - "-" "python-requests/2.9.1"

incident_lookback_time 144 hours:
web.1    | [2017-10-05 09:07:45 +0000] [99102] [WARNING] iris.api Fetched 12001 incidents from DB in 0.3 seconds
web.1    | [2017-10-05 09:07:45 +0000] [99102] [WARNING] iris.api Total time to fetch from DB + parse 12001 recent incidents: 0.35 seconds
web.1    | <IP_ADDRESS> - - [05/Oct/2017:09:07:45 -0500] "POST /v0/response/email HTTP/1.1" 204 - "-" "python-requests/2.9.1"

incident_lookback_time 336 hours:
web.1    | [2017-10-05 09:09:28 +0000] [99599] [WARNING] iris.api Fetched 27994 incidents from DB in 0.49 seconds
web.1    | [2017-10-05 09:09:28 +0000] [99599] [WARNING] iris.api Total time to fetch from DB + parse 27994 recent incidents: 0.54 seconds
web.1    | <IP_ADDRESS> - - [05/Oct/2017:09:09:28 -0500] "POST /v0/response/email HTTP/1.1" 204 - "-" "python-requests/2.9.1"

@jrgp, any thoughts on this? If preferred, I could make this whole feature subject to an on/off config option and default it to off.


[master] Fix WFLY-8548

@frickeo reported an issue in WildFly EJB security annotation processing and submitted a pull request a while back https://github.com/wildfly/wildfly/pull/9922. I happened to have a look at it today and what's reported in there is indeed a genuine issue, IMO. The issue can cause wrong security roles to be checked for, by the container, when dealing with invocations on EJB methods, when method level EJB security annotations are applicable.
The credit for investigating and fixing this issue should go to @frickeo. The fix provided by @frickeo has been included in this PR. The sole purpose of my PR is to add a proper testcase, to test the presence of the bug and verify the fix. The JIRA for this issue is https://issues.jboss.org/browse/WFLY-8548.

org.wildfly.test.integration.elytron.audit.TCPSyslogAuditLogTestCase] testSuccessfulAuth
[04:27:23][testSuccessfulAuth] java.lang.AssertionError: Successful authentication was not logged
[04:27:23]
[testSuccessfulAuth] java.lang.AssertionError: Successful authentication was not logged
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.wildfly.test.integration.elytron.audit.AbstractSyslogAuditLogTestCase.testSuccessfulAuth(AbstractSyslogAuditLogTestCase.java:59)

Failure doesn't look related.

retest this please


Remove Plug dependency (except for tests)

@belltoy thanks for contribution. I have decided to keep Plug as a dependency. The TODO in the README was prior to Plug 1.0 when I was getting many PRs to update the Plug dependency. Now that it's stable, I'm okay with keeping it in.
There is actually one more place I'm using Plug - it's main use (besides testing) - was to automatically parse the content-type.
I may revisit this later.
Thanks again!


how to hint DateTime field

code
@derive {Nestru.Decoder, hint: %{created_at: DateTime}}

error info
 ** (RuntimeError) Expected a map value received "2024-03-29T18:42:5
9.816000Z" instead. Can't convert it to a DateTime struct.

expect help me do this
|> update_in([Access.key!(:created_at)], fn prev ->
        {:ok, datetime, _tz} = DateTime.from_iso8601(prev)
        datetime
      end)

Hi @lost22git , thanks for adding the issue.
This has been a challenge. I released a new version of the library 1.0.1, which supports serialization between structs and binaries. This eventually led to the renaming of interface functions, see the release notes in README.md.
Here is an example how to support DateTime which I believe covers your case.
Please, let me know if you have any other issues!


Merge pull request #2045 from microsoft/4.7-staging

Merge 4.7 staging into master
Fixes #
Proposed Changes

Testing

The target branch is an old working branch belonging to @Zerryth.
@Zerryth if you could delete your old branch that would be great.
Closing this PR.


wrong short-circuiting implementation in bool.and & bool.or

The current implementation of short-circuiting in bool.and and bool.or is wrong. It should consider the ρ object first, not its application to the first element of the x vararg.
This makes the datarization process of the bool.and and bool.or less efficient and conceptually wrong.

@IngeniariusSoftware thanks for finding the bug

Job gh:cqfn/eo#235 is not assigned, can't get performer

You @nlchar/z need to have one of these roles in order to do "Request order start": ARC, PO; I'm sorry to say this, but at the moment you've got no roles in this project

@yegor256/z everybody who has role DEV is banned at #235; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@yegor256/z everybody who has role DEV is banned at #235; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@0crat out

@0crat out (here)

@yegor256 Job gh:cqfn/eo#235 is not assigned, can't get performer


Fixed backspace repeat with TextEdit

Closes https://github.com/emilk/egui/issues/2415.

Good catch, that's quite irritating. I think the arrow keys should probably also heed repeats. I'll merge this and fix that soon after


chore(main): release 10.1.2

:robot: I have created a release beep boop
10.1.2 (2024-11-22)
Bug Fixes

keymaps: completion menu is now <c-_> (24f296b)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v10.1.2
:sunflower:


fix(app): Implement sign out option in the app

https://user-images.githubusercontent.com/7862595/215026207-9e19a981-f452-4f1d-9d09-f06e5ce9da47.mov
Signed-off-by: Talwinder kaur<EMAIL_ADDRESS>

Codecov Report
Base: 90.81% // Head: 90.81% // No change to project coverage :thumbsup:

Coverage data is based on head (ec47b91) compared to base (74d8d52).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #217   +/-   ##
=======================================
  Coverage   90.81%   90.81%
=======================================
  Files          42       42
  Lines        1633     1633
=======================================
  Hits         1483     1483
  Misses        104      104
  Partials       46       46

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


What is file_annotation parameter of chat_postMessage method?

I want to know how file_annotation parameter works with chat_postMessage method. I already search it on issue and the document of the chat_postMessage method, but there was no appropriate information about it.
    def chat_postMessage(
        self,
        *,
        channel: str,
        text: Optional[str] = None,
        ...
        file_annotation: Optional[str] = None, # > How can I use this?
        ...
        **kwargs,
    ) -> SlackResponse:

In my case, I use bolt SDK to handle the message events on socket mode, and want to send message to another channels if the message was sent by a user. However, if the file is attached to the origin message, there was no ways to send the same file to the bot message, which was sent by bot with bolt SDK.
I assume to use the file_annotation parameter in the chat_postMessage method, but there was no information to teach how it works. Isn't is supported yet?

Hi @0417taehyun 👋🏻 Thanks for the question!
I'm done some research and learned that the file_annotation parameter was once part of the chat.PostMessage API. However, it's now been deprecated and removed. So it's no longer available in the public Slack API.
We didn't notice that the parameter was removed, which is why it's still part of the Bolt for Python framework. I'll create an issue for us to remove it in an upcoming release!
Thanks for catching this! 🙇🏻

@mwbrooks
Then how can I connect files with messages with message by using chat_postMessage method?
I want to send message by bot with files. Isn't it possible?

@0417taehyun As long as your bot user has access to a file (e.g., the file is shared in a public channel where your app's bot user is a member), your app can share the file by posting a message with a permalink of the file. More specifically, you can have a file's permalink in a message's top-level text property.


Upgrade to 23.1 with NIC tunables

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[x ] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x ] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
When upgrading from 22.7 to 23.1, I had tunables set for an I350 NIC on all 4 ports. With these in place, the ports do not initialize properly upon reboot after upgrade. After reverting the system to 22.7 and removing the tunables, the system upgraded without issue. I was able to add the flow control tunables back after upgrade, but adding the eee_control back and hitting apply will lock up the interface and ports will not initialize properly upon a reboot. I can log in directly to the box locally and use ifconfig to bring the ports down and up and they work until reboot. It's possible flow control did not need to be removed before upgrade, but I did not have time to test as it is a production system.
dev.igb.0.fc = 0
dev.igb.0.eee_control = 0
To Reproduce
Steps to reproduce the behavior:
Ensure the tunables above are added via the GUI and system is working normally on 22.7
Use the GUI to upgrade from 22.7 to 23.1
Upon reboot you will not have access to any network functionality
Expected behavior
The tunables above worked without issue in 22.7 and it would be expected that you could complete an upgrade to 23.1 without having to first remove those tunables. Until resolved, a warning should be added to the upgrade dialog or tunables should be removed by the upgrade process.
Describe alternatives you considered
Not using tunables
Screenshots
N/A
Relevant log files
N/A
Additional context
N/A
Environment
OPNsense 22.7.11 (amd64, OpenSSL).
Dell Wyse 5070 Extended - Intel J5005
Network Intel® I350 x4

This is mainly a driver issue. Not using the tunables is recommended.

You can still set them manually from /boot/loader.conf.local if you really need to.


Add kubeflow addon

From https://github.com/kubermatic/flowmatic

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

/assign @toschneck

/assign @toschneck


add source files (datagrid.css, datagrid.js)

Do you think its possible to add here source files of css and javascript? It can be useful for my own theme.
Thanks

In master


Remove dependency on bigtable_client.

We needed a function (internal::MakeStatusFromGrpcError) in the bigtable_client library, so I added it as a dependency. We should refactor the function to a new library. Note that the function cannot be moved to google_cloud_cpp_common as it depends on gRPC and we do not want Cloud Storage to depend on gRPC.

WIP branch
The clang-tidy build now passes. The cmake-super build does not pass. For some reason, this integration test is failing:
[ RUN      ] DatabaseAdminClient.DatabaseBasicCRUD
/v/google/cloud/spanner/integration_tests/database_admin_integration_test.cc:47: Failure
Value of: project_id.empty()
  Actual: true
Expected: false
[  FAILED  ] DatabaseAdminClient.DatabaseBasicCRUD (0 ms)
[----------] 1 test from DatabaseAdminClient (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] DatabaseAdminClient.DatabaseBasicCRUD

 1 FAILED TEST

The integration tests for Spanner can only run against production, and they need to have a number of environment variables set for them to run (GOOGLE_CLOUD_PROJECT

Sorry, I don't know what I clicked that closed this issue.

Are you running these manually? Typically the integration tests are disabled except for the integration and coverage builds.

No I am not. I'm running it with ./ci/kokoro/docker/build.sh and the tests fail for the WIP branch, so either I'm doing something wrong or it pulled in a google-cloud-cpp update that breaks something.

My bad. I misunderstood. The cmake-super build does not exclude the integration tests.

We are not running that build in the CI system, we should fix both these issues. Thanks for bringing it up.


chore: remove GLBufferPool

The change for this was introduced for an older version of LWJGL where the application will crash when you attempted to delete an VBO buffer. I think its unnecessary traffic to the gpu since we need to clear the vbo but the pointer is still bound to the buffer.
The game should work the same way when walking around the world and ensure the terasology does not crash since this was the original what the change was intended to fix.
https://github.com/MovingBlocks/Terasology/commit/0e8c8d563d45e42c3a0079b3fa6d0d98f81ecae3

This mainly just affects ChunkMesh.

This currently crashes the game for me when starting CoreGameplay with a RuntimeException: Shader 'engine:post' failed to compile for features '[]'
From the logs:
13:20:41.276 [main] INFO  o.t.e.rendering.opengl.GLSLShader - Dumped Shader Path: logs/shaders/fragment_engine-post_0.glsl
13:20:41.277 [main] WARN  o.t.e.rendering.opengl.GLSLShader - Shader 'engine:post' failed to compile for features '[]'.

Shader Info:
0(466) : error C7616: global function texture3D is removed after version 140
0(468) : warning C7533: global variable gl_FragData is deprecated after version 120

13:20:41.280 [main] INFO  o.t.e.rendering.opengl.GLSLShader - Dumped Shader Path: logs/shaders/fragment_engine-post_0.glsl
13:20:41.282 [main] ERROR o.t.engine.core.modes.StateLoading - Error while loading org.terasology.engine.core.modes.loadProcesses.InitialiseWorld@6238b10a
java.lang.RuntimeException: Shader 'engine:post' failed to compile for features '[]'.

Shader Info:
0(466) : error C7616: global function texture3D is removed after version 140
0(468) : warning C7533: global variable gl_FragData is deprecated after version 120

	at org.terasology.engine.rendering.opengl.GLSLShader.compileShader(GLSLShader.java:350)
	at org.terasology.engine.rendering.opengl.GLSLShader.registerAllShaderPermutations(GLSLShader.java:267)
	at org.terasology.engine.rendering.opengl.GLSLShader.lambda$recompile$1(GLSLShader.java:133)
	at org.terasology.engine.core.subsystem.lwjgl.LwjglGraphicsManager.asynchToDisplayThread(LwjglGraphicsManager.java:130)
	at org.terasology.engine.rendering.opengl.GLSLShader.recompile(GLSLShader.java:132)
	at org.terasology.engine.rendering.ShaderManagerLwjgl.addShaderProgram(ShaderManagerLwjgl.java:155)
	at org.terasology.engine.rendering.ShaderManagerLwjgl.addShaderProgram(ShaderManagerLwjgl.java:147)
	at org.terasology.engine.rendering.ShaderManagerLwjgl.initShaders(ShaderManagerLwjgl.java:51)
	at org.terasology.engine.rendering.world.WorldRendererImpl.initRenderingSupport(WorldRendererImpl.java:178)
	at org.terasology.engine.rendering.world.WorldRendererImpl.<init>(WorldRendererImpl.java:161)
	at org.terasology.engine.core.subsystem.lwjgl.LwjglRenderingSubsystemFactory.createWorldRenderer(LwjglRenderingSubsystemFactory.java:32)
	at org.terasology.engine.core.modes.loadProcesses.InitialiseWorld.step(InitialiseWorld.java:158)
	at org.terasology.engine.core.modes.StateLoading.update(StateLoading.java:240)
	at org.terasology.engine.core.TerasologyEngine.tick(TerasologyEngine.java:495)
	at org.terasology.engine.core.TerasologyEngine.mainLoop(TerasologyEngine.java:456)
	at org.terasology.engine.core.TerasologyEngine.runMain(TerasologyEngine.java:432)
	at org.terasology.engine.core.TerasologyEngine.run(TerasologyEngine.java:398)
	at org.terasology.engine.Terasology.main(Terasology.java:179)

Seems not be related to this change but induced by https://github.com/MovingBlocks/Terasology/pull/4616/
Can retry to test this after that issue is fixed.


Genesis Checkpoint Inspection tool

Description
The tool allows to inspect the genesis checkpoint in an interactive way.
It organizes:

Validators
Sui Distribution.
Objects & Packages

and asserts:

total sui supply
staked sui are in legit validators pools
genesis objects's storage fund is 0

Also added an example to generate genesis checkpoint for testing. See Test Plan.
Test Plan
cargo run --example generate-genesis-checkpoint
cargo run --bin sui genesis-ceremony examine-genesis-checkpoint

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[ ] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes
Adding a tool to allow inspect the genesis checkpoint in an interactive way.

✨ great idea ✨
I'll find some time this afternoon to test it out. Thanks for creating this!


[DOC] Fix backporting issue(s) from PR 2212

What do you want to do?
Fix missed cherrypicks in docs backports. E.g. https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker#related-links

Tell us about your request. Provide a summary of the request and all versions that are affected.
Please review the pages touched by https://github.com/opensearch-project/documentation-website/pull/2212 for all version backports and clean up the cherrypicks. It might have just been with the Docker install page but all of the backports should still be checked (probably good enough to go to each backport PR, do CTRL+F and look for each of the following strings:
This one shows the current change from your branch HEAD:
<<<<<< HEAD

This is the delimiter between your HEAD and MAIN:
=======

This one indicates the state of the doc at the origin:
>>>>>>> main

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.
n/a

Fixed by #2580


Make migration has "DOUBLE PRECISION" resulting in Unexpected token, expected ","

Encountering SyntaxError: Unexpected token, expected "," when trying to make migration. Seems like there's "DOUBLE PRECISION" instead of DOUBLE

not enough info i didnt get the idea where does that happen and how?
send me some example of model making this problem @knivore

Apologies, have attached the model code below:
`/* jshint indent: 2 */
const Sequelize = require('sequelize');
module.exports = (sequelize, DataTypes) => {
return Geolocation.init(sequelize, DataTypes);
}
class Geolocation extends Sequelize.Model {
static init(sequelize, DataTypes) {
super.init({
geo_id: {
autoIncrement: true,
type: DataTypes.BIGINT,
allowNull: false,
primaryKey: true
},
latitude: {
type: DataTypes.DOUBLE,
allowNull: true
},
longitude: {
type: DataTypes.DOUBLE,
allowNull: true
},
created_date: {
type: DataTypes.DATE,
allowNull: false
},
created_by: {
type: DataTypes.STRING(255),
allowNull: false
},
updated_date: {
type: DataTypes.DATE,
allowNull: false
},
updated_by: {
type: DataTypes.STRING(255),
allowNull: false
}
}, {
sequelize,
tableName: 'geolocation',
version: true,
createdAt: 'created_date',
updatedAt: 'updated_date'
});
return Geolocation;
}
}
`
When I try to run the following command - "sequelize-mig migration:make -n initial" for the first time, I get the following in the terminal.
`
SyntaxError: Unexpected token, expected "," (114:40)
112 |         "geolocation",{
113 | "geo_id": { "type": Sequelize.BIGINT, "field":"geo_id", "primaryKey":true, "allowNull":false, "autoIncrement":true },

114 | "latitude": { "type": Sequelize.DOUBLE PRECISION, "field":"latitude", "allowNull":true },
|                                        ^
115 | "longitude": { "type": Sequelize.DOUBLE PRECISION, "field":"longitude", "allowNull":true },
116 | "created_date": { "type": Sequelize.DATE, "field":"created_date", "allowNull":false },
117 | "created_by": { "type": Sequelize.STRING(255), "field":"created_by", "allowNull":false },
`

@knivore ok ok got it and i think i knew where is the problem but i dont know how should we make a solution... I will check it

This should fi the problem #63 please try it so i can close this issue


Command-E "Search Selection" is honored by Atom on OSX

A very common scenario for text handling on OSX is the following:

Select some text in App 1 using Command-E.
Switch to App 2.
Jump to next occurrence of the text using Command-G.

This scenario is very efficient, fast, and does not require any "panel" or "dialog" or any visible UI. It works with almost all apps on OSX, at least apps that are a good OSX citizen. Command-E, Command-Tab (switch), Command-G, and here is your string.
Atom is not a good citizen in this scenario.
It understands Command-E, but only within its own little environement: it does not inherit search strings from other apps, nor does it shares its own with other applications. It really should.
Atom 1.0.7, MacOS 10.10.5 (14F27).

This looks like a duplicate of https://github.com/atom/atom/issues/4650 – please subscribe there for updates.


Remove pre-delete-job from AzureBroker

Description

removed pre-delete-job

Related issue(s)

https://github.com/kyma-project/kyma/issues/1737

I provisioned AzureBroker and deprovsioned it afterwards. Both actions were successful and I confirmed that pre-delete job was not triggered during deprovisioning action
kubectl -n qa get po -w
NAME                                                              READY     STATUS    RESTARTS   AGE
azure-service-broker-redis-0                                      1/1       Running   0          59s
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   1/1       Running   0          59s
azure-service-broker-redis-0   1/1       Terminating   0         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       Error     0         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       Error     1         1m
azure-service-broker-redis-0   0/1       Terminating   0         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       CrashLoopBackOff   1         1m
azure-service-broker-redis-0   0/1       Terminating   0         1m
azure-service-broker-redis-0   0/1       Terminating   0         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       Terminating   1         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       Terminating   1         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       Terminating   1         1m
hb-azure-service-broker-default-82793724-3a8a-11e9-b1-azurlr6zg   0/1       Terminating   1         1m


Consistent boilerplate code in Sandcastle demos

Some Sandcastle examples use outdated or inconsistent boilerplate code. This can be fixed be clicking the Save As button in each demo and replacing the code.
This would make a good beginner issue.
More details and an example from @emackey in https://github.com/AnalyticalGraphicsInc/cesium/pull/6337/

The other changes here are just from re-saving the demo directly from a Sandcastle instance. Would be great to have an intern go through and re-save all the demos at some point, so the boilerplate code all matches what Sandcastle produces (or write a build script to do it).

To avoid manual labor in the future, we should be able to automate this process so that we re-save all Sandcastle examples with a single npm command, just like we do with sortRequires.  Should be possible with either headless Chrome or Electron.

I could probably automate this for you guys. Not sure I totally understand what you mean, though.
Are you saying that there's a mismatch between the code displayed on the sandcastle screen and the code in the actual html files?

@srwiseman Thanks for the offer!  The Sandcastle system has a "Save" functionality built-in, and the way it works is a little unorthodox.  You can write or modify some Sandcastle code directly in your web browser, testing it with F8 and using the integrated Cesium docs etc., and then when you're done, you click "Save As" from the top toolbar.  Sandcastle offers you a button "Save HTML file..." and you end up "downloading" your own code from the client-side-only web page.
But you end up downloading more than just your own code.  Sandcastle attaches some boilerplate HTML and JS, called the Sandcastle "Bucket."  This means that what you've downloaded is not just a loose snipped of JavaScript, it's a full HTML file with your code embedded in a script tag.  The resulting file can be (and should be) saved directly to the Sandcastle gallery folder.  The saved HTML can be viewed as a stand-alone page right from there (although it still needs to be hosted, not just read from the filesystem there).  The next time you run the Cesium build, it will find the new save and sweep it into the Sandcastle Gallery listing, so it can be loaded back into Sandcastle right from the gallery bar.
Here are some known problems:

The Sandcastle "Bucket" does change every so often, and when it does, ideally every gallery demo that exists should be updated to the new bucket code.  Typically the old ones still load fine, so no one bothers to update them.  A manual load-then-save from Sandcastle itself will cause the bucket code to be reset to the current contents.

Sometimes, well-meaning developers actually try to modify or whitespace-format the saved gallery HTML directly, like in some kind of text editor or other horrible app that isn't official Sandcastle.  Savages, I tell you.

So, what's needed is a build script that will run some of the load & save code from Sandcastle.  It would, one at a time, import each gallery demo, use the Sandcastle load code to strip away the bucket code that was saved into the demo itself, and then use Sandcastle save code to add a copy of the current bucket back to the loaded demo.  The result can then be saved back to the gallery in-place, overwriting the previous version.
With this script, one could make a change to the actual bucket, and then the script would rip through the entire collection of Sandcastle demos and update the bucket code in each of them.
Criteria for success is:  I should be able to load any demo from the gallery, click "Save As", and save the demo back over itself, and git should say that nothing has changed.  We worked hard to make Sandcastle capable of round-trips with no changes per trip, but it only works when the thing being loaded has the correct bucket code.

Gotcha! Thanks for the awesome explanation.
Here's my proposed short-term solution:
I'll go through and manually re-save all of the demos so that they have a consistent boilerplate code. I'll then do something like:
git update-index --assume-unchanged Apps/Sandcastle/gallery/*.html
That way the html files stay tracked, but git won't pick up future changes when you do an add/commit. When someone changes the boilerplate code, they have to do:
git update-index --no-assume-unchanged Apps/Sandcastle/gallery/*.html
Then re-save the html files.
Once that short-term solution is in place, I can work on a longer-term solution in the form of a script that automatically deploys the current boilerplate code to all sandcastle html files.
How does that sound?

I don't think we're pressed for a short-term solution here.  The individual demos get updated a lot more frequently than the bucket code, so we don't want developers to mess with git update-index just to update a particular demo.
I think just the long-term solution, in the form of a script that updates the bucket code throughout the gallery, is all that's needed here.  Thanks!

Ah, understood. You're right, If the demos themselves are not relatively static, then git update-index isn't viable. I'll see if I can come up with a script!

I haven't forgotten about this! I'm just very busy with some other projects. I'll do this soon!

No worries @srwiseman! No rush here, whenever you have some time to look at it that will be great =)


Add completion rules.

Two options.

There could be RuleSet properties for what to do on completion of running a ruleset.
There could be dedicated rules for what to do on completion. One or more of these rules could be added.

I tend to prefer the second option.

Note though that these rules should not be run when running rulesets within the web client.


Need distinction between content types in Activity Feed

User story
Different actions in the 'activity feed' tab all have the same colored tag. This makes it hard to scan the feed when looking for specific types of actions e.g. when looking for all payment requests it would be easier to spot them if they were tagged with money icon.
Describe the solution you'd like in Hypha
Differentiate content types in Activity Feed
Priority

Low priority (annoying, would be nice to not see)

Affected roles

Staff

Ideal deadline
January 2022

Can you include a list of the different activity feed content-types in the above?


Extending ontology with sex, ethnicity and age categories

Hi all,
We're in the process of allowing users of our tool to use the Evidence and Conclusion Ontology to annotate pathways with more detailed information. Recently we stumbled on a paper, asking -omics databases for sex, ethnicity and age specific annotations.  We would like to do this in a structured manner through using ECO, however couldn't find terms describing this (see also this issue).
Would it be possible to add this information in your ontology?

Hi Denise,
Thanks for considering ECO to annotate pathways. We will take a look at the paper you mentioned and discuss your request in our next meeting which will be in three weeks (some of us will be out on vacation in coming weeks).
If you want to use just the terms male, female, sex, ethnicity and age, there is Semanticscience Integrated Ontology (SIO), which can be useful.
Will get back to you soon.
Thanks,
Suvvi

Forgot to ask, could you also let us know with an example how you would want to annotate pathways with sex, ethnicity and age details. It will be very helpful for us to understand your needs. Thanks again!

Hi @nsuvarnaiari , thank you for getting back to me so soon and considering my request.
I've checked the SIO, but that's not what we're after. The SIO mentions the term "ethnicity" for example, but does not provide subclasses (Caucasian, African-American etc.). We were thinking in the line of adding cell line/culture evidence:

What we want to do, is add an annotation on interactions within a pathway, stating from what type of evidence this interaction is derived. For example, a publication describes a causal link between gene X and disease Y, where participants in the study were all male (probably no genotyping was performed to determine XY), 90% Caucasian participants (again probably no genotyping was performed, this is often done with a questionnaire), within an age range of 48 +- 8 years (again no reliable measure for this, mostly based on information from participants). As you can see from this example, there is some uncertainty on the study group and factors attributed to this group itself, which could be nice to capture in a pathway model, for example with the existing term ECO:0005613 inference by association of genotype from phenotype. However, if we would want to include the main features of this study group (results are relevant for Caucasian males between 40-56) within the model, we would need additional terminology. I'm not sure which cut-offs to use for the age-groups (some might be relevant for Inherited Metabolic Disorders for example, since these present differently between newborns, kids, and adults); this could be extended with age ranges relevant for certain sexes (e.g. post/pre menopausal for women, pregnancy trimesters, and other hormonal changes greatly influencing biological measurements). For ethnicity, I found this paper interesting: Bhopal RGlossary of terms relating to ethnicity and race: for reflection and debateJournal of Epidemiology & Community Health 2004;. Hope my explanation helps, if not let me know :).

That's an interesting point, but hard to see how it could be easily articulated in ECO. A specialized term could be derived from ECO:0005613 (inference by association of genotype from phenotype), to something like "gender-biased inference by association of genotype from phenotype", but providing more granularity (e.g. the percentage of male/female) is not something that ECO is set up to capture (in the same way it is not meant to capture proportions of reagents in a Western blot).
An interesting derivative here would be to think of biases as an orthogonal entity type, since they are not really "evidence" in a meaningful way, nor assertion types either, but they modulate both. So a subontology could be derived for experimental biases, and terms derived by cross-linking bias, evidence and assertion terms.

@ivanerill : yes, the % is not something we also think we should capture (because it's also depending on amount of study participant, if the findings are statistically significant).... that is a bit too detailed for now. But having a term stating "male sex-biased ..." vs "female sex biased" would already be a great addition. And this being considered evidence yes/no --> I leave that up to your group (since its in the name of the ontology ;) ). However I do feel it is a certain level of evidence, as finding result in cell cultures are considered to be of less evidential value as compared to the same findings in modeling organism, which is then of less evidential value as findings in humans (with again a difference between preclinical and clinical studies).... to make it all even more complex...

Another interesting paper I just came across: The misuses of “biological sex” by Katrina Karkazis, Lancet 2019.

@DeniseSl22
Upon further review of this issue, we don't think the types of terms you are looking for (e.g. "male-sex biased...")  are a good fit for ECO. They don't seem to be evidence for an assertion so much as modifiers to attach to results (which themselves might be evidence for an assertion.)
With regard for terms for sex, gender, ethnicity you might want to consider SNOMED CT for this - which we have used to capture some of this information for other projects.
With regard to your comment - "However I do feel it is a certain level of evidence, as finding result in cell cultures are considered to be of less evidential value as compared to the same findings in modeling organism, which is then of less evidential value as findings in humans..." - we could have (and do have some) terms describing cell culture, animal model, and other system derived evidence, what you seem to be wanting is some kind of confidence assessment, which ECO is not set up to provide.
I'm going to close this issue, but please reopen it if you'd like to discuss this further - we'd be happy to explore further as needed.
Thanks,
Michelle


[Feature]: Add listener on Mini App close

Is your feature request related to a problem? Please describe.
I need to handle the mini app close (send a request to save user progress), but I am struggling to find a way to do it.
Overall I found a workaround:

I make the BackButton visible.
And adding event onClick:

Telegram.WebApp.BackButton.onClick(function() {
    saveUserProgress();
    Telegram.WebApp.close();
});

Describe the solution you'd like
I'd like to have something that natively allows to listen these events:
Telegram.WebApp.onEvent('mini_app_close', function() {
     saveUserProgress();
});

Describe alternatives you've considered
I am mostly considering implementing this using vanilla js telegram-web-app.js library.
But I would also agree with using telegram-apps/sdk or alternative
Additional context
I am using the Unity WebGL application, which loads using vanilla JS,  so to not add complexity I would like to keep my stack the same.

Will be implemented in https://github.com/Telegram-Mini-Apps/telegram-apps/issues/553


CLA class efficient frontier issues

Hi,
Attached is the jupyter notebook highlighting 2 issues:

Plotting of efficient frontier will need weights to be computed via max_sharpe or min_volatility? Setting weights will not work?
CLA class does not work for a portfolio with only 2 assets?

cla_ef_issue.zip

Hi @edgetrader, really appreciate the testing as always!
As for point 1, what exactly are you trying to achieve? Is it that you want to plot the red cross at a different location? The efficient frontier itself does not depend on a particular set of weights.
As for number 2, it is indeed a silly bug. Basically, I used some lazy logic to determine whether the user was passing a list of bounds e.g [(0,1), (0, 0.8), (0.3, 0.6), ...] or passing a single (lower, upper) pair. I will fix this now.
Best,
Robert

Hi,
For point 1, I had just wanted to plot the efficient frontier.  However, the function is complaining that the weights are not yet computed.
Hence, for now, to plot the efficient frontier, I will need to run min_volatility or max_sharpe first.  Setting weights will not work too.

Regarding the weights not being computed, you are right – that's a bug. I've fixed that now.
Setting weights is not meant to change anything – the efficient frontier is not a function of a given set of weights, it is meant to be generated from a large number of weight combinations.


Introduce snap on resize

Is your feature request related to a problem? Please describe.
For esthetic and readability reasons it I'd like to make sure that my diagram is nicely organized. This includes that participants and text annotations that are related have the same size.

This does not only apply to participants but to sub processes and text annotations, too.
Describe the solution you'd like

[x] Snap on resize to similar corners of neighboring elements

Describe alternatives you've considered
We could ditch this and rely on snap to grid #1019 for the work. This, however, assumes that snap to grid is available (and all existing diagrams are snapped).
Additional context
This should fix https://github.com/camunda/camunda-modeler/issues/609
This should fix https://github.com/camunda/camunda-modeler/issues/608

Closed via https://github.com/camunda/camunda-modeler/pull/1404.


StreamKit would sometimes stop reading chat

StreamKit would sometimes stop reading chat. App need to be closed and re-opened for it to start again.
It might be something to do with the audio player being stuck.

This should be fixed or at least reduced on the upcoming 0.8 release.


Update axios in historian and gitrest

Description
Companion to https://github.com/microsoft/FluidFramework/pull/18312 to update axios in historian and gitrest.
Reviewer Guidance
The review process is outlined on this wiki page.

@znewton @pradeepvairamani (I just added you, I know) friendly ping on this one. Nothing blocking, but would like to merge it so the CVE in axios<1.6.0 is fully addressed in the repo.


operator awx-operator (2.0.1)

Thanks for submitting your Operator. Please check the below list before you create your Pull Request.
New Submissions

[ ] Are you familiar with our contribution guidelines?
[ ] Have you packaged and deployed your Operator for Operator Framework?

[ ] Have you tested your Operator in all supported installation modes?
[ ] Have you considered whether you want to use semantic versioning order?
[ ] Is your submission signed?
[ ] Is operator icon set?

Updates to existing Operators

[ ] Did you create a ci.yaml file according to the update instructions?
[ ] Is your new CSV pointing to the previous version with the replaces property if you chose replaces-mode via the updateGraph property in ci.yaml?
[ ] Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?
[ ] Have you tested an update to your Operator when deployed via OLM?
[ ] Is your submission signed?

Your submission should not

[ ] Modify more than one operator
[ ] Modify an Operator you don't own
[ ] Rename an operator - please remove and add with a different name instead
[ ] Modify any files outside the above mentioned folders
[ ] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[ ] Description of the managed Application and where to find more information
[ ] Features and capabilities of your Operator and how to use it
[ ] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[ ] Human readable name and 1-liner description about your Operator
[ ] Valid category name1
[ ] One of the pre-defined capability levels2
[ ] Links to the maintainer, source code and documentation
[ ] Example templates for all Custom Resource Definitions intended to be used
[ ] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file an issue against this repo and explain your need
2 For more information see here

I'm seeing no such host errors in the logs.  Maybe the service that spins up these test openshift envs is temporarily down?
I do a retry.

/retest

/merge

@J0zi @mvalarh do you know why this didn't auto-merge? all of the tests have passed.
After this merges, I will re-base https://github.com/redhat-openshift-ecosystem/community-operators-prod/pull/2631 and it should also pass.

I just did a force push to this branch to re-kick CI the normal way to see if that will make this auto-merge.

/retest

/merge possible

/merge possible


Changed from CO2/person to CO2e/person

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (9b1131f) 82.53% compared to head (5214a34) 80.95%.
Report is 2 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #29      +/-   ##
==========================================
- Coverage   82.53%   80.95%   -1.58%
==========================================
  Files          14       16       +2
  Lines         750      793      +43
==========================================
+ Hits          619      642      +23
- Misses        131      151      +20

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


pkg: validate: validate Username not empty in ImageStatus

Kubernetes rely on that Username field to provide RunAsUser, we need to
validate runtimes correctly return it. We had recently an issue in
CRI-O for that.
@feiskyer @mrunalp PTAL
Signed-off-by: Antonio Murdaca<EMAIL_ADDRESS>

:+1:

@feiskyer @Random-Liu do you know why the test pull the image but show an empty Username field in status?

@runcom 1002 is actually a Uid instead of username, could we change the validation to check Uid instead?

Yup, blame on me

We need a test for Username as well, I'll add one

@feiskyer actually, there's no UID field in image status, that's why I was asking

@runcom See https://github.com/kubernetes/kubernetes/blob/ada9400915e2828fdf1c22f1723d5fcc345180d2/pkg/kubelet/apis/cri/runtime/v1alpha2/api.proto#L972-L978

blame on me again, I've fixed the PR, thanks!

ready now

Can we put the Dockerfile into this repo? I can build one and push to gcr kubernetes test image repo.

Yup, I can do it, can we do the same for the PR I've opened in kubernetes/kubernetes as well? @Random-Liu

@Random-Liu this PR now contains your PR #252 - I've added the Dockerfile under images/image-user/.... When that's merged,I'll rebase this one

#252 is merged. Please update your PR and I'll review. :)

@Random-Liu updated, please push test images on gcr.io or otherwise this will fail :)

@Random-Liu PTAL again please

@runcom Image pushed.

@Random-Liu took care of your comments and this is now green


Update Cargo.toml

Checklist:

[ ] Updated source's version for individual source changes
[ ] Updated all sources' versions for template changes
[ ] Set appropriate nsfw value
[ ] Did not change id even if a source's name or language were changed
[ ] Tested the modifications by running it on the simulator or a test device

should be in #287


Multiprocessing - Possible threaded bug?

Recent changes in python3.9 changed how multiprocessing  works and this seems to prevent its use from within an existing thread. jimi uses this and we should test to see if we are affected?
Redesign of multiprocessing or using async if the issue exists
This does not affect concurrency just threaded

Tested on Python 3.9.2 for windows no issues found