Added ScriptType listen and unlisten API for easier Event subscription handling

Fixes https://github.com/playcanvas/engine/issues/4910
Test project with the debug version of the engine already included. It has buttons to change scene (destroying entities) and to disable/enable entities with scripts that are listening for mouse and touch events.
https://playcanvas.com/project/1078046/overview/listen-api-test-project
PR includes automated tests as well
I confirm I have read the contributing guidelines and signed the Contributor License Agreement.

Would be nice to see this merged, saves a ton of time/nerves handling events :+1:
Also required to rather easily fix Seemore scene change issue as described in https://github.com/playcanvas/engine/issues/5409#issuecomment-1597271712
Good work as always :100:

Anyone got time to review this one for 1.65? :)

Just a comment related to this issue: I've made a small override of on and once and another method of event handler, to return object with off method, and in a large projects with long legacy we have not encountered any issue.
And it made it a breeze to manage events that way.

Just a comment related to this issue: I've made a small override of on and once and another method of event handler, to return object with off method

Interesting, can you share full example? Is it only an empty object otherwise? If so, what do you think about just returning a function which - when called - removes the event?
Just a little bit like React useEffect with a different life-cycle/context:
  useEffect(() => {
    function handleStatusChange(status) {
      setIsOnline(status.isOnline);
    }
    ChatAPI.subscribeToFriendStatus(props.friend.id, handleStatusChange);
    // Specify how to clean up after this effect:
    return function cleanup() {
      ChatAPI.unsubscribeFromFriendStatus(props.friend.id, handleStatusChange);
    };
  });

Example of usage:
let evt = entity.on('trigger', () => { });

// sometime after
evt.off();

@Maksims That is what the Editor does in various places. As an example:
const events = [];

events.push(editor.on('someevent', ...));
events.push(editor.on('otherevent', ...));

events.push(panel.on('clear', () => {
    for (const event of events) {
        event.unbind();
    }
    events.length = 0;
});

@Maksims That is what the Editor does in various places. As an example:
const events = [];

events.push(editor.on('someevent', ...));
events.push(editor.on('otherevent', ...));

events.push(panel.on('clear', () => {
    for (const event of events) {
        event.unbind();
    }
    events.length = 0;
});

Indeed.
As I've designed Editor's events API based on exactly this pain of managing events. So within Editor's context - it proved to be a great and flexible approach.

Just a comment related to this issue: I've made a small override of on and once and another method of event handler, to return object with off method, and in a large projects with long legacy we have not encountered any issue.

With that in mind, do we want to go ahead with this PR or explore other possibilities?
If so, what is being purposed here? Going with @Maksims comment on returning an object with an off method requires the user to still hand this manually in an destroy/disable/enable scenario which I wanted to avoid as that's what people tend to forget/not know how to handle.

With that in mind, do we want to go ahead with this PR or explore other possibilities?

Personally I want this merged and not explore 100 possibilities which eventually just get stuck in a rut. I already use it and then I have to patch every PR using this if I want to load for example Seemore to test bug fixes against non-trivial scenes.
You already made sure that this is extensible in future, so nothing is lost - and if someone feels like extending upon this, that can be better done in individual PR's.

Managing events - is somewhat advance topic. It is required in more complex scenarios, as ~80% of events don't need to be managed.
PlayCanvas does not break existing public APIs, so adding an API - should be very considered.
Also, it is important to avoid "multi-path" in API design - to prevent branching during learning. If it is possible to achieve same thing different ways - it is harder to learn such bloated APIs.
Unfortunately this particular issue - has no easy solution.
Proposed API stands out to me personally as a utility library, rather than a first-class API of PlayCanvas way of doing things.
It also does not handle a case where event is already unbound (using off), but will be still in a list unless it is unbound only using unlisten method. This leads to GC being unable to free things that are reachable from the event function scope in some cases.

It also does not handle a case where event is already unbound (using off), but will be still in a list unless it is unbound only using unlisten method. This leads to GC being unable to free things that are reachable from the event function scope in some cases.

Practically the unlisten method isn't necessary because the point is to call ScriptType#listen once and then being able to forget about it. Because pretty much everyone forgets (or has forgotten) to off their on's.
When the entity (or only the script component) is deleted, the GC gets rid of every ScriptType instance including of course every ScriptType#_listeners.

It also does not handle a case where event is already unbound (using off), but will be still in a list unless it is unbound only using unlisten method. This leads to GC being unable to free things that are reachable from the event function scope in some cases.

Practically the unlisten method isn't necessary because the point is to call ScriptType#listen once and then being able to forget about it. Because pretty much everyone forgets (or has forgotten) to off their on's.
When the entity (or only the script component) is deleted, the GC gets rid of every ScriptType instance including of course ScriptType#_listeners.

This is only one of the cases for cleaning events.
There are many cases where re-binding events is necessary during lifetime of script component. E.g. bind on enable and unbind on disable. If it would be created using listen method and then unbound using off, it will lead to stalled references that will not be GC'ed, and developer will not know about it without knowing the details of the implementation of API.
So either it should handle remove of events that were created using listen or the alternative API design would be preferred.

This API also unbinds the underlying event during disable and enable.
You are right in its a utility function where the developer can write one line to listen for events that automatically unbinds on disable and destroy and rebind on enable.
Throughout the years I've done support and moderated the forums, the amount of boilerplate code needed to manage events and number of topics created around destroying entities and changing scenes because of dangling events is high.
It's been complained about enough by developers from studios and beginners that it warrants a solution like this where its one call, managed and part of the core API. Not an extra script that is added.

This API also unbinds the underlying event during disable and enable.

What if it is not desired (some events should be called during disabled state)? But unbind on destroy is desired?
This covers very specific use case. And does not cover few different scenarios, this will lead to still a need to develop own solutions when this does not cover.
Sorry for being critical here, I just believe there is a simpler building block API can be designed that is more flexible and less narrow, while providing a tools to cover different behaviours and needs of a developer.

Then they can use the 'raw' events that currently exist. If there is a solution that doesn't require the developer to have manually listen for the destroy/enable/disable events, then I be happy to hear it but haven't seen any suggestions that cover that use case.
While this is a specific use case, it's a very common one.

So, an combine API of returning events handle with listen utility, with handling case of removing event handler from listen list - probably would be a better solution here.
Also, provide an optional mechanic to handle enable/disable when using listen.
Such way developers would use common way to create events, as well as have a utility that does not have hidden behaviours that can bite.
Something like:
this.linkEvent(app.on('event', () => { }));

We are linking an event to this script instance.
Also, we can link at any point, and unlink using the same event handle.
So it covers the main purpose of this API. Without introducing new events API (alternative way of subscribing).

If we go this route, we would have to be okay breaking API for the on/off functions on the EventHandler as they currently set up for chaining

Lets say I have a script like this, two on's and two listen's:
HoloVideoPlayer.prototype.initialize = function () {
    this.on("attr:url", this.reload, this);
    this.on('destroy', this.cleanUp, this);
    this.listen(this.app.mouse, "mousemove", this.mousemove, this);
    this.listen(this.app.mouse, "mousedown", this.mousedown, this);
    // allocate all kinds of things down the line...
};

I work to ensure that all of the operations done by initialize are undone by cleanUp. Pretty much allocating resources, textures, mesh instances, meshes, entities, events on self and others:
HoloVideoPlayer.prototype.reload = function () {
    this.cleanUp();
    this.initialize();
}

Now something becomes obvious:
HoloVideoPlayer.prototype.cleanUp = function () {
    this.off();
    this.unlisten(this.app.mouse, "mousemove", this.mousemove, this);
    this.unlisten(this.app.mouse, "mousedown", this.mousedown, this);
    // delete other stuff...
}

EventHandler#off is way easier to work with than ScriptType#unlisten. If we had 20 of each, we would still have only one this.off(), but 20 unlisten's.
Considering that we want to make it a quick/easy interface, maybe we need to improve unlisten?

Here is an example of how I believe such API could work:
Script.prototype.initialize = function () {
    this.bindEvents();
};

// handling swap becomes easier
Script.prototype.swap = function (old) {
    // old script events on script instance are removed, so no need to remove them
    // old script linked events are unsubscribed automatically too
    this.bindEvents();
};

Script.prototype.bindEvents = function() {
    this.on('attr:property', this.onProperty);

    // should trigger only when this script is enabled, and be ignored when this script is disabled
    this.linkEvent(this.app.mouse.on('mousedown', this.onMouseDown, this), true);

    // should trigger while this script is either enabled or disabled
    // also, if `otherEntity` gets destroyed, it will lead to `off` on that event,
    // which should automatically be removed from this script linked events
    this.linkEvent(otherEntity.on('someEvent', () => { }, this));
};

The API would be:
// these are changes to existing API
EventHandler.on(); // returns event handle
EventHandler.once(); // returns event handle

// new APIs
class EventHandle // handle of the event
EventHandle.off(); // unbind an event
EventHandle.silenced // if true, event will not be triggered

ScriptType.linkEvent(eventHandle, [silenceWhenDisable]); // method to link event to script instance life. Optionally can be silenced when script instance is disabled. This method returns eventHandle.
ScriptType.unlinkEvents(); // unbind (off) all linked events, this method is ran automatically when script instance is destroyed

EventHandler.off(); // should ensure linked event handles to script instances to be removed

Such API and implementation, will eliminate hanging references of linked event handler that were off'ed elsewhere, also will handle cases where source of the event removes such event (e.g. entity that we subscribe to is destroyed). Additionally it allows developers to have any complexity for event handle's life. And it does not introduces alternative way of making events, which means integrating this approach to existing code, is as easy as adding ScriptType.linkEvent( in front of existing lines of code.
The only problem with such approach, is that it requires a breaking change - EventHandler to return EventHandle instance instead of scope of which it was called. Which I have not ever seen to be used by other developers. So to handle such transition, we can create a global project setting, that will define the behaviour, and by default for all existing projects or if not defined will default to old way. For new projects will be enabled by default. For existing projects it will be possible to switch in settings.
Potential impact of such breaking change is extremely tiny. As nor the engine uses "chaining" (which is not chaining) in its code base, nor I personally seen any other project used it.

Do we have an example elsewhere in the engine where creation of an object is 'linked' in a similar way that we need to be considerate of pattern wise?

Do we have an example elsewhere in the engine where creation of an object is 'linked' in a similar way that we need to be considerate of pattern wise?

I believe this pattern would be new to the engine API design.

I inspected the idea of adding a EventHandler#destroy method a bit, which would fire a this.fire('destroy') and three classes which extend EventHandler already have it:
https://github.com/playcanvas/engine/blob/c55ab2065c5b127e95cdc8898925ec3edea7a36b/src/platform/sound/manager.js#L117-L125
https://github.com/playcanvas/engine/blob/c55ab2065c5b127e95cdc8898925ec3edea7a36b/src/platform/graphics/graphics-device.js#L359-L369
This one feels odd, kinda semantically wrong - remove implies out of hierarchy, but still "working"? Destroying implies you shouldn't touch any properties any longer, because the object is... well... destroyed:
https://github.com/playcanvas/engine/blob/c55ab2065c5b127e95cdc8898925ec3edea7a36b/src/framework/xr/xr-plane.js#L94-L96
At the same time, EventHandler.destroy could always call this.off()? A quick look over the code base looks like many classes do that already, so we could even reduce the amount of code, pretty much removing duplicate code here and there :thinking:

XrPlane is "remove", because it is not destroyed, as its data is still available to the developer. But it is removed from tracking by the underlying system.

Closing PR as it's not clear of the direction that this should go in. Can alway reopen later down the line.

Closing PR as it's not clear of the direction that this should go in. Can alway reopen later down the line.

I think we can just forget about the references in _listeners, it's not even a bug and rather simple to debug using Chrome's Heap Snapshot tool... and to put it bluntly, no one else cares about that, the engine is full of examples.
I saw for example exactly one case trying to get rid of the this.app reference:
https://github.com/playcanvas/engine/blob/6f82ce824643fa0fae5ec31009c3a50b3594524f/src/framework/script/script-registry.js#L24-L27
No other class would ever consider that.
But forgetting to actually remove event listeners is causing bugs.
When I implemented this myself, I implemented some other stuff and came across this event off'ing bug aswell:
https://github.com/playcanvas/engine/blob/6f82ce824643fa0fae5ec31009c3a50b3594524f/src/framework/components/camera/system.js#L33-L47
https://github.com/playcanvas/engine/blob/6f82ce824643fa0fae5ec31009c3a50b3594524f/src/framework/components/camera/system.js#L170-L174
Two little code snippets of the same file, anyone wants to guess the bug here?
And this example brings me to another point: instead of in ScriptType, we should add this in EventHandler, because "events are easy to forget for everyone".
EventHandler#unlisten:
    /**
     * Removes a listener for an event on an EventHandler that was added by {@link ScriptType#listen}.
     *
     * @param {EventHandler} [eventHandler] - EventHandler that was originally listened to.
     * @param {string} [name] - Name of the event that was originally listened to.
     * @param {HandleEventCallback} [callback] - Function that was used as the callback when the event was originally
     * listened to.
     * @param {object} [scope] - Object that was used as the scope when the event was originally listened to
     * @example
     * this.unlisten(this.app.mouse, 'mousemove', this.onMouseMove, this);
     * @see {@link ScriptType#listen} to listen to events on an EventHandler.
     */
    unlisten(eventHandler, name, callback, scope) {
        for (let i = 0; i < this._listeners.length; ++i) {
          const l = this._listeners[i];
          if (
            (!eventHandler || l.eventHandler === eventHandler) &&
            (!name         || l.name         === name        ) &&
            (!callback     || l.callback     === callback    ) &&
            (!scope        || l.scope        === scope       )
          ) {
            this._listeners.splice(i, 1);
            l.eventHandler.off(l.name, l.callback, l.scope);
          }
        }
    }

With this change, we can simply call this.unlisten() to remove all remote events, which is the equivalent for the common this.off() for local events.
Anyway, that's my current direction I'm thinking in, would be happy to see more discussion around this... IMO this is rather important to have and I couldn't find another idea as appealing as listen/unlisten for non-this event handling.

I think I found a legit example for listenEvenWhenDisabled:
https://github.com/playcanvas/engine/blob/e5c6903d1645d1875d5e047843c3caef3a643d1c/src/scene/shader-lib/program-library.js#L33-L56
Not directly applicable here (since it's not a script), but when it comes to resource handling, we woud want to destroy shaders or other resources.
In the script I also don't see a off call, might be an event leak?
And as usual, we don't remove device from the instance, "leaking" just as much as listen/unlisten would do - so I don't see a strong argument against it.

Made a PR to push "chaining" away, that will lead to better APIs and events management in the future.


Codebase cleanup and assembly refactoring

EDIT: This PR needs to be tested before merging it.
I just cleaned up parts of the codebase, hoping to make an improvement to overall readability. I hope it's not too biased with my own tastes. If so, please feel free to reject any change and I will rewrite the patch cherry-picking only the commits you deem acceptable.
Summary

On e74a4ec:

Added VMX_EXIT_ prefix to VMX exit reasons enum. Goal is to avoid polluting the global namespace with ambiguous names such as "CPUID_INSTRUCTION" that are confusing without context (e.g. it could refer to the "cpuid" opcode, or the "cpuid" exit reason, or anything else involving that instruction).
Removed _INSTRUCTION suffix from VMX exit reasons enum. Goal is to keep consistency with exit_* handlers (which don't have such such suffix). It's also less verbose this way.
Added a couple of missing exit reasons (57-64), for the sake of completeness.
Renamed exceptions to VECTOR_XX, to remain consistent with the Intel SDM.
Fixed few English grammar issues.

On 6f693d4:

Updated the exit handler list and config_t to use designated initializers
Fixed small typos.

On 10597c3:

Replaced platform-specific *.asm files with general NASM files.
Using NASM macros to make most assembly functions cross-platform (32-bit/64-bit), and to automate the generation of certain functions, via function_* macros.
Replaced the vmx_error_t with vmx_result_t enum type with VMX_SUCCEED (0), VMX_FAIL_INVALID (1), VMX_FAIL_VALID (2) values. Transforming EFLAGS into these values happens in the assembly functions via the macro vmx_check. The new enum type vmx_error_t contains the error codes for VMX_FAIL_VALID. This change required renaming a couple of variables.
Replaced vmx_error_t cpu_vmentry_failed(...) to void cpu_vmentry_failed(...) since the return value is #useless and always ignored by the caller.
Renamed load_kernel_ldt to set_kernel_ldt to remain consistent with IDT/GDT setters.
Moved few VMX-specific function declarations from asm.h to vmx.h.

As always, based on the changes you accept, I'll rewrite the Git history to avoid polluting the repository.

I hope it's not too biased with my own tastes.

Not at all! On the contrary, I'm really grateful that you took your time to improve the code quality of HAXM. I can see that you spot something wrong and actually went on to fix it--that's really admirable.
I've reviewed the first three patches and I think they are good to go. Please allow me more time on the last one, which is especially exciting because it is another big step toward a unified assembler for all supported platforms.

Sorry for the issues, I was waiting for the new PR to be merged to rebase and review+test the code. There's still an annoying issue under 32-bit configuration regarding calling conventions. I'll fix it and let you know when everything's ready.

Sure.

There's still an annoying issue under 32-bit configuration regarding calling conventions.

I see. I managed to dig out your old comment about the previous Win32 linking issue. Not sure if it's related to this one:

The problem is that haxlib.vcxproj and IntelHaxm.vcxproj are stdcall by default, and tests.vcxproj is cdecl by default.
As mentioned earlier, the solution would be to explicitly stating the correct calling convention. However, changing tests.vcxproj to stdcall causes an issue with Google Test, and changing haxlib/IntelHaxm to cdecl causes an issue with MASM code.
The only solution left was explicitly stating that these two emulator functions are stdcall, but that's a suboptimal fix. I'm still thinking how to properly deal with this situation.

I also noticed that the new ia32.asm uses the fastcall convention for Win32, whereas emulate_ops.asm (fastop_dispatch()) uses cdecl. How do you choose the calling convention?

I wonder if we should split this PR into multiple ones, so we can merge commits early, as soon as they become ready. Right now we have a small PR #69, which is ready to merge but has some conflicts with this PR, at least with commit b67284a (vmx_ops.asm). If we can merge part of this PR first, I think you'll have an easier time rebasing the rest of it.

Sorry, last week and part of this week I was/will be mostly AFK for work reasons. Don't worry about merging other PRs first, I don't mind rebasing. Also, I expect to finish this PR next weekend. Thanks for your other comments, I will try to update my patch as soon as possible :)

Thanks for merging the recent PRs, they had really useful stuff! Rebasing this patch was really simple. Sorry for the delays, I don't like keeping PRs idling, but I got overwhelmed by real-life work.
I've finished all the features I wanted to implement so far. There's more to come, but that's for another PR since this one is already big enough. Since your last review, I've fixed the issues brought up by your comments and additionally:

On 881c2f9:

Implemented asm_vmxrun and asm_invept.

On d555074:

Replaced the vmx_error_t with vmx_result_t enum type with VMX_SUCCEED (0), VMX_FAIL_INVALID (1), VMX_FAIL_VALID (2) values. Transforming flags into these values occurs in the assembly functions via the macro vmx_check. Consequently, vmx_error_t contains the error codes for VMX_FAIL_VALID. This change required renaming a couple of variables.
Replaced vmx_error_t cpu_vmentry_failed(...) to void cpu_vmentry_failed(...) since the return value is useless and always ignored by the callet.
Renamed load_kernel_ldt to set_kernel_ldt to remain consistent with IDT/GDT setters.
Moved few VMX-specific function declarations from asm.h to vmx.h.

Once you approve the patch, I'll rewrite the history into three commits

Yes, that's a good plan. Please feel free to incorporate the miscellaneous changes (spelling, coding style, etc.) into these three at your convenience.

All done, patch history is clean now. An overview of every change is updated at https://github.com/intel/haxm/pull/64#issue-195689921.

Thanks! Clang was able to spot a few more issues when we tried to build this PR on Mac (sorry I still don't know how to comment on unchanged code):

paniced => panicked

https://github.com/AlexAltea/haxm/blob/10597c3a90c0d41b26d23bcd17a69c83633c7d9e/include/darwin/hax_mac.h#L200

vmx_error_t => vmx_result_t (I'm actually very surprised that VC++ didn't catch this--we should check our compiler settings)

Multiple occurrences in core/include/cpu.h and core/cpu.c

%lx => %x (because vmx_result_t is essentially an int):

https://github.com/AlexAltea/haxm/blob/10597c3a90c0d41b26d23bcd17a69c83633c7d9e/core/cpu.c#L453
https://github.com/AlexAltea/haxm/blob/10597c3a90c0d41b26d23bcd17a69c83633c7d9e/core/cpu.c#L553
https://github.com/AlexAltea/haxm/blob/10597c3a90c0d41b26d23bcd17a69c83633c7d9e/core/cpu.c#L561
https://github.com/AlexAltea/haxm/blob/10597c3a90c0d41b26d23bcd17a69c83633c7d9e/core/cpu.c#L699

In addition, my colleague @wcwang ran a few tests on Windows, and some of them resulted in a host freeze. He can tell you more about that.

When I launched Debian in QEMU, which guest image is debian_squeeze_i386_desktop.qcow2, the host would be frozen once the launch command was entered. No QEMU window popped up or logs output in command prompt. Similar results occurred on Android virtual device launch. You may try to reproduce this issue on Windows.

I was able to fix the compiler errors and add the two new .asm files to Xcode.

I looked into a few of them, and they are actually C wrappers around asm_xxx, but only exist for Windows (windows/{i386, amd64}/wrapper.c). We could add a asm/wrapper.c for Mac too, but I guess it would be better to move these wrappers to core/ so as to further reduce code duplication?

Thanks for the report, testing and for the macOS-specific patch. I'll update my PR, and try to find and fix the bugs found by @wcwang.

I'm actually very surprised that VC++ didn't catch this--we should check our compiler settings

Seems like a compiler bug in MSVC, see minimal example at: https://godbolt.org/g/KDjcGJ. Other compilers (gcc/clang/icc) either throw warnings/errors for this. Even MSVC throws an error when building as C++. I've reported the issue, but let's see if they even bother with it, as they tend to heavily prioritize C++ over C.

I guess it would be better to move these wrappers to core/ so as to further reduce code duplication?

Will do so.

Seems like a compiler bug in MSVC, [...] I've reported the issue, but let's see if they even bother with it, as they tend to heavily prioritize C++ over C.

Thanks for the information! Before they fix this bug, we'll just continue to rely on Clang to detect coding errors.
Regarding the host freeze issue, I wonder how you plan to debug it and narrow it down. We'd like to help, but all we are sure about now is that the bug lies in the third/last commit, which is very big. To make debugging easier, could you consider splitting it further, e.g.:

Replacing the old vmcs.{asm, c} with vmx_ops.asm, vmx_error_t with vmx_result_t.

With some more effort, this one could be split into 2-3 commits, beginning with the introduction of vmx_result_t and its integration with the old assembly code, before plugging in vmx_ops.asm.

Replacing the old ia32.{asm, c}, segments.{asm, c} with ia32.asm, moving wrapper.c into core/.

Or if you have a better idea, let us know :-)

BTW:

You can reproduce the host freeze bug on Windows regardless of the guest OS used, or even with no guest OS (i.e. the boot ROM test): qemu-system-x86_64 -accel hax
My colleague @junxiaoc managed to get rid of the host freeze issue by restoring the old vmcs.asm, but the guest still wouldn't boot up. This means we probably have bugs in both vmx_ops.asm and the new ia32.asm. If they are in separate patches, we can deal with them one at a time.

We just tried to only pick up ia32.asm(original ia32.asm and segments.asm) related change, qemu could boot up with guest OS.
So freeze issue might be related with vmx_ops or vmx_result_t related changes.

Thanks a lot for tracking the bug down, I've fixed and the last commit and pushed it back.

We'd like to help, but all we are sure about now is that the bug lies in the third/last commit, which is very big. To make debugging easier, could you consider splitting it further [...]

Sorry I couldn't made it in time, I will be mostly AFK until tomorrow.

All fixed.

I'd actually prefer calling the macOS kernel function, even though it would create platform-dependent code paths

I fully agree with your reasoning. I've reverted that change and used #ifndef __MACH__ to remove the generic assembly-based function from macOS.

Thanks! The new code looks all good, and works well on Windows. However, most Mac tests are broken, resulting in a host kernel panic. We're still trying to root cause the issue. For now, let me post the macOS kernel panic logs, in case you can spot an obvious problem.
So far we've seen two kinds of macOS kernel panic:
a)
*** Panic Report ***
panic(cpu 3 caller 0xffffff8008987d3d): Kernel trap at 0x0000000000000189, type 14=page fault, registers:
CR0: 0x000000008001003b, CR2: 0x0000000000000189, CR3: 0x0000000040a1d055, CR4: 0x00000000003627e0
RAX: 0x0000000000000000, RBX: 0xffffff8044613800, RCX: 0x0000000000000000, RDX: 0x0000000000000663
RSP: 0xffffff913b4839b0, RBP: 0x0000000000000000, RSI: 0x0000000000000018, RDI: 0x0000000000000000
R8:  0x0000000000000000, R9:  0x0000000000000000, R10: 0x0000000000000000, R11: 0x0000000000000000
R12: 0x000000002c100801, R13: 0xffffff913b483a40, R14: 0xffffff802bdd9c00, R15: 0x0000000000000000
RFL: 0x0000000000010086, RIP: 0x0000000000000189, CS:  0x0000000000000008, SS:  0x0000000000000010
Fault CR2: 0x0000000000000189, Error code: 0x0000000000000010, Fault CPU: 0x3, PL: 0, VF: 0

Backtrace (CPU 3), Frame : Return Address
0xffffff913b483460 : 0xffffff800886d166
0xffffff913b4834b0 : 0xffffff8008995824
0xffffff913b4834f0 : 0xffffff8008987b10
0xffffff913b483570 : 0xffffff800881f180
0xffffff913b483590 : 0xffffff800886cbdc
0xffffff913b4836c0 : 0xffffff800886c99c
0xffffff913b483720 : 0xffffff8008987d3d
0xffffff913b4838a0 : 0xffffff800881f180
0xffffff913b4838c0 : 0x189

BSD process name corresponding to current thread: qemu-system-x86_

Mac OS version:
17F77

Kernel version:
Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64
Kernel UUID: CC1699EE-71B5-3AF1-AAA3-6DBD16222B92
Kernel slide:     0x0000000008600000
Kernel text base: 0xffffff8008800000
__HIB  text base: 0xffffff8008700000
System model name: MacBookPro14,2 (Mac-CAD6701F7CEA0921)

System uptime in nanoseconds:<PHONE_NUMBER>9496
last loaded kext at<PHONE_NUMBER>8307: com.intel.kext.intelhaxm	1 (addr 0xffffff7f8e030000, size 229376)
last unloaded kext at<PHONE_NUMBER>1238: com.intel.kext.intelhaxm	7.2.0 (addr 0xffffff7f8e030000, size 126976)

b)
*** Panic Report ***
mp_kdp_enter(): 2, 3, 4 TIMED-OUT WAITING FOR NMI-ACK, PROCEEDING
panic(cpu 2 caller 0xffffff801fb87d3d): Kernel trap at 0xffffff801faba11c, type 14=page fault, registers:
CR0: 0x000000008001003b, CR2: 0x0000000000000016, CR3: 0x00000001b34d410b, CR4: 0x00000000003627e0
RAX: 0x0000000000000000, RBX: 0xffffff802029c670, RCX: 0xffffff9156a83900, RDX: 0x0000000000000004
RSP: 0xffffff9156a83820, RBP: 0xffffff9156a838f0, RSI: 0x0000000000000004, RDI: 0xffffff802029c670
R8:  0x0000000000000ff4, R9:  0x0000000000000000, R10: 0xffffc90000022000, R11: 0xffffffff81141e00
R12: 0x0000000000000000, R13: 0x0000000000000000, R14: 0x0000000000000004, R15: 0x0000000000000000
RFL: 0x0000000000010046, RIP: 0xffffff801faba11c, CS:  0x0000000000000008, SS:  0x0000000000000010
Fault CR2: 0x0000000000000016, Error code: 0x0000000000000000, Fault CPU: 0x2, PL: 0, VF: 0

Backtrace (CPU 2), Frame : Return Address
0xffffff9156a832d0 : 0xffffff801fa6d166
0xffffff9156a83320 : 0xffffff801fb95824
0xffffff9156a83360 : 0xffffff801fb87b10
0xffffff9156a833e0 : 0xffffff801fa1f180
0xffffff9156a83400 : 0xffffff801fa6cbdc
0xffffff9156a83530 : 0xffffff801fa6c99c
0xffffff9156a83590 : 0xffffff801fb87d3d
0xffffff9156a83710 : 0xffffff801fa1f180
0xffffff9156a83730 : 0xffffff801faba11c
0xffffff9156a838f0 : 0xffffff801fa78e9a
0xffffff9156a83940 : 0xffffff802005e456
0xffffff9156a83960 : 0xffffff7fa55407ba
No mapping exists for frame pointer
Backtrace terminated-invalid frame pointer 0xffff88005fc03f98
      Kernel Extensions in backtrace:
         com.intel.kext.intelhaxm(1.0)[C0020148-1BDC-365A-8793-D893A66DA7CB]@0xffffff7fa553c000->0xffffff7fa5573fff

BSD process name corresponding to current thread: qemu-system-x86_

Mac OS version:
17F77

Kernel version:
Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64
Kernel UUID: CC1699EE-71B5-3AF1-AAA3-6DBD16222B92
Kernel slide:     0x000000001f800000
Kernel text base: 0xffffff801fa00000
__HIB  text base: 0xffffff801f900000
System model name: MacBookPro14,2 (Mac-CAD6701F7CEA0921)

System uptime in nanoseconds:<PHONE_NUMBER>83
last loaded kext at<PHONE_NUMBER>54: com.intel.kext.intelhaxm	1 (addr 0xffffff7fa553c000, size 229376)
last unloaded kext at<PHONE_NUMBER>79: com.intel.kext.intelhaxm	7.2.0 (addr 0xffffff7fa553c000, size 126976)

The manual address/offset patching method works, so this vmx_get_rip issue is probably the last bug we need to fix :-)
I inspected vmx_ops.o with otool and nm, but did not spot anything abnormal...
If we compare the original (vmcs.{asm, c}) and current implementations of this assembly routine, there are two main differences:
i. The old code uses the lea instruction instead of mov.
ii. The old code declares exit_entry as a global label (MASM EXIT_ENTRY:: rather than EXIT_ENTRY:, GAS EXIT_ENTRY: rather than something like 1b:).
Changing exit_entry to a global label doesn't seem to help. I'm now trying to figure out how to make lea work...

All done!


Change weights and add defaults

Anything missing?

I think it is ok for now. Maybe we find something else, but we can make another PR if it occurs.


Error uploading files using v3.3.2 - Multipart parsing error: Multipart: Final boundary missing.

With the last update (and removal of old images - why, btw?) to 3.3.2 we can not upload files anymore properly without triggering the following error:
[error] 24#24: *338 [client <IP_ADDRESS>] ModSecurity: Access denied with code 400 (phase 2). Matched "Operator `Eq' with parameter `0' against variable `REQBODY_ERROR' (Value: `1' ) [file "/etc/modsecurity.d/modsecurity.conf"] [line "60"] [id "200002"] [rev ""] [msg "Failed to parse request body."] [data "Multipart parsing error: Multipart: Final boundary missing."] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "<IP_ADDRESS>"]

Is this some kind of a regression or did something change with the rules that would cause this behaviour change?
Using the 3.3.2-nginx image.

Hi we experienced the same issues,
and regarding the audit logging it has to do with the :
SecDataDir ${MODSEC_DATA_DIR}
SecTmpDir ${MODSEC_TMP_DIR}
SecUploadDir ${MODSEC_UPLOAD_DIR}

The Owasp is trying to write this to non existing folders by default it works for us by overwriting these env vars to /tmp

+1.

Hi we experienced the same issues, and regarding the audit logging it has to do with the :
SecDataDir ${MODSEC_DATA_DIR}
SecTmpDir ${MODSEC_TMP_DIR}
SecUploadDir ${MODSEC_UPLOAD_DIR}

The Owasp is trying to write this to non existing folders by default it works for us by overwriting these env vars to /tmp

Did you just override them in the custom config section in the nginx.conf file? Or rather in the env variables of the dockerfile?

Changing the values in the Dockerfile worked for me. Ideally, this should still be fixed in the repo itself tho.

Closing.


Initial pass at adding a toggle to show mode changes as a line chart

Closes #640.
This is the first pass at showing mode changes as a line chart similar to how Raven works.
There's more work to go on this issue, right now horizontal rules aren't working properly and the correct way to implement this is to generate a new type of Y axes which we're not currently doing.

Looks like src/utilities/view.test.ts > generateDefaultView will need to be updated to reflect the new view schema?

It's calling the correct createTimelineXRangeLayer function which includes the new view property... I'll keep investigating.

One thing that would be really useful here would be to draw the horizontal ticks as is done for line layers. Think that could go in this PR or want to split it into a new ticket?

One thing that would be really useful here would be to draw the horizontal ticks as is done for line layers. Think that could go in this PR or want to split it into a new ticket?

I was looking into this a little yesterday afternoon, I can see if it's viable quickly for this PR otherwise I'll add it to the next ticket.


Call to a member function getBotUsername() on null in …vendor/longman/telegram-bot/src/Request.php

Required Information

Operating system: CentOS
PHP version: 7.0
PHP Telegram Bot version: 0.52
Using MySQL database: no
Update Method: Webhook
Self-signed certificate: no

Expected behaviour
I have a Telegram bot, which is located on domain1.com. Telegram bot tested and works fine. Also I have a system (build with Laravel Lumen), which is located on domain2.com and it is try to send Telegram messages without user interaction as described in wiki (https://github.com/php-telegram-bot/core/wiki/How-to-send-message-to-channel-user-without-user-interaction). Both domains has valid SSL certificates.
So, to teach system send Telegram messages, I'am using a tutorial from wiki too (https://github.com/php-telegram-bot/core/wiki/Install-on-Laravel). Messages is send by Laravel jobs, so my job is
<?php

namespace App\Jobs;
use Longman\TelegramBot\Request;
use Longman\TelegramBot\Telegram;
use Longman\TelegramBot\Exception\TelegramException;

class ProcessTelegramMessageJob extends Job
{
    const TELEGRAM_API_KEY = 'BOTKEY';
    const TELEGRAM_BOT_NAME = 'BOTNAME';

    protected $telegram;
    protected $data;

    public function __construct($data)
    {
        $this->data = $data;

        try {

            $this->telegram = new Telegram(self::TELEGRAM_API_KEY, self::TELEGRAM_BOT_NAME);
            $this->telegram->handle();

        } catch (TelegramException $e) {}
    }

    public function handle()
    {
        $data = [

            'chat_id' => $this->data['user_id'],
            'text' => $this->data['message'],
        ];

        $result = Request::sendMessage($data);
    }
}

Actual behaviour
When Laravel queue try to dispatch a job ProcessTelegramMessageJob it is process infinitelly. In log I see a message Call to a member function getBotUsername() on null in …vendor/longman/telegram-bot/src/Request.php. How can I fix this problem? Thank you in advance!

I'am trying to initialize Request by telegram object manually, but code $result = Request::initialize($this->telegram)->sendMessage($data); gives an error Call to a member function sendMessage() on null

Hi @esomkin
When you create a new Telegram object, the Request class gets initialised already, no need to manually do that. You get the error because Request::initialize(...) does not return anything.
Try just using Request::sendMessage(...) after you've created your Telegram object!
Also, in your initial message, I think you want $this->telegram->handle(); to be $this->handle();, right?
Or does the webhook call link into your job?

Hi @noplanman
Thank you for answer, I'am using Request::sendMessage(...) too, as you can see in my question. Handle method of a Job class start automatically, $this->telegram->handle() is removed, but result is the same - error in console.

@noplanman It is strange but, message send successful only if I place send code into constructor.

I think you didn't read my comment properly...
You want $this->telegram->handle(); to be $this->handle();.
Otherwise the method obviously isn't called.
Are you new to PHP programming?

@noplanman I read your comment, and I understand it good. Handle method of a Laravel job is called automatically by Laravel, that is why I place sendMessage here. $this->telegram->handle(); I add as in hook.php, it can be removed and it does not matter. The question is why it works only if place creating new Telegram object and Request::messageSend in __construct together?

@noplanman Please check discussion about this problem https://stackoverflow.com/questions/48702352/call-to-a-member-function-getbotusername-on-null-in-vendor-longman-telegram/48702699?noredirect=1#48702699

Riiight @ Laravel calling handle() method 👍
You could also just call Request::initialize($this->telegram); first in your handle() method, before sending the message.
Then it should work.

Suggestion: Could we add an exception throw when Request is not initialized @noplanman ? 🤔

@noplanman Ok, thank you for your answers!

So we can close off here then, right?

@noplanman I think, yes


M07-LAB01-Ex02: Playbook is gone

Contact Details
<EMAIL_ADDRESS>What happened?
Exercise : 02
Task: 02
Step: 09
Description of issue: Playbook is gone already
But we can access directly this link.
https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/Teams/Playbooks/Post-Message-Teams
Lab
Lab 07 Exercise 02 Create a Playbook
Relevant screenshots
paste here 😉
Do you want to help us? 👏

[ ] Create a Pull Requestt by following this project's Contribution Guide

I found the Teams folder was moved under Sentinel/Solutions due to issue/PR #6029.

Fixed.


Public Accessory meta changes

@alex-emerich @imchairmanm  - I checked and it looks like these changes were not adopted. Are you seeing the same on your end?

hi @mkrasne2 . It looks like we approved, but there was an additional merge step that you needed to do, "squash and merge". It looks like now there are conflicts because of file changes. It should give you instructions to resolve and then merge if you press "open this in Github Desktop". I believe you are doing this through the app.

I was able to resolve these locally, but unable to push the changes to this PR, so #332 should contain all of the correct changes.


docs: intended README links

Related Issues
Navigating links README to atomic-cli and atomic-server.
closes #
Checklist

[ ] Add changelog entry linking to issue, describe API changes
[ ] Add or update tests if needed
[ ] Update docs if needed

Thanks!


Dynamic NTK RoPe scaling support

transformers merged https://github.com/huggingface/transformers/pull/24653 only dynamic NTK RoPe scaling (NTKv2), would be nice to have it in exllama.

Yes, this is planned. Stay tuned.


Add Installation scripts for complete beginners and/or non-technical users (Implementing #83)

Brief Summary:
This pull request adds two installation scripts (install_puterbot.sh and install_puterbot.ps1) that allow complete beginners and/or non-technical users to easily download and install OpenAdapt on their machines.
Approach:

My approach was to create two separate installation scripts for Windows and Mac users respectively (install_puterbot.sh and install_puterbot.ps1).
The scripts check if Git and Python 3.10 are already installed on the system and if not, they proceed to download and install them. After that, the OpenAdapt repository is cloned, a Python virtual environment is created, and the required packages and libraries are installed.
Finally, the database migration is run, and the test suite is executed.

Assumptions:

The user has internet access.
The user has administrative privileges on their system.
The user is running the appropriate script on either macOS or Windows Operating System.

Integration of external libraries:

I did not integrate any external libraries. This is because, the scripts only require the built-in shell commands of macOS and PowerShell.
However, the user needs to use the curl command to download the installation script for macOS and the iwr (Invoke-WebRequest) command to download the installation script for Windows from the OpenAdaptAI Github repository.

Other Comments:

For new users who might not be familiar with the technical specifics of installing OpenAdapt, I think these scripts will assist lower the entry barrier.

Thank you @KrishPatel13 !
It looks like this PR includes changes related to https://github.com/MLDSAI/puterbot/issues/83 (install scripts) and https://github.com/MLDSAI/puterbot/issues/47 (scrubbing).
Can you please split these into separate PRs?

Done!, For this PR, I have removed the unnecessary commits.

Thank you! Can you please push your scrubbing code to a separate PR?

It us not completed yet. That is why I did not make a seprate PR yet. Is that fine ?

@KrishPatel13 that's fine, make a separate PR when it's ready. I think this one can be closed.

I have also made a PR on here: https://github.com/OpenAdaptAI/install/pull/3

Thanks for asking @KrishPatel13 ! What do you think about creating a new PR with the install repo as a git submodule? Once that's done we can link back to this and close it.

@KrishPatel13 can you please provide an update on the status of this PR?

@KrishPatel13 can you please prioritize addressing my comments on this PR so that we can get it merged?

Sure!
I just finished the Scrubbing issue and I am lkdging a new PR for it.
As soon as it is done, I will finish up this Issue.
The reason It was on a hault was that there is a change that needs to be made for install scripts.
Now we are using the spacy model
en_core_web_trf.
Anyways, since the scrubbing is now completed. I am going to finish this uo today! :)

Addressed in https://github.com/MLDSAI/OpenAdapt/issues/212


Cannot find Tensor() function to update Model to Keras2 API

The code gets warning
UserWarning: Update your Model call to the Keras 2 API: Model(inputs=Tensor("in..., outputs=Tensor("de...)
But the Tensor function cannot be found in the Keras models.

from keras.layers import Input, Dense
from keras.models import Model
from keras.datasets import mnist
import numpy as np
num_hidden = 16
num_col_img = 28
num_row_img = 28
num_all_img = num_col_img*num_row_img
input_img = Input(shape=(num_all_img, ))
op_encode = Dense(num_hidden, activation='relu')(input_img)
op_decode = Dense(num_all_img, activation='sigmoid')(op_encode)
autoencoder = Model(input=input_img, output=op_decode)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

Tensor is the type. The warning is about the argument renaming between Keras2 and Keras1
so :
autoencoder = Model(input=input_img, output=op_decode)
become
autoencoder = Model(inputs=input_img, outputs=op_decode)

@Dref360 Great, now I understand. Sorry for not noticing the difference between inputs and input. I thought I need to change the type. Thank you very much.

Yeah, I was confused as well.


fix: gatsby - moved build message to footer

alert in the header is way too distracting + causes the mobile menu buggy
theres no need to have it that big, lets put it into the footer :)

put gatsby/config/gatsby-node.ts back please

Heh, i removed it kinda by mistake :D But we have to adjust it... every single time theres some type changed, the whole compilation stops and the only way is to rebuild the whole FE 😢

I think we should remove it… except the export { createPages } from './create-pages'; part. 😁

Well, I use DISABLE_TYPE_CHECK env variable, but there's a catch – there's a === with a string so you have to define the env variable as a string not a boolean.

ah, thats why it failed :D okay, ill try to adjust it, we will keep the lint up to CI

You were editing the footer so I'd like to encourage you to do these two things:

remove the footer__brand class
change col-12 col-sm-6 to just col-12

There's a card in Trello for it, I hope.
Or we can do it some other day. 🤷♂ It's up to you.


Remove unnecessary type parameters

There is no reason to have type parameters that we bound to concrete types:
julia> isconcretetype(UInt8)
true

julia> isconcretetype(Int)
true

This is similar to using where {FT <: Float64}, we might as well not have the type parameters, and not waste compiling a generic method with type assertions.
This PR removes these unnecessary type parameters. xref:

FWIW, this bit also a bit odd: where I<:Int64, since this means I is either Union{} or Int, but seems like it would always just be an Int?

Unfortunately, this is a breaking change, but fixing upstream breakages with this should be easy.

bors r+


Error when downloading any video from youtube ( HTTPSConnectionPool(host='rr7---sn-pouxgn8t51-jx3z.googlevideo.com', port=443): Read timed out..)

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
An error occurred when downloading any video from youtube with any parameters.
HTTPSConnectionPool(host='rr7---sn-pouxgn8t51-jx3z.googlevideo.com ', port=443): Read timed out..
VPN is not enabled, firewall is not blocking.
The command is simple, for example:
yt-dlp.exe -fb https://youtu.be/b8fFRX0T38M?si=P6ODhoZ0xYjf9SUM
Any ideas?
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp.exe -f b -vU https://youtu.be/b8fFRX0T38M?si=P6ODhoZ0xYjf9SUM
[debug] Command-line config: ['-f', 'b', '-vU', 'https://youtu.be/b8fFRX0T38M?si=P6ODhoZ0xYjf9SUM']
[debug] Encodings: locale cp1251, fs utf-8, pref cp1251, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [f0993391e] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: none
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1829 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[youtube] Extracting URL: https://youtu.be/b8fFRX0T38M?si=P6ODhoZ0xYjf9SUM
[youtube] b8fFRX0T38M: Downloading webpage
WARNING: [youtube] Webpage contains broken formats (poToken experiment detected). Ignoring initial player response
[youtube] b8fFRX0T38M: Downloading ios player API JSON
[youtube] b8fFRX0T38M: Downloading player 0e90208b
[youtube] b8fFRX0T38M: Downloading web player API JSON
[debug] Loading youtube-nsig.0e90208b from cache
[debug] [youtube] Decrypted nsig SuzNm2Yw7mLk-dnI => EbgyQgCKo3S1Sw
[debug] Loading youtube-nsig.0e90208b from cache
[debug] [youtube] Decrypted nsig TbXos2Ja_Jm14gYr => WY5tiFN-rdzhMw
[youtube] b8fFRX0T38M: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[info] b8fFRX0T38M: Downloading 1 format(s): 18
[debug] Invoking http downloader on "https://rr7---sn-pouxgn8t51-jx3z.googlevideo.com/videoplayback?expire=1722331308&ei=TFyoZqusIKjxzN0P4KSD-Ac&ip=<IP_ADDRESS>&id=o-AEUdpy3KJMwqj4kq1oBO6lF95da0zJ8wP5I6VHGAWXhi&itag=18&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=RN&mm=31%2C29&mn=sn-pouxgn8t51-jx3z%2Csn-n8v7knel&ms=au%2Crdu&mv=m&mvi=7&pcm2cms=yes&pl=18&initcwndbps=328750&bui=AXc671KAGD8QnAZIoLv_ilGiRpXGtL_5BOyEHkTGDlMYPP5ST8_30wymjvyGRT0p8mU-qebggm03BTmA&spc=NO7bAQOhhkeB030NCGwHhHi5mdKZEXiu7WnCaDjlVsCWd23-HSoVZ8S5-aL-4Vg&vprv=1&svpuc=1&mime=video%2Fmp4&ns=RvhqXg5rDmLKHUCrrW8L_74Q&rqh=1&gir=yes&clen=74069110&ratebypass=yes&dur=3043.532&lmt=1704987221750088&mt=1722309431&fvip=7&c=WEB&sefc=1&txp=5438434&n=EbgyQgCKo3S1Sw&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=AJfQdSswRQIgHOKTTOPavd-haZdHyUMYU1lGFaMoVF5qwtUJlVKTYRACIQDAYa1D7BzsIP4vtgxi6wys-8yVn3WwxD4UyA6_PBJIfA%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&lsig=AGtxev0wRQIgFvjx7c8ZrUB1pv-jARpP98pZYz81s8FIEk-oAq2l_r0CIQD3mSt8YW0aGr9X4wGdluDaLhenJ8UutdwLf9DE_kfb0A%3D%3D"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: .NET 8 Web API & Entity Framework 🚀 Full CRUD Course (with Code-First Migrations & SQL Server) [b8fFRX0T38M].mp4
[download]   1.4% of   70.64MiB at    1.06MiB/s ETA 01:05[download] Got error: HTTPSConnectionPool(host='rr7---sn-pouxgn8t51-jx3z.googlevideo.com', port=443): Read timed out.. Retrying (1/10)...
[download] Destination: .NET 8 Web API & Entity Framework 🚀 Full CRUD Course (with Code-First Migrations & SQL Server) [b8fFRX0T38M].mp4
[download]   2.6% of   70.64MiB at   33.37KiB/s ETA 35:12[download] Got error: HTTPSConnectionPool(host='rr7---sn-pouxgn8t51-jx3z.googlevideo.com', port=443): Read timed out.. Retrying (2/10)...
[download] Destination: .NET 8 Web API & Entity Framework 🚀 Full CRUD Course (with Code-First Migrations & SQL Server) [b8fFRX0T38M].mp4
[download]   4.0% of   70.64MiB at   31.97MiB/s ETA 00:02

The question can be closed
The problem is with Google's non-functioning caching servers in Russia.
It works via VPN.

Ref: #10443


Fix 1805

Adding the missing documentation and fixing a link that we missed on previous reviews.
Previous PRs: https://github.com/biocore/qiita/pull/2087 & https://github.com/biocore/qiita/pull/2062.
@josenavas & @wasade, wanna do another review?
BTW, the new link is shown as:

and the documentations looks like:

Coverage increased (+0.08%) to 91.665% when pulling 8ef50c1d618d3e5819457b941c531934ccf1c091 on antgonza:fix-1805 into 27950465f44a5e945bf4001ddb40bde805789f8d on biocore:master.

Coverage increased (+0.07%) to 91.655% when pulling e6a65a3e245be14ea33975bfe660e360b268f6d8 on antgonza:fix-1805 into 27950465f44a5e945bf4001ddb40bde805789f8d on biocore:master.

@antgonza and I pair-programmed this - do we think that we need another reviewer or with @wasade 's review is enough?

I think if @wasade or someone else reviews we should be OK.


Add macro description; Add macro hover which shows description

This pull-request adds a description field to the macro definitions in the definition files.
Then it also adds a 'hover' for macros (currently only for SugarCube2), which will display any existing description (which supports Markdown).
This would allow embedding documentation/quick-examples and, well, descriptions of what the macro is meant for. Useful for not having to look at the main documentation, and also useful when you're working on a file with others who may add their own widgets/macros.
I initially ran into some issues with determining the macro-name under the cursor, but used collect to acquire a list of macros and finding the entry that intersected the current position.
I imagine a more efficient implementation of collect for finding a single macro that intersects with a position could be implemented, but I don't think it is a major issue.
Example in a json file:
"linkreplace": {
	"name": "linkreplace",
	"container": true,
	"description": "Creates a link that when clicked replaces itself with its content.\n\nExample: `<<linkreplace \"Say hello\">>Hi!<</linkreplace>>`"
},

Example in yaml file
    thing:
      name: thing
      description: |-
        The universe revolves around this macro.

        Example: `<<thing>><</thing>>`
      container: true

Picture:

Through some effort, it will only show the description on hover of certain parts. Ex: on <<linkreplace "Testing">>Text<</linkreplace>>, it would only show hover on: <<linkreplace, >>, <</linkreplace>>. My initial version had it also show hover when you put your mouse over "Testing" but I felt that was undesirable and so have the more complex implementation that this pull-request is.
Note: This Pull Request does not add any documentation to the inbuilt SugarCube2 macros, but simple examples could be written up easily. (There's also the option of embedding the official docs as the descriptions).
Note2: It would be nice if this made the macro highlighted when it is showing the description. Currently vscode automatically highlights a 'word', but that does not match macros (ex: the 'word' for vscode stops at - characters, which are not uncommon in macro names).
Suggestions (These are some suggestions for the project, but perhaps should have been opened as issues):
- Provide a testing project, either in the main repo or another repo, that exhibits a good amount of behavior. This would allow easier testing.
- Some of the functions (ex: sugarcube-2's 'macroList') would benefit from explicit return types as typescript fails to guess them.
- There are some minor formatting weirdness. Could find some formatter for VSCode and just let it auto-format?
- It would be nice if the extension was also uploaded to: https://github.com/eclipse/openvsx which is a vendor-neutral VSCode marketplace. (Arch-Linux's vscode uses that).

Yeah, the collect function is sort of expensive, making a list of everything even when only one is needed. It was needed for the error diagnosis, specifically illegal opening/closing tags, but now that I think about it, having a way to get only what's needed would be nice. I'll look into it.
For documentation, I'll probably write bits for the core library myself, and then link to the relevant part of the official docs. Would be neater to keep things concise on hover.
For highlighting the opening and closing tags of container macros, I used decorators. So they could come in handy for this too. I'll see about hooking one in.
Lastly, I read over the changes and saw checks for maliciously named macros. I don't think they're needed, as macros can only ever start with letters and have dashes/underscores in them. And that is checked for by the macroRegex.

As for project suggestions:

That's a very good idea. I have one that I test on, so I'll clean that up and include it in the project.
I think they don't have return types to keep TS from guessing them and throwing errors... I know... Was a total noob with TS when I started with the project, getting rid of errors was priority... Will look into it.
I just use VSC's built in formatter for JS... I did install prettier at some point and removed it because I didn't like it. Probably remains of that period... Will look into it.
Will do.

Thank you for all of this! I'd at some point added JSDocs support to add suggestions and documentation while typing macros. Because that messes with VSC's inbuilt word suggestions (which is sorta nice for writers) I scrapped it. But hover tips are a nice alternative! It is very much appreciated.

Yeah, the collect function is sort of expensive, making a list of everything even when only one is needed. It was needed for the error diagnosis, specifically illegal opening/closing tags, but now that I think about it, having a way to get only what's needed would be nice. I'll look into it.
For documentation, I'll probably write bits for the core library myself, and then link to the relevant part of the official docs. Would be neater to keep things concise on hover.
For highlighting the opening and closing tags of container macros, I used decorators. So they could come in handy for this too. I'll see about hooking one in.
Lastly, I read over the changes and saw checks for maliciously named macros. I don't think they're needed, as macros can only ever start with letters and have dashes/underscores in them. And that is checked for by the macroRegex.

As for project suggestions:

That's a very good idea. I have one that I test on, so I'll clean that up and include it in the project.
I think they don't have return types to keep TS from guessing them and throwing errors... I know... Was a total noob with TS when I started with the project, getting rid of errors was priority... Will look into it.
I just use VSC's built in formatter for JS... I did install prettier at some point and removed it because I didn't like it. Probably remains of that period... Will look into it.
Will do.

Thank you for all of this! I'd at some point added JSDocs support to add suggestions and documentation while typing macros. Because that messes with VSC's inbuilt word suggestions (which is sorta nice for writers) I scrapped it. But hover tips are a nice alternative! It is very much appreciated.

Pushed the change that removes the Object.hasOwnProperty checks.

I just use VSC's built in formatter for JS... I did install prettier at some point and removed it because I didn't like it. Probably remains of that period... Will look into it.

I noticed this because the formatter I have for javascript/typescript (which I think is the built-in-one?) tried removing some extra spaces or reformatting lines in a few places. (Ex: extension.ts line 185 has two spaces before the curly bracket). It doesn't really matter much but it did confuse me for a moment when I noticed change-highlights in the editor.

Pushed the change that removes the Object.hasOwnProperty checks.

I just use VSC's built in formatter for JS... I did install prettier at some point and removed it because I didn't like it. Probably remains of that period... Will look into it.

I noticed this because the formatter I have for javascript/typescript (which I think is the built-in-one?) tried removing some extra spaces or reformatting lines in a few places. (Ex: extension.ts line 185 has two spaces before the curly bracket). It doesn't really matter much but it did confuse me for a moment when I noticed change-highlights in the editor.

Alrighty. I've merged the changes. Will publish a new version after finishing up with all the descriptions, and other changes here and there in the README and stuff.

Alrighty. I've merged the changes. Will publish a new version after finishing up with all the descriptions, and other changes here and there in the README and stuff.


Revert "MWPW-137282 - [Action Items] Centered in the grid/scroller"

Reverts adobecom/milo#2237
As per messages in milo-stage there is a request to revert the PR
Please see details for issue related to the PR in comment section of https://github.com/adobecom/milo/pull/2284
(Jira- MWPW-148020)

Verified the revert PR raised :
https://main--cc--adobecom.hlx.page/products/illustrator?milolibs=revert-2237-sartxi-MWPW-137282-action-centering

https://main--cc--adobecom.hlx.page/products/premiere?milolibs=revert-2237-sartxi-MWPW-137282-action-centering

https://stage--cc--adobecom.hlx.page/creativecloud/tools/graphic-design-software?milolibs=revert-2237-sartxi-MWPW-137282-action-centering


Flux combined maintainers list for CNCF

Minor housekeeping. This has already been addressed in the Flux Due Diligence incubation document. However, it is not reflected in CNCF project maintainers CSV because that requires a single file to draw from. And that is the primary file used by CNCF to address all project maintainer requests.
Example<EMAIL_ADDRESS>mailing list. While various maintainers were added to this mailing list informally, officially it should correspond to people listed on this centralized CNCF CSV. Ref CNCF Service Desk ticket CNCFSD-995.
PS in this file I used YAML so we can script out formatting this in whatever other ways we need to. Also using YAML anchors and pointers to keep the repeating list simple, reflecting what we did in a narrative style within the various fluxcd GitHub org MAINTAINER files. But if you have another preferred format, I'm not too picky about it as long as it's easy to maintain and serves the Flux project's needs 😄

In the interest of moving this forward, maybe can you file an issue with a check list of the next steps? Happy to merge this afterwards. :-)

@dholbach I agree with all of the above.
Added checklist of follow-up items in https://github.com/fluxcd/community/issues/135


Missing the step that performs the apply torizon packages on the Dockerfiles of the Rust templates

The following apply torizon packages step are missing on the Dockerfiles of the Rust templates:
# your regular RUN statements here
# Install required packages
RUN apt-get -q -y update && \
    apt-get -q -y install \
# DO NOT REMOVE THIS LABEL: this is used for VS Code automation
    # __torizon_packages_prod_start__
    # __torizon_packages_prod_end__
# DO NOT REMOVE THIS LABEL: this is used for VS Code automation
    && apt-get clean && apt-get autoremove && \
    rm -rf /var/lib/apt/lists/*
    ```

Duplicates: #121


Deligate equality to other if conversion to Unit fails.

This fixes issue #149 by returning NotImplemented if other is not or cannot be converted to a Unit.  By doing this Unit allows other to make the decision or in the case that neither can determine equality Python will fallback on identity.

Coverage decreased (-0.1%) to 88.489% when pulling 63bbb1a8a6da137e39719c56f94bb034bae6d662 on mrshannon:fix_issue_149 into 812009c9d13b25b11cad6118e05f4c0f27d4ac7d on SciTools:master.

Coverage decreased (-0.1%) to 88.489% when pulling 63bbb1a8a6da137e39719c56f94bb034bae6d662 on mrshannon:fix_issue_149 into 812009c9d13b25b11cad6118e05f4c0f27d4ac7d on SciTools:master.

Coverage decreased (-0.1%) to 88.489% when pulling 63bbb1a8a6da137e39719c56f94bb034bae6d662 on mrshannon:fix_issue_149 into 812009c9d13b25b11cad6118e05f4c0f27d4ac7d on SciTools:master.

@mrshannon Sorry for the very tardy response... I clearly missed this, but let's bank it now (finally) and include it in a patch release

@mrshannon LGTM :+1:


Update docs to reflect recent changes on directory structure

Signed-off-by: Hai Yan<EMAIL_ADDRESS>Description
This PR updates documentation for recent directory structure change, including the new script to run data prepper and the changes to configuration file location.
Issues Resolved
Resolves #1785
Check List

[ ] New functionality includes testing.
[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #1786 (f70ef79) into main (95115a6) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1786      +/-   ##
============================================
- Coverage     93.97%   93.94%   -0.03%
  Complexity     1454     1454
============================================
  Files           188      188
  Lines          4263     4263
  Branches        344      344
============================================
- Hits           4006     4005       -1
- Misses          176      177       +1
  Partials         81       81

Impacted Files
Coverage Δ

.../org/opensearch/dataprepper/pipeline/Pipeline.java
88.40% <0.00%> (-4.35%)
:arrow_down:

...rwarder/discovery/AwsCloudMapPeerListProvider.java
95.06% <0.00%> (+2.46%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Pre-RFC: Save and load state for Iterator

On my project, I recently found a problem that I can't implement a comfortable interface that do operation on Iterator directly using existing API.
/* not work because iterator state is changed after each operation */
pub fn do_operateion<'a, T, U>(input_iter: &mut T, output_iter: &mut U)
where
    T: Iterator<Item = &'a f32>,
    U: Iterator<Item = &'a mut f32>,
{
    let temp = input_iter.skip(2).map(...).sum();
    output_iter.step_by(2).zip(input_iter).for_each(|a, b| *a * temp);
    output_iter.skip(1).step_by(2).zip(input_iter).for_each(|a, b| *a * temp * 123.0);
}

Hence the idea of adding StateSavableIterator trait:
pub trait StateSavableIterator: Iterator {
    type IteratorState;

    fn save(&self) -> IteratorState;
    fn load(&mut self, state: &IteratorState) -> Self;
}

So that I can:
pub fn do_operateion<'a, T, U>(input_iter: &mut T, output_iter: &mut U)
where
    T: StateSavableIterator<Item = &'a f32>,
    U: StateSavableIterator<Item = &'a mut f32>,
{
    let input_init = input_iter.save();
    let out_init = output_iter.save();
    let temp = input_iter.load(input_init).skip(2).map(...).sum();
    output_iter.load(out_init).step_by(2).zip(input_iter.load(input_init)).for_each(|a, b| *a * temp);
    output_iter.load(out_init).skip(1).step_by(2).zip(input_iter.load(input_init)).for_each(|a, b| *a * temp * 123.0);
}

We don't have to pass &slice anymore! This trait benefits to writing more generic and readable API.

slice::IterMut<T> could not implement StateSavableIterator for exactly the same reason.

To give a more helpful answer, if output_iter in your code is indeed slice::IterMut, consider passing the actual slice instead and calling .iter_mut() on it twice. This will let the borrow checker make sure that the lifetimes of the two iterators do not overlap. (Until NLL lands you’ll also likely need to add a {…} anonymous block to limit the scope of the first iterator.)

I don't see any case where Clone isn't viable but a load-and-save state is both viable and sound.

Closing as per the above comments, this sort of interface does not seem to solve the problem.


Callbacks for custom behaviour during simulation

My usage of the library (e.g. zero-order optimisation) requires the definition of a score associated with a simulation.
This method defines a score as a function that takes the simulation time and the state of the dynamics (data) at a given frame to return the score of the model for that frame. As of now the score is independent of previous values, but this can be implemented easily.
The definition of such through callbacks which are called pre, during and post simulation has the following advantages:

Customisation of the functions that define a score, given the state of the model.
Storage of variables during simulation
Adaptability for different types of callbacks that a user might be interested in.

Current usage:
def score_func(t: float, data: mujoco.MjData) -> float:
        return 0.1

scb = ScoreCallback(score_func) # score function = f: (t, data) -> score
tcb = TrackerCallback(["qpos"], False) # (list of tracked variables, print variables on step)

# Define simulator and set initial position
mujoco_instance = MujocoSimulator()
mujoco_instance.load_model(
    robot_model,
    s=[-0.55, 0],
    xyz_rpy=np.asarray([0, 0, initial_height, 0.5, 0, 0]),
    floor_opts={}
)
s, ds, tau = mujoco_instance.get_state()
t = mujoco_instance.get_simulation_time()
H_b = mujoco_instance.get_base()
w_b = mujoco_instance.get_base_velocity()
mujoco_instance.set_visualize_robot_flag(False)
t = 0
scb.on_simulation_start()
tcb.on_simulation_start()
while t < 4:
  # Reading robot state from simulator
  s, ds, tau = mujoco_instance.get_state()

  t = mujoco_instance.get_simulation_time()

  # Step the simulator
  mujoco_instance.step()
  scb.on_simulation_step(t, mujoco_instance.data)
  tcb.on_simulation_step(t, mujoco_instance.data)

scb.on_simulation_end()
tcb.on_simulation_end()

Ideally the list of callbacks should be passed to the constructor of MujocoSimulator and the callback functions should be called internally (e.g. run method)

New EarlyStoppingCallback to finish a simulation early.
Usage:
sim = MujocoSimulator()
sim.load_model(...)

cb = EarlyStoppingCallback(lambda t, data: t > 3.0)
sim.run(6.0, callbacks=[cb])

An idea for a useful callback is the ContactCallback which theoretically captures all the contacts that happened at a given frame. This should either store, alert or return all the contacts of interest during the simulation.
Input: The user passes a list of objects (str) for which he is interested of gathering contact information of (for example: ("plane", "model.right_foot" or "plane", "model.base"). If nothing is passed, all contacts can be considered by default (memory expensive) or no contacts.
Store: the class contains a list of ContactInfo for the tracked objects.
Alert: the callback raises a log, event or whatever whenever a contact between the objects has taken place, if an alert flag is active.
My doubt is whether to keep this logic inside a callback or implement it internally to the mujocoSimulator class.

Currently facing a problem on TrackerCallback where tracked variables are not stored correctly. I'd like to recall that this callback takes care of storing the values of variables of interest through a simulation, in the following way:

Constructor: asks for a list of identifiers of the variables to be tracked during simulation

During simulation:

On start: reset the lists for storage
On step: retrieve the value of the variable and append it to the respective list
On end: -

Return/expose the tracked variables' values

The problem is how the values of the variables are stored.  We'd obviously expect the values of a variable to be stored in the following way:
t0: [v0]
t1: [v0, v1]
...
tN: [v0, v1, ..., vN]

but for some reason, they are stored in the following way:
t0: [v0]
t1: [v1, v1]
...
tN: [vN, vN, ..., vN]

and so the returned values are all the same.
Below is an example:
Output

qpos: [0.         0.         1.49996076 1.         0.         0.
 0.        ]
qpos: [0.         0.         1.49988228 1.         0.         0.
 0.        ]
qpos: [0.         0.         1.49976456 1.         0.         0.
 0.        ]
qpos: [0.        0.        1.4996076 1.        0.        0.        0.       ]
qpos: [0.        0.        1.4994114 1.        0.        0.        0.       ]
qpos: [0.         0.         1.49917596 1.         0.         0.
 0.        ]
qpos: [0.         0.         1.49890128 1.         0.         0.
 0.        ]
qpos: [0.         0.         1.49858736 1.         0.         0.
 0.        ]
qpos: [0.        0.        1.4982342 1.        0.        0.        0.       ]
qpos: [0.        0.        1.4978418 1.        0.        0.        0.       ]
qpos: [0.         0.         1.49741016 1.         0.         0.
 0.        ]

Time: [0.002, 0.004, 0.006, 0.008, 0.01, 0.012, 0.014, 0.016, 0.018000000000000002, 0.020000000000000004, 0.022000000000000006]

Q position: [1.49741016 1.49741016 1.49741016 1.49741016 1.49741016 1.49741016
 1.49741016 1.49741016 1.49741016 1.49741016 1.49741016]

The lines starting with qpos are the runtime positions which are displayed correctly and different at each frame.
The final list is the history of the values, they are all the same and equal to the last value

:heavy_check_mark:  Solved!
Thanks to @fils99's enormous help, I found out that the values were stored as references inside the TrackerCallback and were modified at the beginning of each iteration when mujoco.mj_step was called. Adding a simple copy.deepcopy when appending the values in the callback class fixed the issue.


Remove v0.6 endpoints, added v0.8 endpoint

some lint issues :)

fyi I just rebased

Should we also update the readme / docs? Otherwise lgtm

Should we also update the readme / docs? Otherwise lgtm

let's do it in separate PR (starkware haven't merged the updates yet)


Google Source: Process Typing Indicator Events via Websocket

Is your feature request related to a problem? Please describe.
We should process typing indicator events so relevant conversational interfaces such as our Airy Inbox UI are able to indicate to logged in users that relevant contact is still typing.
Describe the solution you'd like
Relevant events are being received in the following format:
{
  "agent": "brands/BRAND_ID/agents/AGENT_ID",
  "requestId": "REQUEST_ID",
  "conversationId": "CONVERSATION_ID",
  "customAgentId": "CUSTOM_AGENT_ID",
  "sendTime": "SEND_TIME",
  "userStatus": {
    "isTyping": true | false,
    "createTime": "CREATION_TIME",
  },
}

Source: https://developers.google.com/business-communications/business-messages/guides/build/events#receive
We should process and send them via the web socket but not via the webhook to avoid noise as mostly synchronous clients benefit from such real time events (compare with Slack RTM API vs. Slack Events API).
Describe alternatives you've considered
Additional context

Same comment as for the Facebook ticket: https://github.com/airyhq/airy/issues/2122#issuecomment-877141751

Copying @chrismatix comment from https://github.com/airyhq/airy/issues/2122:
Processing typing indicators on the websocket/webhook

In case the agent was typing, the contact would not be present in payload.metadata, but maybe that's rather weak and puts more pressure on the clients. Or would you rather suggest to move the typed_at property into the contact object in that case?

The entire metadata object is always present. And yes, putting typed_at on the contact field makes sense 👍

Also from the perspective of the clients it might be really hard to screen the payload object for keys to determine whether typed_at is defined or not. Looking at the way you implemented tags and channel updates having their own types as documented here (https://airy.co/docs/core/api/websocket#channel) it might justify having a separate event type instead of just sending a metadata event?

Introducing a separate event
In our taxonomy conversation metadata is any optional data associated with a conversation. That's not true for tags and channels, which is why we made them separate entities. Therefore I am against introducing a separate entity for that.
Clients' ease of parsing
In Javascript languages, this is fairly easy to do, but I see how this can become cumbersome at some point. There are ways of making this look and behave a bit more like the Google userStatus webhook event, but I would discuss and plan them once we get there.

@chrismatix - I agree with the general logic you apply regarding the question whether it should be a separate event, however I think we cannot fully ignore that while tags and channels are separate entities and don't relate to a conversation in the way conversation metadata does, we are looking at events here that we receive in a specific format as outlined above.
Maybe we should try to transform the relevant event only as much as needed and stay agnostic about the event as much as possible.
Given that we market Airy as an open source conversational platform with unified messaging APIs, I think what we cannot do is simply swallow events that people might want to receive had they decided to not use Airy in between the source and their app, but connected their app directly to the source.
Obviously, the more we transform the event instead of just passing it through, the easier it gets to work with our solution and to remove logic from the clients (e.g. to differentiate between Google typing indicators and typing indicators from other sources such as Slack, etc.). However, the more we transform these events and create our own metadata updates from them, the more we need to support on our end.
So in this case, I would support your perspective and implement it as suggested, but we need to be careful not to prevent more complex use cases like the Slack typing indicators where there is added complexity because of several participants in a channel and in the end it's relevant to know who is typing.
Even within the Airy inbox app, we could benefit from showing User Agent A that User Agent B is typing, so we should design the metadata event in a way that the clients have a chance to understand what is happening and who is doing what.

Sorry for leaving this hanging for so long @steffh. Fully agree with your points and would thus layout the conversation metadata like so:
{
  // ... conversation
  metadata: {
    typing: {
      "user/conversation-id": "last typing time"
    }
  }
}

conversation-id here indicates that the contact was typing last, whereas user id indicates the same for the agent. This design would also work for multi-participant sources in the future. Let me know what you think.

@steffh in an effort to close this: Do you agree with the above design?


[BugFix] fix sorting of overflowed nullable column

What type of PR is this：

[x] bug
[ ] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #6746
Problem Summary(Required) ：

Problem:

The sorting of nullable column is divided into two steps: at first partition the null elements and non-null element, then sort them individually
A basic assumption of sorting null element is, the datum of null element should be the same, so the sorting procedure does not need to differentiate null element and not-null element
But the exception is, the NullableColumn could not ensure this property: value of null element are same
For some special expression, it could generate a NullableColumn whose value are not default value

Solution:

NullableColumn::update_null_as_default before sorting a nullable column
Column::update_default would update the null element to default value if necessary

@Mergifyio backport branch-2.3


feat(pkger): add parser support for notification rules

[x] CHANGELOG.md updated with a link to the PR (not the Issue)
[x] Well-formatted commit messages
[x] Rebased/mergeable
[x] Tests pass

tests failing upstream b/c of go vendor issue. This is not related to this PR work. Should not have any affect on any upstream stuffs, no new vendored code here, nor any changes to existing stuffs being used by idpe


Translate April 17 report into German

Hello @nextstrain/german!
The situation report from 2020-04-17  is ready to be translated to German. Sorry it’s a little late this week. The report focuses on the USA. We do have a significant linguistic diversity here, but I’ll let you decide if you want to translate it. In general, I do think it’s worth understanding how the virus spreads in other regions of the world, but I’m not doing the hard work of translation.
Branch to translate on: sit-rep_de
File to translate: https://github.com/nextstrain/translations/blob/sit-rep_de/narratives/translations_ncov_sit-rep_de_2020-04-17.md You can translate directly in this file, but please remember to push often as other people may be working on the translation!
Link to preview: https://nextstrain.org/community/narratives/nextstrain/translations@sit-rep_de/ncov/sit-rep/de/2020-04-17
Twitter post to translate:
“This week’s #COVID19 situation report covers the outbreak in the United States. It is now available in German. Thank you @, @, and @ for the rapid translation.”
Logistics
Please update the translations board as you work on the translation: https://github.com/orgs/nextstrain/projects/7
Please assign yourself to the translation if you’re working on it this week.
When a translation is ready for publication -- at minimum meaning two people have worked on it and both agree that it’s ready -- please create a pull request to merge to master and tag me and the issue in it!
Discussion board for your team is here: https://github.com/orgs/nextstrain/teams/german
Standard rules for translation:
Note that the file to translate is in Markdown. In general, it is important not to change any formatting or links. (Although it is okay to update Wikipedia or CDC links to the appropriate language version.) Comments to help with translation are noted in comment symbols like <! -- Comment -->.
Please add your name under translators: with a tab, dash, and new line, formatted just like authors are. Please also add a link to your professional page (can use Twitter) under translatorLinks:.
Note: You may know of other great resources available in German; please don’t add them to the situation report. We need to keep content the same across all translations, so we can be confident in the accuracy of what we publish. Thanks for understanding.
At minimum, I require two sets of eyes on every translation; one of those people should have a background in phylogenetics. Although if you have more, that’s fine too.
Feel free to use a live, online editor like hackmd (sign in with GitHub) to translate together.
As always a hearty thank you! I am really enjoying working with you all, even if it’s just digitally. Take care!

Balancing time it would take to translate this and value (as the report is on the US) I won't be working on it this week. Looking forward to a new report on Europe though and happy to help with that!

@astridgall I would have time to translate it :) Would you have time proofreading it? :)

@Hirthe yes, I can proffread layer today. Let me know when you are finished

@astridgall I am done :) Have a nice Sunday evening :)

@Hirthe Thanks! Enjoy the rest of your Sunday too! I'll proofread now

@cassiawag This is ready. Here the tweet:
Diese Woche behandelt der #COVID19 Lagebericht den Ausbruch in den Vereinigten Staaten. Er ist nun in Deutsch verfügbar. Vielen Dank an @AstridGAGall und Elisabeth Hirth für die schnelle Übersetzung.

Published and tweeted: https://nextstrain.org/narratives/ncov/sit-rep/de/2020-04-17
Thank you both so much!!


Change the charting backend from bqplot to plotly

Colab already has plotly pre-installed. We can probably switch the charting backend from bqplot to plotly. The plotly FigureWidget provides similar functionality to bqplot.
https://plotly.com/python/figurewidget-app/

It seems FigureWidget is not supported by Colab. We can show a non-widget version of a plotly chart in a widgets.Output (interactive), but we can't observe interactions like a widget to, e.g., click a point in a scatter plot and have it return the feature observation info.
Here is a Colab notebook for testing: https://colab.research.google.com/drive/1X2zePfh-zO6yIWKcwOraYKrTaQe30Pzf?usp=sharing
Colab supports custom widgets, but I can't get FigureWidget to work. I'll register a bug with the Colab team.

The chart module has been added in #1855. Closing this issue now.