Crash in checkpointed C++ Pennant on multiple nodes

I'm seeing a crash in C++ Pennant when I run it on multiple nodes on Sapling. The same code appears to pass when I run it through MPICH in a Docker container. This is with Legion_NETWORKS=mpi.
pennant: /scratch2/eslaught/resilience/legion/runtime/realm/transfer/transfer.cc:161: bool Realm::TransferIteratorBase<N, T>::done() [with int N = 1; T = long long int]: Assertion `inst_impl->metadata.is_valid()' failed.

Backtrace:
(gdb) bt
#0  0x00007f2778f2223f in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=req@entry=0x55fc183c3920, rem=rem@entry=0x55fc183c3920)
    at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:78
#1  0x00007f2778f27ec7 in __GI___nanosleep (requested_time=requested_time@entry=0x55fc183c3920, remaining=remaining@entry=0x55fc183c3920) at nanosleep.c:27
#2  0x00007f2778f27dfe in __sleep (seconds=0) at ../sysdeps/posix/sleep.c:55
#3  0x00007f2779d80afd in Realm::realm_freeze (signal=6) at /scratch2/eslaught/resilience/legion/runtime/realm/runtime_impl.cc:183
#4  <signal handler called>
#5  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#6  0x00007f2778e67859 in __GI_abort () at abort.c:79
#7  0x00007f2778e67729 in __assert_fail_base (fmt=0x7f2778ffd588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7f277a405000 "inst_impl->metadata.is_valid()",
    file=0x7f277a4038f8 "/scratch2/eslaught/resilience/legion/runtime/realm/transfer/transfer.cc", line=161, function=<optimized out>) at assert.c:92
#8  0x00007f2778e78fd6 in __GI___assert_fail (assertion=0x7f277a405000 "inst_impl->metadata.is_valid()", file=0x7f277a4038f8 "/scratch2/eslaught/resilience/legion/runtime/realm/transfer/transfer.cc",
    line=161, function=0x7f277a418c58 "bool Realm::TransferIteratorBase<N, T>::done() [with int N = 1; T = long long int]") at assert.c:101
#9  0x00007f2779a2a20a in Realm::TransferIteratorBase<1, long long>::done (this=0x7f271411c360) at /scratch2/eslaught/resilience/legion/runtime/realm/transfer/transfer.cc:161
#10 0x00007f2779a2a310 in Realm::TransferIteratorBase<1, long long>::step (this=0x7f271411c360, max_bytes=8, info=..., flags=0, tentative=false)
    at /scratch2/eslaught/resilience/legion/runtime/realm/transfer/transfer.cc:258
#11 0x00007f27799247dd in Realm::XferDes::default_get_requests (this=0x7f2710008980, reqs=0x7f27702b63d8, nr=1, flags=0) at /scratch2/eslaught/resilience/legion/runtime/realm/transfer/channel.cc:2126
#12 0x00007f277995c12e in Realm::FileXferDes::get_requests (this=0x7f2710008980, requests=0x7f27702b63d8, nr=1) at /scratch2/eslaught/resilience/legion/runtime/realm/transfer/channel_disk.cc:81
#13 0x00007f277995c4a9 in Realm::FileXferDes::progress_xd (this=0x7f2710008980, channel=0x55fc18391f10, work_until=...) at /scratch2/eslaught/resilience/legion/runtime/realm/transfer/channel_disk.cc:119
#14 0x00007f277995e80c in Realm::XDQueue<Realm::FileChannel, Realm::FileXferDes>::do_work (this=0x55fc18391f48, work_until=...)
    at /scratch2/eslaught/resilience/legion/runtime/realm/transfer/channel.inl:157
#15 0x00007f2779c3e875 in Realm::BackgroundWorkManager::Worker::do_work (this=0x7f27702b69c0, max_time_in_ns=-1, interrupt_flag=0x0) at /scratch2/eslaught/resilience/legion/runtime/realm/bgwork.cc:621
#16 0x00007f2779c3c19e in Realm::BackgroundWorkThread::main_loop (this=0x55fc17ef9670) at /scratch2/eslaught/resilience/legion/runtime/realm/bgwork.cc:125
#17 0x00007f2779c3ffa0 in Realm::Thread::thread_entry_wrapper<Realm::BackgroundWorkThread, &Realm::BackgroundWorkThread::main_loop> (obj=0x55fc17ef9670)
    at /scratch2/eslaught/resilience/legion/runtime/realm/threads.inl:97
#18 0x00007f2779de8d36 in Realm::KernelThread::pthread_entry (data=0x55fc18281210) at /scratch2/eslaught/resilience/legion/runtime/realm/threads.cc:781
#19 0x00007f2778e06609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#20 0x00007f2778f64133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

This is control_replication 3a37d58ce612ef48d969f5acbccdd1db54310595.
Edit: Forgot to mention I'm running in DEBUG with -lg:safe_ctrlrepl 2 -lg:inorder, in case that matters. I see no errors reported.

Can you generate logs with -level dma=2,xplan=1,inst=1 and put them somewhere I can read them?

Logs are here:
/scratch2/eslaught/resilience/build/pennant_*.log

If you want a frozen process:
Process 3015968 on node g0004.stanford.edu is frozen!

One minor correction to my original comment: I actually get an error with -lg:safe_ctrlrepl 2. This does not happen with -lg:safe_ctrlrepl 1 so I'm not sure if this is a real error or not:
[1 - 7fc000088800]    2.113099 {5}{runtime}: [error 607] LEGION ERROR: Specific control replication violation occurred from member argument (from file /scratch2/eslaught/resilience/legion/runtime/legion/legion_context.cc:13274)

Anyway, the logs came from a run with -lg:safe_ctrlrepl 1, so the error is the one I originally reported.

Here is a copy being executed on rank 1:
[1 - 7f6b8c232800]    1.282884 {2}{xplan}: created: plan=0x7f6b28432ec0 domain=IS:<0>..<0>,dense srcs=2 dsts=2
[1 - 7f6b8c232800]    1.282902 {1}{xplan}: created: plan=0x7f6b28432ec0 srcs[0]=field(7700, inst=400000000000002f, size=4)
[1 - 7f6b8c232800]    1.282908 {1}{xplan}: created: plan=0x7f6b28432ec0 srcs[1]=field(7200, inst=400000000000002f, size=8)
[1 - 7f6b8c232800]    1.282913 {1}{xplan}: created: plan=0x7f6b28432ec0 dsts[0]=field(7700, inst=4000000000800001, size=4)
[1 - 7f6b8c232800]    1.282918 {1}{xplan}: created: plan=0x7f6b28432ec0 dsts[1]=field(7200, inst=4000000000800001, size=8)
[1 - 7f6b8c232800]    1.283070 {1}{xplan}: analysis: plan=0x7f6b28432ec0 dim_order=[0] xds=2 ibs=0
[1 - 7f6b8c232800]    1.283089 {1}{xplan}: analysis: plan=0x7f6b28432ec0 xds[0]: target=0 inputs=[inst(400000000000002f,0+1)] outputs=[inst(4000000000800001,0+1)]
[1 - 7f6b8c232800]    1.283095 {1}{xplan}: analysis: plan=0x7f6b28432ec0 xds[1]: target=0 inputs=[inst(400000000000002f,1+1)] outputs=[inst(4000000000800001,1+1)]
[1 - 7f6b8c232800]    1.283120 {2}{dma}: dma request 0x7f6b284254c0 created - plan=0x7f6b28432ec0 before=8000800005300033 after=8000800005100051
[1 - 7f6b8c232800]    1.283128 {2}{dma}: dma request 0x7f6b284254c0 ready - plan=0x7f6b28432ec0 before=8000800005300033 after=8000800005100051
[1 - 7f6b8c232800]    1.283134 {2}{dma}: dma request 0x7f6b284254c0 started - plan=0x7f6b28432ec0 before=8000800005300033 after=8000800005100051

and here is rank 0 deleting the destination instance before the copy can finish:
[0 - 7f3d539bf800]    1.271682 {1}{inst}: instance layout: inst=4000000000800001 layout=Layout(bytes=12, align=32, fields={7200=0+0, 7700=1+0}, lists=[[<0>..<0>->affine(<8>+0)], [<0>..<0>->affine(<4>+8)]])
[0 - 7f3d539bf800]    1.272260 {1}{inst}: allocation completed: inst=4000000000800001 offset=0
[0 - 7f3d539bf800]    1.272291 {2}{inst}: instance created: inst=4000000000800001 external=file(name='checkpoint.0.lr.7..dat', mode=1, offset=0) ready=0
[0 - 7f3d539bf800]    1.283918 {2}{inst}: instance destroyed: inst=4000000000800001 wait_on=0
[0 - 7f3d539bf800]    1.292461 {1}{inst}: deallocation completed: inst=4000000000800001
[0 - 7f3d50413800]    1.292696 {2}{inst}: releasing local instance:<PHONE_NUMBER>800001

@lightsighter what would you suggest looking at next?

What kind of memories are the instances in?

The target instance is an external file.

@elliottslaughter I think we need to fix the control replication violation first if you're not doing an index space attach.

Actually it is probably not that. @streichler is it the first file instance deletion that causes this?

 This bug has been here for more than 6 months at least. I don't understand why the CI is just picking it up now.

For what it's worth, this is not in CI. And that's true at multiple levels: we're not running this branch of Pennant (at all), we don't have it with multiple nodes, and we don't have it with checkpointing. Checkpointing in particular seems to be stressing the disk backend harder than we've done before.
Is there anything I can do in the meantime? E.g., would it help for me to cache the region trees instead of destroying them after I'm done?

I don't think so, the way normal instance deletion was refactored to automatically pick up dependences on all the events for operations still using an instance, but for external instances we have a separate path to pass through the event precondition representing all the deletion preconditions, but we lost the ability to pass that event through with the refactoring.

Testing a fix here: https://gitlab.com/StanfordLegion/legion/-/merge_requests/689

Merged, pull and try again.

After pulling, I'm able to get past the checkpoints. But during shutdown, the program appears to crash with:
pennant: /scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc:799: Realm::EventMerger::~EventMerger(): Assertion `!is_active()' failed.
[0 - 7fa4b29f5800]    7.760653 {6}{event}: Event<PHONE_NUMBER>200000 destroyed with current local waiters
[0 - 7fa4b29f5800]    7.760697 {6}{event}:   waiting on<PHONE_NUMBER>200006: 0x7fa44c4509c8

Backtrace:
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f5ab0a29859 in __GI_abort () at abort.c:79
#2  0x00007f5ab0a29729 in __assert_fail_base (fmt=0x7f5ab0bbf588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7f5ab201cb63 "!is_active()",
    file=0x7f5ab201c738 "/scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc", line=799, function=<optimized out>) at assert.c:92
#3  0x00007f5ab0a3afd6 in __GI___assert_fail (assertion=0x7f5ab201cb63 "!is_active()", file=0x7f5ab201c738 "/scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc", line=799,
    function=0x7f5ab201cb40 "Realm::EventMerger::~EventMerger()") at assert.c:101
#4  0x00007f5ab1807399 in Realm::EventMerger::~EventMerger (this=0x7f5a55c77590, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc:799
#5  0x00007f5ab18086ca in Realm::GenEventImpl::~GenEventImpl (this=0x7f5a55c77560, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc:960
#6  0x00007f5ab182b430 in Realm::DynamicTableNode<Realm::GenEventImpl, 65536ul, Realm::UnfairMutex, unsigned long long>::~DynamicTableNode (this=0x7f5a55bfb010, __in_chrg=<optimized out>)
    at /scratch2/eslaught/resilience/legion/runtime/realm/dynamic_table.inl:51
#7  0x00007f5ab182b462 in Realm::DynamicTableNode<Realm::GenEventImpl, 65536ul, Realm::UnfairMutex, unsigned long long>::~DynamicTableNode (this=0x7f5a55bfb010, __in_chrg=<optimized out>)
    at /scratch2/eslaught/resilience/legion/runtime/realm/dynamic_table.inl:51
#8  0x00007f5ab1953fcf in Realm::DynamicTable<Realm::DynamicTableAllocator<Realm::GenEventImpl, 11ul, 16ul> >::~DynamicTable (this=0x55deb43ca040, __in_chrg=<optimized out>)
    at /scratch2/eslaught/resilience/legion/runtime/realm/dynamic_table.inl:73
#9  0x00007f5ab194591f in Realm::RuntimeImpl::~RuntimeImpl (this=0x55deb43c9f80, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/runtime_impl.cc:844
#10 0x00007f5ab1944195 in Realm::Runtime::wait_for_shutdown (this=0x7ffc164d6b88) at /scratch2/eslaught/resilience/legion/runtime/realm/runtime_impl.cc:646
#11 0x00007f5ab47a592c in Legion::Internal::Runtime::start (argc=9, argv=0x7ffc164d71f8, background=false, supply_default_mapper=true) at /scratch2/eslaught/resilience/legion/runtime/legion/runtime.cc:29667
#12 0x00007f5ab429f185 in Legion::Runtime::start (argc=9, argv=0x7ffc164d71f8, background=false, default_mapper=true) at /scratch2/eslaught/resilience/legion/runtime/legion/legion.cc:7651
#13 0x000055deb3323ee7 in ResilientLegion::Runtime::start (argc=9, argv=0x7ffc164d71f8, background=false, supply_default_mapper=true) at /scratch2/eslaught/resilience/src/resilience/resilience.cc:1837
#14 0x000055deb3318012 in main (argc=9, argv=0x7ffc164d71f8) at /scratch2/eslaught/resilience/examples/pennant/src/main.cc:130

If I add -lg:inorder, the program hangs at shutdown instead.

For what it's worth, C++ Circuit hits this too now.

How do you reproduce it with circuit? If we're failing it with circuit then Legion Spy checks for leaking user events should be failing in the CI.

Note: this is checkpointed Circuit, and checkpointed Pennant. The plain versions are working right now as far as I know.
To reproduce with Circuit:
srun -n 1 -N 1 -c 40 -p gpu --exclusive --pty bash --login
cd /scratch2/eslaught/resilience/build
mpirun -n 2 -oversubscribe -x REALM_FREEZE_ON_ERROR=1 $PWD/examples/circuit/circuit -lg:safe_ctrlrepl 1

To reproduce with Pennant:
srun -n 1 -N 1 -c 40 -p gpu --exclusive --pty bash --login
cd /scratch2/eslaught/resilience/build
mpirun -n 2 -oversubscribe -x REALM_FREEZE_ON_ERROR=1 $PWD/examples/pennant/pennant -n 4 -f $PWD/../examples/pennant/test/sedovsmall/sedovsmall.pnt -lg:safe_ctrlrepl 1

Feel free to fiddle with the flags as needed.
Note that REALM_FREEZE_ON_ERROR seems to be insufficient to catch this, so you may need to add -lg:delay 60 to attach the debugger.
Feel free to rebuild if you need. I have no other active reproducers at the moment.
srun -n 1 -N 1 -c 40 -p gpu --exclusive --pty bash --login
cd /scratch2/eslaught/resilience/legion/build
make install -j20
cd /scratch2/eslaught/resilience/build
make clean && make -j20

This is not a user event leak. Here is the bad event from a run done with legion spy logging:
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f8c9a2b0859 in __GI_abort () at abort.c:79
#2  0x00007f8c9a2b0729 in __assert_fail_base (fmt=0x7f8c9a446588 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x7f8c9b754b63 "!is_active()", file=0x7f8c9b754738 "/scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc", line=799, function=<optimized out>) at assert.c:92
#3  0x00007f8c9a2c1fd6 in __GI___assert_fail (assertion=0x7f8c9b754b63 "!is_active()", file=0x7f8c9b754738 "/scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc", line=799, function=0x7f8c9b754b40 "Realm::EventMerger::~EventMerger()") at assert.c:101
#4  0x00007f8c9af3f399 in Realm::EventMerger::~EventMerger (this=0x7f8c39c16cd0, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc:799
#5  0x00007f8c9af406ca in Realm::GenEventImpl::~GenEventImpl (this=0x7f8c39c16ca0, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/event_impl.cc:960
#6  0x00007f8c9af63430 in Realm::DynamicTableNode<Realm::GenEventImpl, 65536ul, Realm::UnfairMutex, unsigned long long>::~DynamicTableNode (this=0x7f8c39bff010, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/dynamic_table.inl:51
#7  0x00007f8c9af63462 in Realm::DynamicTableNode<Realm::GenEventImpl, 65536ul, Realm::UnfairMutex, unsigned long long>::~DynamicTableNode (this=0x7f8c39bff010, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/dynamic_table.inl:51
#8  0x00007f8c9b08bfcf in Realm::DynamicTable<Realm::DynamicTableAllocator<Realm::GenEventImpl, 11ul, 16ul> >::~DynamicTable (this=0x5596945675c0, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/dynamic_table.inl:73
#9  0x00007f8c9b07d91f in Realm::RuntimeImpl::~RuntimeImpl (this=0x559694567500, __in_chrg=<optimized out>) at /scratch2/eslaught/resilience/legion/runtime/realm/runtime_impl.cc:844
#10 0x00007f8c9b07c195 in Realm::Runtime::wait_for_shutdown (this=0x7ffe9d9590e8) at /scratch2/eslaught/resilience/legion/runtime/realm/runtime_impl.cc:646
#11 0x00007f8c9dee6dd8 in Legion::Internal::Runtime::start (argc=9, argv=0x7ffe9d9597f8, background=false, supply_default_mapper=true) at /scratch2/eslaught/resilience/legion/runtime/legion/runtime.cc:29667
#12 0x00007f8c9d9dd40d in Legion::Runtime::start (argc=9, argv=0x7ffe9d9597f8, background=false, default_mapper=true) at /scratch2/eslaught/resilience/legion/runtime/legion/legion.cc:7651
#13 0x0000559693a17787 in ResilientLegion::Runtime::start (argc=9, argv=0x7ffe9d9597f8, background=false, supply_default_mapper=true) at /scratch2/eslaught/resilience/src/resilience/resilience.cc:1837
#14 0x00005596939d0723 in main (argc=9, argv=0x7ffe9d9597f8) at /scratch2/eslaught/resilience/examples/circuit/circuit.cc:276
(gdb) p/x this->me
$6 = {
  id = {
    value = 0x800000000a900000
  }
}

Now if I search for all the events with that prefix in the Legion Spy logs, I see this:
log_0.spy:[0 - 7f8c92764800]   33.573149 {3}{legion_spy}: Rt User Event 800000000a900003 423
log_0.spy:[0 - 7f8c92764800]   33.573738 {3}{legion_spy}: Rt User Event Trigger 800000000a900003
log_0.spy:[0 - 7f8c92764800]   33.575345 {3}{legion_spy}: Event Event<PHONE_NUMBER>a00015 800000000a900004
log_0.spy:[0 - 7f8c92764800]   33.575354 {3}{legion_spy}: Event Event<PHONE_NUMBER>a0000a 800000000a900004
log_0.spy:[0 - 7f8c92764800]   33.575358 {3}{legion_spy}: Event Event<PHONE_NUMBER>900014 800000000a900004
log_0.spy:[0 - 7f8c92764800]   33.575363 {3}{legion_spy}: Event Event<PHONE_NUMBER>90001c 800000000a900004
log_0.spy:[0 - 7f8c92764800]   33.575378 {3}{legion_spy}: Event Event 0 800000000a900005
log_0.spy:[0 - 7f8c92764800]   33.575383 {3}{legion_spy}: Event Event 800000000a900004 800000000a900005
log_0.spy:[0 - 7f8c92764800]   33.575591 {3}{legion_spy}: Fill Events 430 49 2 2 800000000a900005 800000000a900006 0
log_0.spy:[0 - 7f8c92764800]   33.575601 {3}{legion_spy}: Fill Field 800000000a900006 2<PHONE_NUMBER>900012
log_0.spy:[0 - 7f8c92764800]   33.575838 {3}{legion_spy}: Rt User Event 800000000a900007 430
log_0.spy:[0 - 7f8c92764800]   33.576277 {3}{legion_spy}: Event Event 800000000a900006<PHONE_NUMBER>f00028
log_0.spy:[0 - 7f8c92764800]   33.579480 {3}{legion_spy}: Event Event 800000000a900008<PHONE_NUMBER>f00067
log_0.spy:[0 - 7f8c92764800]   33.582937 {3}{legion_spy}: Event Event 800000000a900006<PHONE_NUMBER>a0000d
log_0.spy:[0 - 7f8c92758800]   35.107016 {3}{legion_spy}: Rt User Event 800000000a90000b 6
log_0.spy:[0 - 7f8c92764800]   35.249895 {3}{legion_spy}: Rt User Event Trigger 800000000a90000b
log_0.spy:[0 - 7f8c92764800]   35.654172 {3}{legion_spy}: Event Event 800000000a900006<PHONE_NUMBER>100010
log_0.spy:[0 - 7f8c99de6800]   36.124919 {6}{event}: Event 800000000a900000 destroyed with current local waiters
log_0.spy:[0 - 7f8c99de6800]   36.124982 {6}{event}:   waiting on 800000000a90000c: 0x7f8c341b99f8
log_1.spy:[1 - 7fcaabbfd800]   33.577523 {3}{legion_spy}: Rt User Event Trigger 800000000a900007

Legion Spy logs all (both runtime and application) user events and when they are triggered, so we can see that the last user event for this generational event was 0xb and it was triggered more than a second before we hit the error at the end. Furthermore, if we run the user event leak detection on these log files, it says that all the user events where triggered:
mebauer@g0004:/scratch2/eslaught/resilience/build$ python /scratch2/mebauer/legion/tools/legion_spy.py --leaks log_*spy
Reading log file log_0.spy...
Reading log file log_1.spy...
Reducing top-level index space shapes...
 |||||||||||||||||||||||||||||||||||||||||||||||||||| 100.0%
Done
Computing refinement points...
Dim 1: |||||||||||||||||||||||||||||||||||||||||||||||||||| 100.0%
Dim 2: |||||||||||||||||||||||||||||||||||||||||||||||||||| 100.0%
Done
Computing physical reachable...
 |||||||||||||||||||||||||||||||||||||||||||||||||||| 100.0%
Done
Performing user event leak checks...
Legion Spy analysis complete.  Exiting...

So we haven't leaked any user events anywhere in Legion. It's unclear exactly what created the event 800000000a90000c because Legion Spy doesn't log the runtime events (other than the user events and their triggers), so we can't see what made that.  Interestingly when I try to call Realm::show_event_waiters(0) from the process, I end up hitting a pure virtual method:
OperationTable(node=0) {
}
PRINTING ALL PENDING EVENTS:
Event<PHONE_NUMBER>d00000: gen=28 subscr=0 local=1+0 remote=0
pure virtual method called

which I think is because this generational event data structure has probably already been deleted, so we can't really see the all the event waiters to determine what might have caused this event not to be triggered.

I added logging to Legion Spy to dump all the runtime events from meta-tasks and event mergers, and now there is more weirdness. Here is the bad event:
(gdb) p/x this->me
$1 = {
  id = {
    value = 0x8000000007700000
  }
}

and here is what we see in the Legion Spy logs:
log_0.spy:[0 - 7fad283b6800]  137.870022 {3}{legion_spy}: Event Event<PHONE_NUMBER>100001<PHONE_NUMBER>700001
log_0.spy:[0 - 7fad283b6800]  137.870054 {3}{legion_spy}: Meta Task: 12<PHONE_NUMBER>700001<PHONE_NUMBER>800001
log_0.spy:[0 - 7fad283aa800]  137.870815 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700002 6
log_0.spy:[0 - 7fad283aa800]  137.870981 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700002
log_0.spy:[0 - 7fad283aa800]  138.081206 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700003 6
log_0.spy:[0 - 7fad283b6800]  138.094665 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700003
log_0.spy:[0 - 7fad283b6800]  138.095105 {3}{legion_spy}: Event Event<PHONE_NUMBER>700004<PHONE_NUMBER>300006
log_0.spy:[0 - 7fad283b6800]  138.099207 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700005 252
log_0.spy:[0 - 7fad283b6800]  138.099262 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700005
log_0.spy:[0 - 7fad283b6800]  138.099329 {3}{legion_spy}: Meta Task: 55<PHONE_NUMBER>200029<PHONE_NUMBER>700006
log_0.spy:[0 - 7fad283b6800]  138.135477 {3}{legion_spy}: Meta Task: 2<PHONE_NUMBER>700007<PHONE_NUMBER>30000a
log_0.spy:[0 - 7fad283b6800]  138.142423 {3}{legion_spy}: Meta Task: 107 0<PHONE_NUMBER>700008
log_0.spy:[0 - 7fad283b6800]  138.154137 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700009 260
log_0.spy:[0 - 7fad283b6800]  138.155526 {3}{legion_spy}: Event Event<PHONE_NUMBER>700009<PHONE_NUMBER>20000b
log_0.spy:[0 - 7fad283b6800]  138.190956 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700009
log_0.spy:[0 - 7fad283b6800]  138.192791 {3}{legion_spy}: Meta Task: 46 0<PHONE_NUMBER>70000a
log_0.spy:[0 - 7fad283b6800]  138.300734 {3}{legion_spy}: Ap User Event<PHONE_NUMBER>70000b 296
log_0.spy:[0 - 7fad283b6800]  138.309020 {3}{legion_spy}: Event Event<PHONE_NUMBER>70000b<PHONE_NUMBER>800031
log_0.spy:[0 - 7fad283b6800]  138.333125 {3}{legion_spy}: Ap User Event Trigger<PHONE_NUMBER>70000b
log_0.spy:[0 - 7fad283b6800]  138.333138 {3}{legion_spy}: Event Event<PHONE_NUMBER>900018<PHONE_NUMBER>70000b
log_0.spy:[0 - 7fad283b6800]  138.398785 {3}{legion_spy}: Reservation 1f00000000000002<PHONE_NUMBER>800032<PHONE_NUMBER>70000c
log_0.spy:[0 - 7fad283b6800]  138.398797 {3}{legion_spy}: Reservation 1f00000000000003<PHONE_NUMBER>70000c<PHONE_NUMBER>900019
log_0.spy:[0 - 7fad283aa800]  139.793996 {3}{legion_spy}: Ap User Event<PHONE_NUMBER>70000d 6
log_0.spy:[0 - 7fad283b6800]  139.803595 {3}{legion_spy}: Ap User Event Trigger<PHONE_NUMBER>70000d
log_0.spy:[0 - 7fad283b6800]  139.804014 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>70000e 352
log_0.spy:[0 - 7fad283b6800]  139.804079 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>70000e
log_0.spy:[0 - 7fad283b6800]  139.804108 {3}{legion_spy}: Event Event<PHONE_NUMBER>b0000d<PHONE_NUMBER>70000f
log_0.spy:[0 - 7fad283b6800]  139.804114 {3}{legion_spy}: Event Event<PHONE_NUMBER>70000e<PHONE_NUMBER>70000f
log_0.spy:[0 - 7fad283b6800]  139.804138 {3}{legion_spy}: Meta Task: 12<PHONE_NUMBER>70000f<PHONE_NUMBER>700010
log_0.spy:[0 - 7fad283b6800]  139.805538 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700011 355
log_0.spy:[0 - 7fad283b6800]  139.805594 {3}{legion_spy}: Event Event<PHONE_NUMBER>700011<PHONE_NUMBER>b00012
log_0.spy:[0 - 7fad283b6800]  139.819417 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700012 358
log_0.spy:[0 - 7fad283b6800]  139.820059 {3}{legion_spy}: Event Event<PHONE_NUMBER>700012<PHONE_NUMBER>a00020
log_0.spy:[0 - 7fad283b6800]  139.828857 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700012
log_0.spy:[0 - 7fad283b6800]  139.829677 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700013 362
log_0.spy:[0 - 7fad283b6800]  139.829796 {3}{legion_spy}: Meta Task: 76<PHONE_NUMBER>700013<PHONE_NUMBER>200012
log_0.spy:[0 - 7fad283b6800]  139.830791 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700013
log_0.spy:[0 - 7fad283b6800]  139.874090 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700014 361
log_0.spy:[0 - 7fad283b6800]  139.874122 {3}{legion_spy}: Event Event<PHONE_NUMBER>700014<PHONE_NUMBER>800036
log_0.spy:[0 - 7fad283b6800]  139.887964 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700014
log_0.spy:[0 - 7fad283b6800]  139.893241 {3}{legion_spy}: Meta Task: 80<PHONE_NUMBER>10004e<PHONE_NUMBER>700015
log_0.spy:[0 - 7fad283b6800]  139.894422 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700016 370
log_0.spy:[0 - 7fad283b6800]  139.894505 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700016
log_0.spy:[0 - 7fad283b6800]  139.894518 {3}{legion_spy}: Event Event<PHONE_NUMBER>200036<PHONE_NUMBER>700017
log_0.spy:[0 - 7fad283b6800]  139.894640 {3}{legion_spy}: Event Event<PHONE_NUMBER>700016<PHONE_NUMBER>30001e
log_0.spy:[0 - 7fad283aa800]  139.910727 {3}{legion_spy}: Event Event<PHONE_NUMBER>700018<PHONE_NUMBER>200037
log_0.spy:[0 - 7fad283b6800]  139.911310 {3}{legion_spy}: Meta Task: 12 0<PHONE_NUMBER>700019
log_0.spy:[0 - 7fad283b6800]  139.932218 {3}{legion_spy}: Meta Task: 80<PHONE_NUMBER>60001f<PHONE_NUMBER>70001a
log_0.spy:[0 - 7fad283aa800]  139.987409 {3}{legion_spy}: Meta Task: 107 0<PHONE_NUMBER>70001b
log_0.spy:[0 - 7fad283b6800]  139.989881 {3}{legion_spy}: Event Event<PHONE_NUMBER>70001a<PHONE_NUMBER>70001c
log_0.spy:[0 - 7fad283b6800]  139.989891 {3}{legion_spy}: Event Event<PHONE_NUMBER>300037<PHONE_NUMBER>70001c
log_0.spy:[0 - 7fad283b6800]  139.989926 {3}{legion_spy}: Event Event<PHONE_NUMBER>70001c<PHONE_NUMBER>200037
log_0.spy:[0 - 7fad283b6800]  139.993140 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>70001d 388
log_0.spy:[0 - 7fad283b6800]  139.993304 {3}{legion_spy}: Event Event<PHONE_NUMBER>70001d 800000000a200003
log_0.spy:[0 - 7fad283b6800]  140.001798 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>70001d
log_0.spy:[0 - 7fad283b6800]  140.004323 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>70001e 412
log_0.spy:[0 - 7fad283b6800]  140.004367 {3}{legion_spy}: Meta Task: 81<PHONE_NUMBER>70001e<PHONE_NUMBER>500026
log_0.spy:[0 - 7fad283b6800]  140.026007 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>70001e
log_0.spy:[0 - 7fad283b6800]  140.027003 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>70001f 409
log_0.spy:[0 - 7fad283b6800]  140.027429 {3}{legion_spy}: Event Event<PHONE_NUMBER>70001f 800000000d800002
log_0.spy:[0 - 7fad283b6800]  140.028115 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700020 388
log_0.spy:[0 - 7fad283b6800]  140.028153 {3}{legion_spy}: Meta Task: 81<PHONE_NUMBER>700020<PHONE_NUMBER>800040
log_0.spy:[0 - 7fad283b6800]  140.029822 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700020
log_0.spy:[0 - 7fad283b6800]  140.030521 {3}{legion_spy}: Event Event 800000000e400002<PHONE_NUMBER>700021
log_0.spy:[0 - 7fad283b6800]  140.030531 {3}{legion_spy}: Event Event 800000000d700008<PHONE_NUMBER>700021
log_0.spy:[0 - 7fad283b6800]  140.030550 {3}{legion_spy}: Meta Task: 55<PHONE_NUMBER>700021<PHONE_NUMBER>100011
log_0.spy:[0 - 7fad283b6800]  140.030878 {3}{legion_spy}: Event Event<PHONE_NUMBER>700022 800000000e600004
log_0.spy:[0 - 7fad283b6800]  140.048840 {3}{legion_spy}: Meta Task: 107 0<PHONE_NUMBER>700023
log_0.spy:[0 - 7fad283b6800]  140.177899 {3}{legion_spy}: Rt User Event<PHONE_NUMBER>700024 416
log_0.spy:[0 - 7fad283b6800]  140.209723 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700024
log_0.spy:[0 - 7fad283b6800]  140.209826 {3}{legion_spy}: Ap User Event<PHONE_NUMBER>700025 416
log_0.spy:[0 - 7fad283b6800]  140.213500 {3}{legion_spy}: Ap User Event Trigger<PHONE_NUMBER>700025
log_0.spy:[0 - 7fad283b6800]  140.213509 {3}{legion_spy}: Operation Events 424 0<PHONE_NUMBER>700025
log_0.spy:[0 - 7fad283b6800]  141.651392 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>400019
log_0.spy:[0 - 7fad283b6800]  141.703900 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>800013
log_0.spy:[0 - 7fad283b6800]  141.797655 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>f00035
log_0.spy:[0 - 7fad283b6800]  141.954432 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025 800000000c200006
log_0.spy:[0 - 7fad283b6800]  142.294153 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025 800000000a900012
log_0.spy:[0 - 7fad283b6800]  142.310372 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>c0004a
log_0.spy:[0 - 7fad283b6800]  142.311877 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>800015
log_0.spy:[0 - 7fad283b6800]  142.321460 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>d00031
log_0.spy:[0 - 7fad283b6800]  142.322856 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025 800000000de00012
log_0.spy:[0 - 7fad283b6800]  142.324221 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025 800000000d90001d
log_0.spy:[0 - 7fad283b6800]  142.326478 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>d00057
log_0.spy:[0 - 7fad283b6800]  142.327923 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>c0001c
log_0.spy:[0 - 7fad283b6800]  142.330301 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>40001e
log_0.spy:[0 - 7fad283b6800]  142.331748 {3}{legion_spy}: Event Event<PHONE_NUMBER>700025<PHONE_NUMBER>700025
log_0.spy:[0 - 7fad2a5cf800]  142.779088 {6}{event}: Event<PHONE_NUMBER>700000 destroyed with current local waiters
log_0.spy:[0 - 7fad2a5cf800]  142.779162 {6}{event}:   waiting on<PHONE_NUMBER>700026: 0x7facc43a8998
log_1.spy:[1 - 7f801c723800]  139.815890 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>700011
log_1.spy:[1 - 7f801c723800]  140.037146 {3}{legion_spy}: Rt User Event Trigger<PHONE_NUMBER>70001f

What's interesting about this is that the last event that Legion thinks it made for this generational event was<PHONE_NUMBER>700025 here:
log_0.spy:[0 - 7fad283b6800]  140.209826 {3}{legion_spy}: Ap User Event<PHONE_NUMBER>700025 416
log_0.spy:[0 - 7fad283b6800]  140.213500 {3}{legion_spy}: Ap User Event Trigger<PHONE_NUMBER>700025

As far as I can tell, Legion never saw generation 0x26 for this generational event. @streichler am I interpreting this correctly? Is Realm still using events internally as part of its shutdown process?

Can you add event=2,task=2,dma=2 to the logging and maybe that'll tell us where this event is coming from?  There should not be any internal events in the shutdown process.

Here is the bad event:
(gdb) p/x this->me
$1 = {
  id = {
    value = 0x800000000c900000
  }
}

And here is the output:
log_0.spy:[0 - 7fe708018800]   37.366559 {1}{event}: event not ready: event=800000000c900000/1 owner=0 gen=0 subscr=0
log_0.spy:[0 - 7fe708018800]   37.366568 {3}{legion_spy}: Meta Task:<PHONE_NUMBER>00c600001 800000000c900001
log_0.spy:[0 - 7fe708018800]   37.366977 {3}{legion_spy}: Meta Task:<PHONE_NUMBER>00c900001 800000000cd00001
log_0.spy:[0 - 7fe700658800]   37.367259 {1}{event}: event subscription: node=1 event=800000000c900001
log_0.spy:[0 - 7fe700658800]   37.367268 {1}{event}: event subscription recorded: node=1 event=800000000c900001 (> 0)
log_0.spy:[0 - 7fe700658800]   37.375751 {1}{event}: remote trigger of event 800000000c900001 from node 1
log_0.spy:[0 - 7fe700658800]   37.375760 {1}{event}: event triggered: event=800000000c900001 by node 1 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   37.376724 {2}{event}: user event created: event=800000000c900002
log_0.spy:[0 - 7fe708018800]   37.376731 {2}{event}: user event trigger: event=800000000c900002 wait_on=0
log_0.spy:[0 - 7fe708018800]   37.376738 {1}{event}: event triggered: event=800000000c900002 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   37.376747 {3}{legion_spy}: Event Event<PHONE_NUMBER>000013 800000000c900002
log_0.spy:[0 - 7fe708018800]   37.376753 {3}{legion_spy}: Event Event<PHONE_NUMBER>c0001d 800000000c900002
log_0.spy:[0 - 7fe708018800]   37.376769 {3}{legion_spy}: Event Event<PHONE_NUMBER>b0000a 800000000c900002
log_0.spy:[0 - 7fe708018800]   37.376802 {2}{event}: user event created: event=800000000c900003
log_0.spy:[0 - 7fe708018800]   37.376807 {2}{event}: user event trigger: event=800000000c900003 wait_on=0
log_0.spy:[0 - 7fe708018800]   37.376812 {1}{event}: event triggered: event=800000000c900003 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   37.376819 {3}{legion_spy}: Event Event 800000000c900002 800000000c900003
log_0.spy:[0 - 7fe708018800]   37.376827 {2}{event}: user event trigger: event=800000000110006e wait_on=800000000c900003
log_0.spy:[0 - 7fe708018800]   37.376863 {3}{legion_spy}: Event Event 800000000c900003<PHONE_NUMBER>10006e
log_0.spy:[0 - 7fe708018800]   37.587844 {2}{event}: event merging: event=800000000c900004 wait_on=800000000410003d
log_0.spy:[0 - 7fe708018800]   37.587855 {2}{event}: event merging: event=800000000c900004 wait_on=800000000d200004
log_0.spy:[0 - 7fe708018800]   37.587866 {1}{event}: received trigger merged event=800000000c900004 left=3 poisoned=0
log_0.spy:[0 - 7fe708018800]   37.587871 {3}{legion_spy}: Event Event<PHONE_NUMBER>10003d 800000000c900004
log_0.spy:[0 - 7fe708018800]   37.587875 {3}{legion_spy}: Event Event 800000000d200004 800000000c900004
log_0.spy:[0 - 7fe708018800]   37.587910 {1}{event}: event not ready: event=800000000c900000/4 owner=0 gen=3 subscr=0
log_0.spy:[0 - 7fe708018800]   37.587917 {3}{legion_spy}: Reservation 1f00010000000004 800000000c900004<PHONE_NUMBER>e00017
log_0.spy:[0 - 7fe708018800]   37.588079 {3}{legion_spy}: Copy Events 429 51 2 2 800000000c900004<PHONE_NUMBER>a00013
log_0.spy:[0 - 7fe70075c800]   37.818015 {1}{event}: received trigger merged event=800000000c900004 left=2 poisoned=0
log_0.spy:[0 - 7fe700658800]   38.761766 {1}{event}: received trigger merged event=800000000c900004 left=1 poisoned=0
log_0.spy:[0 - 7fe700658800]   38.761774 {1}{event}: event triggered: event=800000000c900004 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   38.879200 {2}{event}: user event created: event=800000000c900005
log_0.spy:[0 - 7fe708018800]   38.879204 {3}{legion_spy}: Rt User Event 800000000c900005 456
log_0.spy:[0 - 7fe708018800]   38.879248 {2}{event}: event merging: event=8000000007400034 wait_on=800000000c900005
log_0.spy:[0 - 7fe708018800]   38.879253 {1}{event}: event not ready: event=800000000c900000/5 owner=0 gen=4 subscr=0
log_0.spy:[0 - 7fe708018800]   38.879267 {3}{legion_spy}: Event Event 800000000c900005<PHONE_NUMBER>400034
log_0.spy:[0 - 7fe708018800]   38.881828 {2}{event}: user event trigger: event=800000000c900005 wait_on=0
log_0.spy:[0 - 7fe708018800]   38.881835 {1}{event}: event triggered: event=800000000c900005 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   38.881858 {3}{legion_spy}: Rt User Event Trigger 800000000c900005
log_0.spy:[0 - 7fe708018800]   39.100887 {2}{event}: user event created: event=800000000c900006
log_0.spy:[0 - 7fe708018800]   39.100892 {2}{event}: deferring user event trigger: event=800000000c900006 wait_on=800000000f100007
log_0.spy:[0 - 7fe708018800]   39.100909 {1}{event}: received trigger merged event=800000000c900006 left=2 poisoned=0
log_0.spy:[0 - 7fe708018800]   39.100914 {3}{legion_spy}: Event Event 800000000f100007 800000000c900006
log_0.spy:[0 - 7fe708018800]   39.100926 {2}{task}: task 0x7fe6a805ad70 created: func=4 proc=1d00000000000000 arglen=24 before=800000000c900006 after=800000000f600003
log_0.spy:[0 - 7fe708018800]   39.100942 {1}{event}: event not ready: event=800000000c900000/6 owner=0 gen=5 subscr=0
log_0.spy:[0 - 7fe708018800]   39.100948 {3}{legion_spy}: Meta Task: 12 800000000c900006 800000000f600003
log_0.spy:[0 - 7fe708018800]   39.136493 {1}{event}: received trigger merged event=800000000c900006 left=1 poisoned=0
log_0.spy:[0 - 7fe708018800]   39.136498 {1}{event}: event triggered: event=800000000c900006 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe700658800]   39.136867 {2}{task}: task 0x7fe6a805ad70 ready: func=4 proc=1d00000000000000 arglen=24 before=800000000c900006 after=800000000f600003
log_0.spy:[0 - 7fe708018800]   39.136886 {2}{task}: task 0x7fe6a805ad70 started: func=4 proc=1d00000000000000 arglen=24 before=800000000c900006 after=800000000f600003
log_0.spy:[0 - 7fe708018800]   39.137081 {2}{task}: task 0x7fe6ac015b70 created: func=4 proc=1d00000000000000 arglen=40 before=800000000d90000a after=800000000c900007
log_0.spy:[0 - 7fe708018800]   39.137090 {1}{event}: event not ready: event=800000000c900000/7 owner=0 gen=6 subscr=0
log_0.spy:[0 - 7fe708018800]   39.137103 {3}{legion_spy}: Meta Task: 76 800000000d90000a 800000000c900007
log_0.spy:[0 - 7fe708018800]   39.137461 {2}{task}: task 0x7fe6a805ad70 completed: func=4 proc=1d00000000000000 arglen=24 before=800000000c900006 after=800000000f600003
log_0.spy:[0 - 7fe708018800]   39.138821 {2}{task}: task 0x7fe6ac015b70 ready: func=4 proc=1d00000000000000 arglen=40 before=800000000d90000a after=800000000c900007
log_0.spy:[0 - 7fe708018800]   39.138880 {2}{task}: task 0x7fe6ac015b70 started: func=4 proc=1d00000000000000 arglen=40 before=800000000d90000a after=800000000c900007
log_0.spy:[0 - 7fe708018800]   39.138945 {2}{task}: task 0x7fe6ac015b70 completed: func=4 proc=1d00000000000000 arglen=40 before=800000000d90000a after=800000000c900007
log_0.spy:[0 - 7fe708018800]   39.138956 {1}{event}: event triggered: event=800000000c900007 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   39.139113 {2}{event}: user event created: event=800000000c900008
log_0.spy:[0 - 7fe708018800]   39.139117 {3}{legion_spy}: Rt User Event 800000000c900008 488
log_0.spy:[0 - 7fe708018800]   39.139255 {2}{event}: user event trigger: event=800000000c900008 wait_on=0
log_0.spy:[0 - 7fe708018800]   39.139260 {1}{event}: event triggered: event=800000000c900008 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   39.139264 {3}{legion_spy}: Rt User Event Trigger 800000000c900008
log_0.spy:[0 - 7fe708018800]   39.139280 {2}{event}: user event created: event=800000000c900009
log_0.spy:[0 - 7fe708018800]   39.139285 {2}{event}: user event trigger: event=800000000c900009 wait_on=0
log_0.spy:[0 - 7fe708018800]   39.139290 {1}{event}: event triggered: event=800000000c900009 by node 0 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   39.139295 {3}{legion_spy}: Event Event 800000000f100009 800000000c900009
log_0.spy:[0 - 7fe708018800]   39.139338 {2}{event}: user event trigger: event=8000000006600027 wait_on=800000000c900009
log_0.spy:[0 - 7fe708018800]   39.139387 {3}{legion_spy}: Event Event 800000000c900008<PHONE_NUMBER>600028
log_0.spy:[0 - 7fe708018800]   39.163595 {1}{event}: event not ready: event=800000000c900000/10 owner=0 gen=9 subscr=0
log_0.spy:[0 - 7fe708018800]   39.163605 {3}{legion_spy}: Meta Task: 107 0 800000000c90000a
log_0.spy:[0 - 7fe70075c800]   39.164255 {1}{event}: event subscription: node=1 event=800000000c90000a
log_0.spy:[0 - 7fe70075c800]   39.164260 {1}{event}: event subscription recorded: node=1 event=800000000c90000a (> 9)
log_0.spy:[0 - 7fe70075c800]   39.164264 {1}{event}: event subscription immediate trigger: node=1 event=800000000c90000a (<= 9)
log_0.spy:[0 - 7fe70075c800]   39.167108 {1}{event}: remote trigger of event 800000000c90000a from node 1
log_0.spy:[0 - 7fe70075c800]   39.167114 {1}{event}: event triggered: event=800000000c90000a by node 1 (poisoned=0)
log_0.spy:[0 - 7fe708018800]   39.168739 {2}{event}: event merging: event=800000000c90000b wait_on=2000000003900008
log_0.spy:[0 - 7fe708018800]   39.168751 {1}{event}: received trigger merged event=800000000c90000b left=2 poisoned=0
log_0.spy:[0 - 7fe708018800]   39.168761 {1}{event}: event not ready: event=800000000c900000/11 owner=0 gen=10 subscr=0
log_0.spy:[0 - 7fe70b29d800]   40.130569 {6}{event}: Event 800000000c900000 destroyed with current local waiters
log_0.spy:[0 - 7fe70b29d800]   40.130633 {6}{event}:   waiting on 800000000c90000b: 0x7fe6a42fae28
log_1.spy:[1 - 7f41f9659800]   37.367416 {2}{task}: task 0x7f41a4040f50 created: func=4 proc=1d00010000000000 arglen=208 before=800000000c600001 after=800000000c900001
log_1.spy:[1 - 7f41f9659800]   37.367424 {1}{event}: event not ready: event=800000000c900000/1 owner=0 gen=0 subscr=0
log_1.spy:[1 - 7f41f9659800]   37.367442 {2}{task}: task 0x7f41a40411d0 created: func=4 proc=1d00010000000000 arglen=208 before=800000000c900001 after=800000000cd00001
log_1.spy:[1 - 7f41f9659800]   37.367457 {1}{event}: event not ready: event=800000000c900000/1 owner=0 gen=0 subscr=1
log_1.spy:[1 - 7f41f9659800]   37.373149 {2}{task}: task 0x7f41a4040f50 ready: func=4 proc=1d00010000000000 arglen=208 before=800000000c600001 after=800000000c900001
log_1.spy:[1 - 7f4200b56800]   37.374535 {2}{task}: task 0x7f41a4040f50 started: func=4 proc=1d00010000000000 arglen=208 before=800000000c600001 after=800000000c900001
log_1.spy:[1 - 7f4200b56800]   37.375059 {2}{task}: task 0x7f41a4040f50 completed: func=4 proc=1d00010000000000 arglen=208 before=800000000c600001 after=800000000c900001
log_1.spy:[1 - 7f4200b56800]   37.375066 {1}{event}: event triggered: event=800000000c900001 by node 1 (poisoned=0)
log_1.spy:[1 - 7f41f9659800]   37.377515 {2}{task}: task 0x7f41a40411d0 ready: func=4 proc=1d00010000000000 arglen=208 before=800000000c900001 after=800000000cd00001
log_1.spy:[1 - 7f4200b56800]   37.377804 {2}{task}: task 0x7f41a40411d0 started: func=4 proc=1d00010000000000 arglen=208 before=800000000c900001 after=800000000cd00001
log_1.spy:[1 - 7f4200b56800]   37.377896 {2}{task}: task 0x7f41a40411d0 completed: func=4 proc=1d00010000000000 arglen=208 before=800000000c900001 after=800000000cd00001
log_1.spy:[1 - 7f41f9659800]   37.378011 {1}{event}: event update: event=800000000c900001 poisoned=0:{}
log_1.spy:[1 - 7f41f9659800]   39.163947 {2}{task}: task 0x7f41a404afe0 created: func=4 proc=1d00010000000000 arglen=80 before=0 after=800000000c90000a
log_1.spy:[1 - 7f41f9659800]   39.163956 {1}{event}: event not ready: event=800000000c900000/10 owner=0 gen=1 subscr=1
log_1.spy:[1 - 7f41f9659800]   39.163965 {2}{task}: task 0x7f41a404afe0 ready: func=4 proc=1d00010000000000 arglen=80 before=0 after=800000000c90000a
log_1.spy:[1 - 7f4200b56800]   39.165477 {2}{task}: task 0x7f41a404afe0 started: func=4 proc=1d00010000000000 arglen=80 before=0 after=800000000c90000a
log_1.spy:[1 - 7f4200b56800]   39.165531 {2}{task}: task 0x7f41a404afe0 completed: func=4 proc=1d00010000000000 arglen=80 before=0 after=800000000c90000a
log_1.spy:[1 - 7f4200b56800]   39.165537 {1}{event}: event triggered: event=800000000c90000a by node 1 (poisoned=0)
log_1.spy:[1 - 7f41f9659800]   39.165966 {1}{event}: event update: event=800000000c900009 poisoned=0:{}
log_1.spy:[1 - 7f41f9659800]   39.167620 {1}{event}: event update: event=800000000c90000a poisoned=0:{}

I only see three lines with the bad generation:
log_0.spy:[0 - 7fe708018800]   39.168739 {2}{event}: event merging: event=800000000c90000b wait_on=2000000003900008
log_0.spy:[0 - 7fe708018800]   39.168751 {1}{event}: received trigger merged event=800000000c90000b left=2 poisoned=0
log_0.spy:[0 - 7fe70b29d800]   40.130633 {6}{event}:   waiting on 800000000c90000b: 0x7fe6a42fae28

I have no idea how it could have been made without going through the Legion Spy logging on the Legion side.

can you add realm=1 to the logging as well?

So I'm not sure why Realm is calling it an event merger, it's actually a Legion call to Event::ignorefaults that is creating the bad event. I think I understand what is happening. There is some dangling instance deletions that are not getting folded up into the preconditions RealmRuntime::shutdown. It's really hard to know when an instance is finally deleted...
@streichler: If I ask for a profiling callback for an instance deletion that will trigger a user event that is a precondition to RealmRuntime::shutdown, could I have a race with Realm trying to shut things down before the profiling callback task is done running?

Yes, "laundering" events does require a merger.  And as long as you don't initiate any new work or wait on any events after you trigger the user event, you should be fine w.r.t. races.  A "flush" task needs to run on the processor that's running the profiling callback before things start getting torn down.

A "flush" task needs to run on the processor that's running the profiling callback before things start getting torn down.

Yes, that should work.

I think I finally have a plan to address this issue that doesn't involve asking for a callback from Realm on every instance deletion.

@elliottslaughter Please try again now that this has merged:
https://gitlab.com/StanfordLegion/legion/-/merge_requests/745

I fixed half the problem, but you're being tricky with your external attaches and I wasn't considering them in the garbage collection effects. Fix tomorrow.

Pull and try again.

Fixed, thanks!


fix: Removed FsEventWatcher

The FsEventWatcher seems to be for MacOS only while RecommendedWatcher seems to be os agnostic, please confirm on your device though!

Thank you! This is great.


operation_unfiltered index

Closes #538
Following #477 and #401
This reverts commit ae5c078610d528425af22336bd049610f7b24033.

Currently this PR is breaking because of IndexType.

Hey, @xflpt!
This feature is mostly done. However, I can't include it officially in the next minor release because of the schema change. But we're in Python, so I came up with a bizarre hack:
if env.NEXT:
    class IndexType(Enum): ...
else:
    class IndexType(Enum): ...  # type: ignore[no-redef]

Now you can use two different incompatible schemas, with the IndexType.operation_unfiltered member and without it. To use the new schema, add DIPDUP_NEXT=1 to the env variables.
$ DIPDUP_NEXT=1 dipdup run

That way, you'll have your own secret feature in the DipDup codebase 🤫
Could you please test this stuff from the branch?

That's awesome, thank you so much for that :)


Make GitLab ELT Incremental

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/5
Originally created by @jschatz1 on 2018-07-25 21:59:41

The current implementation of the Gitlab ELT is implemented as:

Downloads CSV files from a GCS bucket
Decompress the files
Integrate the CSV using the corresponding strategy (upsert or overwrite)

The bulk of this work is in the Pseudonymizer component of GitLab: we need to make it export only new data, from the last export. One way to do it would be to persist some kind of cursor (MAX(id) is a natural one for numeric id, MAX(created_date) can also work) and instead of walking through all the data set, start the extraction from this cursor.
We already output metadata files in the pseudonymizer run, we could either add this to the metadata, or create a cursor.yml that tracks this.
The pseudonymizer would then:

Read the provided cursor file (either provided at invocation or fetched from the latest run or default cursors)
Extract starting at the cursor
Export the updated cursor
Upload the cursors along the data

There should be a way to invalidate this cursor, for any of these cases (this might be a follow-up MR, you can manually delete the cursor file):

An entity has changed:

New entity
New attribute
Changed transformation

cc @tayloramurphy @mbergeron

View 7 previous comments from the original issue on GitLab


Support classic and alternate import syntax in flogo.json

What kind of change does this PR introduce? (check one with "x")
[] Bugfix
[x] Feature
[] Code style update (formatting, local variables)
[] Refactoring (no functional changes, no api changes)
[] Other... Please describe:

This PR is required to be merged before https://github.com/project-flogo/cli/pull/32

What is the current behavior?
Currently imports in the "imports" array of flogo.json file follow the Go import path syntax (including aliases).
Hence it is not possible to know what is the path to the underlying module (if it exists) neither to specify a custom version neither to specify a custom version which will be handled with go mod plumbering instead of "legacy" go get.
This complicates the interaction of the Flogo CLI with go mod commands for instance in the flogo create command and it prevents the flogo.json to recreate an application in a reproducible way.
What is the new behavior?
In PR https://github.com/project-flogo/cli/pull/32, a proposal is made to create a new syntax which would be more verbose while remaining readable.
This syntax is aimed at keeping good default values (no version = latest, no alias by default).
The only mandatory add-on is the : when the original import path does not point to a Go module like the github.com/project-flogo/contrib/activity/log contribution.
So
    "github.com/project-flogo/contrib/activity/log",

becomes
    "github.com/project-flogo/contrib:/activity/log",

Notice the : separating the path to the module and the relative import path.
Notice also that the github.com/project-flogo/flow import is unchanged as it already points to a Go module and it is expected to use latest version.
Update : this new syntax format can totally be mixed with the old one. Using the new one gives more flexibility to manage dependencies under the hood. Hence I suggest to name those two compatible syntaxes the classic syntax and the alternate syntax.

I updated the names of the syntaxes: the classic syntax and the alternate syntax
@fm-tibco : I reverted the examples/engine/flogo.json and created examples/alt/engine/flogo.json instead. So the default behaviour is to use the classic syntax. This way the alternate syntax will only be used by advised users.


Metric: Issues by Status

This issue tracks the completeness of the metric.
Business Question: How many issues where open in a certain period of time?
How to calculate: Issue Count by Status File details of metrics here
Tasks

[ ] Create/update backend to fetch Issues by Status
[ ] Visualize it in the frontend

@Lehcar this might be a good one to dive into! I think that, given the data we're already pulling to calculate issue resolution time, we can also calculate how many issues were open 30, 60, and 90 days ago. That'd let us do some charting/trends around whether issues are going up or down for a given repo. Let me know if that makes sense!

I'm a bit confused what this line const issues = result.filter((issue) => !issue.pull_request); does here. Is it saying that any issue without a PR is an open issue, or does graphQL return both pull_requests and issues, and we're just filtering out the PRs?

Also, here's my first pass/draft of how to do this: https://github.com/sbv-world-health-org-metrics/sbv-world-health-org-metrics/compare/main...issues-by-status, but I think I'd like to pair on this to make sure my approach is ok/to make sure I'm understanding this correctly 😅

We are going to be closing this issue as out of scope for now.
We need true historical data in order to correctly calculate these metrics.


Bump OPF size to 10000

This is failing because of https://github.com/SciML/SciMLBenchmarks.jl/pull/735


Make queue first step of workflow

Building  on #316
This might help us out when we have the issue of two branches being updated at once, causing overlapping runs on a particular database.
The current failure message indicates that some work needs to be done on the CircleCI side. We might also need to add an API key to get their working, as per the docs

Orb<EMAIL_ADDRESS>not loaded. To use this orb, an organization admin must opt-in to using third party orbs in Organization Security settings.

I think I have the permissions to turn this on, but I'd want to double check first.
Alternative thought:

Make the schema dbt_utils_integration_tests_redshift_{{ env_var('CIRCLE_PR_NUMBER') }}
Drop the schema as the final step of run_test.sh. (Or: Every few months I log into each database and drop a ton of schemata.)

Orb<EMAIL_ADDRESS>not loaded. To use this orb, an organization admin must opt-in to using third party orbs in Organization Security settings.

I think I have the permissions to turn this on, but I'd want to double check first.
Alternative thought:

Make the schema dbt_utils_integration_tests_redshift_{{ env_var('CIRCLE_PR_NUMBER') }}
Drop the schema as the final step of run_test.sh. (Or: Every few months I log into each database and drop a ton of schemata.)

I had that thought too, but I think if the orb works nicely, we should just use that. It feels a little less error-prone for some reason

I had that thought too, but I think if the orb works nicely, we should just use that. It feels a little less error-prone for some reason

Closing due to lack of interest :)


Add mwu's SSH key

@bors-servo r+

:pushpin: Commit fd8237d has been approved by larsbergstrom

:hourglass: Testing commit fd8237d with merge 2175aee...

:sunny: Test successful - travis


ODK 1.2.31 update and HPO check

fixes #3460

Updating to latest ODK (1.2.31)
Adding check for HP compatibility. This check only looks at HP axioms themselves and exclude external axioms, which make problems extremely unlikely. The check is also disconnected from the rest fo the checks, so while iitHub action QC may complain, the build wont fail because of it.

This introduces only the usual (non-functional) ODK improvements and that HPO compatibility check! Would be good to merge, because I am working on some other improvements as well :) Thanks

merged, branch deleted
On Mon, Dec 13, 2021 at 5:17 PM Nico Matentzoglu @.***>
wrote:

This introduces only the usual (non-functional) ODK improvements and that
HPO compatibility check! Would be good to merge, because I am working on
some other improvements as well :) Thanks
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/mgijax/mammalian-phenotype-ontology/pull/3483#issuecomment-992961103,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACTYTUKLYJDDKMXFLBGLIP3UQZWI5ANCNFSM5J3ADE5Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


"Code of conduct" section made interactive

When clicked on the "Code of Conduct" section, user will be redirected to "https://github.com/HimanshuNarware/CareerZunction_Intern/blob/e360e9bb3bb7a0fdc384506d0657fcd6a0fe7c30/CODE_OF_CONDUCT.md"
Pertinent to issue #420

Resolve Conflicts and what about MIT License one? @ShravaniPathak

Can you provide me with the link for the MIT License @PSS2134


Better display of Unknown Error

Ref https://github.com/fractal-analytics-platform/fractal-server/issues/842
The one in that issue is likely a KeyError, which should be part of the error message.

import traceback

class MyError(RuntimeError):
    pass

try:
    meta = {}
    meta["image"]
except Exception as e:
    print(traceback.format_exc())

try:
    raise MyError("Error message")
    meta["image"]
except Exception as e:
    print(traceback.format_exc())

writes
Traceback (most recent call last):
  File "/tmp/x.py", line 10, in <module>
    meta["image"]
KeyError: 'image'

Traceback (most recent call last):
  File "/tmp/x.py", line 16, in <module>
    raise MyError("Error message")
MyError: Error message

Ref 487fbb693cf67947a727ae2faa0630f76b8d6a95


DateTime picker for both date and time in a single component

Feature request type
New component
Component name
DateTimePicker
Is your feature request related to a problem?
In some cases, is useful to use a picker for date and time without having to set two independent components and then combine both values in a single DateTime.
This also gives the user the idea that those two values (date & time) are for the same concept:

date & time of birth
date & time of some event that happened

Describe the solution you'd like
The idea is to have a component that combines the selection of date and time in a single component.
This component should be possible to bind to a DateTime and have options to select date and time like this examples:

Have you seen this feature anywhere else?
In UI component libraries like Material UI.
Also, in base html using the input type="datetime-local"
Describe alternatives you've considered
Use both DatePicker and TimePicker and later combine both values in a single DateTime.
This requires some extra logic.
Pull Request

[X] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

This is duplicate of #1082
closing, you still can create a PR tho, just reference #1082 instead in the PR


COMP: Fix uninitialized vector warnings

Fix uninitialized vector warnings.
Fixes:
Modules/Numerics/Statistics/include/itkDistanceMetric.h:101:45:
warning: 'm' may be used uninitialized [-Wmaybe-uninitialized]

and
Modules/Numerics/Statistics/include/itkMeasurementVectorTraits.h:70:72:
warning: 'm' may be used uninitialized [-Wmaybe-uninitialized]

and
Modules/Numerics/Statistics/include/itkMembershipFunctionBase.h:97:45:
warning: 'm' may be used uninitialized [-Wmaybe-uninitialized]

and
Modules/Numerics/Statistics/include/itkSample.h:121:45:
warning: 'm' may be used uninitialized [-Wmaybe-uninitialized]

raised for example in:
https://open.cdash.org/viewBuildError.php?type=1&buildid=8240358
PR Checklist

[X] No API changes were made (or the changes have been approved)
[X] No major design changes were made (or the changes have been approved)

Converted to a draft, as I think I need some suggestion here to provide an appropriate fix.

Simply using a list-based initialization, as proposed in the first commit, yields type conversion narrowing warnings:
https://open.cdash.org/viewBuildError.php?buildid=8241864
The vectors in some of these templates can be of any type, including itk::Array, itk::FixedArray, itk::Point, itk::RGBPixel, itk::VariableLengthVector, itk::Vector, or even std::vector.

Not all of them share the same initialization methods, e.g. fill vs. Fill vs. MakeFilled).
Not all of them have the ValueType alias (e.g. to do MeasurementVectorType m{ static_cast<MeasurementVectorType::ValueType>(0.0) };).

So for itk::itkDistanceMetric.h and itk::MembershipFunctionBase I could get away with it using a loop as the length is a member variable; however, such is not the case for itk::MeasurementVectorTraits or itk::Sample, where the vector can contain other more exotic types.
Not sure whether attempting to cast to all of the possible types in itk::MeasurementVectorTraits::IsResizableto then initialize the vector properly is possible/the only way here, or how types can be handled in the case ofitk::Sample`.
@N-Dekker, @dzenanz, @thewtex, @blowekamp I would be very grateful if you could chime in here :-)

@jhlegarreta I found a somewhat similar pull request of yours  😃: PR #2393 commit 7548a390d71dc25b80e792703ee3cfea17853a99 "COMP: Fix missing initializer warning", March 2021. At that time, the warnings were solved by value-initialization, using an empty pair parentheses ().  Similarly, you could also do:
const auto m = MeasurementVectorType();

For the record, the warning appears to be introduced with GCC 11. Minimal example: https://godbolt.org/z/nsbGd9TnM
struct MyVector
{
  int data[42];
};

bool
IsResizable(const MyVector &)
{
  return false;
}

int
main()
{
  MyVector v;
  return IsResizable(v) ? 1 : 0;
}

@jhlegarreta I found a somewhat similar pull request of yours smiley: PR https://github.com/InsightSoftwareConsortium/ITK/pull/2393 commit https://github.com/InsightSoftwareConsortium/ITK/commit/7548a390d71dc25b80e792703ee3cfea17853a99 "COMP: Fix missing initializer warning", March 2021.

I see that the fix also came after a few comments and unsuccessful attempts, and a suggestion of yours :-).
The loop-based change resulted here in segfaults, probably because the size member variable having a value that does not apply to the local variable :expressionless:.
I prefer not to introduce the const style change here. Let's see if this makes the CIs happy and fixes the warnings in the dashboard first. The style change may then be applied in another PR.

I believe the compiler should not have produced those warnings in the first place. I consider it a compiler bug. My example at #3707 (comment) obviously does not use an uninitialized value, and still GCC 11 produces the warning.
I guess the next generation compilers might produce a warning saying that the initializations that are added by this PR are unnecessary! smiley_cat
But for now, I understand that we need to address today's compiler warnings, so OK, approved +1

@N-Dekker Thank you! BTW, gcc 12.2 does the same. Probably the reason is that the struct is passed  to the IsResizable function and compiler doesn't check what happens there and gives -Wmaybe-uninitialized. There is also -Wuninitialized, could be triggered e.g.
#include <iostream>
struct MyVector
{
  int data[42];
};

int main(int, char **)
{
  MyVector v;
  int x = v.data[5] + 1;
  std::cout << x << std::endl;
  return 0;
}

Strange thing is that for the above example -Wall is required, IMHO, there is serious error here with an undefined result.
Unfortunately gcc 12.2 brings many new warnings to ITK build.

Another finding, for completeness, if MyVector is passed without const there is no warning
struct MyVector
{
  int data[42];
};

bool
IsResizable(MyVector &)
{
  return false;
}

int
main()
{
  MyVector v;
  return IsResizable(v) ? 1 : 0;
}

if MyVector is passed without const there is no warning

Thanks Mihail, but that makes (some) sense... at least, it would make sense if the function definition of IsResizable(MyVector &) would be elsewhere (e.g., hidden away in a separate library).
A non-const parameter might still be assigned to inside the function (IsResizable, in our example), before using its value. So then the warning that it "may be used uninitialized" would be less relevant.


use jinja with extensio on sliders

Try to use jinja with extension to use the slider as an extra html file where you pass the disired parameters

https://stackoverflow.com/questions/9404990/how-to-pass-selected-named-arguments-to-jinja2s-include-context

https://jinja.palletsprojects.com/en/3.1.x/extensions/#with-extension

Changed in version 2.9: This extension is now built-in and no longer does anything.
maybe not neccesary to install extension


passive watcher esm module

unbuild uses
await writeFile(output + '.mjs', `${shebang}import jiti from 'jiti';\nexport default jiti(null, { interopDefault: true })('${entry.input}');`)

but this results in
import jiti from 'jiti';
export default jiti(null, { interopDefault: true })('../src/index.ts');

which only do default export for esm modules (not named ones) i tried to figure it out how solve this but seems is not possible what you thing

Hi @mdbetancourt. Thanks for feedback. Actually jiti preserves default + named exports but we cannot just imported them as named in stub mode. I guess we can improve this by emulating named exports using static analyze with mlly.

Also facing the same 😞

facing this issue
Are they installed?
Failed to resolve import<EMAIL_ADDRESS>from "../dist/index.mjs". Does the file exist?

Used antfu's starter

same

same

I think what I am seeing is this same issue, originally though it was a jiti problem, but moving details here. I am not sure stubs can currently be used with ESM as trying to use named exports doesn't work for stubs but using the default export doesn't work when built.

clone https://github.com/nuxt/framework
run npx yarn install
run npm yarn stub
create test.mjs at the root with contents

import { findPath } from "@nuxt/kit";

console.log(findPath);

run node index.mjs

you will see the error SyntaxError: The requested module '@nuxt/kit' does not provide an export named 'findPath'
Then run npm run build and node index.mjs again you will see findPath resolves fine.

some :/ "dev": "unbuild --stub"

I am also experiencing the issue, "Failed to resolve import" when using the stubbed project from a vite project, that depends on the stubbed project through a "link:..." dependency. It seems like vite is not able to resolve the file protocol import of jiti. Is there any new information on this, I am also using the ts-starter from antfu.
Btw. the same issue also appears in larger projects like https://github.com/harlan-zw/unlighthouse for me, so maybe there is just a configuration problem on our end?

also facing the same issue

I'll just chime in here to point out that @pi0 has since implemented the following:

I guess we can improve this by emulating named exports using static analyze with mlly. (Generating named exports in stub)

So, the jiti ESM stub does provide named exports now. For any of you experiencing related problems, it's worth first making sure you're on the latest version of unbuild first.
Now, with that said, unfortunately, I think we will continue to run into issues with the stubs with different bundlers, etc. because we can't necessarily rely on the stub executing in the context of Node.js. You can see similar issues described here.
I love the stub approach over a watch process (especially in monorepos) and have been relying on something similar from Preconstruct for a while now. Unfortunately, I'm just not sure it's possible to reproduce that functionality for ESM bundles when they are consumed by Next.js or Nuxt, etc.
Preconstruct's CJS stub works with Next.js because it doesn't import any Node-specific packages (fs, module, etc.). jiti would have to be re-worked significantly to add an option to produce stubs that don't rely on those modules. Even then, I think Preconsctruct's CJS stub only works on account of the require hook (from pirates, which jiti also uses) — but I don't think we can ever get an ESM stub working that way with the bundlers that power Next.js and Nuxt (because require interop in ESM requires createRequire from the module Node.js module). Maaaybe if the stub leveraged an ESM loader instead of the require hook it could work in some contexts, but I'm not sure.

I have to agree; this is an issue.  We need the standard watch option here.  I'm trying to replace tsc with unbuild for the vue support it adds, but without a watch feature, its making dev rather challenging.

Let's track active watcher via https://github.com/unjs/unbuild/issues/281


Do_predict does not make predictions for all instances in the test.tsv

Hi. I am running below command to make predictions using my trained BERT binary classifier. The code executed without any error. However, when I look into the test_results.tsv file, the number of predictions are less than the total number of sentences in test.tsv. To be exact, I have 1110 sentences in the test.tsv and only 1109 in test_results.tsv. I do not know which sentence is missing. So, how can I find out which prediction is for which sentence?
python bert/run_classifier.py  --task_name=mrpc  --do_predict=true  --data_dir=./data  --vocab_file=$BERT_BASE_DIR/vocab.txt  --bert_config_file=$BERT_BASE_DIR/bert_config.json  --init_checkpoint=$TRAINED_CLASSIFIER  --max_seq_length=128  --output_dir=./bert_output/

Hi @Mahhos : Does the tsv file have headers as the first line? If so, essentially, you'll get 1109 predictions excluding the headers.
If that's not the case, try adding an id column to the tsv and match it with the predictions so that you'll find the missing sentence.

Hi @AbinayaM02 and @Mahhos, have you found any reason for this issue? And do we know which prediction is missing - first row, middle or the last row from test set? Thanks.


Add footer with the link to this repository

Add footer with the link to this repository.

Implemented in #15.


NoMethodError when using route helpers

Hi,
Im getting a NoMethodError: undefined method `[]' for nil:NilClass
in rspec-testing when using path helpers. I have included the controller via the controller-method, and tried setting the type to :cell, but it doesn't help.
Using Rails 5.1 and Slim.

I have the same issue

@erikaxel
Workaround for the moment is
let(:controller) { double('controller', url_options: {}) }

I found that it depends on the order you put cells-rails and rspec-cells into the Gemfile.
If cells-rails goes first then it works, if it's after rspec-cells then it doesn't
Which implies that rspec-cells has an implied dependency on cells-rails.

The order in the Gemfile matters? :fearful:


[FEATURE]: Improve karavi-metrics logs to display the number of pulled metrics

Describe the solution you'd like
Current logs give time to collect and push data :
2021/01/12 10:47:04 Looking up system ID 31ba0b0125517f0f Name powerflex01
2021/01/12 10:47:04 gatherMetrics took 2.25µs
2021/01/12 10:47:04 pushMetrics took 812ns
2021/01/12 10:47:04 GetSDCStatistics took 22.456348ms
2021/01/12 10:47:04 gatherPoolStatistics took 3.107µs
2021/01/12 10:47:04 pushPoolStatistics took 1.026µs
2021/01/12 10:47:04 gatherPoolStatistics took 1.742µs
2021/01/12 10:47:04 pushPoolStatistics took 729ns
2021/01/12 10:47:04 GetStoragePoolStatistics took 49.552945ms

While debugging an issue with the otel-collector, I was unsure of the actual metrics pushed.
Is it possible to add to this log the number of pulled/pushed metrics. For example:
2021/01/12 10:47:04 gatherMetrics XXX took 2.25µs

where XXX is the number of metrics.

Keeping as a low priority and will address in a future release.

Adding the "help wanted" label to solicit help from the community to help with contributions for this feature.


global javascript variable pass from php

why code broken syntastic ?
<?php $qwe = "qwe";?>
<script>
var qwe = <?php echo $qwe;?>;
</script>

but if in function insert , than it work norm
<?php $qwe = "qwe"; ?>
<script>
function qwe()
{
var qwe = <?php echo $qwe;?>;
}
</script>

Perhaps ask on a PHP forum?

sorry
this problem in "vim-javascript" plugin
pls delete issue


Running with Holographic remoting

Thanks for sharing this project amazing work ! i was wondering if you have any idea if it is possible to combine this project with holographic custom sample https://github.com/microsoft/MixedReality-HolographicRemoting-Samples/tree/main/player/sample  in order to run the app using holographic remoting

Thank you for the kind words :)
Yes I do believe this is possible, but I do not know how much engineering effort would be required and do not have the bandwidth currently to attempt building this.

Thank you for your answer i will give it a try and let you know if i manage  to make it work


feat(cmd): create ggshield sca scan ci command

Create the ggshield sca scan ci command.
Some code was refactored in order to allow the reuse in sca scan ci of functions used for secret scan ci.
Seeing as there are now 3 sca commands, the structure of cmd/sca was changed to split commands into differents files, as it is done in cmd/iac.
Issue: SCA-853

Only some details, we're almost done !


get patient doesn't create json dictionary

*directory not dictionary. the json folder needs to be created by the script.

was already fixed before b4ed3b429b1e58341afbb36e6a6e0e171b02a0fd


Make storage names for OAuth ticket catalog objects configurable

Back-port of #5274

@apereocas-bot runci


Travis integration

Tests are ready to run: ./bin/phpunit

https://travis-ci.com/M6Web/Tornado
Done in https://github.com/M6Web/Tornado/commit/d40449ce9eec82ec2589252e537cf8da698e09b5


Fix the swap_refcnt test on linux.

Fix the swap_refcnt test on linux.
The problem here is that we were performing a naive negative FileCheck test for
retain/release. In certain modes, we would not have any retains/releases along
normal control paths but would have retains on unreachable paths. This test only
is trying to test if normal code paths have this issue.
To work around this issue, I created a small utility pass that prunes all
non-unreachable instructions from blocks with an unreachable terminator. This is
useful functionality in general when analyzing SIL since often times one will
have large fatal error blocks that disguise the true behavior of the
function. In this specific case, I just pipe in the normal sil output and run it
through sil-opt. sil-opt then runs just the utility pass and I then FileCheck
that sil-opt output.
rdar://30181104

@swift-ci smoke test and merge

Mmm... just noticed that CI is down. Sorry for the noise! (Watching mailing list).

The CI is alive!!!

Took a quick look:
PASS: Swift(linux-x86_64) :: SILOptimizer/swap_refcnt.swift (7691 of 10050)


ListItemAttachments Control - fixing the OnClick handler when clicking on the document card

Q
A

Bug fix?
[x ]

New feature?
[ ]

New sample?
[ ]

Related issues?
fixes #1541

What's in this Pull Request?
Problem statement
In the earlier version, When trying to open a file on the ListItemAttachments control, the URL contains duplicate parameters between the absolute url and the serverRelativeURL as below
Result URL:
https://TENANT.sharepoint.com/sites/SITE/_layouts/15//sites/SITE/Lists/LISTNAME/Attachments/ID/FILENAME.msg?web=1
Solution
The problem is caused in the OnClick handler of DocumentCard, while parsing the URL using encodeURIComponent (As below)
 onClickHref={!openAttachmentsInNewWindow && `${encodeURIComponent(file.ServerRelativeUrl)}?web=1`}
onClick={openAttachmentsInNewWindow && (() => window.open(`${encodeURIComponent(file.ServerRelativeUrl)}?web=1`, "_blank"))}

Screen shot of the old code

By removing this, was able to make it work again.
onClickHref={!openAttachmentsInNewWindow && `${(file.ServerRelativeUrl)}?web=1`}
onClick={openAttachmentsInNewWindow && (() => window.open(`${(file.ServerRelativeUrl)}?web=1`, "_blank"))}

Screen shot of the corrected code

hence making the url be like https://TENANT.sharepoint.com/sites/site/Lists/LISTNAME/Attachments/ID/FILENAME.msg?web=1
Thanks,
Nishkalank

@NishkalankBezawada many thanks for the fix, I tested this and it was working well.
I have also tracked down when the encodeURIComponent was introduced and found this was related to #1484. I tested uploading a file with the # character in the name and the character was removed, and opening the file worked as expected. I have also tried adding that file as an attachment via SharePoint UI and there was an error message saying that the character was invalid. Seems to me that SharePoint internal validation is now taking care of the special character and this is no longer an issue, so your PR does not seem to introduce #1484 again. All good :)
(Never thought I had to test so many things for a PR that only removed a function to encode a string 😆)

Many thanks @joelfmrodrigues  for the merge and your insightful comments.
I also saw that SharePoint handles validation; therefore, I removed the encodeURIComponent from the code to test, and it worked. I hope this is a long-lasting solution.
Thanks,
Nishkalank


RANGER-5021: Add timeouts to jobs in CI

What changes were proposed in this pull request?
Add timeouts to jobs in CI
How was this patch tested?
CI Run 2 successful with timeout of 15 mins.

@kumaab  - would this change result in CI failure when build doesn't complete in 30 minutes?

@mneethiraj
Yes, that's right. Should we increase the time across different stages?

@mneethiraj Yes, that's right. Should we increase the time across different stages?

I suggest increasing overall timeout to 1 hour, to avoid unnecessary failures due to any slowness in build and testing.

@adoroszlai
Could you please review this PR, thanks!

Yes, with multi-threaded builds build-8 and build-11, runs generally last around 20 mins but sometimes it may run longer may be due to m2 cache not being present and time it takes to populate m2. docker-build is faster with caches in place.
CC: @mneethiraj

Thanks @kumaab, I didn't notice cache miss.  That should explain it.


Are there any way to "export" the trained parameters and their final performance?

Dear  Zhang ,Crysple and other developers:
Thank you for for your great work! The NNI is such an awesome system that all my friends using it!
But I think it could be more powerful if it can show the "hyper-parameters surface" which it had run .
Here is a relevant project: https://github.com/lucfra/FAR-HO The heat map is cool, but a 3D version would be even better.
But it might be impossible to draw the picture, as the tuner doesn't traverse a path for optimization for all parameters, but it would be helpful to have a "one-button" way to export trained parameters and their final performance in form of text or .csv files.

@HudsonHuang Thanks for using NNI. We will support to export trained parameters and their final performance in form of json file.

We support to download parameters in UI in the recent v0.4 release, please try it out.


Privacy Manifests

https://developer.apple.com/support/third-party-SDK-requirements
There is 'Toast' in the library list that Apple should provide Privacy Manifests. Is this library correct??

+1 see

Just checked the repo, I suppose this kind of UI component library won't need a privacy manifest, and I could not find any usage of the required reasons API.

Sorry for the delay on this. I'll get this updated in the coming days.

Sorry for the delay on this. I'll get this updated in the coming days.

Thanks a lot, @scalessec!
Sidetrack:
Just wonder if this thing is actually needed, as I can't find any usage of the APIs.

Sorry for the delay on this. I'll get this updated in the coming days.

Thanks a lot, @scalessec!
Sidetrack: Just wonder if this thing is actually needed, as I can't find any usage of the APIs in the codebase.

Wouldn't it be necessary to prove that there isn't?

Yeah, better to just have it even if there are no offending APIs.

Yeah, better to just have it even if there are no offending APIs.

Thank you for taking care of me even though you are busy.
Happy New Year.

I pushed a first pass at this (and fixed all unrelated warnings), but need a bit more time for regression testing. I'll cut a new release once that's done.

@scalessec If any help/assistance is needed, please feel free to tag me, currently available this week and early next week.

I just pushed 5.1.0. I'd appreciate if you could give it a test drive if you have a moment. I confirmed the new PrivacyInfo.xcprivacy resource is copied into the target when using SPM, CocoaPods, and Carthage. One thing to note—when I integrated the lib into a new app, archived, and then attempted to Generate Privacy Report, it generated an empty PDF. I tried with multiple 3rd party libraries (not just Toast-Swift), and always had the same result. I'm not sure what's going on with that, but as far as I can tell this should meet Apple's requirement. Please let me know if you find otherwise. Thanks again for opening this issue.

I just pushed 5.1.0. I'd appreciate if you could give it a test drive if you have a moment. I confirmed the new PrivacyInfo.xcprivacy resource is copied into the target when using SPM, CocoaPods, and Carthage. One thing to note—when I integrated the lib into a new app, archived, and then attempted to Generate Privacy Report, it generated an empty PDF. I tried with multiple 3rd party libraries (not just Toast-Swift), and always had the same result. I'm not sure what's going on with that, but as far as I can tell this should meet Apple's requirement. Please let me know if you find otherwise. Thanks again for opening this issue.

The issue might be the problem that we don't have any declared reasons?
Not sure the 3rd party libraries you used. But agree with you that this should fulfill Apple's requirement.
Will try to test it tomorrow, thanks!

Update: It works fine on my applications, but might require more results from others.

Update: It works fine on my applications, but might require more results from others.

@tommyming
I'm in the same situation as @scalessec
I also tried 'Generation Privacy Report' after Arcive.
But the PDF didn't show anything.
I installed more libraries with Privacy Manifest added for testing.
As I've been testing so far, all of them were not displayed in PDF except for the 'ZIPFoundation' library.
For the 'SDWebImage' library, even though I set it up in PrivacyInfo to use 'File Timestamp', it does not appear in the PDF.
I used SPM.
How should Privacy Manifest look if it works properly??

Not sure in this case.
Since Apple does not provide any sample of a successful report/failure report, I performed the below testing.
I tried to add only Toast-Swift, and included it with some privacy usage in the app's plist, but the generated report PDF is also empty.
So my assumption here is: If the report is empty, then it means successful?
To reproduce a failure case, I tried to install AWS Amplify, which does not declare data collection types in the privacy manifest, and the report will show the error correctly.
In this case, my assumption is there will be errors shown in the report if you really missed something or set it up wrong.
I haven't tried on the additional data case, so please feel free to provide more info.

Thanks for your help, @tommyming @wlxo0401 !

Some updates about the privacy manifest:
https://developer.apple.com/news/?id=3d8a9yyh
Seems the App Store will start to warn developers if there are any issues about the privacy manifest in the app.
If there are any issues about this repo, I think people will start to report.

See this for resolution:
https://apnspush.com/add-privacy-manifest-sdk
For cocoapods, you should put privacy manifest file into a resource bundle, but not spec.resources item:
spec.resource_bundles = {'Toast-Swfit' => ['resource/*.xcprivacy']}
the reason at the bottom of this page:
https://developer.apple.com/forums/thread/733537

@freefa thanks for the info.
Maybe someone who imports toast using cocoapods can try to test on this approach? Thanks.

@tommyming I've test it and create a merge request  https://github.com/scalessec/Toast-Swift/pull/215

@scalessec Some updates about the privacy manifest:
Seems Apple doesn't recommend an empty one, as I go through the discussion of RxSwift about the privacy manifest.
https://github.com/ReactiveX/RxSwift/issues/2567 (Yeah an extremely long discussion)
Do you have any recommendations on this? I am not sure in this case, since Apple's guidelines are unclear...

Thank for the updates, I'll do my best to get this resolved this weekend.


Change adapter default ports

FEATURE REQUEST
All adapters default ports are 8180 for http and 8181 for gRPC.
If someone tries to run couple of those adapters locally on his machine, with default settings, there is a port clash. It would be nice to have default ports different, so it would work without any need to set those ports.
Another thing that would be nice is to have ports split by protocol. Something like the example below:
7000 - users grpc
7001 - things grpc
8000 - ws adapter
9000 - users http
9001 - things http
9002 - http adapter
Please note that this change would affect docker compose configuration as well as run.sh script

We should also document this change and announce it loud as we would probably brake something on the users side


Viosock : recv stuck in client, while the socket is being SHUTDOWN from host

Describe the bug
on client side, stuck while call the recv, even using the latest driver code
To Reproduce
Run attachment application as below description
server.txt
mv server.txt server.c and compile it to run on linux host
run server.c  on the Linux host
client.txt
mv client.txt client.cpp and compile it to run in windows guest
Expected behavior
on client side, recv can return after the host SHUTDOWN the connection
Screenshots
on server ,  the socket is being close

on client windows,  the recv stuck:

Host:
seems no related to the host and qemu version and command line options etc.

Disto:  ubuntu

Kernel 6.2.0-31-generic

QEMU   emulator version 7.0.50

QEMU

qemu-system-x86_64
...
-device vhost-vsock-pci,guest-cid=3
...

VM:

Windows  win10 18363.592
vsock
the latest vsock driver

the problem i found when i comment out the server.txt line 192, which sleep 10 second after close the socket
what if i comment out the line 193, which do a write on host side, the recv stuck issue on client side is disappear

Hello,
can you provide a kernel memory dump for the WIndows VM?

Hello,
can you provide a kernel memory dump for the WIndows VM?

I config the kernel memory dump in windows, but it seems there is no system crash or a  BSOD  which generate the dump file?

Hello,
can you provide a kernel memory dump for the WIndows VM?

I config the kernel memory dump in windows, but it seems there is no system crash or a BSOD which generate the dump file?

With a bit more config, you should be able to initiate the system crash (thus, generate a kernel memory dump) manually. I think the Keyboard method should be best for your case.
https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/forcing-a-system-crash
Also, can you point me to the version of the Virtio drivers you are using (especially the viosock.sys)? If you compiled them yourself, I would be glad to get viosock.sys and viosock.pdb (the debug symbols for the driver) if you have these files.
Thanks.

Hello,
can you provide a kernel memory dump for the WIndows VM?

I config the kernel memory dump in windows, but it seems there is no system crash or a BSOD which generate the dump file?

With a bit more config, you should be able to initiate the system crash (thus, generate a kernel memory dump) manually. I think the Keyboard method should be best for your case. https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/forcing-a-system-crash
Also, can you point me to the version of the Virtio drivers you are using (especially the viosock.sys)? If you compiled them yourself, I would be glad to get viosock.sys and viosock.pdb (the debug symbols for the driver) if you have these files.
Thanks.

I test the latest version driver on the github, which seems the bug can be reproduce too
I will try to get a kernel memory dump later

dump00.zip
dump01.zip
dump02.zip
dump03.zip
the dump00-04 contain the windows vm kernel memory dump file, and viosock.sys, viosock.pdb file i use
i reproduce the test flow and trigger the windows to dump the memory

@MartinDrab  any progress for the issue now? or is there any info that i need to provide?
BTW i checkout the code in Socket.c  as below, which comment out some lines
which i guess if it relate to this kind of issue?
when the guest recv the SHUTDOWN form peer, and by then, guest close the recv end by force?
VIOSockShutdownFromPeer(
    PSOCKET_CONTEXT pSocket,
    ULONG uFlags
)
{
....
//     if (uDrain & VIRTIO_VSOCK_SHUTDOWN_SEND)
//         VIOSockReadDequeueCb(pSocket, WDF_NO_HANDLE);
//     if (uDrain & VIRTIO_VSOCK_SHUTDOWN_RCV)
//     {
//         //TODO: dequeue requests for current socket only
//     }

Then I try to uncomment the line above, and do a WdfRequestComplete(ReadRequest, STATUS_CANCELLED); in function VIOSockReadDequeueCb, but i currently cannot figure out where should i do this. or is it doable like this?

DMP and PDB files should be enough for me. I will look into it tomorrow and let you know.

Hello,
I reproduced the issue also with both server and the client on one (Windows) VM. Looking at the code, the shutdown signal is sent during socket close, however, the recv request seems not to be completed. I will look into it deeper.

You may try #1036, that should fix your issue.


PKS v1.1 - Configure Ops Man Document - Create Network Step

Document URL: https://docs.pivotal.io/runtimes/pks/1-1/gcp-om-config.html
PKS Version: 1.1
Problem Section: Step 5: Create Networks Page
Problem Description: The Add Network section of the tile also has a subnet section. The document does not mention anything about it, which is confusing. Also, the subnet section has availability zones to be selected with checkboxes for all of them. The document also does not have any reference to it. Please add these details as it is really confusing in its current format.

Thanks @parth-pandit. The PKS v1.1 docs are no longer live, but I believe this issue has been addressed in the PKS v1.2+ docs. For example, https://docs.pivotal.io/pcf/om/2-3/gcp/config-manual.html. Closing.


Can't retrieve fromSenderDomain using Bicep for Azure Email Communication Service

Is your feature request related to a problem? Please describe.
To send an email using the Azure Email Communication Service, a sender address is required. Currently it can only be retrieved manually through azure portal. This is currently affecting our automated DevOps provisioning & deployment pipelines as there's no way to fetch it using bicep. This introduces a lot of manual work to manually add them in the KeyVault.
Describe the solution you'd like
Expose the fromSenderDomain property in bicep template which is available in Azure Portal

resource emailServiceAzureDomain 'Microsoft.Communication/emailServices/domains@2023-03-31' existing = {
  parent: emailService
  name: 'AzureManagedDomain'
}
output fromSenderDomain = emailServiceAzureDomain.properties.fromSenderDomain

But, if there's already a way, please let me know.
Cheers!

Is the fromSenderDomain something that can be looked up using the az CLI or the Az powershell module? If so, a deployment script may be an option.

Is this extract from the Portal from "JSON View" on the resource? Does it specify an API version that might not be documented yet? (If so it might still work from Bicep even though the VS Code extension will issue a warning.)

Thanks @jeskew. You are right @ahelland. I'm able to retrieve it from the bicep itself. My confusion rose from the documentation and the VS Code warning. I wonder if the documentation could be fixed https://learn.microsoft.com/en-us/azure/templates/microsoft.communication/emailservices/domains?pivots=deployment-language-bicep


Interactive Tests via client sometimes hang after webdriverjs 2.47.1

Update to selenium-webdriver 2.47.1 occasionally causes elementexplorer to crash when tested with the client (e.g. interactive tests):
> node scripts/interactive_tests/interactive_test.js

/Users/ralphj/protractor/node_modules/q/q.js:126
                    throw e;
                    ^
Command <"browser.get(\"http://localhost:8081\")"> did not receive a response in 10s.

I am unable to reproduce when running with element explorer manually, so I do not believe this should be an issue for users.
@hankduan ccing in case you have any ideas.

Hey Julie.
My apologies to use this channel to contact you, but i don't find your e-mail.
I would love to contribute to Protractor. I'm an product designer and o think i can build an very nice new website for the Protractor. :)
What do you think? How could I help?
Thank you.  :)

Can you see if 2.48.x fixes it for you? I ran npm test a few times and never encountered this.

Yup, as far as I can tell it's fixed. Closing.


Whether it is possible to configure the duration of the alarm to the present in the alarm rules

是否可以实现在告警规则上配置发出告警到目前的持续时间，如何在rule文件配置去获取当前时间并减去触发时间
Whether it is possible to configure the duration of the alarm to the present in the alarm rules, and how to obtain the current time and subtract the trigger time in the rule file configuration

Thanks for your report. It looks as if this is actually a question about usage and not development.
It makes more sense to ask questions like this on the prometheus-users mailing list rather than in a GitHub issue. On the mailing list, more people are available to potentially respond to your question, and the whole community can benefit from the answers provided.


Not able to import modules

After successfully installing all the requirements, I am not able to run the script ./main.py:

Traceback (most recent call last):
File "./main.py", line 3, in
from youtube import Youtube
File "/home/fariss/smd/youtube.py", line 13, in
from moviepy.editor import *
File "/home/fariss/.local/lib/python3.6/site-packages/moviepy/editor.py", line 23, in
import imageio
File "/home/fariss/.local/lib/python3.6/site-packages/imageio/init.py", line 22, in
from .core import FormatManager, RETURN_BYTES
File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/init.py", line 10, in
from .util import Image, Array, Dict, asarray, image_as_uint, urlopen
File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/util.py", line 27, in
import numpy as np
File "/home/fariss/.local/lib/python3.6/site-packages/numpy/init.py", line 142, in
from . import core
File "/home/fariss/.local/lib/python3.6/site-packages/numpy/core/init.py", line 91, in
raise ImportError(msg.format(path))
ImportError: Something is wrong with the numpy installation. While importing we detected an older version of numpy in ['/home/fariss/.local/lib/python3.6/site-packages/numpy']. One method of fixing this is to repeatedly uninstall numpy until none is found, then reinstall this version.

I also don't know what packages to install manually.

Hi @kaletaa,
I did just that. It got rid of the numpy error. However, I am still not able to run the script, here's the output:
fariss@fariss:~$ ./main.py
Traceback (most recent call last):
  File "./main.py", line 3, in <module>
    from youtube import Youtube
  File "/home/fariss/smd/youtube.py", line 13, in <module>
    from moviepy.editor import *
  File "/home/fariss/.local/lib/python3.6/site-packages/moviepy/editor.py", line 26, in <module>
    imageio.plugins.ffmpeg.download()
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 79, in download
    get_remote_file(fname=fname, directory=directory, force_download=force_download)
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/fetching.py", line 130, in get_remote_file
    _fetch_file(url, filename)
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/fetching.py", line 189, in _fetch_file
    % os.path.basename(file_name)
OSError: Unable to download 'ffmpeg-linux64-v3.3.1'. Perhaps there is a no internet connection? If there is, please report this problem.

So, I tried installing 'ffmpeg-linux64-v3.3.1' manually, and I am getting this:
fariss@fariss:~$ python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import imageio
>>> imageio.plugins.ffmpeg.download()
Imageio: 'ffmpeg-linux64-v3.3.1' was not found on your computer; downloading it now.
Try 1. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 23339008/45929032 bytes (50.8%)Error while fetching file: The read operation timed out.
Try 2. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 7340032/45929032 bytes (16.0%)Error while fetching file: The read operation timed out.
Try 3. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 3907584/45929032 bytes (8.5%)Error while fetching file: The read operation timed out.
Try 4. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 1032192/45929032 bytes (2.2%)Error while fetching file: The read operation timed out.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 79, in download
    get_remote_file(fname=fname, directory=directory, force_download=force_download)
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/fetching.py", line 130, in get_remote_file
    _fetch_file(url, filename)
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/fetching.py", line 189, in _fetch_file
    % os.path.basename(file_name)
OSError: Unable to download 'ffmpeg-linux64-v3.3.1'. Perhaps there is a no internet connection? If there is, please report this problem.
>>>
>>> imageio.plugins.ffmpeg.download()
Imageio: 'ffmpeg-linux64-v3.3.1' was not found on your computer; downloading it now.
Try 1. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 25337856/45929032 bytes (55.2%)Error while fetching file: The read operation timed out.
Try 2. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 17154048/45929032 bytes (37.3%)Error while fetching file: The read operation timed out.
Try 3. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 32776192/45929032 bytes (71.4%)Error while fetching file: The read operation timed out.
Try 4. Download from https://github.com/imageio/imageio-binaries/raw/master/ffmpeg/ffmpeg-linux64-v3.3.1 (43.8 MB)
Downloading: 4505600/45929032 bytes (9.8%)Error while fetching file: The read operation timed out.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/plugins/ffmpeg.py", line 79, in download
    get_remote_file(fname=fname, directory=directory, force_download=force_download)
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/fetching.py", line 130, in get_remote_file
    _fetch_file(url, filename)
  File "/home/fariss/.local/lib/python3.6/site-packages/imageio/core/fetching.py", line 189, in _fetch_file
    % os.path.basename(file_name)
OSError: Unable to download 'ffmpeg-linux64-v3.3.1'. Perhaps there is a no internet connection? If there is, please report this problem.

It can't see the ffmpeg binary, yeah. I'm trying to fix this bug too, I've opened another issue in this repo about it ( #8 )

Problem fixed. Thanks.

Closed issue.