Memory leak?

Hi!
I observer that both of our running emqttd instances grow all the time. We compiled the broker ourselves as we have made small changes to the redis auth.
We compiled the commit 88edf1a02b946af70b8bf3c3ae29f08f6bef4b2a with erlang/otp 18.3. I read about a memleak in OTP 18.0 but the issue is fixed in 18.3.
These are the changes we made: https://github.com/Jaumo/emqttd_plugin_redis/commit/faa90c712ef35dca31d1101ad0e34479e363d0a1
I can't imagine that this is responsible for the growing memory usage, actually it is mostly copy+paste so that redis auth + client-id auth can work together.
This is our config:
https://gist.github.com/brstgt/90bfd75325e873fdab0da207ac837318
What else?
We use only qos0, no retained messages, only 3 or 4 persistent sessions, the rest is not persistent.
The connections/clients are NOT constantly growing. These are our normal client numbers, they don't vary too much: http://cl.ly/1k1D1g3N1q2w. I cant imagine this mem usage is normal for this "hand full" of clients: http://cl.ly/1x300j2U3Q3V.
This is what emqttd_ctl vm memory tells me:
memory/total            :<PHONE_NUMBER>
memory/processes        :<PHONE_NUMBER>
memory/processes_used   :<PHONE_NUMBER>
memory/system           : 372380400
memory/atom             : 654217
memory/atom_used        : 637305
memory/binary           : 143626640
memory/code             : 16070506
memory/ets              : 52078152
Need more information? I am happy to do everything I can :)

You can see mem comsumption over time: http://cl.ly/1t2b2w1d193y
The drops / holes are restarts or when node was not running

Hi @brstgt. Steps:

Check shutdown count of TCP listeners:

./bin/emqttd_ctl listeners

Metrics and Stats of the broker:

 ./bin/emqttd_ctl broker metrics
./bin/emqttd_ctl broker  stats

Load emqttd_recon plugin and check bin_leak:

./bin/emqttd_ctl plugins load emqttd_recon
./bin/emqttd_ctl recon bin_leak

Machine 1
./bin/emqttd_ctl listeners
listener on http:8084
acceptors       : 4
max_clients     : 16384
current_clients : 9218
shutdown_count  : [{destroy,1362607},{parser_error,3},{keepalive_timeout,8446}]
listener on mqtts:8883
acceptors       : 64
max_clients     : 512000
current_clients : 5432
shutdown_count  : [{idle_timeout,4102},{destroy,286804},{einval,7},{keepalive_timeout,1242576},{closed,2317122},{protocol_bad_connect,6}]
listener on mqtt:1883
acceptors       : 64
max_clients     : 512000
current_clients : 5
shutdown_count  : [{destroy,2},{keepalive_timeout,5284},{closed,7}]
listener on http:18083
acceptors       : 4
max_clients     : 512
current_clients : 4
shutdown_count  : []
./bin/emqttd_ctl broker metrics
bytes/received          :<PHONE_NUMBER>0
bytes/sent              :<PHONE_NUMBER>38
messages/dropped        : 82
messages/qos0/received  : 151496137
messages/qos0/sent      : 50125251
messages/qos1/received  : 0
messages/qos1/sent      : 0
messages/qos2/received  : 0
messages/qos2/sent      : 0
messages/received       : 151496137
messages/retained       : 3
messages/sent           : 50125251
packets/connack         : 141227451
packets/connect         : 139771330
packets/disconnect      : 135977286
packets/pingreq         : 28897727
packets/pingresp        : 30987307
packets/puback/received : 0
packets/puback/sent     : 0
packets/pubcomp/received: 0
packets/pubcomp/sent    : 0
packets/publish/received: 151496137
packets/publish/sent    : 50125251
packets/pubrec/received : 0
packets/pubrec/sent     : 0
packets/pubrel/received : 0
packets/pubrel/sent     : 0
packets/received        : 456147752
packets/sent            : 222345281
packets/suback          : 5272
packets/subscribe       : 5272
packets/unsuback        : 0
packets/unsubscribe     : 0
./bin/emqttd_ctl broker stats
clients/count       : 5590
clients/max         : 9132
retained/count      : 3
retained/max        : 3
routes/count        : 5570
routes/max          : 9075
sessions/count      : 4
sessions/max        : 5
subscribers/count   : 5587
subscribers/max     : 9117
subscriptions/count : 5586
subscriptions/max   : 9115
topics/count        : 5620
topics/max          : 9132
./bin/emqttd_ctl recon bin_leak
{<10038.263.0>,-186,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.265.0>,-171,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.268.0>,-167,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.270.0>,-155,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.31489.6776>,-130,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.22032.2681>,-122,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.294.0>,-113,
[emqttd_sm_16,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.402.2720>,-109,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.19137.2627>,-106,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.275.0>,-103,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.290.0>,-103,
[emqttd_sm_12,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.29105.2680>,-96,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.264.0>,-95,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.288.0>,-92,
[emqttd_sm_10,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.279.0>,-90,
[emqttd_sm_1,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.10164.2755>,-90,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.11918.2742>,-88,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.24457.2599>,-87,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.12523.2711>,-87,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.31602.2133>,-86,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.21138.2389>,-83,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.3353.2674>,-83,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.25358.2749>,-83,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.274.0>,-82,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.14298.2672>,-80,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.11634.2046>,-79,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.7721.2755>,-77,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.7332.2680>,-76,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.25898.2720>,-76,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29385.2705>,-75,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.21358.2762>,-75,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.15449.2691>,-74,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.281.0>,-73,
[emqttd_sm_3,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.6246.0>,-68,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.4533.1825>,-68,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.25933.2735>,-67,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.287.0>,-66,
[emqttd_sm_9,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.8410.2141>,-66,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.15195.2534>,-66,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.15869.2688>,-65,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.293.0>,-64,
[emqttd_sm_15,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.17070.2537>,-64,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.4371.2708>,-64,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.6444.2727>,-64,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.20508.2672>,-62,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5847.2686>,-62,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29369.2659>,-61,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.31537.2707>,-61,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.22717.2714>,-61,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.10747.2733>,-61,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.266.0>,-60,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.22450.2293>,-60,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.8114.2521>,-60,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.261.0>,-59,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.24024.2485>,-59,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.7268.2703>,-59,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.10060.2762>,-59,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.16594.2141>,-58,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.14301.2720>,-58,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.18841.2748>,-58,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.284.0>,-56,
[emqttd_sm_6,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.12867.2676>,-56,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.16788.2710>,-56,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.26171.2353>,-55,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.8403.2615>,-55,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29651.2702>,-55,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.282.0>,-54,
[emqttd_sm_4,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.285.0>,-54,
[emqttd_sm_7,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.23722.2515>,-54,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.283.0>,-53,
[emqttd_sm_5,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.291.0>,-53,
[emqttd_sm_13,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.26909.2750>,-53,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.273.0>,-52,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.8959.2241>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.20714.2483>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5787.2616>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.26923.2631>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.26704.2681>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.28799.2728>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.18572.1506>,-51,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.16498.2621>,-51,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.2981.2694>,-51,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.289.0>,-50,
[emqttd_sm_11,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.6362.2276>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1681.2498>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.22832.2559>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.8549.2583>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5078.2654>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.14780.2672>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.12974.2681>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.26899.2723>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.17298.2586>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.292.0>,-48,
[emqttd_sm_14,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.19531.2087>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.14012.2450>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29476.2601>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.16094.2653>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.4321.2656>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29348.2701>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.6691.2767>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
/bin/emqttd_ctl vm memory
memory/total            :<PHONE_NUMBER>
memory/processes        :<PHONE_NUMBER>
memory/processes_used   :<PHONE_NUMBER>
memory/system           : 363574328
memory/atom             : 654217
memory/atom_used        : 637932
memory/binary           : 120219608
memory/code             : 16084429
memory/ets              : 51237696
Machine 2
./bin/emqttd_ctl listeners
listener on http:8084
acceptors       : 4
max_clients     : 16384
current_clients : 10725
shutdown_count  : [{destroy,1256045},{parser_error,8},{keepalive_timeout,9430}]
listener on mqtts:8883
acceptors       : 64
max_clients     : 512000
current_clients : 5386
shutdown_count  : [{idle_timeout,3959},{destroy,264721},{einval,10},{keepalive_timeout,1220077},{closed,2266196}]
listener on mqtt:1883
acceptors       : 64
max_clients     : 512000
current_clients : 6
shutdown_count  : [{keepalive_timeout,5284},{closed,7}]
listener on http:18083
acceptors       : 4
max_clients     : 512
current_clients : 3
shutdown_count  : []
./bin/emqttd_ctl broker metrics
bytes/received          :<PHONE_NUMBER>7
bytes/sent              :<PHONE_NUMBER>61
messages/dropped        : 79
messages/qos0/received  : 151479440
messages/qos0/sent      : 49597967
messages/qos1/received  : 0
messages/qos1/sent      : 0
messages/qos2/received  : 0
messages/qos2/sent      : 0
messages/received       : 151479440
messages/retained       : 3
messages/sent           : 49597967
packets/connack         : 140991978
packets/connect         : 139646022
packets/disconnect      : 135902177
packets/pingreq         : 28763075
packets/pingresp        : 30780795
packets/puback/received : 0
packets/puback/sent     : 0
packets/pubcomp/received: 0
packets/pubcomp/sent    : 0
packets/publish/received: 151479440
packets/publish/sent    : 49597967
packets/pubrec/received : 0
packets/pubrec/sent     : 0
packets/pubrel/received : 0
packets/pubrel/sent     : 0
packets/received        : 455795990
packets/sent            : 221376016
packets/suback          : 5276
packets/subscribe       : 5276
packets/unsuback        : 0
packets/unsubscribe     : 0
./bin/emqttd_ctl broker stats
clients/count       : 5532
clients/max         : 9130
retained/count      : 3
retained/max        : 3
routes/count        : 5514
routes/max          : 9069
sessions/count      : 4
sessions/max        : 5
subscribers/count   : 5530
subscribers/max     : 9109
subscriptions/count : 5530
subscriptions/max   : 9109
topics/count        : 5572
topics/max          : 9125
./bin/emqttd_ctl recon bin_leak
{<10038.57.0>,-288,
[ssl_manager,
{current_function,{gen_server,loop,6}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.270.0>,-160,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.261.0>,-152,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.271.0>,-152,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.268.0>,-145,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.269.0>,-135,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.273.0>,-133,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.267.0>,-131,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.272.0>,-121,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.259.0>,-118,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.3640.2651>,-117,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.8341.2642>,-115,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.288.0>,-113,
[emqttd_sm_12,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.6502.2689>,-102,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1885.2722>,-101,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29718.2704>,-100,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.292.0>,-95,
[emqttd_sm_16,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.2235.2526>,-94,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.17690.2527>,-94,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.31198.2681>,-94,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.11455.2714>,-94,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.17872.2350>,-93,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.7080.2649>,-93,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.7418.2695>,-92,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.10320.2661>,-90,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.167.0>,-89,
[{current_function,{disk_log,loop,1}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.22486.1009>,-89,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.281.0>,-88,
[emqttd_sm_5,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.16334.2685>,-88,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.287.0>,-86,
[emqttd_sm_11,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.262.0>,-82,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.13476.2619>,-82,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.12028.2694>,-82,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.23444.2700>,-82,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29311.2585>,-80,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.15480.2672>,-80,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.280.0>,-78,
[emqttd_sm_4,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.13298.2628>,-77,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.291.0>,-76,
[emqttd_sm_15,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.3773.2669>,-76,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.12746.2670>,-74,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.18283.2571>,-73,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.7975.2673>,-70,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.265.0>,-69,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.8382.2141>,-69,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.289.0>,-68,
[emqttd_sm_13,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.3035.2607>,-68,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1225.2641>,-68,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.258.2679>,-68,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.284.0>,-67,
[emqttd_sm_8,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.29081.2710>,-65,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.30039.2726>,-65,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.266.0>,-63,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.286.0>,-63,
[emqttd_sm_10,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.26098.2535>,-62,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5887.2679>,-62,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.12205.2714>,-62,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.14834.2612>,-61,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.20791.2712>,-61,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.17441.2650>,-60,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.22976.771>,-59,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.2740.2472>,-59,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.4747.2328>,-58,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1940.2584>,-58,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.29046.2646>,-58,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.285.0>,-57,
[emqttd_sm_9,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.7180.2614>,-57,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.20864.2678>,-57,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5596.2729>,-56,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.290.0>,-55,
[emqttd_sm_14,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.2028.2622>,-55,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.26378.2666>,-55,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.277.0>,-54,
[emqttd_sm_1,
{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.30182.2478>,-54,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.9892.2635>,-54,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.25325.2700>,-54,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.2803.2728>,-54,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5232.2637>,-53,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.25193.2659>,-53,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.18853.2654>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.21965.2676>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.18184.2680>,-52,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.8305.2634>,-51,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.19359.2658>,-51,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.263.0>,-50,
[{current_function,{gen_server2,process_next_msg,1}},
{initial_call,{proc_lib,init_p,5}}]}
{<10038.23951.2493>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.24567.2571>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.21513.2661>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.26646.2687>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.13943.2744>,-50,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.15692.2039>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.6901.2410>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1416.2494>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.8639.2558>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.32254.2628>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1046.2662>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.5529.2690>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.22357.2698>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.24309.2728>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.19374.2314>,-48,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}

Forgot current mem usage on machine2:
./bin/emqttd_ctl vm memory
memory/total            :<PHONE_NUMBER>
memory/processes        :<PHONE_NUMBER>
memory/processes_used   :<PHONE_NUMBER>
memory/system           : 377000072
memory/atom             : 662409
memory/atom_used        : 651355
memory/binary           : 137413840
memory/code             : 16318505
memory/ets              : 43881304
Unfortunately I have to restart the nodes now, otherwise they will be killed by the kernel in the next hours.

@brstgt, more questions ;)

The release of emqttd broker:
./bin/emqttd_ctl status

The pool_size of emqttd_plugin_redis?

./bin/emqttd_ctl status
Node<EMAIL_ADDRESS>is started
emqttd 1.0.1 is running
cat plugin.config
[
{emqttd_plugin_redis, [
{eredis_pool, [
  %% ecpool options
  {pool_size, 8},
  {auto_reconnect, 2},

  %% eredis options
  {host, "<IP_ADDRESS>"},
  {port, 22122},
  {database, 0},
  {password, ""}
]},

%% HMGET mqtt_user:%u password
{authcmd, ["HGET", "mqtt_user:%u", "password"]},

%% Password hash algorithm: plain, md5, sha, sha256, pbkdf2?
{password_hash, plain},

This option is required for our own plugin changes to auth with client id
{clients_file, "etc/clients.config"},
%% SMEMBERS mqtt_acl:%u
{aclcmd, ["SMEMBERS", "mqtt_acl:%u"]},

%% If no rules matched, return...
{acl_nomatch, deny},

%% Load Subscriptions form Redis when client connected.
{subcmd, ["HGETALL", "mqtt_subs:%u"]}

]}
].
Still need more? ;)

@brstgt. It seems the memory leak is caused by Erlang SSL library? The result of 'recon bin_leak':
{<10038.6901.2410>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
{<10038.1416.2494>,-49,
[{current_function,{gen_fsm,loop,7}},{initial_call,{proc_lib,init_p,5}}]}
...

There are some gen_fsm processes that will not release the refc binaries timely. I searched the whole project and found the ssl_connection.erl is implemented as gen_fsm behaviour.
Hi @phanimahesh, please help check the issue:)

If you mean this bug i thought, it was fixed in OTP 18.3.
If I can help you, just let me know!

@brstgt, please post the process count and memory allocation:
./bin/emqttd_ctl vm
./bin/emqttd_ctl recon allocated

As mentioned, i had to restart the instances, so the following output does not match the stats in the first comment:
Machine 1
./bin/emqttd_ctl vm
cpu/load1               : 3.90
cpu/load5               : 3.81
cpu/load15              : 3.80
memory/total            :<PHONE_NUMBER>
memory/processes        :<PHONE_NUMBER>
memory/processes_used   :<PHONE_NUMBER>
memory/system           : 307688752
memory/atom             : 654217
memory/atom_used        : 638268
memory/binary           : 65292104
memory/code             : 16053963
memory/ets              : 69945816
process/limit           : 2097152
process/count           : 32591
io/max_fds              : 1048576
io/active_fds           : 1
./bin/emqttd_ctl recon allocated
binary_alloc/current: 138972080
driver_alloc/current: 14453680
eheap_alloc/current :<PHONE_NUMBER>
ets_alloc/current   : 108825520
fix_alloc/current   : 390630320
ll_alloc/current    : 225184552
sl_alloc/current    : 42240944
std_alloc/current   : 7637936
temp_alloc/current  : 2230264
binary_alloc/max    : 527207344
driver_alloc/max    : 59280304
eheap_alloc/max     :<PHONE_NUMBER>
ets_alloc/max       : 330599344
fix_alloc/max       : 662473648
ll_alloc/max        : 244321064
sl_alloc/max        : 68697008
std_alloc/max       : 57445296
temp_alloc/max      : 43169784
Machine 2
./bin/emqttd_ctl vm
cpu/load1               : 2.63
cpu/load5               : 3.11
cpu/load15              : 3.35
memory/total            :<PHONE_NUMBER>
memory/processes        :<PHONE_NUMBER>
memory/processes_used   :<PHONE_NUMBER>
memory/system           : 313433944
memory/atom             : 662409
memory/atom_used        : 651612
memory/binary           : 73551840
memory/code             : 16058187
memory/ets              : 65090192
process/limit           : 2097152
process/count           : 34198
io/max_fds              : 1048576
io/active_fds           : 2
./bin/emqttd_ctl recon allocated
binary_alloc/current: 135302064
driver_alloc/current: 19958704
eheap_alloc/current :<PHONE_NUMBER>
ets_alloc/current   : 115903408
fix_alloc/current   : 352881584
ll_alloc/current    : 222563112
sl_alloc/current    : 44600240
std_alloc/current   : 16550832
temp_alloc/current  : 2230264
binary_alloc/max    : 509119408
driver_alloc/max    : 81562544
eheap_alloc/max     :<PHONE_NUMBER>
ets_alloc/max       : 275549104
fix_alloc/max       : 638880688
ll_alloc/max        : 243534632
sl_alloc/max        : 70495152
std_alloc/max       : 66096048
temp_alloc/max      : 45697016

After running emqttd for some time

run bin/emqttd_ctl recon bin_leak and note the pids of the first few entries.
In a shell, run sys:get_status(list_to_pid("<replace.with.pid>")). to inspect the internal state of the processes. Specifically, note the parent and ancestors.

I expect all or most of them to have a common ancestor, most likely a one_for_one supervisor. Identifying it will point to the source of leak.

@phanimahesh, do your App connect to the broker with MQTT Over WebSocket(SSL)?

Hmmm. I tried but what happens is this:

root@push1:/opt/emqttd-jaumo/erts-7.3/bin# ./erl -name<EMAIL_ADDRESS>-cookie emqttdsecretcookie -remsh<EMAIL_ADDRESS>Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:16:16] [async-threads:10] [kernel-poll:false]
*** ERROR: Shell process terminated! (^G to start new job) ***

When i pres ^G this happens:

User switch command
--> sys:get_status(list_to_pid("<10038.24464.2600>")).
Unknown command
-->

SSL: Yes we use SSL for public connections and no-ssl for internal connections

Lets try a different approach:

I could try using non-ssl connections for external connections as well, at least to prove that there is a SSL problem or not
What erlang/OTP Version do you use to compile emqttd official distribution? I could also try to compile against a different version.

Just let me know what helps you most!

@emqplus We have a separate node for ssl stripping, and emqttd gets plain traffic. Also we are running on OTP 17, so can't reproduce. Our memory usage is very stable.
@brstgt ^G puts you into job control mode. h or ? should show you the job control help. You are not able to connect to the remote shell because of bad flag. The cookie must be passed as -setcookie emqttdsecretcookie instead of -cookie ...

Hi @brstgt,

Okay, let's try non-ssl connections. I reviewed the pubsub, session and client modules of the broker, and ensure that we called GC when processing a MQTT message.

The binary packages of official emqttd release are built on R18.2+

My Skype: feng.emqtt.io, we could setup a chatroom to discuss this issue.
@phanimahesh, Thanks for you help:)

@emqplus I sent you a contact request.

I now moved mqtt sessions to non-SSL. Unfortunately I can't move the websockets to non-SSL as this leads to a browser security policy breach. First insights:
Already in the startup phase the mem consumption is MUCH lower. Before, the broker grew to about 600-700MB (used). This is probably due to the high connection rate on startup as all clients do reconnect automatically when the broker disconnects because of the restart.
Now, with mqtt connections the memory consumption in the first minute is about 210 MB (used).
Even after the startup phase, with mqtts, the broker is constantly growing with about 10MB every few seconds. With mqtt, the usage stays at around 270-280MB.
Unfortunately I cannot really tell if the websocket SSL connection leak as well, but if so, the broker grows much slower probably because the connection rate of websockets is MUCH lower than the mqtt conn-rate.

@phanimahesh, What do you use for "SSL stripping"? I'd rather prefer to let emqttd do the SSL work but as long as there is a such a severe mem leak I am thinking about a workaround.

Hi @brstgt, I noticed some strange metrics:
messages/dropped : 82
messages/received : 151496137
messages/sent : 50125251
The messages received is about 3 times than sent, but there are seldom dropped!  Do your application server subscribes '#' to process messages from clients?

Copying from our discussion on skype. to maintain a record here.
We use a dedicated HAProxy to do ssl stripping for both mqtt and websocket connections on emqttd.

Since avoiding SSL in emqttd, the leak became smaller. The daemon is growing far slower than before.
Unfortunately it IS STILL growing. All I can see is that the number of processes grows and grows. It seems to me that normally, the number of processes should be approximately double as high as the number of current connections, right? At least this is the case in the startup phase. But then over the time, the number of processes grows and grows.
How can we debug that?

Note the number of processes at each step.

Start a clean emqttd.
Connect exactly one or two clients.
Send some messages.
Disconnect.
Reconnect
Disconnect
Wait for session to expire

The difference between first and last readings should be zero. It will help if you set the session expiry to a very small time. Repeat with clean_session false and true and using mqtt and websockets. Atleast one of them should show the leak, and that will tell us where to look for it in the code.

@brstgt, If your MQTT clients connect to the broker with 'Clean Session = False' flag, the number of session processes will grow.

Thanks for you reply.
@phanimahesh I tried, but nothing suspicious happened. The number of processes went up and down as expected. Tested all kind of clients we are supporting.
@emqplus We use ONLY clean sessions
Maybe it has to do sth with the unique number of clients? However I recognized a suspicious behaviour. See the screencast: http://cl.ly/253Q2e2L0V05.
Currently I am disabling misc modules to see if it changes anything but no success until now.

Clean_session true only means clean any existing session and create a new one for this client id. After the client disconnect the session process still stays alive, and will die only after session expiry or when the same clientid connects with clean session true.
If there is no observable process leak due to client connections and disconnections, check your plugins. Also make sure it's not because of session expiry being set to a large value.

Clean session: It actually means, that this session is not persistent, so there sould not be taken any session timeout into account, if clean_session is true. See here http://www.hivemq.com/blog/mqtt-essentials-part-7-persistent-session-queuing-messages
Anyway I've set the session timeout to 1m and it doesn't make a difference. The only plugin I have activated is redis auth with some copy + paste customization to allow both redis auth + auth by ip/client_id. You can see the plugin here (https://github.com/Jaumo/emqttd_plugin_redis). The diff to the original plugin is not huge. I only merged a few functions from here https://github.com/emqtt/emqttd/blob/master/src/emqttd_auth_clientid.erl.
Here you can see the complete config: http://cl.ly/1Y2u3H3i2F0G. If it helps, I could even give you access to our production system.
From my understanding as a developer/devop (unfortunately not erlang), it is definitely not a normal behaviour that the number of processes is constantly growing even if the number of concurrent connection stays roughly the same and there is no persistence at all. To me it looks like a bug.

By the way: I updated to version 1.1 with OTP 18.3, same behaviour. Processes (and memory) is constantly growing. My first intention was to compile the 1.0.x with OTP 17.x but then I recognized that the latest version requires OTP18 so I decided to stay with 18 for future compat.

I've just re-read the protocol, and you are right. The session must be cleaned on disconnection if clean_session is true.
Checked the diff and config, no issues there. Atleast none were obvious to me. The issue should be reproducible on a clean emqttd without plugins then.
The number of processes is not supposed to grow constantly. It will stabilise when connections stabilises, and variations should be explainable by timeouts. There's definitely some bug.
Do you have any stats on how fast the processes increase with time?

Not yet, but if it helps i will gather the stats and write them in a time series DB. Will do that on monday and will be able to present some data maybe a day later.
Until then: Let me know what else I can do to find a solution :)

If you are going to collect stats, please record number of connected clients, and disconnected clients so far bucketed by reasons (as shown by emqttd_ctl listeners), along with number of processes. I'm trying to gauge the severity. Also hoping that the numbers may hint at something. A correlation between one particular kind of disconnections and extra processes, for example.
Thanks for being patient.

Would it be possible to add the disconnect stats to the dashboard listeners route (http://host:18083/api/listeners)?
I'd prefer to collect the stats via http/json as the stats are collected from another node in PHP.
Maybe other users would also benefit from showing these stats in the dashboard.

Stats are collecting from now on. Unfortunately without listener-disconnect stats as these aren't available from dashboard API.

disconnection stats are important, without them we won't be able to see if a particular disconnection is the cause, by not cleaning up processes properly.
I'm on vacation, I can add the listener disconnect stats to the api next week. Till then, see if you notice any patterns on processes vs clients.

Thats totally fine! I wont have the time to check that before next week
anyway!
2016-06-09 19:39 GMT+02:00 J Phani Mahesh<EMAIL_ADDRESS>
disconnection stats are important, without them we won't be able to see if
a particular disconnection is the cause, by not cleaning up processes
properly.
I'm on vacation, I can add the listener disconnect stats to the api next
week. Till then, see if you notice any patterns on processes vs clients.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/emqtt/emqttd/issues/561#issuecomment-224970319, or mute
the thread
https://github.com/notifications/unsubscribe/AAUQZ_6wC5hfQuGnClddi9jO5dgNkhL5ks5qKE-4gaJpZM4Ij7nE
.

Hi There! Are you back from holidays?
I have some stats right now, the graphs show 7 days:
http://cl.ly/3W0B0y143i0z
http://cl.ly/3Y3i2l34411J
Unfortunately the memory graphs are rather fresh but clearly show a pattern.
Need more? What about the listener disconnect stats via http?

Hi @brstgt, let's discuss on Skype;)

Hello @brstgt , I've been a bit busy lately, and haven't been able to meet my estimates for the past couple of weeks. I'm back from vacation but am moving to a new home, so my replies may be slightly delayed. Let's continue the conversation on skype, and we can post a summary here for any readers.

@brstgt, we found a process leak issue in the emqttd_dashboard plugin: https://github.com/emqtt/emqttd_dashboard/pull/64
Do you poll these HTTP APIs frequently?
/api/clients	List of all connected clients
/api/sessions	List of all sessions
/api/topics	List of all Topics
/api/routes	List of all routes
/api/subscriptions	List of all Subscriptions

Has been fixed in 1.1.3


SR300 - depth - duplicated objects in-front of the camera

Required Info

Camera Model
{ SR300 }

Firmware Version
(Uknown)

Operating System & Version
{Linux (Ubuntu 16)

Kernel Version (Linux Only)
(4.4.0-116-generic)

Platform
PC

SDK Version
{ Uknown }

Issue Description
I am trying to read only the depth information (ideally as a 2D greyscale image) using opencv. I have managed to plot the camera feed image but I am getting duplicated objects in-front of the camera.
Code :
#include <librealsense2/rs.hpp> // Include RealSense Cross Platform API
#include <opencv2/opencv.hpp>   // Include OpenCV API

int main(int argc, char * argv[]) try
{

    // Declare RealSense pipeline, encapsulating the actual device and sensors
    rs2::pipeline pipe;
    // Start streaming with default recommended configuration
    pipe.start();

    using namespace cv;
    const auto window_name = "Display Image";
    namedWindow(window_name, WINDOW_AUTOSIZE);

    while(cvGetWindowHandle(window_name))
    {
        rs2::frameset frames = pipe.wait_for_frames(); // Wait for next set of frames from the camera
        //rs2::frame ir_frame = frames.first(RS2_STREAM_INFRARED);
        rs2::frame depth_frames = frames.get_depth_frame();

        // Query frame size (width and height)
        const int w = depth_frames.as<rs2::video_frame>().get_width(); // 480
        const int h = depth_frames.as<rs2::video_frame>().get_height(); //640

        // Create OpenCV matrix of size (w,h) from the colorized depth data
        Mat image(Size(w, h), CV_8UC1, (void*)depth_frames.get_data(), Mat::AUTO_STEP);

        // Update the window with new data
        imshow(window_name, image);
        waitKey(1);
        }

    return EXIT_SUCCESS;
}

catch (const rs2::error & e)
{
    std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
    return EXIT_FAILURE;
}
catch (const std::exception& e)
{
    std::cerr << e.what() << std::endl;
    return EXIT_FAILURE;
}

Results :

Is there any quick fix for that ??

Hi @kmonachopoulos ,
You are using the wrong format for the matrix (CV_8UC1), it should be CV_8UC3.
Please have a look at the opencv examples for more info

Hello @zivsha I have already tried that and get : Segmentation fault (core dumped)

Try running the imshow example and see if it works.

Hi @kmonachopoulos ,
The depth frame native format Z16 stands for 16bit/unsigned/little endian.
To render it with grayscale levels you need to convert it to a similar type OpenCV matrix type, probably CV_16UC1
In the demos the depth frame is often converted into RGB space to enhance visualization traits:
16bit grayscale->24bit RGB (8bit per channel).
In these cases the CV_8UC3 would be the appropriate conversion type, like in the following example.

Thank you @ev-mp ! CV_16UC1 solved the issue !

Solved .. Closing ..


Calculating code metrics through MSBuild does not work

For calculating code metrics I have included Microsoft.CodeAnalysis.Metrics 2.9.4 nuget package to my csproj project.
Before executing msbuild from the command line, I have called "nuget.exe restore", and it correctly found MSBuild and restore package:

"MSBuild auto-detection: using msbuild version '<IP_ADDRESS>702' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\bin'.
Restoring NuGet package Microsoft.CodeAnalysis.Metrics.2.9.4. ..."

Then, from command line I am calling msbuild with /t:Metrics option:

"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\bin\MSBuild.exe" /p:Configuration=Debug /t:Metrics My.sln "

which ended with following error:

"Unhandled Exception: System.ArgumentException: Directory "C:\Program Files (x86)\Microsoft Visual Studio\2019\TestAgent\MSBuild\Current\Bin" does not exist
Parameter name: msbuildPath
at Microsoft.Build.Locator.MSBuildLocator.RegisterMSBuildPath(String msbuildPath)
at Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults()
at Metrics.Program.d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)"

It looks like Metrics.exe was started from msbuild itself and can't find it location, but nuget.exe did?

MSBuildLocator was the right place to file this. If you'd like the Metrics folks to look into a workaround on their side, I believe the right repo is https://github.com/dotnet/roslyn-analyzers.

Duplicate of microsoft/MSBuildLocator#73


Royalties

Hi,
We intend to use this library - mostly the flags - in an application. Are there any royalties for the flags SVG?

Feel free to use it.. and if you make a few gazillions I wouldn't mind getting my share :)

Will do. Unfortunately (or fortunately) I am only a humble employee, but I'll do what I can ;)


stopped working 11/30/2021

After 14 sunny days I'm back in the Chrome Incognito dungeon.

Long story short - Google killed all incognito theme customisation back in September 2021. This is a deliberate choice on their part. I suspect this is the first step in killing all theme customisation, as you can now set themes colours from within Settings anyway, so there is no real reason to keep third-party themes around. However, there is no way to change the incognito theming through themes or through Settings - it will always be dark.
You can temporarily reverse this change by going to chrome://flags/#incognito-brand-consistency-for-desktop and setting to Disabled. However, Chrome flags do not stick around for long, especially those related to appearances. When the flag goes, the functionality will revert back.
These recent developments mean that Material Incognito Light Theme is officially deprecated. It will no longer be receiving any updates from now on. When the Chrome flag disappears, I will pull this theme from the Chrome Web Store shortly after, as it will serve essentially no purpose anymore.
Thank you for using my theme in the meantime; I hope you enjoyed it while it lasted.

Dear Fiddle,
Thanks much for this information.  Strangely your theme plugin did work when I first installed it November.
Now your temporary workaround, with the flag, is working and what a pleasure it is to be able to scan through long lists of bookmark names displayed black-on-white.
What was Google's reason for preventing black-on-white themes for Incognito?
Google makes less money from Incognito use.  It can dissuade sensitive users by tormenting them with white-on-black, while looking cool to a probably penniless fringe.  (I hope there is a less cynical explanation.)
Sincerely,
Dennis Couzin
From: Fiddle @.>
Sent: Monday, 6 December 2021 01:20
To: Fiddle-N/material-incognito-light-theme @.>
Cc: dcouzin @.>; Author @.>
Subject: Re: [Fiddle-N/material-incognito-light-theme] stopped working 11/30/2021 (Issue #3)
Long story short - Google killed all incognito theme customisation back in September 2021. This is a deliberate choice on their part. I suspect this is the first step in killing all theme customisation, as you can now set themes colours from within Settings anyway, so there is no real reason to keep third-party themes around. However, there is no way to change the incognito theming through themes or through Settings - it will always be dark.
You can temporarily reverse this change by going to chrome://flags/#incognito-brand-consistency-for-desktop and setting to Disabled. However, Chrome flags do not stick around for long, especially those related to appearances. When the flag goes, the functionality will revert back.
These recent developments mean that Material Incognito Light Theme is officially deprecated. It will no longer be receiving any updates from now on. When the Chrome flag disappears, I will pull this theme from the Chrome Web Store shortly after, as it will serve essentially no purpose anymore.
Thank you for using my theme in the meantime; I hope you enjoyed it while it lasted.
—
You are receiving this because you authored the thread.
Reply to this email directly, https://github.com/Fiddle-N/material-incognito-light-theme/issues/3#issuecomment-986333302, or https://github.com/notifications/unsubscribe-auth/AMOEZQMUU7UUS6PZMIYZGOLUPP6SXANCNFSM5JC6M3QA.
Triage notifications on the go with GitHub Mobile for https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

From what I can tell, it's about delivering a consistent "Incognito" brand. Incognito isn't just a mode in Chrome anymore, it's a brand that is getting applied to other products, like Gboard. Keeping the colour always the same is part of that Incognito brand identity.
I don't agree with the change, but it is what it is.

  Incognito acquired a comically dysfunctional interface.  For the coolest users only.
Thanks for your help.
Dennis
From: Fiddle @.>
Sent: Monday, 6 December 2021 20:39
To: Fiddle-N/material-incognito-light-theme @.>
Cc: dcouzin @.>; Author @.>
Subject: Re: [Fiddle-N/material-incognito-light-theme] stopped working 11/30/2021 (Issue #3)
From what I can tell, it's about delivering a consistent "Incognito" brand. Incognito isn't just a mode in Chrome anymore, it's a brand that is getting applied to other products, like Gboard. Keeping the colour always the same is part of that Incognito brand identity.
I don't agree with the change, but it is what it is.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/Fiddle-N/material-incognito-light-theme/issues/3#issuecomment-987120083 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AMOEZQPFOCJODSCZSGRVVTTUPUGOXANCNFSM5JC6M3QA .
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675  or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .


Inheritance nested class from interface contains rxjs reference, breaks constructor type checking

TypeScript Version:  3.1.0-dev.20180728

Search Terms: type check, rxjs, nested class
Code
import { BehaviorSubject } from "rxjs";

interface IAsset {
  title: string;
}

export interface ICategory<TAsset extends IAsset> {
    categories: BehaviorSubject<ICategory<TAsset>[]>;
}

class Foo {
    private y = 1;
    static Bar = class implements ICategory<IAsset> {
        constructor(foo: Foo) { }

        categories = new BehaviorSubject<typeof Foo.Bar[]>([]);
    }
}

console.log(new Foo.Bar(1));

Expected behavior: bug.ts:20:25 - error TS2345: Argument of type '1' is not assignable to parameter of type 'Foo'
Actual behavior: Compilation false succesful
Playground Link: https://stackblitz.com/edit/ts-check-bug-incorrect
(removing implements fix issue: https://stackblitz.com/edit/ts-check-bug-correct)
Related Issues: did not find any related issues

if you turn on --noImplicitAny flag you should see errors:
'Bar' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer.
'categories' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly


Future: make migration of data types defined by plugin packages possible

Motivation
Currently, database migrations are only possible for data types that are defined in aiida-core. This means that e.g. data plugins that are a part of a plugin package like aiida-quantumespresso can not change the way the data is stored in the SQL database and repository.
Desired Outcome
This item will have been resolved once it is possible to migrate data types that come with a plugin instead of being defined in aiida-core.
Impact
A couple of example use cases are:

There are already plenty of data types defined in plugin packages at the moment, whose storage can now not be changed (e.g. the UpfData in aiida-pseudo, ForceConstantsData in aiida-quantumespresso). In case we want to for example add some information to the database to make it easier to query for, this is currently not possible.

New data plugins are also being implemented (see e.g. https://github.com/aiidateam/aiida-quantumespresso/pull/849), where the best solution on how to store the data in the database/repository isn't always clear. Any new data type that is now implemented outside aiida-core needs to be very careful in making this choice, and it's always possible to overlook issues with the design, complicating the development of new data types in plugin packages.

There are several data types currently defined in aiida-core that we may want to move to a more appropriate plugin package (see Background). One barrier to doing so is the fact that once we do this it will be impossible to change their storage.

Complexity
Implementing this feature has already been discussed at several previous coding events, and so far no straightforward solution has been found. So it's fair to say this is a quite complex issue that will take at least a few weeks of dedicated time to solve.
Background
One context in which this feature was discussed is the move of the materials science-related data types outside of aiida-core to make it more field agnostic:
https://github.com/aiidateam/aiida-core/issues/2686
Progress
There have been some brain storming sessions on how to proceed with this in previous coding weeks, but as far as I know no real progress has been made towards implementing this feature.

@sphuber if I remember correctly, you already had a look at this? Do we have some notes on the topic somewhere?


It shall raise exception when value is out of scope

Given the following program:
    public static void foo() throws Exception {
        String str = String.valueOf(Integer.MAX_VALUE);
        short sh = JSON.parseObject(str, short.class);
        System.out.println(sh);
    }

FastJson will return -1 when the value is out of the scope of short type. Suppose it shall raise an exception in such case.

The above code is for short type overflow, the behaviour is correct when it is for int type overflow:

for all primitive type overflow testing, the following types cannot pass:

byte
double
float
short

byte/short type overflow already raised exception. Can you tell me how you test float / double overflow?

They come up with special Float and Double value:

Just FYI, in my param value resolver (aimed to bind form/request param) I have the logic to check the Infinity:

bug fixed. please use last version https://github.com/alibaba/fastjson/releases/tag/1.2.21

@wenshao  The issue with Float has been fixed, but Double still have the same issue


Chase further tests which are looking for a Biggus array

For example: iris.tests.test_pp_module.TestPPCopy test_deepcopy_field_deferred

https://github.com/marqh/iris/pull/41


Static site + Vuex

Hi! :) I'm creating a Nuxt site using wordpress + netlify.
At first (in dev mode) each page I call the menu and footer api, to populate menu and footer. (duh) Now I'm using vuex, so I called this only in layout/default.vue.
But can I call the vuex before generate pages? Because now, when I deploy to netlify, I need to call my api to populate vuex. :/

Wow, thats awesome. :) I don't realize what is middleware. Thanks, @manniL!


Can it support pre issued commands when terminal is opened?

I am wondering if it supports executing some pre issued commands as soon as terminal is opened. For example: ls , go to some specific folder, or executing some applications.
I know you can always open the terminal and manually enter those commands. But it would be great if it can support executing those commands in advance. One use case is letting user open a debug session and all the related debug configurations are sent and enabled once terminal is open so the user does not need to manually enable debug commands.

See #94 for an example of how to do this


verification page

Description
Create a verification page
Follow the Guidelines and code of conduct when your pushing changes.
Folder structure:
Verificarion pages\verification_page_01\
Screenshots
No response
Additional information
No response

Hey @Jaideep25-tech Can I work on this issue

yeah bro you can but remember the folder structure, guidelines and commit message!

Hey @Jaideep25-tech verification page is like OTP verification page ??

yeah bro


bugfix: gProfiler excessive restarts when run in a container and failing to grab the mutex

Change exit code to 0 when failing to grab the mutex. This help gprofiler container avoid
excessive restarts when running using K8s or docker
Description
Exiting with exit code 0 when failing to grab the mutex.
Checklist:

[ ] I have read the CONTRIBUTING document.
[x] I have updated the relevant documentation.
[ ] I have added tests for new logic.

Also change the documentation to use --restart=on-failure?

I've also made a maximum of 10 retries by default FYI

I've also made a maximum of 10 retries by default FYI

Cool. I wasn't aware of this feature. Can you please check from which Docker version is it available? (we refrained from adding --pull=always because it's too new and it caused docker run to fail in many servers; hence it shouldn't be in the "standard" command)

I've also made a maximum of 10 retries by default FYI

Cool. I wasn't aware of this feature. Can you please check from which Docker version is it available? (we refrained from adding --pull=always because it's too new and it caused docker run to fail in many servers; hence it shouldn't be in the "standard" command)

Couldn't find it. I guess it's old enough to be used here.

@Jongy I resolved conflict in README please reapprove.

@Jongy You know what are this problems in the CI?
GProfiler CI / test (3.6) (pull_request) Failing after 41s — test (3.6)
It fails on async_profiler_env.sh

@Jongy You know what are this problems in the CI?
GProfiler CI / test (3.6) (pull_request) Failing after 41s — test (3.6)
It fails on async_profiler_env.sh

Nope. Will check in the next PR


Add zoom in/out buttons

Added zoom in/out buttons, as part of JWST DADF sprint.
Need help with figuring out the following:

[x] How to make + and - shortcuts to actually also do the zooming, similar to key-binding?
[x] How to automatically deactivate/untoggle the button after it is pressed?

UPDATE: Works now but only for glue >= 0.9

@pllim - you are running up against issues which I'm just in the process of solving in https://github.com/glue-viz/glue/pull/1085. Regarding the issues you are seeing here:
Shortcut only works once: this is because for MouseModes, the shortcut enables/disables the MouseMode, but you don't really want a mouse mode since you don't need the mouse actions. In #1085, I've introduced CheckableMode and NonCheckableMode - in your case you would want to inherit from NonCheckableMode
No deactivate: this is added in #1085
The issue is that currently is that #1085 isn't quite ready, but I'll try and finish it up while traveling tomorrow. If I don't manage to finish it then, I'll finish it on Saturday (sprint day at EuroSciPy). Would that work for you? In the mean time, just make sure it works the first time you press it, and it'll be easy to adapt this PR to work properly once #1085 is ready and merged.

@astrofrog , ah, okay! No problem. I think the zooming already works, so should be easy to update when your PR is merged. Thank you and safe travels!

@pllim - can you try updating this to see if it works with my branch in https://github.com/glue-viz/glue/pull/1085? Since my above comment, I changed the name of the classes I had mentioned above, so in this case you want to just sub-class Tool (which doesn't stay checked).

@astrofrog , is this holding up anything? If not, I'll get to this when I can, as the sprint is over.

@pllim - no, I'll go ahead and merge https://github.com/glue-viz/glue/pull/1085 once it passes, so this isn't holding anything back. Sorry for being slow to fix all this - often things that come up as part of sprints can't be solved immediately during the sprint... I tend to be lagging by one sprint!

@astrofrog , no problem. It is a common problem among all sprinters for such a sprint. I'll get back to you before JWST launces, I promise. :wink:

@astrofrog , I made the changes as documented here. Obviously, this is not compatible with glue < 0.9 but I do not intend it to, will this be a problem for glue-exp?
Using the latest dev version of glue, it is unable to load my tools:
.../glueviz-0.9.0.dev3097-py3.5.egg/glue/external/qt.py:6: GlueDeprecationWarning: The glue.external.qt subpackage is deprecated - see the v0.9 release announcement for more details
  "release announcement for more details", GlueDeprecationWarning)
WARNING:root:Message signing is disabled.  This is insecure and not recommended!
Traceback (most recent call last):
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/app/qt/mdi_area.py", line 61, in dropEvent
    new_layer(layer)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/app/qt/mdi_area.py", line 51, in new_layer
    self._application.choose_new_data_viewer(layer)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/app/qt/application.py", line 675, in choose_new_data_viewer
    return self.do(cmd)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/core/application_base.py", line 207, in do
    self._cmds.do(command)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/core/command.py", line 126, in do
    result = cmd.do(self._session)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/core/command.py", line 216, in do
    v = session.application.new_data_viewer(self.viewer, self.data)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/mpl_widget.py", line 39, in wrapper
    result = func(*args, **kwargs)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/core/application_base.py", line 82, in new_data_viewer
    c = viewer_class(self._session)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/data_viewer.py", line 30, in __call__
    obj.initialize_toolbar()
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/image/qt/viewer_widget.py", line 376, in initialize_toolbar
    super(ImageWidget, self).initialize_toolbar()
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/data_viewer.py", line 242, in initialize_toolbar
    mode_cls = viewer_tool.members[tool_id]
KeyError: 'Zoom In'

What did I do wrong? Please advise.

@astrofrog , now it is complaining about self._axes. How do I access the matplotlib axes with this refactored toolbar? Thanks!
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/toolbar.py", line 111, in trigger
    self.active_tool = tool
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/toolbar.py", line 66, in active_tool
    self.activate_tool(new_tool)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/mpl_toolbar.py", line 162, in activate_tool
    super(MatplotlibViewerToolbar, self).activate_tool(mode)
  File ".../glueviz-0.9.0.dev3097-py3.5.egg/glue/viewers/common/qt/toolbar.py", line 84, in activate_tool
    tool.activate()
  File ".../glue_exp-0.1.dev0-py3.5.egg/glue_exp/tools/zoom_buttons/zoom_buttons.py", line 45, in activate
    auto_zoom(self._axes, zoom_type='out')
AttributeError: 'ZoomOutTool' object has no attribute '_axes'

@pllim - all tool objects have access to .viewer, so you can access the axes with self.viewer.axes - I should clarify that in the docs.

@astrofrog , thanks for all your help! It works now. I don't know how glue-exp ties to glue version. As long as you are okay with this not being backward compatible for glue < 0.9, then feel free to merge.

@pllim - this repo is just used for experimental plugins so we aren't too fussy about required glue versions (this is not something we advertise widely to users). So this is fine!

@pllim - oops I'm sorry for not merging this sooner!


Under what circumstances do we need to zero out memory?

In codegen, we frequently do a CREATE_MEMSET() to zero out any memory we've allocated with AllocaInst:

// initializing memory needed for older kernels
// memset so verifier doesn't complain about reading uninitialized stack

I see that we don't always do this (e.g. sometimes we allocate structs, and jump straight into CreateStore() without zeroing it out).
Should we skip the memset if we're on a sufficiently high kernel version?
After what kernel version do we cease to need the memset?
I expect there'll be a small number of cases where the memset makes sense for safety:

if we store 32-bit value into a 64-bit buffer
if we probe_read_str() content into a buffer (it may copy content only until it encounters a null byte — dunno where the docs are to confirm this)

But in cases where we know we're gonna overwrite every byte of the buffer, it'd be nice to be able to skip the memset.
I suspect LLVM already optimizes away memsets where it can see we're overwriting the whole buffer. But for https://github.com/iovisor/bpftrace/pull/1360, there's a technique we use to zero out large maps (probe-read some zeroes from userspace), and I'm less confident that LLVM can optimize that away. This technique was inherited from an RFC of @mmarchini's, so I don't fully understand when/why it becomes necessary to use a probe_read rather than memset.
I guess what I'm looking for is to know "could codegen be simplified". If memset is always needed, then maybe it should be put inside our AllocaInst helper (and for scratch maps, which https://github.com/iovisor/bpftrace/pull/1360 uses in similar ways to AllocaInst). If it's only needed on older kernels, maybe it should be activated by a preprocessor macro. if probe_read is preferred over memset under some conditions, maybe we should generalize the memset to a "zero out buffer" helper.

I don't have anything authoritative to say but here are some of my thoughts:

I think it depends on if the verifier is smart enough to figure out the prog only ever reads from initialized memory

bpf_probe_read*() can fail if it requires a page fault, so the verifier cannot say for certain memory is initialized after a bpf_probe_read*()

I'm a bit scared to upset the verifier in older kernels b/c we probably arrived at this state from trial and error

Other projects (like libbpf) have quite complicated vmtest frameworks to run CI on different kernel versions and distros
If we can get vmtest infra for bpftrace then I'd be a lot more comfortable shipping any codegen change that can pass CI


Update self-upgrade script to work on target branches other than 'main'.

Currently, only default branch 'main' is supported.

/approve


GameCube {HuiJia USB box}

Just a config for the USB GameCube controller adapter I own. I tried to make the name/layout match the existing GameCube controller config.

Merged manually due to lack of platform specification in PR.


Implement Update Operation

Issue #, if available:
https://github.com/aws-controllers-k8s/community/issues/1181
Description of changes:
This implements update support for the opensearchservice controller.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/ok-to-test


close gaps left by non-deterministic elements.comp monoid

This change fixes the gaps described in #62 for Gio.
However, I haven't figured out how to squeeze the FLAG_END_PATH flag into the scene.rs tag encoding. There is already the is_fill flag, but it's encoded as separate Rust enums, and I don't want to add another variant dimension (not-end-of-path vs end-of-path).
Please advice?

Ok, I've done #69 so I can encode bits into the tag word, something that I think will generally be useful. I also locally patched this to set the flag for the last element on close_path, and I get this result:

There are two issues here. One is stray segments from stroked paths that aren't closed. This is as expected from reading the patch. Also, there are more artifacts related to closed paths not being watertight. I haven't tracked these down either.
So I have questions about how carefully this patch was tested. Were you able to get the tiger to render artifact free?

Ugh, that's bad. No, I didn't test the tiger because I didn't figure out a way to pass the end path flag in Rust. This change does work great for all of Gio's tests, even with non-trivial transformations and lots of text. I don't use strokes yet, though, which is why I didn't notice the open path flaw.
Now, I can dig into the tiger rendering glitches if you like, but it sounds to me you've found a better fix than this change. If so, let's just abandon this change and not waste more time on it.


别名库炸了？

检查发现 https://api.yuzuchan.moe/maimaidx 似乎没有返回正确值
Traceback (most recent call last):
  File "/usr/local/python3/lib/python3.9/site-packages/starlette/routing.py", line 671, in lifespan
    async with self.lifespan_context(app):
  File "/usr/local/python3/lib/python3.9/contextlib.py", line 181, in __aenter__
    return await self.gen.__anext__()
  File "/usr/local/python3/lib/python3.9/site-packages/nonebot/drivers/fastapi.py", line 165, in _lifespan_manager
    await self._lifespan.startup()
  File "/usr/local/python3/lib/python3.9/site-packages/nonebot/drivers/_lifespan.py", line 35, in startup
    await self._run_lifespan_func(self._startup_funcs)
  File "/usr/local/python3/lib/python3.9/site-packages/nonebot/drivers/_lifespan.py", line 29, in _run_lifespan_func
    await cast(ASYNC_LIFESPAN_FUNC, func)()
  File "/home/www/arsbot/src/thirdparty/nonebot_plugin_maimaidx/__init__.py", line 32, in get_music
    await mai.get_music_alias()
  File "/home/www/arsbot/src/thirdparty/nonebot_plugin_maimaidx/libraries/maimaidx_music.py", line 339, in get_music_alias
    self.total_alias_list = await get_music_alias_list()
  File "/home/www/arsbot/src/thirdparty/nonebot_plugin_maimaidx/libraries/maimaidx_music.py", line 279, in get_music_alias_list
    alias_data = await maiApi.get_alias()
  File "/home/www/arsbot/src/thirdparty/nonebot_plugin_maimaidx/libraries/maimaidx_api_data.py", line 124, in get_alias
    return await self._request('GET', self.MaiAliasAPI + '/maimaidxalias')
  File "/home/www/arsbot/src/thirdparty/nonebot_plugin_maimaidx/libraries/maimaidx_api_data.py", line 29, in _request
    res = await session.request(method, url, **kwargs)
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_client.py", line 1530, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_client.py", line 1719, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/python3/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/python3/lib/python3.9/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout

别名库一切正常，检查自身网络情况

确实是我网络问题，但是很奇怪，明明有 except UnknownError: 但是访问失败会让整台bot都运行不起来


FW Attitude controller Diagram

Hi guys,
We are are trying to get better understanding of the FF gains in the FW flight. I saw a old diagram from @tumbili for the attitude controller but could not find it anywhere in the dev guide or other place. Any ideas? If there is no such diagram created with graphic tools I could create them for the dev guide and provide them to @hamishwillee

@bkueng Thoughts on this?
@tubeme Where would you put this in the guide?

We should standardize on a particular tool and gradually get these together for all controllers. It's a fairly common request.
Options that immediately come to mind are https://www.draw.io/ for simplicity or using latex that lives in the repository. I think our best chance at keeping it up to date is to have the documentation in each module, but that won't be as easy to modify.
@priseborough any suggestions?

@dagar When you say "in each module" you mean storing the image in the github tree? Or were you talking out where you render it - ie here somewhere: https://dev.px4.io/en/middleware/modules_main.html ?
Does this sort of thing modify regularly - if not, then having in source is not a necessity.
Using a consistent tool makes a lot of sense.  I like draw.io and other web based systems a lot, but I'm not sure how we'd manage access consistently in an "open source"/free world. The "best" solution is an open source diagrammer that allows you to store the source in github (for import by anyone). Dia was my goto for that, but it is a bit dated now. Anyone tried anything else  -  some options http://merabheja.com/best-free-tools-for-creating-flowcharts/

Yes in each module means in the git tree. For example the FW attitude controller diagram would live right here in the FW attitude controller module. https://github.com/PX4/Firmware/tree/master/src/modules/fw_att_control
No it doesn't change that often, but I've seen quite a few bits of documentation spreadout across other directories in git (https://github.com/PX4/Firmware/tree/master/Documentation), various hosted services, old wiki, discuss posts, now the new wiki, etc. Ultimately I think it needs to live right in the source next to the thing it's describing. It can and should still be posted to the dev guide, but this would be the source.

No it doesn't change that often, but I've seen quite a few bits of documentation spreadout across other directories in git (https://github.com/PX4/Firmware/tree/master/Documentation), various hosted services, old wiki, discuss posts, now the new wiki, etc. Ultimately I think it needs to live right in the source next to the thing it's describing. It can and should still be posted to the dev guide, but this would be the source.

There is definitely a problem there, but the solution is arguable.  I look at https://github.com/PX4/Firmware/tree/master/Documentation and see unmanaged documentation that no one is likely to discover until too late, which has no clear owner, which may be out of date, which has not context. IMO all of that should be moved into the devguide and the directory removed.
In some cases it can be useful to keep the docs alongside source, but it still should go into our guides. via some form of import. If it isn't in the guides then we should work to remove it - that includes old wikis, but probably not old discussion boards.

We should definitely have controller diagrams on the dev-guide. If they're kept high-level they don't change often.
FYI @Stifael

Is it possible to have the controller block diagram held with the source with links in the dev guide? Many of our controllers (eg TECS) are in libraries
I would not expect architecture diagrams to change often. Something in the code code comments

@hamishwillee @dagar I agree with @priseborough that this diagrams does not change very often and in the future they will change even less. So why make it harder to maintain with a lot of tools etc. Placing a simple JPG with the diagram is fair enough. I could do the diagrams in a couple of days with the Corel and I could make them look pretty catchy. We need not more than 4-5 diagrams for the different aspects of the controller. BTW APM did it with the very basic black and white look but you get the concepts right away.

@hamishwillee We could create a separate section called PX4 Controller Diagrams and place all the diagrams at one place. There are only certain people that need them and this way it will be much easier to find what you need with the rule of the "three clicks". Arranged this way google will make a good indexing as well and it will pop up in the search right away.

If somebody can sketch the diagrams on paper, I will start making them digital. I could not find any diagram except this. BTW this diagram was dug deeper in a gitter chat and google search found it right away when I searched for the FF gain of PX4.
https://gitter.im/PX4/VTOL/archives/2015/05/22

Thanks all!

Yes it is possible and easy to keep the images in source, but I would prefer not to do this. It is better to maintain all the docs in one place, if only so that people know what will be affected by moving, deleting or whatever the image.

In the general case I am pragmatic - if an image had to belong in source we would live with it.

@tubeme Thanks for offering to do the images.

@dagar @Stifael @bkueng Can you sketch them out for Vasil, or point me to best people to ask for help?

I could do the diagrams in a couple of days with the Corel and I could make them look pretty catchy. We need not more than 4-5 diagrams for the different aspects of the controller. BTW APM did it with the very basic black and white look but you get the concepts right away.

Generally a documentation set looks better if text and diagramming is consistent. As not everyone has Corel it is better to have images that aren't too funky :-) Upshot, keep it simple - use the basics. Add colour and funkyness where it adds to clarity.

We could create a separate section called PX4 Controller Diagrams and place all the diagrams at one place. There are only certain people that need them and this way it will be much easier to find what you need with the rule of the "three clicks". Arranged this way google will make a good indexing as well and it will pop up in the search right away.

I agree with this grouping, at least initially (if we had any documentation about individual controllers then splitting by controller would make sense)
But where in the structure should this live - we don't actually have much real information about controllers? This isn't middleware. We could do (one of)

Concepts > Controllers >  Controller Diagrams
Flight Stack > Controllers > Controller Diagrams

The whole devguide structure is going to get revisited, so open to ideas.

note:
P-ctrl = proportional controller (not position controller)
PID-ctrl = proportional, integral, derivative controller
let me know if it needs more clarification

@tubeme Here is some work done a few month ago. It's quite accurate: http://discuss.px4.io/t/position-and-attitud-controllers-diagram/3186

@Stifael @bresch Thanks I will start making them in Corel. I will make them in one style, pretty simple one @hamishwillee without blink blink :). I will keep the CDR sources so we could change them with time. Probably we could create a space somewhere to keep the CDR files so they are available for future edit.
@bresch  I see this graph is from the MC ATT controller. How about the FW ATT controller. There is no D in the FW so there should be some difference.
We noticed that the FF gain in the FW controller have connection to the whole PI chain. Ie. if we lower the FF too low, the ATT becomes sluggish and not adequate enough for good stabilization. Increasing the FF gain gives much better stabilization? Any ideas?

Thanks all.

Presumably the different gains etc map to parameters.(?). I assume it would be useful to map these to the diagrams? If so:

propose do this in the document outside of the diagram
Can I have the mapping :-)

Can I have the list of controllers that we aim to create? ... so we can make sure they all get done.

@tubeme Thanks!

I will keep the CDR sources so we could change them with time. Probably we could create a space somewhere to keep the CDR files so they are available for future edit.

Good idea. If they aren't huge we could reasonably put them in the gitbook assets alongside the generated image.

@tubeme Indeed, for the FW controller, the attitude loop uses a proportional gain only (P-controller) and the rates loop uses a PI-controller with feedforward:
https://github.com/PX4/ecl/blob/b0300b97235d4c5907bd1ad8878550540403c933/attitude_fw/ecl_roll_controller.cpp#L130-L132
The feedforward isn't directly affecting the PI loop but improves a lot the tracking. Let's take for example a roll maneuver: moving the ailerons produces a torque around the longitudinal axis that accelerate the body. As the rollrate increases, the roll damping contribution will limit the maximum rollrate with the current deflection. (Ultra-simplified, the roll moment is: L = C_ld*aileron - C_lp*rollrate)
The I term of the controller might be able to handle this, but using integral for normal operation is never a good solution. The better approach is then to use a feedforward that will directly deflect the ailerons with roughly the correct amplitude and the PI-controller will just need to compensate for errors.
Furthermore, on an aircraft, the roll damping derivative term is usually so high that even without a PI-controller (Attitude loop directly controlling the ailerons) you might not see a big difference. That's also why piloting manually an aircraft feels more like piloting in rates than in acceleration.

@bresch It might be useful to capture some of that information (https://github.com/PX4/Firmware/issues/8316#issuecomment-346413502) alongside the image :-)

@hamishwillee Here is the first diagram.
If you like the style I will continue with the rest.
@bresch Which diagram to use from the link you gave me ( http://discuss.px4.io/t/position-and-attitud-controllers-diagram/3186). There are couple of diagrams? Can you provide me with legend of the parameters? Do you have a diagram of your explanation about the FW controller and the FF gain, how does it differ from the MC implementation?
Thanks
@Stifael can you take a look to see if I got that correct?

@tubeme Yes, I'll check and give you the corresponding parameters.
About the diagram you just did:

I would suggest to use a different letter styles for the letter P in order to avoid confusion between position and P gain (e.g. use  for the position vector)
I don't think "- Ctrl" is needed in the controller rectangles, it would be lighter and clear enough to put P in a triangle  (to indicate a gain) and PID in a rectangle.
The "Phi" greek letter used to define yaw should be in lowercase () as the uppercase is usually used to represent the attitude vector.

@tubeme Awesome! Is there a way to allow to edit this in a cloud tool, like draw.io?

@tubeme Yes looks great :-). It displays pretty well in gitbook too!

We need to have an easy way to change if needed. Normally we suggest draw.io.
@bresch suggestions seem reasonable.
I like the legend. Will be good to see the parameter mapping too.
As this will be embedded in the docs, we may not need the heading and logo.
Presumably this is just MC - not FW?

So what documentation should accompany this? The information above from bresch would seem useful. Other ":thoughts"

What are the effects of tuning different values
Where to go in the code to play with things
What can/should be modified/not modified
How the controller maps to the quadcopter model - ie why the maths works..

@tubeme looks good. I think you forgot the position estimate label which is subtracted from the desired position setpoint.
I have some comments about documentation for math related implementation.
Nomenclature
There are quite a lot of math symbols used in px4, and I am sure the amount of symbols will rather increase than decrease. Therefore I suggest to have a full list of Symbols within the documentation. This list would also include acronyms and abbreviations such as MPC, MC and so on.
Notation
In the position control diagram I labeled vectors with an arrow. I only did that because that was the most obvious, easiest way to label a vector when drawing something by hand. However, for a documentation purpose it might be better to find a different kind of notation, such as having vectors boldface and scalars non-boldface. The really important part, however, is to be consistent with all symbols.
I attached my own Nomenclature and Notation as example.
nomenclature.pdf
notation.pdf

@Stifael Could not agree more. Are you happy for us to steal your nomenclature and notation as a starting point for PX4?
@tubeme

for a documentation purpose it might be better to find a different kind of notation, such as having vectors boldface and scalars non-boldface.

What do you think would be the best thing to do in this case?

Are you happy for us to steal your nomenclature and notation as a starting point for PX4?

Of course

Thanks all let me think with all the info and I will post you with the idea about notations and nomenclature.

@hamishwillee @Stifael @tubeme I started to write a glossary containing symbols and acronyms useful in the context of PX4. I've been highly inspired by several aeronautic, control and robotics books I read and try to avoid confusion in the notation as much as possible.
Here is my fist version on the glossary: https://github.com/bresch/Glossary/blob/master/main.pdf
It needs to be ordered and sorted correctly and there is still a lot missing. Please comment here or in the repository so we can quickly have a good basis to start to write schematics.

@bresch @Stifael Probably best if those of you with more experience in using these symbols decide if you're happy with the set.
My two bits:

Is there any benefit in sorting by Latin, Greek, whatever? I would have thought perhaps by "Domain".
I like that @Stifael version has units.
How do we turn the two sets into one?

With respect to the acroymns, we can probably add them to the gitbook glossary. Care needs to be taken to limit that glossary size because Gitbook will actually cross link all the items, and as you can imagine the more you have the slower the process is. Eventually it will also fall over.

About the gitbook glossary let me check, I never used it so far.

I have. It is very limited, and massively slows builds. Only matters to those of use who work locally, but is a complete pain.

@tubeme Any ETA on next iteration/Is this waiting on me/others for anything?
The discussion of the maths is just as important as the diagram. I've also had a request to capture the information about how the controllers work in a webinar format. If you guys are interested in helping with that, please email me hamishwillee at gmail dot com, or catch me on slack (hamishwillee)

Are we with these on draw.io yet? We need a workflow where control engineers can alter the diagram with minimal friction.

@tubeme Would you like me to copy your diagram to draw.io for iteration - just to speed the process and make easier for anyone/everyone to edit?

@tubeme Any ETA on next iteration/Is this waiting on me/others for anything?

Appreciate you are busy. But if you don't respond I'll move what you have done across so we can continue progressing this.

OK, I have created a near duplicate of @tubeme controller diagram on draw.io here. Any of you can edit it. Just pausing in case one of you wants to proceed from here?
Image looks like this:

Updated version above has following changes:

PID - Ctrl in box TO PID
p (position) - changed from P
v (velocity) - changed from V
Ψ (attitude vector) (psi upper case)  - changed from q
ψ (yaw angle) (psi lower case) - changed from Ψ

Questions:

F is "thrust" . Is that a reasonable synonym for force? @Stifael "F" is (I believe) more common than "f" as a symbol for force. What should we use here, and is 'thrust' OK for the label?

@bresch Could you update those things you believe to be errors - and the multiplexer you referred to? If you have questions/undecided for me/us then can you raise again?
Rationalising Nomenclature
IMO we use SI and other "standard" symbols where possible.
@Stifael as you are familiar with your nomenclature set, can you see any discrepencies from your set and @bresch set here - I picked you to review because his set is much smaller.

In his version he defines things like yaw angle as euler yaw angle. Is this helpful definition over "yaw angle"? If so lets merge.
If there are discrepancies, are you happy to use his set, or can you propose an alternative?
Last of all, could you make a copy and experiment with your thoughts on vectors?

F is "thrust" . Is that a reasonable synonym for force? @Stifael "F" is (I believe) more common than "f" as a symbol for force. What should we use here, and is 'thrust' OK for the label?

At the end it depends on the nomenclature that we define on. However, the nomenclature and style for the above diagram is most likely not the final solution. For instance, we definitely need to use subscripts.
@bresch has shown to me a good draft for nomenclature. I suggest that we start with this diagram and also add the draft for nomenclature, which both have to be adjusted simultaneously ( instead of waiting for the perfect, final solution).
One issue that I currently see is that the nomenclature definitions (upper-, lower-, right- and left subscripts) and mathematical symbols might not be very well supported by draw.io. Maybe there is a latex plugin, which would help a lot.
One thing that I personally don't like about the above diagram: arrows as vector indication :). I prefer to have vectors and matrices bold-face. It looks just clunky with arrows...

@Stifael Draw.io has a latex plugin, it is enabled, and that is what I am using in that image. To use it you just put the text in back ticks. There is a draw.io example here
Subscripts work fine using _ , superscripts work with ^ .. sometimes you need to be tricky with ( ) to apply them properly.
Could you please modify the image as you would prefer it to be based on what you and bresch believe is the right thing?

@hamishwillee Thanks for the effort! I modified the diagram to match the glossary I made. Please @Stifael , tell me if there is anything you don't like or if you see a mistake.

Are you OK with this?
Yes.

Perhaps an explanation of anything that might not be clear from the diagram
It obviously needs to have a globally accepted px4 nomenclature and notation. @bresch, what is the standing of the nomenclature document that you have been working on?
I also think that it is important that we do not try to explain everything our-self. Certain things are very standard (such as PID), and you can waste a lot of time trying to explain everything compared to just adding a reference link.

Is the hat (^) standard for estimate
Yes and no. It depends on what we define in the nomenclature. However, ^ is widely used in academia to indicate estimate.

Would these be better grouped under a parent of "Concepts"
I also lean towards Concepts.

Should the F in the index be bold (appears to match on your diagram but not on draw.io version s ?

Corrected. I also added Z in the legend and corrected the projection operator

Is the hat (^) standard for estimate? If not, then r subscript e (like r subscript d) might be more readable.

As Dennis said, it is widely used in academia and in most estimation and control books

What words would you like to see appearing with the diagram?

It might be worth mentioning that:

the estimates come from EKF2
this is a standard cascaded position-velocity loop
depending on the mode, the outer (position) loop is bypassed. The position loop is only used when holding position or when the requested velocity in an axis is null (is this correct @Stifael ?).
the integrator in the inner loop (velocity) controller includes an anti-reset windup (ARW) using a clamping method
@Stifael Some details about what's happening in the Conversion block?

I made minor change to the index because the ( ) looked like a symbol - (). OK?

this is a standard cascaded position-velocity loop

It would be good to provide good links to information, tutorials, videos about the standard loop. Anything you recommend?

I also think that it is important that we do not try to explain everything our-self. Certain things are very standard (such as PID), and you can waste a lot of time trying to explain everything compared to just adding a reference link.

Absolutely. I think that the comments @bresch has made are about the right level but that is something for you to resolve.

It obviously needs to have a globally accepted px4 nomenclature and notation.
@bresch, what is the standing of the nomenclature document that you have been working on?

Whatever you guys propose will be the globally accepted "version 1". Lets get it in and iterate as needed.

Some details about what's happening in the Conversion block?

Sure, I can do that.
What is the next step? I can add links, explanation and so on once I know where to put them.

@Stifael I've created a branch on the devguide you should be able to contribute to - see the PR here: https://github.com/PX4/Devguide/pull/400
This has placeholders for the notation and an "initial" spot for the diagrams. Don't feel the structure is fixed - I just wanted to kick off the discussion/iteration.
After reviewing and adding to it, the next-next step is to mock up the other controller diagrams. Volunteers to either do direct in draw.io, or sketch for me to do first iteration?


AppleBusPowerController

I'm using Aero 15w and have burned a bootable usb drive(macOS 10.14.0 included).
But When I tried to boot my computer, it stopped with an prohibit symbol and cannot enter the installation.
This is the latest message showed in Verbose Mode:

AppleBusPowerController ... Power Supply Unavaliable..

I got a little confused and dont know how could I solve this problem.

I have no idea. I don’t have my laptop with me since it’s been sold for over half a year, but you can always ask this on TonyMacX86. People there would love to help you on this error message.


Fix Bar width calculation bug in time series bar charts

Purpose
Bug fix for Bar width calculation in a time series Bar chart which uses the property timeStep.
In the version 0.7.16 and later the bars of a time series created with use of timeStep property would overlap with each other due to a problem in the calculation of the barWidth
    this.ordinalDataChart = {
      x: 'Quarter',
      charts: [
        {
          type: 'bar',
          y: 'Sales',
          color: 'Product',
          mode: 'stacked',
        },
      ],
      legend: true,
      timeStep: 'year',
    };

    this.timeMetadata = {
      names: ['Quarter', 'Sales', 'Product'],
      types: ['time', 'linear', 'ordinal'],
    };

    this.timeData = [
      [1420050600000, 100000, 'Product 1'],
      [1451586600000, 14000, 'Product 1'],
      [1483209000000, 110000, 'Product 1'],
      [1514745000000, 240000, 'Product 1'],
    ];

The above configuration would render the following.

This PR will fix the issue in the calculation of barWidth
fixed:

Test environment
Node.JS v8.8.1, NPM v5.4.2

Thanks for the contribution


"mix compile" fails on std install: lib/surface/components/form.ex:101: the function "inner_block" cannot handle clauses with the -> operator because it is not a macro.

Describe the bug
On a clean Phoenix install, doing a standard surface install as per the instructions results in a compilation error:
== Compilation error in file lib/surface/components/form.ex ==
** (CompileError) lib/surface/components/form.ex:101: the function "inner_block" cannot handle clauses with the -> operator because it is not a macro. Please make sure you are invoking the proper name and that it is a macro
    expanding macro: Surface.sigil_F/2
could not compile dependency :surface, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile surface", update it with "mix deps.update surface" or clean it with "mix deps.clean surface"

How to reproduce it

Create a new clean Phoenix app mix phx.new AppName, follow instructions to execute mix deps.get etc
Add Surface to mix.exs:deps with the suggested line {:surface, "~> 0.9.4"}
Execute the suggested initialization command mix surface.init
The error occurs
Same error occurs when issuing mix surface.init --demo

The behavior you expected
a. mix surface.init initializes
b. I can compile and launch the app with Surface components working
Your Environment
Surface: v0.9.4
LiveView: v0.18.16
Phoenix: v1.7.2
Elixir: v1.14.1

Sorry, I found it is a duplicate of https://github.com/surface-ui/surface/issues/683 . Closing.