OOM with Influx 2.0.7 running inside a docker container with memory constrains

I'm facing an issue with influx 2.0.7 running in a docker container with memory constrains, I'm aware of memory tradeoffs for databases, and I'm ok with it being slower due to limited resources. I tried to limit all memory/buffer parameters that I could to keep it under control, but influx keeps dying from OOM while importing big historical datasets (1.5mi data points) into 5 measurements with about 5 tags.
Bellow my deployment code (ansible)
  - name: "Monitoring | Influxdb"
    docker_container:
      name: influx
      image: influxdb:2.0.7
      restart_policy: unless-stopped
      memory: 4g
      #memory_swap: 5
      #memory_swappiness: 5
      env:
        INFLUXDB_UDP_ENABLED: "true"
        INFLUXDB_REPORTING_DISABLED: "true"
        INFLUXDB_DATA_CACHE_MAX_MEMORY_SIZE: "2G"
        INFLUXDB_DATA_INDEX_VERSION: "tsi1"
        INFLUXDB_CONFIG_PATH: "/etc/influxdb2/influxdb.config"
      command:
        - "--storage-max-concurrent-compactions=1"
        - "--storage-series-file-max-concurrent-snapshot-compactions=1"
        - "--storage-compact-full-write-cold-duration=2h"
        - "--storage-cache-snapshot-write-cold-duration=5m"
        - "--query-max-memory-bytes=107374182"
        - "--http-read-timeout=0"
        - "--http-write-timeout=0"
        - "--bolt-path=/data/bolt"
        - "--engine-path=/data/engine"
        #- "--storage-series-id-set-cache-size=1024"
        - "--storage-retention-check-interval=48h"
        - "--query-queue-size=102400"
        - "--query-concurrency=16"
        - "--reporting-disabled=true"
      ports:
        - "8086:8086"
        - "8089:8089/udp"
      volumes:
        - "/data/backups/:/backups"
        - "influx-data:/data"
        - "/config/influx-configs:/etc/influxdb2/influx-configs"

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

Start influx with above command
Try to load some historical datasets
After a while

[605511.957732] influxd invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0
[605511.957735] influxd cpuset=ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc mems_allowed=0
[605511.957746] CPU: 2 PID: 17527 Comm: influxd Not tainted 4.19.0-16-amd64 #1 Debian 4.19.181-1
[605511.957748] Hardware name: Dell Inc. OptiPlex 3050/0JP3NX, BIOS 1.15.1 12/22/2020
[605511.957749] Call Trace:
[605511.957763]  dump_stack+0x66/0x81
[605511.957768]  dump_header+0x6b/0x283
[605511.957774]  oom_kill_process.cold.30+0xb/0x1cf
[605511.957782]  out_of_memory+0x1a5/0x450
[605511.957788]  mem_cgroup_out_of_memory+0xbe/0xd0
[605511.957794]  try_charge+0x63a/0x780
[605511.957801]  mem_cgroup_try_charge+0x86/0x190
[605511.957807]  ? pagecache_get_page+0x30/0x2c0
[605511.957812]  mem_cgroup_try_charge_delay+0x1c/0x40
[605511.957817]  do_swap_page+0x224/0x8d0
[605511.957822]  __handle_mm_fault+0x87c/0x11f0
[605511.957827]  ? __switch_to_asm+0x41/0x70
[605511.957833]  handle_mm_fault+0xd6/0x200
[605511.957838]  __do_page_fault+0x249/0x4f0
[605511.957844]  ? page_fault+0x8/0x30
[605511.957849]  page_fault+0x1e/0x30
[605511.957854] RIP: 0033:0x4272e6
[605511.957858] Code: ff c6 0f b6 3b 49 89 cb 89 f1 41 89 fc d3 ef 49 83 fb 08 74 0a 0f ba e7 04 0f 83 c3 00 00 00 41 0f a3 cc 90 73 af 4b 8d 3c 0b <48> 8b 3f 48 85 ff 74 a3 49 89 fc 4c 29 cf 48 39 d7 72 98 48 89 5c
[605511.957861] RSP: 002b:00007f65058bf850 EFLAGS: 00010247
[605511.957864] RAX:<PHONE_NUMBER>203016 RBX: 00007f64fa460dc8 RCX:<PHONE_NUMBER>000001
[605511.957867] RDX:<PHONE_NUMBER>001300 RSI:<PHONE_NUMBER>000001 RDI: 000000c05941b908
[605511.957869] RBP: 00007f65058bf8d0 R08: 00007f64fa5bffff R09: 000000c05941b900
[605511.957871] R10: 000000c000062698 R11:<PHONE_NUMBER>000008 R12:<PHONE_NUMBER>0000da
[605511.957872] R13:<PHONE_NUMBER>8448e0 R14:<PHONE_NUMBER>000000 R15:<PHONE_NUMBER>000000
[605511.957876] Task in /docker/ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc killed as a result of limit of /docker/ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc
[605511.957885] memory: usage 3983680kB, limit 4194304kB, failcnt 46943
[605511.957888] memory+swap: usage 8388608kB, limit 8388608kB, failcnt 2457
[605511.957890] kmem: usage 37268kB, limit<PHONE_NUMBER>740988kB, failcnt 0
[605511.957891] Memory cgroup stats for /docker/ebfab5d905bd35fddc9a6931e6b784d213f51c0a180292e17c1b4594ed15dffc: cache:28KB rss:3946172KB rss_huge:1671168KB shmem:0KB mapped_file:0KB dirty:0KB writeback:0KB swap:4405216KB inactive_anon:657844KB active_anon:3288512KB inactive_file:52KB active_file:4KB unevictable:0KB
[605511.957906] Tasks state (memory values in pages):
[605511.957907] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[605511.958069] [  16803]  1000 16803  2572919<PHONE_NUMBER>4480  1103566             0 influxd
[605511.958075] Memory cgroup out of memory: Kill process 16803 (influxd) score 1003 or sacrifice child
[605511.958169] Killed process 16803 (influxd) total-vm:10291676kB, anon-rss:3935536kB, file-rss:52940kB, shmem-rss:0kB
[605513.118538] oom_reaper: reaped process 16803 (influxd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Expected behavior:
Guidance on how to run influxdb with limited memory constrains without suffering such problems.
Actual behavior:
Influx dies and get stuck into a restart loop due to OOM.
Environment info:

System info: Linux 4.19.0-16-amd64 x86_64
InfluxDB version: InfluxDB 2.0.7 (git: 2a45f0c037) build_date: 2021-06-04T19:17:40Z
Other relevant environment details: Running on docker, in a Debian linux on a bare metal machine.

Config:
Configuration is set on command line arguments on the above snippet
Logs:
Logs are quite big and with a lot of very similar entries as bellow
ts=2021-06-30T06:13:52.067412Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2205 duration=38.368ms
ts=2021-06-30T06:13:52.070047Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2150/_00001.wal size=7009272
ts=2021-06-30T06:13:52.100446Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.103596Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2207/000000008-000000001.tsm id=1 duration=0.427ms
ts=2021-06-30T06:13:52.105162Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.105155Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2207/000000006-000000002.tsm id=0 duration=1.140ms
ts=2021-06-30T06:13:52.106545Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2207 duration=38.987ms
ts=2021-06-30T06:13:52.129169Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2206/000000006-000000002.tsm id=0 duration=15.406ms
ts=2021-06-30T06:13:52.129476Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.129518Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2206 duration=78.383ms
ts=2021-06-30T06:13:52.157937Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2208/000000006-000000002.tsm id=0 duration=11.988ms
ts=2021-06-30T06:13:52.158635Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2208 duration=51.819ms
ts=2021-06-30T06:13:52.215262Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.226000Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2151/_00001.wal size=7610704
ts=2021-06-30T06:13:52.253481Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:13:52.262175Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2152/_00001.wal size=6853353
ts=2021-06-30T06:14:05.797081Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2149 duration=13868.619ms
ts=2021-06-30T06:14:05.870651Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:05.882055Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2209/000000007-000000001.tsm id=1 duration=0.894ms
ts=2021-06-30T06:14:05.886611Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2209/000000005-000000002.tsm id=0 duration=3.016ms
ts=2021-06-30T06:14:05.888572Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2209 duration=89.604ms
ts=2021-06-30T06:14:06.038251Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.047253Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2153/_00001.wal size=6063149
ts=2021-06-30T06:14:06.194766Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2150 duration=14194.929ms
ts=2021-06-30T06:14:06.252163Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.255755Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/221/_00001.wal size=58935
ts=2021-06-30T06:14:06.334527Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/221 duration=139.432ms
ts=2021-06-30T06:14:06.445857Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2152 duration=14286.931ms
ts=2021-06-30T06:14:06.495199Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.499168Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2151 duration=14369.603ms
ts=2021-06-30T06:14:06.499803Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2154/_00001.wal size=6955768
ts=2021-06-30T06:14:06.532590Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.556118Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2210/000000006-000000002.tsm id=0 duration=2.915ms
ts=2021-06-30T06:14:06.556807Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2210 duration=109.049ms
ts=2021-06-30T06:14:06.619907Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.629603Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2211/000000007-000000001.tsm id=1 duration=0.832ms
ts=2021-06-30T06:14:06.632960Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2211/000000005-000000002.tsm id=0 duration=3.081ms
ts=2021-06-30T06:14:06.633474Z lvl=info msg="Opened shard" log_id=0V2tdsJG000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/data/engine/data/591fc0b0c200d622/autogen/2211 duration=76.374ms
ts=2021-06-30T06:14:06.672583Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.678932Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2155/_00001.wal size=7561101
ts=2021-06-30T06:14:06.787156Z lvl=info msg="index opened with 8 partitions" log_id=0V2tdsJG000 service=storage-engine index=tsi
ts=2021-06-30T06:14:06.801331Z lvl=info msg="Opened file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=filestore path=/data/engine/data/591fc0b0c200d622/autogen/2156/000000002-000000002.tsm id=0 duration=11.179ms
ts=2021-06-30T06:14:06.801960Z lvl=info msg="Reading file" log_id=0V2tdsJG000 service=storage-engine engine=tsm1 service=cacheloader path=/data/engine/wal/591fc0b0c200d622/autogen/2156/_00005.wal size=9032745
#

Performance:
Due to the fact that influxdb get stuck into a restart loop and don't start the http interface makes hard to run the command to get profiler information. I'm awaiting for further instructions on how to do it.
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.

Same issue in my company, we were considering migrating from timescale to influx, but due to this we can't proceed with testing. Apparently timescale is much better at memory management

@danxmoran let me know if you need any additional information regarding this bug

I still having the same problem, simulated the same scenario on timescaledb with 1G of ram, it could keep up with the metrics pretty well. Looking forward to have at least the same performance with influxdb 2 on 8gb of ram.

@opsxcq would you be able to tell us anything more about your historical dataset? You mention the number of series and tags in the original issue which is great information - I'm also wondering about how the data is distributed over time, etc. If you have a sample of the data that you could provide that would be great as well. Also how are you trying to import the data - using the influx CLI or some other way?
@angelademarco similar questions for you...from your description in #21766, I've been running some tests with an EC2 instance with 4GB of ram with influxd running docker and am able to OOM it with certain kinds of data (particularly if the data points are spread out over a large / randomized timeframe) but not with others. I'm synthetically generating my data of course; if you have more information you could share about the data you are experiencing these crashes with that would be very useful! Sounds like you are able to simulate the the data/crashes so it'd be great if there's anything you could provide about how you are doing that I'm sure that would be helpful too!

@wbaker85 yes, it is the historical market data since 1950, on daily intervals (range goes from 100 to around 8000 symbols, it chances over time) + recent data with more granularity (hourly, minutes (1,5,15)) but only for the last few months, I stil didn't try to import all my ticker data.
I split them in two measurements, one is the tickers and the other is "daily" which contains the data for daily prices.

For reference, here's a related recent issue in the forums:
https://community.influxdata.com/t/execution-of-heavy-queries-result-in-a-crash/22637/3
I have also encountered crashes on bigger queries and suppose it's a similar issue, but couldn't investigate further yet.

Just wanted to mention this is still an issue in v2.2 - I can run the same query multiple times, but it appears there is some sort of memory leak and the influx instance eventually uses all available system memory, then causes the entire server to freeze until a  hard restart. Granted I'm not running in docker like OP but still hitting that out of memory error:

Sometimes the server can recover after an hour or so, sometimes it just stays frozen until I notice and restart it.

I think this is happening to every one using influxdb version 2.X, Actually it doesn't happen with the version 1.8.
the memory management on influx 1.8 looks like this.

But in influx 2.3 it just die after a while because run out of memory

@riosje I can confirm that the same issue happens with the influxdb:2.6.0 docker image.

Hello! I can confirm that the same issue happens with the influxdb:2.7.6 docker image.
I have a big database with many shards.
A result of the simple command:
docker run  --user=influxdb -d -p 8086:8086  --name influxdb --env-file env.list -v /home/influxdb:/var/lib/influxdb2 influxdb:2.7.6-alpine

is OOM, because the server will allocate memory for each new WAL and index. The server will allocate 20 GiByte of memory, and it will stop with an error. The instance limit is 20 GiByte.
The workaround:
docker rm influxdb
# remove indexes
find /home/influxdb/engine/data/ -type d -name _series -exec rm -r {} +
find /home/influxdb/engine/data/ -type d -name index -exec rm -r {} +
# start
docker run  --user=influxdb -d -p 8086:8086  --name influxdb --env-file env.list -v /home/influxdb:/var/lib/influxdb2 influxdb:2.7.6-alpine

The server will recreate indexes. The server will allocate only 7 GiByte of memory. It will work well.
The full workaround solution:
[Unit]
Description=InfluxDB Service
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop %n
ExecStartPre=-/usr/bin/docker rm %n
ExecStartPre=/usr/bin/docker pull influxdb:2.7.6-alpine
ExecStartPre=find /home/influxdb/engine/data/ -type d -name _series -exec rm -r {} +
ExecStartPre=find /home/influxdb/engine/data/ -type d -name index -exec rm -r {} +
ExecStart=docker run --rm --user=influxdb -d -p 8086:8086 -m 16g --name %n --env-file env.list -v /home/influxdb:/var/lib/influxdb2 influxdb:2.7.6-alpine
ExecStop=/usr/bin/docker stop %n

[Install]
WantedBy=default.target

Cons:

the restart operation is more slowly

this is wild, Influx is requiring way too much RAM to barely operate, and with the release of Influx V3 they will not fix this, because they need the people buy his new license.
I just moved out to victoriaMetrics, the performance of that solution is really good, also the architecture is way better than influx.

offtop_mode_on

I just moved out to victoriaMetrics
My teammates would like to have a storage without strict cardinality limits. And they would like to use some complex scripts for getting metrics. They are familiar with Flux, but not with MetricQL.

I used NGinx-proxy for data replication operations:
https://gist.github.com/polarnik/cb6f22751e8d1590342198609243c529
And teammates had similar data in VictoriaMetrics and in InfluxDB. This is an old solution.
We are using InfluxDB for raw data and complex Flux queries. And VictoriaMetrics for aggregates and alerts, only. VictoriaMetrics has some limits, too. We use it for clean data, only
offtop_mode_off

I still have the OOM problem.
But I have logs. The root cause of the OOM problem is an operation: "TSI log compaction".
The first message was about "TSI log compaction (start)":
May 14 11:09:23 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:23.317877Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=8 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
The operation "TSI log compaction" finished in two minutes, only:
May 14 11:11:53 influxdb f7d421060a18[742]: ts=2024-05-14T11:11:00.371732Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=8 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
Because the OOM happen:
May 14 11:11:54 influxdb f7d421060a18[742]: memory allocation of 1056 bytes failed
May 14 11:11:54 influxdb f7d421060a18[742]: SIGABRT: abort
May 14 11:11:54 influxdb f7d421060a18[742]: PC=0x7f443269c792 m=149 sigcode=18446744073709551610
May 14 11:11:54 influxdb f7d421060a18[742]: signal arrived during cgo execution
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 677421 [syscall]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.cgocall(0x7f443235d3e0, 0xc218f518a8)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/cgocall.go:157 +0x4b fp=0xc218f51880 sp=0xc218f51848 pc=0x7f44305bab8b
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/libflux/go/libflux._Cfunc_flux_analyze(0x7f42973f04d0, 0x7f380360bb20, 0xc0189b1310)
May 14 11:11:54 influxdb f7d421060a18[742]: #011_cgo_gotypes.go:122 +0x50 fp=0xc218f518a8 sp=0xc218f51880 pc=0x7f4430b661f0
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/libflux/go/libflux.AnalyzeWithOptions.func3(0xc218f51948?, 0x2?, 0x2?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/influxdata/flux@v0.194.5/libflux/go/libflux/analyze.go:142 +0x7d fp=0xc218f518f0 sp=0xc218f518a8 pc=0x7f4430b680bd
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/libflux/go/libflux.AnalyzeWithOptions(0xc027fecc88, {{0x0?, 0x7f44339026c0?, 0x7f44335f69a0?}})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/influxdata/flux@v0.194.5/libflux/go/libflux/analyze.go:142 +0x169 fp=0xc218f519f8 sp=0xc218f518f0 pc=0x7f4430b67d09
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/runtime.AnalyzePackage({0x7f4433a0a9e8?, 0xc1b0121620?}, {0x7f4433a05c30?, 0xc027fecc88})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/influxdata/flux@v0.194.5/runtime/analyze_libflux.go:23 +0xb2 fp=0xc218f51a78 sp=0xc218f519f8 pc=0x7f4430b723f2
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/runtime.(*runtime).Eval(0x7f44363628a0, {0x7f4433a0a9e8, 0xc1b0121620}, {0x7f4433a05c30?, 0xc027fecc88?}, {0x7f44339fc378, 0x7f443639ff20}, {0xc140df0580, 0x2, 0x2})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/influxdata/flux@v0.194.5/runtime/runtime.go:102 +0x85 fp=0xc218f51af8 sp=0xc218f51a78 pc=0x7f4430b742a5
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/lang.(*AstProgram).getSpec(0xc1732f34a0, {0x7f4433a0a9e8, 0xc1b01215f0}, {0x7f44339e35f0?, 0x7f4433475460?})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/influxdata/flux@v0.194.5/lang/compiler.go:446 +0x2e3 fp=0xc218f51c88 sp=0xc218f51af8 pc=0x7f44314b3123
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/flux/lang.(*AstProgram).Start(0xc1732f34a0, {0x7f4433a0a9e8, 0xc1b01213e0}, {0x7f4433a0c310, 0xc16ff75f90})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/influxdata/flux@v0.194.5/lang/compiler.go:484 +0x1c9 fp=0xc218f51e98 sp=0xc218f51c88 pc=0x7f44314b3ca9
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/influxdb/v2/query/control.(*Controller).executeQuery(0xc218f51fa8?, 0xc1d4e4a1a0)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/query/control/controller.go:489 +0x219 fp=0xc218f51f48 sp=0xc218f51e98 pc=0x7f4432188df9
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/influxdb/v2/query/control.(*Controller).processQueryQueue(...)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/query/control/controller.go:447
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/influxdb/v2/query/control.New.func1()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/query/control/controller.go:232 +0x76 fp=0xc218f51fe0 sp=0xc218f51f48 pc=0x7f44321871b6
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc218f51fe8 sp=0xc218f51fe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by github.com/influxdata/influxdb/v2/query/control.New in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/query/control/controller.go:230 +0x9ec
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 1 [chan receive, 120 minutes]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x4b939adcbb1a5?, 0x40000000?, 0x0?, 0x0?, 0x8bb2c97000?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc19a77bbc8 sp=0xc19a77bba8 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.chanrecv(0xc000637440, 0x0, 0x1)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/chan.go:583 +0x3cd fp=0xc19a77bc40 sp=0xc19a77bbc8 pc=0x7f44305bd1ad
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.chanrecv1(0xc000e942a0?, 0x7f4433a0aa20?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/chan.go:442 +0x12 fp=0xc19a77bc68 sp=0xc19a77bc40 pc=0x7f44305bcdb2
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/influxdb/v2/cmd/influxd/launcher.NewInfluxdCommand.cmdRunE.func1()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/cmd/influxd/launcher/cmd.go:127 +0x156 fp=0xc19a77bd00 sp=0xc19a77bc68 pc=0x7f4432256416
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/influxdata/influxdb/v2/kit/cli.NewCommand.func1(0xc000c58900?, {0x7f443639ff20?, 0x4?, 0x7f44326a9e5b?})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/kit/cli/viper.go:54 +0x16 fp=0xc19a77bd10 sp=0xc19a77bd00 pc=0x7f4431059a96
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/spf13/cobra.(*Command).execute(0xc000c65b80, {0xc00011e0b0, 0x0, 0x0})
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842 +0x694 fp=0xc19a77bdf8 sp=0xc19a77bd10 pc=0x7f4430fdc654
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/spf13/cobra.(*Command).ExecuteC(0xc000c65b80)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950 +0x389 fp=0xc19a77beb0 sp=0xc19a77bdf8 pc=0x7f4430fdcc09
May 14 11:11:54 influxdb f7d421060a18[742]: github.com/spf13/cobra.(*Command).Execute(...)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
May 14 11:11:54 influxdb f7d421060a18[742]: main.main()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/root/project/cmd/influxd/main.go:61 +0x50a fp=0xc19a77bf40 sp=0xc19a77beb0 pc=0x7f443228c84a
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.main()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:267 +0x2d2 fp=0xc19a77bfe0 sp=0xc19a77bf40 pc=0x7f44305f1892
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc19a77bfe8 sp=0xc19a77bfe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 2 [force gc (idle), 122 minutes]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000084fa8 sp=0xc000084f88 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goparkunlock(...)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:404
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.forcegchelper()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:322 +0xb8 fp=0xc000084fe0 sp=0xc000084fa8 pc=0x7f44305f1b78
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000084fe8 sp=0xc000084fe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.init.6 in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:310 +0x1a
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 3 [runnable]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goschedIfBusy()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:361 +0x28 fp=0xc000085778 sp=0xc000085760 pc=0x7f44305f1c28
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.bgsweep(0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgcsweep.go:305 +0x151 fp=0xc0000857c8 sp=0xc000085778 pc=0x7f44305dbdd1
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcenable.func1()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000857e0 sp=0xc0000857c8 pc=0x7f44305d0e65
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000857e8 sp=0xc0000857e0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcenable in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:200 +0x66
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 4 [GC scavenge wait]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x975898e?, 0x8286c0?, 0x0?, 0x0?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000085f70 sp=0xc000085f50 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goparkunlock(...)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:404
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.(*scavengerState).park(0x7f4436368800)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000085fa0 sp=0xc000085f70 pc=0x7f44305d95c9
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.bgscavenge(0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000085fc8 sp=0xc000085fa0 pc=0x7f44305d9b79
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcenable.func2()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:201 +0x25 fp=0xc000085fe0 sp=0xc000085fc8 pc=0x7f44305d0e05
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcenable in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:201 +0xa5
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 18 [finalizer wait]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x0?, 0x7f44339dd3d0?, 0x40?, 0xf?, 0x1000000010?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000084620 sp=0xc000084600 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.runfinq()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000847e0 sp=0xc000084620 pc=0x7f44305cfe87
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000847e8 sp=0xc0000847e0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.createfing in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mfinal.go:163 +0x3d
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 19 [GC worker (idle), 2 minutes]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x7f44363a2c40?, 0x3?, 0xcc?, 0x31?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000080750 sp=0xc000080730 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0000807e0 sp=0xc000080750 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000807e8 sp=0xc0000807e0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcBgMarkStartWorkers in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1219 +0x1c
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 34 [GC worker (idle), 2 minutes]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x7f44363a2c40?, 0x3?, 0x2a?, 0xf?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000486750 sp=0xc000486730 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0004867e0 sp=0xc000486750 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0004867e8 sp=0xc0004867e0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcBgMarkStartWorkers in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1219 +0x1c
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 5 [GC worker (idle)]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x22c370d133d90?, 0x1?, 0x7?, 0x79?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000086750 sp=0xc000086730 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0000867e0 sp=0xc000086750 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000867e8 sp=0xc0000867e0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcBgMarkStartWorkers in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1219 +0x1c
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 20 [GC worker (idle), 2 minutes]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x22c1e446c85de?, 0x3?, 0x84?, 0xcb?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000080f50 sp=0xc000080f30 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000080fe0 sp=0xc000080f50 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000080fe8 sp=0xc000080fe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcBgMarkStartWorkers in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1219 +0x1c
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 35 [GC worker (idle)]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x22c3de1809b1a?, 0x1?, 0x65?, 0x7b?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000486f50 sp=0xc000486f30 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000486fe0 sp=0xc000486f50 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000486fe8 sp=0xc000486fe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcBgMarkStartWorkers in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1219 +0x1c
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 6 [GC worker (idle)]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x7f44339d89b0?, 0xc0001540a0?, 0x1a?, 0x14?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000086f50 sp=0xc000086f30 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc000086fe0 sp=0xc000086f50 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000086fe8 sp=0xc000086fe0 pc=0x7f4430625421
May 14 11:11:54 influxdb f7d421060a18[742]: created by runtime.gcBgMarkStartWorkers in goroutine 1
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1219 +0x1c
May 14 11:11:54 influxdb f7d421060a18[742]: goroutine 21 [GC worker (idle)]:
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gopark(0x22c3de1807bf7?, 0x3?, 0x2c?, 0xa1?, 0x0?)
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/proc.go:398 +0xce fp=0xc000081750 sp=0xc000081730 pc=0x7f44305f1d0e
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.gcBgMarkWorker()
May 14 11:11:54 influxdb f7d421060a18[742]: #011/go/src/runtime/mgc.go:1295 +0xe5 fp=0xc0000817e0 sp=0xc000081750 pc=0x7f44305d2a25
May 14 11:11:54 influxdb f7d421060a18[742]: runtime.goexit()

My settings:
            "Env": [
                "INFLUXD_REPORTING_DISABLED=true",
                "INFLUXD_STORAGE_CACHE_SNAPSHOT_WRITE_COLD_DURATION=10m0s",
                "INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=1h0m0s",
                "INFLUXD_STORAGE_COMPACT_THROUGHPUT_BURST=80388608",
                "INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=2",
                "INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=2",
                "INFLUXDB_DATA_INDEX_VERSION=\"tsi1\"",
                "INFLUXDB_DATA_CACHE_SNAPSHOT_MEMORY_SIZE=\"200m\"",
                "INFLUXDB_DATA_MAX_INDEX_LOG_FILE_SIZE=10485760",
                "INFLUXDB_DATA_SERIES_ID_SET_CACHE_SIZE=100",
                "INFLUXD_QUERY_MEMORY_BYTES=304857600",
                "INFLUXD_QUERY_INITIAL_MEMORY_BYTES=10485760",
                "INFLUXD_QUERY_CONCURRENCY=5",
                "INFLUXD_STORAGE_CACHE_MAX_MEMORY_SIZE=1073741824",
                "INFLUXD_STORAGE_CACHE_SNAPSHOT_MEMORY_SIZE=262144000",
                "INFLUXD_QUERY_QUEUE_SIZE=100",
                "INFLUXD_FLUX_LOG_ENABLED=true",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "INFLUXDB_VERSION=2.7.6",
                "INFLUX_CLI_VERSION=2.7.3",
                "INFLUX_CONFIGS_PATH=/etc/influxdb2/influx-configs",
                "INFLUXD_INIT_PORT=9999",
                "INFLUXD_INIT_PING_ATTEMPTS=600",
                "DOCKER_INFLUXDB_INIT_CLI_CONFIG_NAME=default"
            ],

I used some limits (2) for compactions settings:

Duration at which the storage engine will compact all TSM files in a shard if it hasn’t received writes or deletes.

export INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=4h0m0s
my
export INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=1h0m0s

Rate limit (in bytes per second) that TSM compactions can write to disk.
export INFLUXD_STORAGE_COMPACT_THROUGHPUT_BURST=50331648
my
export INFLUXD_STORAGE_COMPACT_THROUGHPUT_BURST=80388608

Maximum number of full and level compactions that can run concurrently. A value of 0 results in 50% of runtime.GOMAXPROCS(0) used at runtime. Any number greater than zero limits compactions to that value. This setting does not apply to cache snapshotting.

export INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=0
my
export INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=2

Maximum number of snapshot compactions that can run concurrently across all series partitions in a database.

export INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=0
my:
export INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=2
I have some duplicate lines in the logs:
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.257996Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=4 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.260277Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=6 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.260525Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=5 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.262446Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=7 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.264009Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=3 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.265600Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=6 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.267097Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=7 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.269065Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=1 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.260268Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=8 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.269662Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=8 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.270772Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=2 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.274354Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=4 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.283160Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=1 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.289673Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=5 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.291732Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=3 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.291732Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=2 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.298355Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=6 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start
May 14 11:09:25 influxdb f7d421060a18[742]: ts=2024-05-14T11:09:25.305572Z lvl=info msg="TSI log compaction (start)" log_id=0p9aevYl000 service=storage-engine index=tsi tsi1_partition=4 op_name=tsi1_compact_log_file tsi1_log_file_id=1 op_event=start

Maybe the server has some race condition problem, because 2 threads work in parallel with a same file.
I'm going to try
INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=1
INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=1

@philjb
I will use settings with MEM Limit and GC settings, and I have disabled INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION:
env.list

GOMEMLIMIT=20GiB
GOGC=10
INFLUXD_REPORTING_DISABLED=true
INFLUXD_STORAGE_CACHE_SNAPSHOT_WRITE_COLD_DURATION=1000d
INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=1000d
INFLUXD_STORAGE_COMPACT_THROUGHPUT_BURST=80388608
INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=1
INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=1
INFLUXDB_DATA_INDEX_VERSION="tsi1"
INFLUXDB_DATA_CACHE_SNAPSHOT_MEMORY_SIZE="200m"
INFLUXDB_DATA_MAX_INDEX_LOG_FILE_SIZE=10485760
INFLUXDB_DATA_SERIES_ID_SET_CACHE_SIZE=100

INFLUXD_QUERY_MEMORY_BYTES=304857600
INFLUXD_QUERY_INITIAL_MEMORY_BYTES=10485760
INFLUXD_QUERY_CONCURRENCY=5
INFLUXD_STORAGE_CACHE_MAX_MEMORY_SIZE=1073741824
INFLUXD_STORAGE_CACHE_SNAPSHOT_MEMORY_SIZE=262144000

INFLUXD_QUERY_QUEUE_SIZE=100
INFLUXD_FLUX_LOG_ENABLED=false

I still use the hack with reindexing instead of compaction:
find /var/lib/influxdb2/engine/data/ -type d -name _series -exec rm -r {} + &&\
find /var/lib/influxdb2/engine/data/ -type d -name index -exec rm -r {} +

the entire command line

docker run --shm-size 2g -m 25GiB --user=influxdb --restart=on-failure --restart unless-stopped --entrypoint '/bin/bash' -d -p 8086:8086 --log-driver=syslog --name influx --env-file env.list -v /home/influxdb:/var/lib/influxdb2 influxdb:2.7.6-alpine --verbose -c "find /var/lib/influxdb2/engine/data/ -type d -name _series -exec rm -r {} + && find /var/lib/influxdb2/engine/data/ -type d -name index -exec rm -r {} + && /entrypoint.sh influxd"

It works well.
I saw the memory allocation error before meeting the memory limit:
memory

My memory limits are:

30 GiByte in a station
25 GiByte in a docker container
20 GiByte in GOMEMLIMIT

The current memory allocation is 5-6 GiByte.
I did't see memory allocation ~= 20 GiByte, I had the allocation ~= 10 GiByte, only, but I had a container-restart-state.

I have too many influxdb threads == 71:
memory

I have calculated how many files they use:
lsof > /tmp/lsof.info2.txt
cat /tmp/lsof.info2.txt | grep influx | awk '{ print $11 }' > /tmp/files.txt
cat /tmp/files.txt| sort > /tmp/files.sorted.txt
cat /tmp/files.sorted.txt | uniq -c > /tmp/counts.txt

71 threads use 73 000 file descriptors.
There are 5 183 000 file descriptors in sum.
The influxdb process and all threads (green threads) could reach some limit, but it might not be the memory limit, but the virtual memory limit or the file descriptor limit.
What do you think ?

I only skimmed through your response - you can set GOMAXPROCS to limit the number of OS threads, but I believe those processes showing in htop are golang's greenthreads - you can't limit those. See https://pkg.go.dev/runtime I don't think the number of green threads should be an issue? I'm not aware of a limit for those from linux.
Influxdb can use a lot of file descriptors - you can raise it with ulimit (as you probably know).

My current recipe

Remove indexes
Disable index compactions

The custom command
docker run --shm-size 2g --user=influxdb --restart=on-failure --restart unless-stopped --entrypoint '/bin/bash' -d -p 8086:8086 --log-driver=syslog --name influx --env-file env.list -v /home/influxdb:/var/lib/influxdb2 influxdb:2.7.6-alpine --verbose -c "find /var/lib/influxdb2/engine/data/ -type d -name index -exec rm -r {} + && /entrypoint.sh influxd"
and the custom config
INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=48h
INFLUXD_STORAGE_SERIES_ID_SET_CACHE_SIZE=0

work well. Docker container restarts every 48 hours, only. I have started it at night. It have restarted at night in 48 hours.
The option
INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=1000d
didn't work well. It was equal to 3h.
env.list

GOMEMLIMIT=25GiB
GOGC=10

INFLUXD_REPORTING_DISABLED=true
INFLUXD_STORAGE_CACHE_SNAPSHOT_WRITE_COLD_DURATION=10m
INFLUXD_STORAGE_COMPACT_FULL_WRITE_COLD_DURATION=48h
INFLUXD_STORAGE_COMPACT_THROUGHPUT_BURST=80388608
INFLUXD_STORAGE_MAX_CONCURRENT_COMPACTIONS=1
INFLUXD_STORAGE_SERIES_FILE_MAX_CONCURRENT_SNAPSHOT_COMPACTIONS=1

INFLUXD_QUERY_MEMORY_BYTES=304857600
INFLUXD_QUERY_INITIAL_MEMORY_BYTES=10485760
INFLUXD_QUERY_CONCURRENCY=5
INFLUXD_STORAGE_CACHE_MAX_MEMORY_SIZE=1073741824
INFLUXD_STORAGE_CACHE_SNAPSHOT_MEMORY_SIZE=26214400
INFLUXD_STORAGE_WAL_MAX_WRITE_DELAY=10m
INFLUXD_STORAGE_WRITE_TIMEOUT=10s
INFLUXD_STORAGE_WAL_MAX_CONCURRENT_WRITES=6
INFLUXD_STORAGE_SERIES_ID_SET_CACHE_SIZE=0

INFLUXD_QUERY_QUEUE_SIZE=100
INFLUXD_FLUX_LOG_ENABLED=false

docker run --shm-size 2g --user=influxdb --restart=on-failure --restart unless-stopped -d -p 8086:8086 --log-driver=syslog --name influx --env-file env.list -v /home/influxdb:/var/lib/influxdb2 influxdb:2.7.6-alpine

The simple command has a memory allocation error. The error occurs at a memory allocation point ~ of 25% (6 GiByte from  25-30 GiByte). A server and a Docker container have available RAM.


Is it possible to train just the projector not the LLM and the encoder?

Currently we don't support that (assuming one always tunes the LLM). We can support finetuning only the encoder in the near future (this strategy is a bit weird to me though). For the time being you can approximate this with super small lora hyperparameters (r and alpha) for LLM.

Gotcha, thanks! @zjysteven, a follow up question.
Is it possible to finetune using Lora and do further lora finetuning on top of that already finetuned model? If so, how would that be done? I tried it and it gave me an error saying many modules are not found. I wonder if it's cuz the peft modules were not merged in in the first finetuning.  Thanks in advance!

First see if the discussion https://github.com/zjysteven/lmms-finetune/issues/29#issuecomment-2313703559 here helps. If not, you can share more details of your exact case and we can go from there to solve it.

thanks @zjysteven I actually tried that, doing further training on top of an already finetuned model by providing the checkpoint. The issue arises after the second training is complete and the saved final model couldn't be ingested by
LlavaNextVideoForConditionalGeneration.from_pretrained
I am in the middle of trying something. After the first lora finetuning is complete. I would merge the lora weights into the base model. And then use the merged model to do the secondary finetuning. The final model of this process seems to be ingestible by LlavaNextVideoForConditionalGeneration.from_pretrained but the model seems collapsed. I am gonna try to look into that more if that's a mistake on my part or not.

I see. Yeah it is something we haven't thoroughly tested, and what you described makes sense to me. Theoretically though if we manually merge lora weights before each new round of finetuning, it should be working smoothly. Will add this merging part to the codebase in the near future.
Meanwhile by "collapsed" what do you exactly mean? I can also see if I have some inputs on helping diagnose this.

@zjysteven I need to test again to make sure, but what I mean by "collapsed" is that the final model started to return gibberish

"Prompt containing options" should be fine, as there are training instances formatted like this in llava-1.5; not exactly sure if this is the case for llava-next-video, but I would assume so.
One tip from llava-1.5 is that they find adding an instructing prompt like "Answer the question with a single word/option" at the end of the prompt can help with the performance (see Sec. 3.2 in "Improved Baselines with Visual Instruction Tuning"). I would suggest to try this.
Another thought I have is you probably want to be careful with the training epochs, learning rate, and lora hyperparams when doing the second-round finetuning for classification. If this round is overly trained (like high learning rate, high lora_r) then I could see the possibility of model overfitting on doing classification, while forgetting how to follow other instructions.

@TonyJiang17 Sorry I missed your last question. Yes you can configure it by certain arguments of huggingface trainer like "save_strategy" and "save_steps" https://huggingface.co/docs/transformers/v4.45.2/en/main_classes/trainer#transformers.TrainingArguments.save_strategy


Add support for secure token storage

Token storage change in latest release of gh
This is a message from the GitHub CLI team, maintainers of gh, writing to inform you that the most recent release of gh contains changes which may affect your extension. The latest release introduces the feature of storing authentication tokens in the system keyring (encrypted storage) instead of in a plain text file.
The keyrings that are supported are:

Keychain on macOS

GNOME Keyring on Linux (Secret Service dbus interface)

Wincred on Windows

This has huge security benefits for the users of our tool and was one of our oldest outstanding issues. Unfortunately this change has the potential to break extensions that rely on utilizing the users authentication token to work.
In order to have continued compatibility with gh there are some actions you, as an extension author, need to take. These actions will depend on the implementation of your extension.
Extensions built in Go using go-gh:

Upgrade your go-gh version to v1.2.1, the latest version.

This can be done using go get<EMAIL_ADDRESS>

Verify that in your extension retrieval of the user authentication token is done using the  auth.TokenForHost function.

If you were previously accessing the authentication token using any other method it will no longer work.
Automatic resolution of the authentication token when using the API clients will continue to work without changes.

All other extensions:

Verify that in your extension retrieval of the user authentication token is done by shelling out to the gh auth token command.

If you were previously accessing the authentication token using the gh config get command, reading the configuration file directly, or any other methods it will no longer work.

As of right now storing the authentication token in the system keyring is an opt-in feature, but in the near future it will be required and at that point if the changes above are not made then your extension will be broken for all users. If you have any questions/concerns about this change please feel free to open a discussion in the gh repo.
Thanks,
The GitHub CLI Team

Thanks for the heads-up! Bumped go-gh to v1.2.1 in https://github.com/lucasmelin/gh-hook/releases/tag/v0.1.3 and verified that only the API client is being used to automatically resolve the API token.


Agency theme skeleton form's not working

Hi,
I've been having the toughest time trying to get the forms to show up properly in the agency theme skeleton page. When I go to "simple_form.yaml" to set "enabled: true" I get the following error.
E_COMPILE_ERROR - Access level to Grav\Plugin\Simple_FormPlugin::mergeConfig() must be protected (as in class Grav\Common\Plugin) or weaker
I'm not much of a web programmer and can't really debug this myself. Would it be possible to update the skeleton to fix the issue?

Looks like the skeleton ships Simple Form 1.2.1 which has issues with the current Grav. Latest release is 1.3.2 and has solved the problem.
Updating the plugin through CLI (bin/gpm update) fixes the problem.
@rhukster does the skeleton needs to be regenerated to use the latest available release of Simple Form?

I updated the simple form, but I think a few things need to change for the theme skeleton as well since I couldn't get the form to pop up.

Check the plugin is enabled, I got it working fine

Thanks! I updated with bin/gpm and worked fine.
On Thu, Mar 10, 2016 at 1:14 AM, Flavio Copes<EMAIL_ADDRESS>wrote:

Check the plugin is enabled, I got it working fine
—
Reply to this email directly or view it on GitHub
https://github.com/getgrav/grav-theme-agency/issues/6#issuecomment-194746893
.

--
Bryan


feat(exporter): Add ndm exporters support from helm chart

Why is this PR required? What issue does it fix?:
This PR adds support for installation of ndm exporters(cluster level and disk level)through helm chart to get metrics from each of the storage nodes in a cluster.
What this PR does?:
NDM exporters installaton support added to be able to build dashboards about storage disk metrics.

By default, the exporters are disabled in the helm charts but can be enabled with of a field in values.yaml i.e ndmExporter.enabled.
This PR also adds service yaml for the individual exporters while installing via helm chart.
This PR is also required for Service Discovery of exporters in Prometheus(will be taken care in the openebs monitoring repo).

Does this PR require any upgrade changes?:
If the changes in this PR are manually verified, list down the scenarios covered::
Installed helm charts locally in a cluster from the root directory of the repo via the following command:
helm install openebs-ndm -n openebs ./deploy/helm/charts/

After successful installation, both ndm cluster exporter and the ndm node exporter got deployed in the cluster with their corresponding services.
master@master-virtual-machine:~/go/src/github.com/Ab-hishek/node-disk-manager$ kubectl get pods -n openebs
NAME                                                              READY   STATUS    RESTARTS   AGE
cspc-operator-6d796d7b8f-9j5pc                                    1/1     Running   0          34d
cstor-storage-d2rf-5c5f79f6b6-7rwk5                               3/3     Running   0          15d
cstor-storage-q7q4-c9fb79bbd-wrm4f                                3/3     Running   0          15d
cvc-operator-849dff65c-lm8px                                      1/1     Running   0          34d
jiva-operator-77d7db7846-blvdm                                    1/1     Running   0          34d
maya-apiserver-fd47769d-qvzjj                                     1/1     Running   0          34d
openebs-admission-server-8589bdffdc-tk4bp                         1/1     Running   0          34d
openebs-cstor-admission-server-5d868c64d4-5k8qx                   1/1     Running   0          34d
openebs-cstor-csi-controller-0                                    6/6     Running   0          34d
openebs-cstor-csi-node-4hds7                                      2/2     Running   0          34d
openebs-cstor-csi-node-9wmq4                                      2/2     Running   0          34d
openebs-jiva-csi-controller-0                                     5/5     Running   0          34d
openebs-jiva-csi-node-ldsqn                                       3/3     Running   0          34d
openebs-jiva-csi-node-qtp22                                       3/3     Running   0          34d
openebs-localpv-provisioner-75cc7566c6-q9brj                      1/1     Running   0          34d
openebs-ndm-cluster-exporter-7bf5ff4797-hmjcb                     1/1     Running   0          3s
openebs-ndm-node-exporter-frmgh                                   1/1     Running   0          3s
openebs-ndm-node-exporter-q4jcn                                   1/1     Running   0          3s
openebs-ndm-operator-c99db7fd-w55n4                               0/1     Running   0          3s
openebs-ndm-qzjpc                                                 1/1     Running   0          3s
openebs-ndm-w4vlk                                                 1/1     Running   0          3s
openebs-provisioner-56979b7b4f-dxxxf                              1/1     Running   0          34d
openebs-snapshot-operator-5c89bb7f-m2jjl                          2/2     Running   0          34d
pvc-e073097c-3d7b-40c7-84bd-99b54fcfa978-target-6979954f67c2wk9   3/3     Running   0          15d

master@master-virtual-machine:~/go/src/github.com/Ab-hishek/node-disk-manager$ kubectl get svc -n openebs
NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                               AGE
admission-server-svc                       ClusterIP   <IP_ADDRESS>      <none>        443/TCP                               34d
cvc-operator-service                       ClusterIP   <IP_ADDRESS>    <none>        5757/TCP                              34d
maya-apiserver-service                     ClusterIP   <IP_ADDRESS>    <none>        5656/TCP                              34d
openebs-cstor-admission-server             ClusterIP   <IP_ADDRESS>   <none>        443/TCP                               34d
openebs-ndm-cluster-exporter-service       ClusterIP   None             <none>        9100/TCP                              8m46s
openebs-ndm-node-exporter-service          ClusterIP   None             <none>        9101/TCP                              8m46s
pvc-e073097c-3d7b-40c7-84bd-99b54fcfa978   ClusterIP   <IP_ADDRESS>     <none>        3260/TCP,7777/TCP,6060/TCP,9500/TCP   15d

Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs
Checklist:

[ ] Fixes #
[x] PR Title follows the convention of  <type>(<scope>): <subject>
[ ] Has the change log section been updated?
[ ] Commit has unit tests
[ ] Commit has integration tests
[ ] (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
[ ] (Optional) If documentation changes are required, which issue on https://github.com/openebs/openebs-docs is used to track them:

Signed-off-by: Abhishek Agarwal<EMAIL_ADDRESS>

To which version should the chart be bumped up?

Codecov Report

Merging #612 (da43825) into master (225bbfd) will decrease coverage by 0.51%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #612      +/-   ##
==========================================
- Coverage   48.11%   47.60%   -0.52%
==========================================
  Files          79       79
  Lines        3720     3720
==========================================
- Hits         1790     1771      -19
- Misses       1770     1789      +19
  Partials      160      160

Impacted Files
Coverage Δ

cmd/ndm_daemonset/probe/eventhandler.go
30.55% <0.00%> (-8.34%)
:arrow_down:

cmd/ndm_daemonset/probe/addhandler.go
71.33% <0.00%> (-3.08%)
:arrow_down:

cmd/ndm_daemonset/probe/udevprobe.go
61.74% <0.00%> (-2.19%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 225bbfd...da43825. Read the comment docs.

Do I need to bump the chart version also? If yes, then to what version should the chart be bumped up.

Chart version can be bumped to 1.6.1. Also the images can also be updated to 1.6.1. Since this release will only be a patch release we can do the version bump also in this PR.


k_yield() exhibits different behavior with CONFIG_SMP

Using the latest master (currently 536e785b932b3b246891c8a8aa0075e387e1ed4a).
The documentation for k_yield() says:

Yield the current thread.  This routine causes the current thread to yield execution to another thread of the same or higher priority. If there are no other ready threads of the same or higher priority, the routine returns immediately.

... but the SMP scheduler (on a single-processor system) does not appear to yield control to other equal-priority threads.
Sample program:
#include <zephyr.h>
#include <sys/printk.h>

#define STACK_SIZE 8192

K_THREAD_STACK_DEFINE(stack0, STACK_SIZE);
K_THREAD_STACK_DEFINE(stack1, STACK_SIZE);
K_THREAD_STACK_DEFINE(stack2, STACK_SIZE);

void thread_func(void *p1, void *p2, void *p3)
{
	int num = POINTER_TO_INT(p1);

	k_busy_wait(500000 * num);

	for (;;) {
		k_busy_wait(1000000);
		printk("(%d)\n", num);
		k_yield();
	}
}

struct k_thread thread0;

struct k_thread thread1;
struct k_thread thread2;

void thread_func0(void *p1, void *p2, void *p3)
{
	k_thread_create(&thread1, stack1, STACK_SIZE, thread_func,
		INT_TO_POINTER(1), NULL, NULL, 0, 0, 0);

	k_thread_create(&thread2, stack2, STACK_SIZE, thread_func,
		INT_TO_POINTER(2), NULL, NULL, 0, 0, 0);
}

void main(void)
{
	k_thread_create(&thread0, stack0, STACK_SIZE, thread_func0,
		NULL, NULL, NULL, 0, 0, 0);
}

Basically, two threads are created of equal priority which busy-loop and print an identifying number periodically, yielding each time through the loop.  (thread_func0 exists solely in case the main thread has a low priority.) With the old scheduler (CONFIG_SMP=n), this works as expected, with the two threads interleaving --
On a qemu_x86 target (CONFIG_SMP=n obviously):
SeaBIOS (version rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org)
Booting from ROM..Optimal CONFIG_X86_MMU_PAGE_POOL_PAGES 7
***** Booting Zephyr OS build v2.0.0-rc3-721-g536e785b932b *****
(1)
(2)
(1)
(2)
(1)
(2)

etc.
On a qemu_x86_64 target (CONFIG_SMP=n):
SeaBIOS (version rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org)
Booting from ROM..(1)
(2)
(1)
(2)

etc.
On a qemu_x86_64 target (CONFIG_SMP=y CONFIG_MP_NUM_CPUS=1):
SeaBIOS (version rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org)
Booting from ROM..(1)
(1)
(1)
(1)
(1)

The second thread is never scheduled.

Did you get a chance to try over #19381?

Did you get a chance to try over #19381?

I clicked this link until my stack overflowed

a5ad00ca82cc6ca4a9636a051399b62c09fc5901 in https://github.com/zephyrproject-rtos/zephyr/pull/18531 fixes this

#18531  (pasted the correct link this time) is merged now, we can close this.


Is requires_grad(discriminator_img, False) infection dimg regulation

In train_seg_gan.py, when train dseg
requires_grad(generator, False)
requires_grad(discriminator_img, False)
requires_grad(discriminator_seg, True)

but in regulation part, dont open the toggle, dose this infection the dimg regularize?
if d_regularize:
    real_img.requires_grad = True
    real_pred = discriminator_img(real_img)
    r1_img_loss = d_r1_loss(real_pred, real_img)

    discriminator_img.zero_grad()
    (args.r1 / 2 * r1_img_loss * args.d_reg_every + 0 * real_pred[0]).backward()

    d_img_optim.step()

hi @linyu0219 , thank you for your question. I think it makes sense to enable to gradient for both discriminators in the regularization, this might be a mistake when we refactor the codebase. Could you share some training results if you have?

@lidaiqing  OK, I haven't trained yet


TLS-config credentials not exported

With some encouragement from @mshioji, I've got a first step in place to support TLS. It is working for HTTP requests. The details are in this issue.
We are having a problem getting the certificate when using the plug-in. The TLS-Config editor provides two ways to include the certificate in the flow: referencing the certificate by file path and uploading the certificate to the flow. The check box labeled "Use key and certificates from local files" controls which is used. When the certificate is referenced by file path, everything works as expected. When uploading the certificate, the certificate is not included in the flows_cred_mcu.json file as expected. Based on what Node-RED itself does, it should be in the cadata property:
    {
        "id": "c1445cf1202e809f",
        "type": "tls-config",
        "name": "",
        "cert": "",
        "key": "",
        "ca": "",
        "certname": "",
        "keyname": "",
        "caname": "ca29.pem",
        "servername": "",
        "verifyservercert": true,
        "alpnprotocol": "",
        "credentials": {
            "cadata": "-----BEGIN CERTIFICATE-----\nMIIEGjCCAwICEQCbfgZJoz5iudXukEhxKe9XMA0GCSqGSIb3DQEBBQUAMIHKMQsw\nCQYDVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZl\ncmlTaWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWdu\nLCBJbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlT\naWduIENsYXNzIDMgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3Jp\ndHkgLSBHMzAeFw05OTEwMDEwMDAwMDBaFw0zNjA3MTYyMzU5NTlaMIHKMQswCQYD\nVQQGEwJVUzEXMBUGA1UEChMOVmVyaVNpZ24sIEluYy4xHzAdBgNVBAsTFlZlcmlT\naWduIFRydXN0IE5ldHdvcmsxOjA4BgNVBAsTMShjKSAxOTk5IFZlcmlTaWduLCBJ\nbmMuIC0gRm9yIGF1dGhvcml6ZWQgdXNlIG9ubHkxRTBDBgNVBAMTPFZlcmlTaWdu\nIENsYXNzIDMgUHVibGljIFByaW1hcnkgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkg\nLSBHMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMu6nFL8eB8aHm8b\nN3O9+MlrlBIwT/A2R/XQkQr1F8ilYcEWQE37imGQ5XYgwREGfassbqb1EUGO+i2t\nKmFZpGcmTNDovFJbcCAEWNF6yaRpvIMXZK0Fi7zQWM6NjPXr8EJJC52XJ2cybuGu\nkxUccLwgTS8Y3pKI6GyFVxEa6X7jJhFUokWWVYPKMIno3Nij7SqAP395ZVc+FSBm\nCC+Vk7+qRy+oRpfwEuL+wgorUeZ25rdGt+INpsyow0xZVYnm6FNcHOqd8GIWC6fJ\nXwzw3sJ2zq/3avL6QaaiMxTJ5Xpj055iN9WFZZ4O5lMkdBteHRJTW8cs54NJOxWu\nimi5V5cCAwEAATANBgkqhkiG9w0BAQUFAAOCAQEAERSWwauSCPc/L8my/uRan2Te\n2yFPhpk0djZX3dAVL8WtfxUfN2JzPtTnX84XA9s1+ivbrmAJXx5fj267Cz3qWhMe\nDGBvtcC1IyIuBwvLqXTLR7sdwdela8wv0kL9Sd2nic9TutoAWii/gt/4uhMdUIaC\n/Y4wjylGsB49Ndo4YhYYSq3mtlFs3q9i6wHQHiT+eo8SGhJouPtmmRQURVyu565p\nF4ErWjfJXir0xuKhXFSbplQAz/DxwceYMBo7Nhbbo27q/a2ywtrvAkcTisDxszGt\nTxzhT5yvDwyd93gN2PQ1VoDat20Xj50egWTh/sVFuq1ruQp6Tk9LhO5L8X3dEQ==\n-----END CERTIFICATE-----\n"
        }
    }

Would you have a look? Thank you!

Walking through the plugins code I see no obvious reason why this particular case should / could fail.
Could you - or perhaps @mshioji - please provide a flow that I can use to verify what's happening?
Thank you in advance!

Thanks for taking a look. The challenge in providing a flow is that the credentials are stripped on export. I've manually put together this flow that should allow you to see the problem that @mshioji reports.

flow.json
[
    {
        "id": "7d0d6d4406798076",
        "type": "tab",
        "label": "Flow 4",
        "disabled": false,
        "info": "",
        "env": [],
        "_mcu": {
            "mcu": false
        }
    },
    {
        "id": "e35ea5b35fd6db16",
        "type": "http request",
        "z": "7d0d6d4406798076",
        "name": "",
        "method": "GET",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "www.moddable.com/press",
        "tls": "fbdc49382f0f04fa",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "_mcu": {
            "mcu": false
        },
        "credentials": {},
        "x": 330,
        "y": 140,
        "wires": [
            [
                "d412213eb76c429a"
            ]
        ]
    },
    {
        "id": "d412213eb76c429a",
        "type": "debug",
        "z": "7d0d6d4406798076",
        "name": "debug 87",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "_mcu": {
            "mcu": false
        },
        "x": 540,
        "y": 140,
        "wires": []
    },
    {
        "id": "ec26cb026ae62900",
        "type": "inject",
        "z": "7d0d6d4406798076",
        "name": "",
        "props": [
            {
                "p": "payload"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": true,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "_mcu": {
            "mcu": false
        },
        "x": 130,
        "y": 140,
        "wires": [
            [
                "e35ea5b35fd6db16"
            ]
        ]
    },
    {
        "id": "fbdc49382f0f04fa",
        "type": "tls-config",
        "name": "",
        "cert": "",
        "key": "",
        "ca": "",
        "certname": "",
        "keyname": "",
        "caname": "ca109.pem",
        "servername": "",
        "verifyservercert": true,
        "alpnprotocol": "",
        "_mcu": {
            "mcu": false
        },
        "credentials": {
            "cadata": "-----BEGIN CERTIFICATE-----\nMIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw\nTzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh\ncmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4\nWhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu\nZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY\nMTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc\nh77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+\n0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U\nA5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW\nT8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH\nB5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC\nB5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv\nKBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn\nOlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn\njh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw\nqHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI\nrU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV\nHRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq\nhkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL\nubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ\n3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK\nNFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5\nORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur\nTkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC\njNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc\noyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq\n4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA\nmRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d\nemyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=\n-----END CERTIFICATE-----\n"
        }
    }
]

Import this flow into the Node-RED Editor
Confirm that the HTTP Request node has a TLS-Config that uses ca109.pem
Deploy
The request should succeed
Now deploy to an ESP32
The certificate isn't exported so the HTTP Request fails

6. The certificate isn't exported so the HTTP Request fails

I cannot confirm this:
I did a fresh install of the whole build system, imported the flow & build it for a simulator. The result - initially - is as expected:

flows.json is created with all required nodes, including the tls-config node.
flows_cred_mcu.json is created as well, holding the data of the certificate:

{
    "credentials": {
        "e35ea5b35fd6db16": {},
        "fbdc49382f0f04fa": {
            "cadata": "-----BEGIN CERTIFICATE-----\nMI[...]GCc=\n-----END CERTIFICATE-----\n"
        }
    }
}

flows.js (created by nodered2mcu) yet doesn't get this data:

	nodes = flow.nodes();
	node = nodes.next().value;	// tls-config - fbdc49382f0f04fa
	node.onStart({
	});

Consequentially the simulator raises with Break: Resource: Resource not found: ca109.der!

I'm suspecting the check in line 1960 of nodered2mcu to return false in case an embedded certificate is used - yet didn't analyze things deeper to have full evidence.

@ralphwetzel – thank you for the detailed analysis. It is good to see that you are getting the credentials exported correctly. I'm not sure why the certificate binary isn't being created. I'll reconfirm that works outside the plug-in first, to have some confidence than nodered2mcu is doing what I expect, then I can try to take a look using the plug-in. I'll let you know what I find.

Good news. It is entirely my bug. :(
I didn't anticipate the separate credentials file in the new code paths to extract data (like the TLS certificate) from the flow. I've fixed that. It will take a few days for that to propagate out as we are finishing up the August release.
Thanks for the help and apologies for the false alarm.


Updating Session Br. scenario for the test app

@adityatMS,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Migration to Tauri?

first of all: this app looks cool as hell.
I started looking for a good epub reader because Calibre is too ugly to bear. then I ran into someone on Reddit who recommended it.
I love the design.
@mignaway I would love to hear your thoughts on Tauri. Electron comes with a RAM price.  I'm writing this more out of curiosity about your thoughts than anything else.

oh, I just saw that you thinking of migrating to React. hell yeah. maybe I will even contribute. (also, dark mode is really a must-have)

Hey, thanks for the compliment, I really appreciate it <3
Yes, at the moment I can't focus on this project, especially since I've just started designing and developing a version that I believe will be 20 times better than the current one, so I'm focused on that instead. Also, my design skills have improved significantly, so I think I can do much better in general. The new app will have many more features and will look even better.
Regarding Tauri, thanks for the advice, I could indeed use Tauri instead of Electron for this new project, thanks a lot!

Anyone can contribute to the project btw, although it's quite difficult since I didn't manage the code very well hahaha

@mignaway I can't find the new project on your public repository. is it private?

glad to hear you put time and effort into this. good luck!


No playback with AMD GPUs

Describe the bug
Resolve Studio installs successfully, launches, but refuses to playback any media imported or created within the editor, with the playback head getting stuck.
With rusticl, the behavior is similar – but with an actual error message:

Installation method used

[x] setup.sh
Manual

[ ] Distrobox
[ ] Toolbox

To Reproduce
Steps to reproduce the behavior:

Install and open Davinci Resolve Studio
Drag a clip in the timeline, or create a title
Try to playback the timeline

Expected behavior
Video playing back and appearing in the preview pane.
System information (please complete the following)

Distro: Fedora Workstation 40
Desktop Environment: GNOME 46 Wayland
GPU: AMD Radeon RX 560 (2 GB)
Version: Studio 19 β2, Studio 17.4.2

I'm having the same GPU error on my RX 7700XT with the free version of Resolve. My error is almost exactly the same except the error code is -1 instead of -57
My install was also done with the setup.sh file for Resolve 18.6.5, using KDE Plasma (Wayland) on Nobara 40 (Fedora based).
Is this something we will have to wait for Black Magic to fix?

Moreso we're waiting on driver updates than DaVinci Resolve updates
error code -1 is happening on ROCm (AMD's official compute stack, currently the default for AMD GPUs with davincibox)
error -57 happens with Mesa's rusticl (Mesa's OpenCL, opt-in experimental option), but until recently (2-3 weeks ago) it was working fine for me, so maybe with Mesa 24.1 rusticl will be working again?

ROCm is still borked, but as of Mesa 24.1.1, rusticl seems to be working with Resolve again.

Testing instructions can be found here in the README

As of #89, rusticl is now the default for AMD GPUs instead of ROCm, since the latter has been a frequent source of problems and even relatively newer AMD GPUs (like the RX 6000 series) aren't officially supported by ROCm.
Since rusticl is the default now, simply update davincibox and retest for this issue, no special steps required.

Closing this as the switch to rusticl seems to have resolved it, but I'm on an RX 6000 series card and not RX 500, so feel free to reopen if the issue persits @jannuary


Getting properties via magic method __get()

Would it be desirable to dynamically create some of the properties with __get() (see manual) so that you can get the values from the response as properties instead of method calls?
It's not that hard to integrate, with backwards compatibility of course, and I would be willing to.
$response = Geocode::make()->address('1 Infinite Loop');

if ($response) {
	echo $response->latitude();
	echo $response->longitude();
	echo $response->formattedAddress();
	echo $response->locationType();
}

// Output
// 37.331741
// -122.0303329
// 1 Infinite Loop, Cupertino, CA 95014, USA
// ROOFTOP

Would become:
$response = Geocode::make()->address('1 Infinite Loop');

if ($response) {
	echo $response->latitude;
	echo $response->longitude;
	echo $response->formattedAddress;
	echo $response->locationType;
}

// Output
// 37.331741
// -122.0303329
// 1 Infinite Loop, Cupertino, CA 95014, USA
// ROOFTOP

Found some spare minutes, see my pull request.


[T4A3][T09-B4] Zachary

Some comments added. Feel free to comment if you have any questions. Once done, you may the PR after reading comments.


Bug fix: Proper Web results shown

Fixes issue #35 i.e. web results are repetitive.
Screenshots for the change:

@AliAyub007 @saurabhjn76 @dynamitechetan Please review it. Thank you so much.


Change file names

Since the code is able to do much more things than just add utterances, I think it makes sense to change the name of the file (now its add-utterances-3-6.py)

Renamed to luis-app.py, since it's make more sense.


Problem starting the server.

I tried following the instructions, lein repl to start the server repl then this happens.
user=> (go)

ExceptionInfo please provide a valid file on your file-system  clojure.core/ex-info (core.clj:4593)

Any ideas?

Did you copy the default end file to /use/local/etc?

Sorry, my bad for not including that in the README instructions. I've updated the README. At the moment, there is no need to start the client or server builds. The only thing with real content is the tutorial itself, which is all in devcards.

Cool thanks!


CB-20713 Skip ORG policy decisions rule(AWS) for datalake backup

Context: Org policy decisions should always be skipped for datalake backup.
Test:
Added a custom result for the simultate policy (com.sequenceiq.cloudbreak.cloud.aws.common.util.AwsIamService#simulatePrincipalPolicy) with a 'denied by organization rules' and then checked if it was ignored and the precheckStoragePermission step in datalake backup was successful.

As I have mentioned in the previous PR, we have already addressed this issue where the customer can set the skipOrgPolicyDecisions flag at the credential level, this will be available for customers in the next (2.68) relesase. See the related PRs:
https://github.com/hortonworks/cloudbreak/pull/13856
https://github.com/hortonworks/hortonworks-cloud/pull/3397
Could you please explain why this is needed in addition to the changes above? Maybe I am missing something.

As I have mentioned in the previous PR, we have already addressed this issue where the customer can set the skipOrgPolicyDecisions flag at the credential level, this will be available for customers in the next (2.68) relesase. See the related PRs: #13856 hortonworks/hortonworks-cloud#3397 Could you please explain why this is needed in addition to the changes above? Maybe I am missing something.

Hello Laszlo,
The change you made would help new customers. How about existing customers?

As I have mentioned in the previous PR, we have already addressed this issue where the customer can set the skipOrgPolicyDecisions flag at the credential level, this will be available for customers in the next (2.68) relesase. See the related PRs: #13856 hortonworks/hortonworks-cloud#3397 Could you please explain why this is needed in addition to the changes above? Maybe I am missing something.

Hello Laszlo,
The change you made would help new customers. How about existing customers?

Why just new customers? You can edit the credential.

Based on @lturcsanyi comment, can we close this?


WARNING Bad subkey: openpgp: invalid data: subkey signature invalid: openpgp: invalid signature: RSA verification failure

I've removed all pgp keys from keybase and I'm still getting this error with any pgp command I run.
my log id: 22abb47d475569d7446ab11c

getting the same, when trying to add new device.

I get the same when someone else pulls my key

:

Facing same issue, quick research didn't provide meaningful resolution.


Make Local variable and parameter final.

We have shared with @arturobernalg in the dev mailing list that we are understaffed to deal with the cosmetic changes he has proposed so far, hence closing the ticket.


GSD Request

--- GSD JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v3.2 to before v5.15.8",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=fd734c6f25aea4b2b44b045e489aec67b388577e",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=474f9a8534f5f89841240a7e978bafd6e1e039ce"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "fd734c6f25aea4b2b44b045e489aec67b388577e",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "474f9a8534f5f89841240a7e978bafd6e1e039ce",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "can: sja1000: fix use after free in ems_pcmcia_add_card()\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.15.8 by commit 474f9a8534f5f89841240a7e978bafd6e1e039ce, it was introduced in version v3.2 by commit fd734c6f25aea4b2b44b045e489aec67b388577e. For more details please see the references link."
}
--- GSD JSON ---

This issue has been assigned GSD-2021-1002585


Expose amator's step option?

This will be helpful if you have to sync scroll position of some elements.

Hey! This use case is now possible by using the custom behavior option: https://github.com/stipsan/scroll-into-view-if-needed#function
A live example can be seen here: https://scroll-into-view-if-needed.netlify.com/#custom-transition


2.3.0 Mirror functional issues

In version 2.3.0, there are some small problems in the processing of the arc. After the position of the arc changes, the position of the starting angle and the ending angle changes, so special processing is needed

It should work now with todays (2019/12/30) commit.

that's very nice,thanks for all of you did


Add tracker please

udp://tracker.empire-js.us/
I believe this is the right place to post this... apologies otherwise.
Up and running. Will take me a while to get your implementation of ws.. It's quite a bit more complex then I expected.
My implementation tracks countries that connect! 👍
http://tracker.empire-js.us/stat
or
http://tracker.empire-js.us/stat.json

The tracker doesn't appear to work for me. Are you sure that's the correct URL?

Huh, I used webtorrent app myself with the server and it worked great... If you click the http link do you see this?:
V1.0.3 - 2 Torrents {2 active}

Successful Downloads: 4

Number of Scrapes to this tracker: 0

Connected Peers: 4

Seeders: 4

Leechers: 0

Countries that have connected:

US

Yes, I see the stats page. But the tracker does not respond to requests. You can test this with webtorrent-cli.
DEBUG=bittorrent-tracker* webtorrent --announce udp://tracker.empire-js.us "magnet:?xt=urn:btih:6a9759bffd5c0af65319979fb7832189f4f3c35d&dn=sintel.mp4"

You can see that there is a request, but no response from the server. Compare that to what happens with another tracker that works:
DEBUG=bittorrent-tracker* ./bin/cmd.js --announce udp://tracker.leechers-paradise.org:6969 "magnet:?xt=urn:btih:6a9759bffd5c0af65319979fb7832189f4f3c35d&dn=sintel.mp4"

You can see there is a response.

Omg. Working late nights on this has killed my brain a bit. I need more coffee....
udp://tracker.empire-js.us:1337
Port 1337.
Silly me. Sorry for that.

Done


soc: arm: nxp_kinetis: ke1xf: convert clock ip source code to new DT API

Convert the code for setting the clock IP source of the various IPs present in the NXP KE1xF SoC series to the new DT API.
Signed-off-by: Henrik Brix Andersen<EMAIL_ADDRESS>

I was think we'd move these to drivers like it is on openisa.

The clock IP source setting is clock-controller specific (and thus SoC specific). It was moved from drivers to SoC previously.

I was think we'd move these to drivers like it is on openisa.

The clock IP source setting is clock-controller specific (and thus SoC specific). It was moved from drivers to SoC previously.

I'm fine w/this, but at some point we should put this back in the drivers.

@galak I'll see if I can come up with a good solution for moving these calls to the drivers instead. For now, I'd like if we could merge this API conversion.


[Bug] Problems with di built-in generics

Describe the bug
I found an interesting bug: di doesn't work with built-in generics.
class Client:
    pass

class Service:
    def __init__(self, clients: list[Client]) -> None:
        self._clients = clients

def _get_clients() -> list[Client]:
    return [Client() for _ in range(3)]

def configure_di(app: Sanic) -> None:
    app.ext.add_dependency(list[Client], _get_clients)
    app.ext.add_dependency(Service)

It throws the following error:
File "/usr/local/lib/python3.11/site-packages/sanic_ext/extensions/injection/injector.py", line 37, in finalize_injections
    injection_registry.finalize(app, constant_registry, router_types)
File "/usr/local/lib/python3.11/site-packages/sanic_ext/extensions/injection/registry.py", line 42, in finalize
    constructor.prepare(
File "/usr/local/lib/python3.11/site-packages/sanic_ext/extensions/injection/constructor.py", line 111, in prepare
    raise InitError(
sanic_ext.exceptions.InitError: Unable to resolve dependencies for 'Service'. Could not find the following dependencies:
  - clients: list[src.di.Client].
Make sure the dependencies are declared using ext.injection. See https://sanicframework.org/en/plugins/sanic-ext/injection.html#injecting-services for more details.
Not all workers acknowledged a successful startup. Shutting down.

Environment (please complete the following information):

OS: [e.g. Ubuntu] MacOS Sequoia
Version [e.g. 0.5.3] Python 3.11, sanic 22.12.0, sanic-ext  23.6.0

I found the issue in sanic_ext/extensions/injection/constructor.py:
if not isclass(annotation):
    missing.append((param, annotation))
    continue

The condition for list[Client] will always be True.
It is necessary to add a new check for built-in generics.

Created pull request
https://github.com/sanic-org/sanic-ext/pull/263


Modify the emissions streaming endpoint to exclude data elements

Definition of Ready

[ ] Does this meet the INVEST criteria?
[ ] Does this have a clearly defined user?
[ ] Does this have an estimate?
[ ] Does this have acceptance criteria?
[ ] Does this have sufficient context for the specific functionality being developed (design, business rules, etc.)
[ ] Does this have a dev link been provided in the design?
[ ] Is this traceable to a feature, epic, or PI?

User Story
As a custom data download user I want the emissions streaming api endpoints to have an exclude parameters So that I can remove columns on the data table
Acceptance Criteria
Given that a user has accessed the emissions swagger page
When the functionality is displayed for the streaming endpoints
Then the endpoints include an exclude parameter
Given that a user has made a GET request to an emissions streaming endpoint with filter criteria for the exclude parameter
When the system returns results
Then the system does not return data for the excluded parameter(s)  (see: custom data download key column  )
Given that a user has made a GET request to an emissions streaming endpoint with invalid filter criteria for the exclude parameter
When the GET request is executed
Then the system will return the appropriate error message (see context for error messages)
Development Notes
Scope includes the steaming endpoints for all emissions   sub types:
hourly
daily
monthly
quarterly
ozone season
annual
MATS
On swagger the exclude parameter should be

a drop down
multi select

Does this Story require 508 Testing?

[ ] Implementing a new authoring tool that will produce code / content for the application
[ ] Implementing a 3rd party widget that will be part of the application
[ ] Adding new / updating custom functionality
[ ] Adding new / updating a page template
[ ] Adding new / replacing existing functionality (unless the replacement has already passed 508 testing)
[ ] Full Site Review prior to a scheduled EPA 508 review
[ ] Planning a video that will appear on the application
[ ] Producing a video that will appear on the application
[ ] Producing Training materials
[ ] Producing documents

Context
(User journeys, design mockups, architectural diagrams, business rules, data element definitions)
: custom data download key column
Validations:

Validate that the provided parameter can be removed

Error Message: One or more excluded values cannot be removed
Example. user enters stateCode into the exclude parameter. StateCode is a key value and cannot be removed.

Validate that the provided parameter is valid

Error Message: : One or more excluded values are not valid.
Example. user enters "sjsj" into the exclude parameter.  "sjsj" is not a valid value
Testing
Test Case:
Tester:
Definition of Done

[ ] Have all Acceptance Criteria been agreed to and validated by the scrum team?
[ ] Has the code been unit tested and peer reviewed?
[ ] Have the functional tests been executed?
[ ] Have all defects been reviewed and dispositioned, resolved, or deferred?
[ ] Is the user story ready to be deployed to the test/staging environments?

@tlageman-epa Please see the context section of this ticket to review the error messages that I drafted. Let me know if you have any questions.

TC628 passed @tlageman-epa


s3 low download speed after migrating to ubuntu 14.04

What platform/OS are you using?
14.04
What compiler are you using? what version?
gcc 5.4.1
What's your CMake arguments?
Can you provide a TRACE level log? (sanitize any sensitive information)
Our instance is on aws ec2. We used to see 1Gb/sec download speed from s3(on c3.8xl instance) when running the same code on ubuntu 12. After migrating to ubuntu 14, with the same code, we only saw 100Mb/sec download speed from s3. If we directly use s3cmd for example, the download speed is still as high as before, so we are wondering could there be any change inside aws-sdk that could potentially cause this issue?

We made some changes in TransferManager recently, but not sure if that's the reason slowing down the speed. Thank you for your questions, we will take a look.

thanks! I was using https://github.com/aws/aws-sdk-cpp/commit/39a03136ef8525bc94d82669432491e653154677 and tried to upgrade to latest master code yesterday. Both showed decreased downloading speed when migrating to ubuntu 14.

You said using the same code. But I wonder if you also mean the same SDK code.
I'm only asking because it seems like we have a regression reported here https://github.com/aws/aws-sdk-cpp/issues/711

Yes, it's both same SDK code and same our code to use the SDK. We just saw difference downloading speed by using ubuntu 12 vs ubuntu 14.

The regression affects uploads only. My bad.

Hi @hrjaco, I'm trying to go clear up the repo and I'd like to ask, are you still facing this issue?

Yes, we can close this one. IIRC it turned out to be an issue on our end, where we has been misusing one param.


GEP: Review Query Param Filter GEP

A/C:

Review https://github.com/kubernetes-sigs/gateway-api/pull/2959/files
Determine if NGF can support proposed API
Contribute to the conversation as to NGF's stance on the API
Review summary with the team.

We should be able to do this using a combination of set and rewrite directives.


Support for lesion masks?

Hello,
I saw in a previous post something about a "lesion-robust tool" and was wondering if, when, and how his can be used to improve segmentation and surface reconstruction. I have a lot of subjects with brain tumors (often GBMs) where segmentation results are good but often contain errors around the lesion. Is this something that's available now?
Thanks,
Travis

Hi Travis, we are indeed working on a lesion robust version that supports lesion masks. We are in the final stages of running experiments and finishing the paper. Usually we make the code public after paper acceptance and final testing. So this can still take months. Maybe we can share an early development version at some point for testing. Send an email.


configurable topology refresh duration

This pull requests a configurable way to topology refresh duration. Two files session.rs and cluster.rs have been modified to set the session configuration and consume the duration configuration respectively.
Note: I am not sure if we need any test cases in this case as the duration is still provided from where its called.

[x] I have split my patch into logically separate commits.
[x] All commit messages clearly explain what they change and why.
[x] I added relevant tests for new features and bug fixes.
[x] All commits compile, pass static checks and pass test.
[x] PR description sums up the changes and reasons why they should be introduced.
[x] I have provided docstrings for the public items that I want to introduce.
[x] I have adjusted the documentation in ./docs/source/.
[x] I added appropriate Fixes: annotations to PR description.

Fixes: #232

Note: I am not sure if we need any test cases in this case as the duration is still provided from where its called.

I don't have a good idea on how to write a good test for this. Usually, a SessionBuilder test + a manual test to see whether the parameter is properly propagated is sufficient.

Hi @piodul, are any further changes needed? Let me know in case you missed this.

Hi @piodul, are any further changes needed? Let me know in case you missed this.

@rishabharyal Sorry, I was out on vacation for the last two weeks.