Increased CPU load limits 10G throughput on DEC850 since upgrade to 24.7

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
I see a noticeable performance impact on 10G routing after upgrading my DEC850 to 24.7. Going back to 24.1 solves the issue.
On prior release (including 24.1) I was able to get "line" performance while having relatively low CPU load on the 8 cores the DEC850 has. Since upgrading to 24.7 the network performance is inconsistent and no longer line performance. CPU core spike at 100% during transfer tests.
To Reproduce
Using a device connected to the LAN (ax1), run a moderate network load to a WAN (ax0) target that can provide consistent performance close to 10G line speed. For my example I used "iperf3 -c speedtest.sea11.us.leaseweb.net -p 5201-5210 -P10" which normally results in about 9.4 to 9.3 Gbits/sec and a well distributed load across CPU cores.

Running the same test on 24.7 results in only 8.8 Gbits/sec down and ~5 Gbits/sec up while maxing out most CPU cores.

Expected behavior
The DEC850 should maintain the same 10G routing performance when upgrading to 24.7. CPU core should not spike to 100% system load.
Describe alternatives you considered
Downgrade to 24.1 or 24.4 resolves the issue.
Environment
I am on a 10G internet connection and use a DEC850. AX0 is the WAN port and AX1 is used as LAN. AX1 has several v-lans. No other ports are in use.
The recommended tuning parameters to get 10G performance are set:
dev.ax.0.iflib.override_nrxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.0.iflib.override_ntxds = 2048, 2048, 2048, 2048, 2048, 2048, 2048, 2048
dev.ax.0.rss_enabled = 1
Same settings for ax.1 as well.

OPNsense 24.7(amd64).
DEC850 (stock 8 core configuration)

can you share the output of a detailed top (at least from 24.7, but both would even be better)?
top -CHIPSn -d2 9999

Some netstat statistics might be practical to have as well:
netstat -Q
netstat -m
netstat -s

I'm also a bit curious if the results vary when changing the number of flows in iperf3 (e.g. -P 4, -P 6, -P 8)
A lot has changed upstream in FreeBSD 14.1, some variations are not unusual.

This is an interesting bug. I created a lab environment using a fresh install on a DEC850 to better debug this issue and remove any WAN speed variabilities or other factors.  WAN side is connected to a server running speedtest -s and LAN side is connected to an iPAD Pro M4 connected to 10G via a Thunderbolt NIC. This combination gives me 100% solid transfer speeds at expected line speeds when the issue is not happening.
I also created a sample configuration file for the DEC850 that I am using for all tests which should make it easy to reproduce the issue. This uses the default password.
test-config.xml.txt
My first tests showed full performance and made me doubt that there is an issue. However, after repeated tests the CPU load was increasing, and speeds were dropping using the exact same speedtest parameters.
I captured the requested details for various speedtest "P" settings while the issue was present:
P 1
top -CHIPSn -d2 9999: P1.txt
netstat -m: NM1.txt
netstat -Q: NQ1.txt
netstat -s: NS1.txt
P 5
top -CHIPSn -d2 9999: P5.txt
netstat -m: NM5.txt
netstat -Q: NQ5.txt
netstat -s: NS5.txt
P 10
top -CHIPSn -d2 9999: P10.txt
netstat -m: NM10.txt
netstat -Q: NQ10.txt
netstat -s: NS10.txt
P 20
top -CHIPSn -d2 9999: P20.txt
netstat -m: NM20.txt
netstat -Q: NQ20.txt
netstat -s: NS20.txt
Rebooted the router and got the speed back temporarily before it slowed down again. The logs at expected speed are here:
P 20 after reboot at expected 9.41 Gbits/sec
top -CHIPSn -d2 9999: R-P20.txt
netstat -m: R-NM20.txt
netstat -Q: R-NQ20.txt
netstat -s: R-NS20.txt
Note: I am not sure that a reboot is the only way to "cure" this. I had a series of low-speed test results associated with high CPU loads but changing the P parameter to a lower value (3) for one run did at least once increase the run at 20.
Let me know what other tests you want me to run.

I just noticed looking at the Px.txt files that the load shown does not reflect the same as shown with top -P during the run. During the test runs the cores were at 100%. Any other way to capture that better? The run after the reboot shows the correct core loads.

To be honest, with iperf3 tests I've always seen variations during testing, partly caused by congestion and partly misalignment of flows (ideally with an 8 core system, 8 flows should be distributed evenly, but in reality that's not always the case for various reasons).
When I'm at the office, I'll connect one of our machines between our standard testbed to see if there's anything unusual. Just one question, your comments seem to suggest cpu consumption increases over time without reboots, is that indeed the case?

Yes, CPU consumption is lower right after a reboot, and I also see expected speed right after a reboot. After a while it slows down. Here are test runs with -P 8.
In the "fault" state is only get 8.64 Gbits/sec and CPU cores spike to 100%:
last pid: 87053;  load averages:  3.11,  1.04,  0.50    up 1+01:11:40  00:06:22
349 threads:   21 running, 294 sleeping, 34 waiting
CPU 0:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
CPU 1:  0.4% user,  0.0% nice, 11.4% system,  5.1% interrupt, 83.1% idle
CPU 2:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
CPU 3:  0.0% user,  0.0% nice, 82.0% system,  0.0% interrupt, 18.0% idle
CPU 4:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
CPU 5:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
CPU 6:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
CPU 7:  0.0% user,  0.0% nice,  100% system,  0.0% interrupt,  0.0% idle
Mem: 80M Active, 216M Inact, 1693M Wired, 56K Buf, 13G Free
ARC: 1068M Total, 91M MFU, 833M MRU, 1684K Anon, 20M Header, 120M Other
840M Compressed, 3846M Uncompressed, 4.58:1 Ratio
Swap: 8192M Total, 8192M Free
PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME     CPU COMMAND
0 root        -60    -     0B  2144K CPU4     4   1:50 100.00% kernel{if_io
0 root        -60    -     0B  2144K CPU7     7   1:21 100.00% kernel{if_io
0 root        -60    -     0B  2144K CPU2     2   1:20 100.00% kernel{if_io
0 root        -60    -     0B  2144K CPU5     5   1:04 100.00% kernel{if_io
0 root        -60    -     0B  2144K CPU0     0   0:57 100.00% kernel{if_io
0 root        -60    -     0B  2144K CPU6     6   0:46 100.00% kernel{if_io
11 root        187 ki31     0B   128K RUN      1  25.1H  86.06% idle{idle: c
0 root        -60    -     0B  2144K CPU3     3   1:38  82.33% kernel{if_io
After a reboot, running the exact same test again gives me higher speeds and lower CPU load:
last pid: 79445;  load averages:  1.22,  0.34,  0.13    up 0+00:01:26  00:11:18
347 threads:   10 running, 298 sleeping, 39 waiting
CPU 0:  0.0% user,  0.0% nice,  7.1% system,  0.0% interrupt, 92.9% idle
CPU 1:  0.0% user,  0.0% nice, 76.0% system,  0.0% interrupt, 24.0% idle
CPU 2:  0.0% user,  0.0% nice, 69.3% system,  0.0% interrupt, 30.7% idle
CPU 3:  0.0% user,  0.0% nice,  0.4% system,  2.4% interrupt, 97.2% idle
CPU 4:  0.0% user,  0.0% nice, 38.6% system,  0.0% interrupt, 61.4% idle
CPU 5:  0.0% user,  0.0% nice, 39.8% system,  0.0% interrupt, 60.2% idle
CPU 6:  0.0% user,  0.0% nice, 41.7% system,  0.0% interrupt, 58.3% idle
CPU 7:  0.0% user,  0.0% nice, 35.0% system,  0.0% interrupt, 65.0% idle
Mem: 141M Active, 125M Inact, 581M Wired, 56K Buf, 15G Free
ARC: 199M Total, 45M MFU, 108M MRU, 11M Anon, 1371K Header, 34M Other
113M Compressed, 235M Uncompressed, 2.09:1 Ratio
Swap: 8192M Total, 8192M Free
PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME     CPU COMMAND
11 root        187 ki31     0B   128K RUN      3   1:24  98.17% idle{idle: c
11 root        187 ki31     0B   128K CPU0     0   1:24  94.64% idle{idle: c
0 root        -60    -     0B  2144K -        1   0:11  73.00% kernel{if_io
0 root        -60    -     0B  2144K CPU2     2   0:10  69.28% kernel{if_io
11 root        187 ki31     0B   128K RUN      4   1:13  65.14% idle{idle: c
11 root        187 ki31     0B   128K CPU7     7   1:18  64.77% idle{idle: c
11 root        187 ki31     0B   128K CPU5     5   1:18  61.57% idle{idle: c
11 root        187 ki31     0B   128K RUN      6   1:17  57.16% idle{idle: c

... and just for fun. Here is a run in the "no fault" state with -P 100 pushing 9.40 Gbits/sec with relatively low CPU load. I do not think that we have a congestion issue here when the router is not impacted by whatever the fault state is that develops over time.
last pid: 15071;  load averages:  1.86,  1.09,  0.57    up 0+00:10:47  00:20:40
346 threads:   12 running, 295 sleeping, 39 waiting
CPU 0:  0.0% user,  0.0% nice, 51.2% system,  0.4% interrupt, 48.4% idle
CPU 1:  0.0% user,  0.0% nice, 44.1% system,  0.0% interrupt, 55.9% idle
CPU 2:  0.0% user,  0.0% nice, 47.6% system,  0.0% interrupt, 52.4% idle
CPU 3:  0.0% user,  0.0% nice, 44.9% system,  0.0% interrupt, 55.1% idle
CPU 4:  0.0% user,  0.0% nice, 43.7% system,  0.0% interrupt, 56.3% idle
CPU 5:  0.0% user,  0.0% nice, 38.6% system,  0.0% interrupt, 61.4% idle
CPU 6:  0.4% user,  0.0% nice, 42.5% system,  0.0% interrupt, 57.1% idle
CPU 7:  0.0% user,  0.0% nice, 42.9% system,  0.0% interrupt, 57.1% idle
Mem: 139M Active, 126M Inact, 635M Wired, 56K Buf, 14G Free
ARC: 189M Total, 41M MFU, 112M MRU, 516K Anon, 1363K Header, 34M Other
116M Compressed, 256M Uncompressed, 2.20:1 Ratio
Swap: 8192M Total, 8192M Free
ax1: Link is UP - 10Gbps/Full - flow control off
ax1: Link is DOWN PRI NICE   SIZE    RES STATE    C   TIME     CPU COMMAND
11 root        187 ki31     0B   128K RUN      6   9:48  58.93% idle{idle: c
11 root        187 ki31     0B   128K RUN      3   9:49  58.48% idle{idle: c
11 root        187 ki31     0B   128K RUN      4   9:38  58.40% idle{idle: c
11 root        187 ki31     0B   128K CPU5     5   9:27  57.62% idle{idle: c
11 root        187 ki31     0B   128K CPU7     7   9:37  57.44% idle{idle: c
11 root        187 ki31     0B   128K CPU1     1   9:37  55.85% idle{idle: c
11 root        187 ki31     0B   128K CPU2     2   9:36  54.03% idle{idle: c
11 root        187 ki31     0B   128K RUN      0   9:40  51.61% idle{idle: c

... and just for fun. Here is a run in the "none fault" state with -P 100 pushing 9.40 Gbits/sec with relatively low CPU load. Does not look like a congestion issue to me. At least it runs fine if the router does not hit this bug and the below performance is the same as I saw on prior releases.
last pid: 15071;  load averages:  1.86,  1.09,  0.57    up 0+00:10:47  00:20:40
346 threads:   12 running, 295 sleeping, 39 waiting
CPU 0:  0.0% user,  0.0% nice, 51.2% system,  0.4% interrupt, 48.4% idle
CPU 1:  0.0% user,  0.0% nice, 44.1% system,  0.0% interrupt, 55.9% idle
CPU 2:  0.0% user,  0.0% nice, 47.6% system,  0.0% interrupt, 52.4% idle
CPU 3:  0.0% user,  0.0% nice, 44.9% system,  0.0% interrupt, 55.1% idle
CPU 4:  0.0% user,  0.0% nice, 43.7% system,  0.0% interrupt, 56.3% idle
CPU 5:  0.0% user,  0.0% nice, 38.6% system,  0.0% interrupt, 61.4% idle
CPU 6:  0.4% user,  0.0% nice, 42.5% system,  0.0% interrupt, 57.1% idle
CPU 7:  0.0% user,  0.0% nice, 42.9% system,  0.0% interrupt, 57.1% idle
Mem: 139M Active, 126M Inact, 635M Wired, 56K Buf, 14G Free
ARC: 189M Total, 41M MFU, 112M MRU, 516K Anon, 1363K Header, 34M Other
116M Compressed, 256M Uncompressed, 2.20:1 Ratio
Swap: 8192M Total, 8192M Free
ax1: Link is UP - 10Gbps/Full - flow control off
ax1: Link is DOWN PRI NICE   SIZE    RES STATE    C   TIME     CPU COMMAND
11 root        187 ki31     0B   128K RUN      6   9:48  58.93% idle{idle: c
11 root        187 ki31     0B   128K RUN      3   9:49  58.48% idle{idle: c
11 root        187 ki31     0B   128K RUN      4   9:38  58.40% idle{idle: c
11 root        187 ki31     0B   128K CPU5     5   9:27  57.62% idle{idle: c
11 root        187 ki31     0B   128K CPU7     7   9:37  57.44% idle{idle: c
11 root        187 ki31     0B   128K CPU1     1   9:37  55.85% idle{idle: c
11 root        187 ki31     0B   128K CPU2     2   9:36  54.03% idle{idle: c
11 root        187 ki31     0B   128K RUN      0   9:40  51.61% idle{idle: c

On my end with the same hardware it looks rather normal after clean boot:
[  5]  33.00-33.62  sec  87.0 MBytes  1.17 Gbits/sec    0    506 KBytes
[  7]  33.00-33.62  sec  86.9 MBytes  1.17 Gbits/sec    0    493 KBytes
[  9]  33.00-33.62  sec  87.1 MBytes  1.17 Gbits/sec    0    476 KBytes
[ 11]  33.00-33.62  sec  87.5 MBytes  1.17 Gbits/sec    0    839 KBytes
[ 13]  33.00-33.62  sec  87.5 MBytes  1.17 Gbits/sec    0    819 KBytes
[ 15]  33.00-33.62  sec  87.5 MBytes  1.17 Gbits/sec    0    472 KBytes
[ 17]  33.00-33.62  sec  88.1 MBytes  1.18 Gbits/sec    0    638 KBytes
[ 19]  33.00-33.62  sec  86.9 MBytes  1.17 Gbits/sec    0    615 KBytes
[SUM]  33.00-33.62  sec   698 MBytes  9.38 Gbits/sec    0

Usually I test with slightly smaller packets using the following iperf3 command, where the firewall is positioned between 2 linux hosts:
iperf3 -c  x.x.x.x -P 8 -Z -t 5000 -M 1350

Load distribution varies between tests (when stopping/starting), but thats to be expected.

Your results match my test when the system is in the "nomal" and not "fault, bug" state. I try to find repro steps to trigger the condition but have not found them, yet.  Just had the router run "normal" for hours. Did a reboot and now cores spike to 100% running the same tests.

I did more testing today and found that using lower "Ps" does show a significant difference in both performance and CPU load between 24.1 and 24.7.
Loads spikes to 100% on several cores on 24.7 and the throughput was noticeable slower as well. At about -P 6 is when performance recovered and was more in line with 24.1.
This was even in what I consider the "no fault state" that would show issue when running with more than 6 streams.
@AdSchellevis Can you try to repro this? Very easy to do by using my test config, booting from a USB key and using the configuration importer. Lover that feature... :-)

We have our own tests, load might vary a bit between versions, but that's not really something we can fix I'm afraid, unless we can put our fingers on a clear bug somewhere.  Over the years we have seen more variations, overall the consistency of the performance does seem to improve to be hoonest.
There is however a logical explanation why less flows result in perceived higher load on cores as the chances two or more of them end up on the same core are just rather high.

Understand that this is a complex issue.  I ran more tests and the best way to describe the "bug" is that load distribution of if_io_tqg is different between 24.1 and 24.7. In 24.7 the new FreeBSD version is very inconsistent in how network load is scales across cores. Running with a few streams shows the issues better e.g. I ran -P 4 -R and got results that were between 5.5 and 9 Gbits/secs.  Looking at top -CHIPS at the same time shows that higher speed results were associated with a better distribution between CPU cores, while the lowest speed runs had 100% of the load on single core.

I finally found the issue and if you agree, we can close this bug.
My configuration (also the example attached) does have RSS enabled for the ax0 and ax1 NICs (dev.ax.X.rss_enabled = 1) but was missing net.inet.rss.enabled =1. This had no noticeable performance impact using 24.1 but certainly has with 24.7.
Thinking more about how loads are distributed, I finally noticed that netisr did not show up in any of the top -CHIPS output.
Adding the above tunable finally turned it on and I can see it now doing what it is supposed to do. :-)
top -CHIPS with net.inet.rss.enabled =1
PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME     CPU COMMAND
11 root        187 ki31     0B   128K CPU0     0   8:33  82.10% idle{idle: cpu0}
11 root        187 ki31     0B   128K RUN      4   8:52  82.04% idle{idle: cpu4}
11 root        187 ki31     0B   128K CPU5     5   8:27  80.28% idle{idle: cpu5}
12 root        -60    -     0B   496K CPU7     7   1:43  75.98% intr{swi1: netisr 7}
12 root        -60    -     0B   496K CPU2     2   1:25  75.76% intr{swi1: netisr 2}
12 root        -60    -     0B   496K WAIT     6   0:56  61.61% intr{swi1: netisr 6}
11 root        187 ki31     0B   128K RUN      1   7:33  51.29% idle{idle: cpu1}
11 root        187 ki31     0B   128K RUN      3   9:19  48.78% idle{idle: cpu3}
12 root        -60    -     0B   496K WAIT     1   2:38  36.39% intr{swi1: netisr 1}
12 root        -60    -     0B   496K WAIT     3   0:45  34.77% intr{swi1: netisr 3}
0 root        -60    -     0B  2144K -        6   0:57  34.42% kernel{if_io_tqg_6}
11 root        187 ki31     0B   128K RUN      2   8:32  21.39% idle{idle: cpu2}
0 root        -60    -     0B  2144K CPU4     4   1:03  16.90% kernel{if_io_tqg_4}
0 root        -60    -     0B  2144K -        0   0:40  16.76% kernel{if_io_tqg_0}
0 root        -60    -     0B  2144K -        3   0:27  16.13% kernel{if_io_tqg_3}
0 root        -60    -     0B  2144K RUN      7   0:32  14.63% kernel{if_io_tqg_7}
0 root        -60    -     0B  2144K -        5   0:35  12.23% kernel{if_io_tqg_5}
0 root        -60    -     0B  2144K -        1   0:24  11.91% kernel{if_io_tqg_1}
11 root        187 ki31     0B   128K RUN      7   8:18   9.35% idle{idle: cpu7}
12 root        -60    -     0B   496K WAIT     5   1:34   6.67% intr{swi1: netisr 5}
11 root        187 ki31     0B   128K RUN      6   8:31   3.97% idle{idle: cpu6}
0 root        -60    -     0B  2144K -        2   0:38   2.74% kernel{if_io_tqg_2}

results vary with rss, the driver itself uses rss when enabled (which is always advisable and enabled by default), net.inet.rss.enabled  balances netisr as well, which in some cases offer better results. We usually test without it on our end, but load distribution is certainly different with it.
I'll close the issue as suggested.


new rule: must-throw-error-type

new rule: must-throw-error-type
Only throw the type Error. Throwing a string can break error handling code.

I wonder that is blocked by the type checker?
import util = require("util");
 util.init();  // it execcutes "util.errorCode = new new CustomError('...');"

throw util.errorCode; // is it ok? or BAD?

Yes, we need the type checker to accurately know if the the thrown object is an error or not.

https://palantir.github.io/tslint/rules/no-string-throw/ ✨ 👍


refactor: enable prometheus middleware

:white_check_mark: Build TheIdServer 4.5.4-feat-open-telemetry1-0013 completed (commit https://github.com/Aguafrommars/TheIdServer/commit/39e37a4e5f by @aguacongas)

:x: Build TheIdServer 4.5.4-feat-open-telemetry1-0014 failed (commit https://github.com/Aguafrommars/TheIdServer/commit/5b225a9b4e by @aguacongas)

:white_check_mark: Build TheIdServer 4.5.4-feat-open-telemetry1-0015 completed (commit https://github.com/Aguafrommars/TheIdServer/commit/91ca41e0a8 by @aguacongas)


[1368] Refactor job params to be consistent

Context

https://trello.com/c/vn4wSnvX/1368-pass-objects-to-jobs

Changes proposed in this pull request
Based on this comment https://github.com/DFE-Digital/register-trainee-teachers/pull/638#discussion_r592265750 and the ticket above. We no longer need to be explicit with passing around ids for model instances since https://github.com/rails/globalid .
Guidance to review

 Pass objects to jobs


feat: Add management of source control

Both applications and organizations can manage the source control independently or by inheriting.
In the tests I have added this repository as the repository_url as you have to use a URI that matches a provider. In the IQ instance I was using the provider at the root organisation is set to GitHub, this might mean the acceptance tests I have added here may fail for others with a different configuration.
As I am not an admin of the instance I was running the tests against I only had limited permissions so only ran the Acceptance tests I have added:
TF_ACC=1 go test ./internal/... -run=TestAccSourceControl
ok      terraform-provider-sonatypeiq/internal/provider 19.834s

This is an attempt to implement #17

All tests passed 🆗
TF_ACC=1 go test -v -cover ./internal/provider/
=== RUN   TestAccApplicationCategoriesDataSource
--- PASS: TestAccApplicationCategoriesDataSource (0.81s)
=== RUN   TestAccApplicationDataSource
--- PASS: TestAccApplicationDataSource (0.62s)
=== RUN   TestAccApplicationResource
--- PASS: TestAccApplicationResource (1.78s)
=== RUN   TestAccApplicationRoleMembershipResource
--- PASS: TestAccApplicationRoleMembershipResource (1.30s)
=== RUN   TestAccApplicationsDataSource
--- PASS: TestAccApplicationsDataSource (0.66s)
=== RUN   TestAccConfigSamlDataSource
--- PASS: TestAccConfigSamlDataSource (0.69s)
=== RUN   TestAccOrganizationDataSource
--- PASS: TestAccOrganizationDataSource (0.60s)
=== RUN   TestAccOrganizationResource
--- PASS: TestAccOrganizationResource (0.97s)
=== RUN   TestAccOrganizationRoleMembershipResource
--- PASS: TestAccOrganizationRoleMembershipResource (1.27s)
=== RUN   TestAccOrganizationsDataSource
--- PASS: TestAccOrganizationsDataSource (0.66s)
=== RUN   TestAccRoleDataSource
--- PASS: TestAccRoleDataSource (0.59s)
=== RUN   TestAccSourceControlApplicationResource
--- PASS: TestAccSourceControlApplicationResource (1.91s)
=== RUN   TestAccSourceControlOrganizationResource
--- PASS: TestAccSourceControlOrganizationResource (1.79s)
=== RUN   TestAccSystemConfigDataSource
--- PASS: TestAccSystemConfigDataSource (0.60s)
=== RUN   TestAccSystemConfigResource
0x140003a6220--- PASS: TestAccSystemConfigResource (0.62s)
=== RUN   TestAccUserResource
--- PASS: TestAccUserResource (1.34s)
PASS
coverage: 57.6% of statements
ok      terraform-provider-sonatypeiq/internal/provider 16.636s coverage: 57.6% of statements


Feature Request: Schedule Multiple Tweets Per Day

If hour and minutes are present in a Tweet's filename, a github action to post that tweet should run at the given datetime. If only Month, Day, Year are in the filename, it should run at the default time of day.
Example:
August 17, 2021.md posts at regular time
August 17, 2021 11:53.md posts at 11:53am GMT
August 17, 2021 14:22.md posts at 2:22pm GMT
It would be even better if there was a config var called 'TIMEZONE' in the .env so instead of GMT it would set it to local time 🤔

hey @reidjs, is anyone working on this one?

Hey there! I don't believe so if you want to take it I would appreciate it a lot


Solving minor UI issues

Fixed some minor UI issues with the dark mode button, "go top" button and "Download PDF" button according to issue #99.
I also slightly adjusted the light mode colour of the "a" tags to make them more readable.

Thank you, I really appreciate it.


Implement Preprocessor

Is your feature request related to a problem? Please describe.
PlantUML comes with a built-in preprocessor official documentation. In order support parsing of all different PlantUML diagrams, this parser should also implement preprocessor instruction parsing and a JavaScript implemented preprocessor (preprocess()).
Describe the solution you'd like

[ ] Add test fixtures for all examples in the  official documentation
[ ] Create a new parser for preprocessor instructions
[ ] Add those preprocessor instruction set to the plantuml parser -> make the normal parser skip over preprocessor instructions.
[ ] Write preprocess(diagram: string): string which runs all preprocessor instructions on a raw diagram

Describe alternatives you've considered

related: #25


Update external repo links

Update external repo locations.

Pull Request Test Coverage Report for Build 248691976

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 39.144%

Totals

Change from base Build 248604515:
0.0%

Covered Lines:
19095

Relevant Lines:
44574

💛  - Coveralls


Implemented sort by column

I had my code ready I had just to implement the HeaderClick Event that I remembered later

I have added the ability to collapse groups in the FileListView.
To do this I had to create an extension for the standard ListView, I had done this before for another project therefore I have uploaded the DLL directly. If you like this idea I can provide source code to have full control over it, otherwise, just leave it as it is or discard the group collapsing totally.

Last commit its a raw workaround to avoid a throw exception on closing, didn't identify why it occurs

You can add it to the project but has to be compiled to be able to be used, I admit that I don't know how to handle it.
I will add the files and see what happens.


Errno: -4058 code ENOENT

Hi guys,
I am trying out this cloudinary plugin but, I am having trouble getting it to work.
I am receiving the following error:
    at D:\projects\personal-website\node_modules\payload-cloudinary-plugin\dist\plugins\cloudinaryPlugin.js:127:23
    at step (D:\projects\personal-website\node_modules\payload-cloudinary-plugin\dist\plugins\cloudinaryPlugin.js:44:23)
    at Object.throw (D:\projects\personal-website\node_modules\payload-cloudinary-plugin\dist\plugins\cloudinaryPlugin.js:25:53)
    at rejected (D:\projects\personal-website\node_modules\payload-cloudinary-plugin\dist\plugins\cloudinaryPlugin.js:17:65)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

I think it's trying to upload the file I am adding to payload before it's every hitting the media directory which is causing it to throw a file not existing error.
My config is set up as directed to the readme.md and my credentials work:
server.ts
const server = express();
const cloudinaryConfig = cloudinary.config({
  cloud_name: '********',
  api_key: '********'',
  api_secret: '********'',
});

server.use(mediaManagement(cloudinaryConfig));

payload.config.ts

.
.
.
  plugins: [
    formBuilder({
      redirectRelationships: [
        'pages',
      ],
    }),
    cloudinaryPlugin(),
  ],
});

thanks for looking into this issue!

disabling localStorage in the payload collection config ended up solving my problem.
this isn't documented anywhere and the cloudinaryService looks like it tries to handle the image saving so maybe this is still a bug?

Same error happening here. Appeared out of nowhere a few days ago, had been working in production until then. Haven't upgraded dependencies on version 2.0.15 of payload. Upgraded to 2.5.0 and it didn't do anything.
Here is an example error message:
[13:04:08] ERROR (payload): APIError: Cloudinary: {"error":{"errno":-2,"code":"ENOENT","syscall":"open","path":"/home/node/media/300_balloon_desktop.png"}}     at /home/node/app/node_modules/payload-cloudinary-plugin/dist/plugins/cloudinaryPlugin.js:126:23     at step (/home/node/app/node_modules/payload-cloudinary-plugin/dist/plugins/cloudinaryPlugin.js:44:23)     at Object.throw (/home/node/app/node_modules/payload-cloudinary-plugin/dist/plugins/cloudinaryPlugin.js:25:53)     at rejected (/home/node/app/node_modules/payload-cloudinary-plugin/dist/plugins/cloudinaryPlugin.js:17:65)     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

I'll have a look and let you know ASAP
Thanks for reporting

cloudinary plugin version: ^0.1.1
payload version: ^2.5.0
here are my package.json dependencies
"@lexical/clipboard": "0.12.5",
"@payloadcms/bundler-webpack": "^1.0.5",
"@payloadcms/db-postgres": "^0.2.2",
"@payloadcms/next-payload": "^0.1.11",
"@payloadcms/plugin-cloud-storage": "^1.1.1",
"@payloadcms/plugin-form-builder": "^1.1.0",
"@payloadcms/richtext-lexical": "^0.5.0",
"@types/express-fileupload": "^1.4.1",
"cloudinary": "^1.41.1",
"copyfiles": "^2.4.1",
"critters": "^0.0.18",
"dotenv": "^16.0.0",
"escape-html": "^1.0.3",
"eslint-config-next": "^14.0.4",
"express": "^4.18.0",
"next": "^14.0.4",
"payload": "^2.5.0",
"payload-cloudinary-plugin": "^0.1.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-hook-form": "^7.43.9",
"react-icons": "^4.8.0",
"react-intersection-observer": "^9.4.4",
"sass": "^1.52.0",
"typescript": "^5.1.3",
"util": "^0.12.5",
"yjs": "^13.6.10"

I just noticed the issue raised after this PR from the payload repo: https://github.com/payloadcms/payload/pull/4218
Just checking if there's an easy fix to avoid issues for the future.

Created PR: https://github.com/finkinfridom/payload-cloudinary-plugin/pull/138

Just released v0.1.2


[Update Request]: Cloudflare Tunnel client (cloudflared)

What type of update are you requesting?
A new version of an existing package
Current Package Identifier
Cloudflare.cloudflared
Package Version
2023.6.1
Please describe the changes you would like to see
There is a new release (2023.6.1) available.
Download Page Url: https://github.com/cloudflare/cloudflared/releases

Created PR 110636 to address this issue.


Refactor Github Actions

This PR refactors github actions by merging the three separate workflows into a single one.

Coverage remained the same at 91.632% when pulling d8780a16780b8d73fc9334ac0b6bab4fa34eb0f5 on m6121:master into 2b419f12970f4301e140165ec4621bd63161483c on rdmorganiser:master.


Disable Creatable

Is your feature request related to a problem?
Our use case is that we would like to allow existing users to invite other users to install our app. They would enter the new users' email address into the widget using the creatable constructor.
Describe the feature you'd like
We would like to disable the ability of the user to tap on the 'createBuilder' text (and create a new user) until a valid email address is entered. Our createBuilder is something like below:
                                                createBuilder: (text) {
                                                  if (EmailValidator.validate(
                                                      text)) {
                                                    return Align(
                                                      alignment:
                                                          Alignment.centerLeft,
                                                      child: Padding(
                                                        padding:
                                                            const EdgeInsets
                                                                .all(8.0),
                                                        child: Text(
                                                            'Invite $text'),
                                                      ),
                                                    );
                                                  } else {
                                                    return const Align(
                                                          alignment: Alignment
                                                              .centerLeft,
                                                          child: Padding(
                                                            padding:
                                                                EdgeInsets.all(
                                                                    8.0),
                                                            child: Text(
                                                                'Invalid email address'),
                                                          ),
                                                        );
                                                  }
                                                },

When the email address is invalid they should not be able to tap on the createBuilder. I imagine enabling/disabling the onTap for the createBuilder would be useful in other circumstances as well.
Describe alternatives you've considered
I have tried disabling with GestureDetector and AbsorbPointer.
Thanks for the great widget!

Hello there @mdavo6.
I'll investigate the issue and get back at you asap !

Thanks @esentis . I hope that explanation made sense.

Tada, I'm back !
With the new version (2.6.5) you can find a new parameter on CreateOptions,  validator (bool Function(T))
e.g
validator: (country) {
  return country.name.length > 2;
},

The above will add only countries with name longer than two characters.


Add Handling of Zero Len Arguments

Currently there is a bug in the add operator for TRT where we receive a zero-length add operator. This PR is to offload add and even any such future ops which might have zero length to VM.

@anijain2305 @trevor-m @zhiics PTAL

LGTM, I would clarify that the bug is in the TVM frontend somewhere which is creating an add op with no args.


Add trace method

Reference: https://github.com/pimterry/loglevel

I love this library for our logging, but the lack of a trace method is something I really miss from libraries I've used in the past.
Expected Behavior
trace is an additional level, more verbose than debug
trace is disabled unless explicitly enabled
Use Case
When building big complicated features, it's really great to have an additional level of logging below debug. In the early stages of development, I'll add log.debug statements at key points in the code. As the flow of data becomes more clear, I start removing some of the more verbose log levels (or simply start commenting them out). Feature gets released and I move on to the next thing.
A few months later, I need to revisit the logic to account for a new use case or to fix a bug. My comments are great, but my log statements allow me to see the data in motion, but I've lost most of the verbose ones for clarity. The same goes for another developer on my team who needs to learn the algorithm.
Having a trace level allows me to keep the verbosity available, without flooding my console by default.
I'll see if I have some time in the next few weekends to take a look ...


Using Windows Forms via PowerShell can crash Win 10 File Explorer

Issue Type: Bug
When I use VSCode to alter PowerShell scripts that have a GUI built with Windows Forms, after a few iterations of running the script in debug mode to view and interact with the GUI, if I switch back to Windows (Win 10, 64bit, current build) and try to use the explorer to view files and folders, the windows are unresponsive. Other applications then become unresponsive. The Task Manager can be launched, but it is unresponsive too and doesn't show or allow any processes or applications to be closed. Only resolution is to restart the PC.
VS Code version: Code 1.25.1 (1dfc5e557209371715f655691b1235b6b26a06be, 2018-07-11T15:43:53.668Z)
OS version: Windows_NT x64 10.0.14393

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz (4 x 2808)

GPU Status
2d_canvas: unavailable_softwareflash_3d: unavailable_softwareflash_stage3d: unavailable_softwareflash_stage3d_baseline: unavailable_softwaregpu_compositing: unavailable_softwaremultiple_raster_threads: unavailable_offnative_gpu_memory_buffers: disabled_softwarerasterization: unavailable_softwarevideo_decode: unavailable_softwarevideo_encode: unavailable_softwarevpx_decode: unavailable_softwarewebgl: unavailable_offwebgl2: unavailable_off

Memory (System)
15.89GB (9.54GB free)

Process Argv
C:\Program Files\Microsoft VS Code\Code.exe

Screen Reader
no

VM
0%

Extensions (1)

Extension
Author (truncated)
Version

PowerShell
ms-
1.8.2

Thanks.


Add mouseMove event to detect/trigger residue:hover

Primary: @karry08
Sequence Viewer needs to display a tooltip when the user hovers over a residue.
Changes:

MSA internally listens to mouseMove events on the canvas to listen to residue changes
It calculates the hovered over residue (using the existing _getClickPos helper) and dispatches a backbone js event residue:hover which can be used by consumers.
Minor clean up that renames _getClickPos to _getResidueAtMouseEvent and _onmousemove to _onmousedrag to better indicate what the helper actually do.

LGTM


Add LoadOrCompute method to Map and MapOf

Codecov Report
Base: 95.26% // Head: 95.01% // Decreases project coverage by -0.25% :warning:

Coverage data is based on head (7ced804) compared to base (e5e826b).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #50      +/-   ##
==========================================
- Coverage   95.26%   95.01%   -0.26%
==========================================
  Files           6        6
  Lines         866      902      +36
==========================================
+ Hits          825      857      +32
- Misses         33       36       +3
- Partials        8        9       +1

Impacted Files
Coverage Δ

map.go
94.85% <100.00%> (+0.22%)
:arrow_up:

mapof.go
93.49% <100.00%> (-0.93%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Creating/Viewing App shows 500

I have just created a new application, and when viewing it shows a 500 error.

This was fixed shortly after this issue was posted. Sorry about that!


Merging master into live

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/faq.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Entity Picture support

It would be awesome if it was possibile to support the entity_picture attribute on the media_player entity, so the media control card could display a picture of what currently is on the TV, like:

logo of the tv channel

logo of the active app

Well that sure looks nice, but where are we supposed to get the images for any tv channel (tv channels are also country specific) and/or any tv app?

Well, this should be part of some technical analysis :)

sounds more like grunt work then technical analysis tbh.. unless more from the community favour this and start offering to help with it (including yourself), it may be too much of a burden for one person to research / implement

I've been thinking more about this, and I do believe it's possible to make such a feature with the use of the TMDB API.
The docs for the daily db exports (of tv networks):
https://developers.themoviedb.org/3/getting-started/daily-file-exports
Example of contents from an export:
{"id":1062,"name":"HBO Nordic"}
{"id":1063,"name":"Sky Atlantic"}
{"id":1064,"name":"RMC Découverte"}

This will give us the list of all TV networks on TMDB, but only their IDs and names. The export for TV networks is 84kb and includes 2685 channels, includes country specific and even local tv channels.
We'd then need to do a API request with the channel id to get the logo URL:
https://developers.themoviedb.org/3/networks/get-network-images
Example of a reply from the API:
{
  "id": 3308,
  "logos": [
    {
      "aspect_ratio": 2.425287356321839,
      "file_path": "/tuomPhY2UtuPTqqFnKMVHvSb724.png",
      "height": 174,
      "id": "5d6bb5e15f2db100105137e7",
      "file_type": ".svg",
      "vote_average": 0,
      "vote_count": 0,
      "width": 422
    }
  ]
}

We can then use file_path to get the logo URL, in various forms:

white no background: https://image.tmdb.org/t/p/h100_filter(negate,000,666)/tuomPhY2UtuPTqqFnKMVHvSb724.png
standard: https://image.tmdb.org/t/p/h100/tuomPhY2UtuPTqqFnKMVHvSb724.png

This would be a pretty cool feature, but the issues I see with it is the fact that we'd need to search through 2500+ channel names to get a match.. we could although create a list with previously matched channels to not repeat old searches..
Pinging @Screwdgeh too as he might also be interested in this.

There is no list that I know of for Tizen TV app logos, but that problem is much simpler then the tv channels one. We could just go through these 2 lists of most popular Tizen TV apps:

https://tekdeeps.com/the-best-apps-for-your-samsung-smart-tv-with-tizen/
http://vixi.tv/the-50-best-samsung-smart-tv-apps-list-2019/

and hardcode the app names + logos as a start, and leave it as a community effort to improve on the app list.

This is indeed very sexy. Just checked it out for Spotify media player and looks sexy as hell!
I have been looking more into this and found various options, none of them ideal though:

TMDB - has a good list of networks, no API call to search for a network (so would have to go off of downloading latest list and then matching ourselves) - the website has a call that returns the networks for a query (https://www.themoviedb.org/search/network?query=eurosport), but unfortunately then in the results list the name is not mentioned again, so further queries would be required to fetch the name based on network id and then do further matching in case of multiple results... So best would be to download latest network list at component initialization and match against that and only fetch and cache images when needed upon a title match - not ideal though! We may choose as part of update to fetch a new list every so often, e.g. once every week. With 84 kB that seems fair enough? I'll contact them to see if we overlooked anything...
Maze - has a good list of networks (https://www.tvmaze.com/networks), but also has no API call to query networks list... similar problem as above - I'll contact them to see if there is maybe something undocumented we could use.
TVDB/TVRage - only outdated list of networks and typically without logos, no go.
Website lyngsat-logo.com - also very extensive list of networks, but no API and copyright protected...

I wonder even if this should be a seperate metadata agent component that would contribute improved media metadata including entity picture support across any media player but not yet sure how that could work (as I believe a component cannot override another components state attributes... adding more state attributes would work though, but ideally it would also improve e.g. the media title). Could even rely on code from Plex/Kodi who do this really well. Anyway... that's too big a scope for how much I'm willing to invest... so no sense in daydreaming...
Let's wait for responses and see... otherwise happy to try along the TMDB route as the most viable.

@Screwdgeh oh, you researched quite a lot, I just made a 40 line app to scrape all images from TMDB (took about 10-15min), it resulted into this file which I uploaded to the repo:
https://github.com/jaruba/ha-samsungtv-tizen/commit/e33c183bae306b8ae83d4212d4e070c263aa3368
There are 1555 TV networks in there with their logo paths from TMDB, the file size is 74kb, I set the property names (tv network names) to lowercase for easier matching. Also TMDB considers "Netflix", "Disney+", "Rakuten TV", etc to be TV networks for some reason, but this is in our advantage as we can now match TV apps from the same file too.
It should be as easy as just reading the JSON file in python and logo_paths[self._source.lower()] and using the TMDB prefixes for images:

white no background: https://image.tmdb.org/t/p/h100_filter(negate,000,666)/tuomPhY2UtuPTqqFnKMVHvSb724.png
standard: https://image.tmdb.org/t/p/h100/tuomPhY2UtuPTqqFnKMVHvSb724.png

Not sure when I'll find more time to try it out, you can try implementing it if you want.

First small tryout with entity picture with URLs from the list... Home Assistant is kind of retarded (or probably I am) - it clearly takes the prevalent color out of the picture and uses that as background color... with a transparent picture and only the logo color the end result is that the background color is always the same as the logo (hence logo not visible).
So we need logo's actually with a well chosen background color!
And another issue is aspect ratio. It supports square images really well, rectangular images partly disappear. So we need to append to image size to match a square ratio as well if we want the entire logo visible.
This:

Becomes:

Will need to fiddle around later...

@Screwdgeh I could make a bot that processes all the pictures.. so it makes them all square and of the same size, sets a background color and a padding.. then put them up somewhere..

@Screwdgeh I left the bot running last night and made 2 new prefixes that we can use (instead of the TMDB ones).

colored logo, white background: https://jaruba.github.io/channel-logos/fff
white logo, dark background: https://jaruba.github.io/channel-logos/212c39

Can you try them out? I left the exact same image filenames as in logo_paths.json, so it should be a drop-in change by just replacing the TMDB image prefixes with the new ones.

Wow, you're quick with this stuff, impressive 😊! This is certainly much better! Some examples:
White background:

I love that the color helps view at a glance which channel you are likely at due to the whole entity changing color. But it does look pretty poor when the logo is just 1 color... which I'm afraid most of them actually are...
Dark background:

The dark looks more calm and streamlined, but does make it harder to see at a glance what is playing.

I think we should make a setting called dark_theme which is False by default. And use the dark version if this is set toTrue. I think the dark ones are clearly not as nice, but there is a need for this for users that use dark lovelace themes.
Can u add the setting too?

@Screwdgeh Any reason why you haven't committed yet? If you got stuck with something tell me how I can help.

Hey @jaruba... sorry had some private things to take care of... I made a commit with the changes. To test it add show_logos: "dark" or show_logos: "color" to the component config.
It looks very good and the image matching is working very well for me!
Some edge cases not covered yet for own channel names, like e.g. combined channel names I get returned for some channels, such as "Veronica HD / Disney XD".
How do you feel about making this long term? Things like:

Fetching images directly from TMDB API/file download and manipulating them on the fly?
Adding possibility to override logos by adding images to a local HA folder 'logos<network name>.png'
Let me also know performance on your end - right now it's a synchronous function, but performance looks absolutely fine on my end (<1s to match a logo). Ideally we move the logo matching to an async function in line with guidelines.

By the way, was your intention to do this on the 'source-logos' branch? I updated to master because I saw your logo_paths json file there too... Only later noticed the branch.

Sorry for insisting, I'm just excited about this feature. I made the other branch to commit the logo_paths.json to it, as I wasn't sure we'll be able to integrate it at first, but I pushed that file through the GH web UI to the master branch by mistake. Now that we know it's doable though, there's no problem to use the master branch for this.
Manipulating the images on-the-fly could be troublesome, we'd either need to start a web server in the addon to export them or pay for a server that will manipulate them for all users. I think the way we do it right now is the most efficient way, and I can update the list and logos in minutes by just running the bot again. When it comes to streaming services and tv channels, it's a rare event for a new one to appear, so updating the list isn't something that needs to be done frequently.
Regarding custom logos, we can (even now) change the logo or add new channel paths by editing logo_paths.json (the bot won't overwrite previous changes to this file, it will only add new entries if available), and adding a new logo to the channel-logos repo I made on GH. I wouldn't give users the possibility of adding logos locally to their setup because I prefer incentivizing the users to collaborate on the component so it leads to better UX for everyone.
Regarding sync vs async, I'll test and give u my results, it's possible that HA gives some warning if we use sync though. (It usually does)

There is, although, one flaw with pre-processing images like we currently do. It doesn't handle rebranding, if a logo changes it can't account for that change, because it only adds new entries. I can make it redo all images every time, but then if we manually edit a logo, that change will be overwritten, we can although add logos for channels that are not on the list, and that change will be persisted until TMDB adds that channel also.
I'll leave it like this for now, and if I see that nobody is interested in changing the TMDB logos, I'll make it handle the rebranding scenario too.

Btw, u should be aware that I'm going to change the default port ( https://github.com/jaruba/ha-samsungtv-tizen/blob/master/custom_components/samsungtv_tizen/media_player.py#L65 ) to 8002 in the next release. Both my 2017 TV and 2019 TV use 8002, the 8001 port hasn't been used for too many years already. Just telling you in case some users will complain about the component breaking for them after this change, in which case they should set the port to 8001 in their config.

Guys, great job!
I have a small comment so far.
For the background, I think we should allow using the primary color of the home assistant theme currently selected, as an alternative to the automatic color.
This will be more consistent with how the media player card looks by default when TV is on:

A dark background might not visually inform the TV is on.

@zappasmart I can make a version of the logos with blue background, but the logos themselves will need to be white (like in the dark examples) because otherwise we can't ensure that HA will use the blue color for the card background, it seems to use the primary color of the image and there's no setting to change that.

If I try to set a white image with transparent background like this one:
https://www.iconsdb.com/icons/preview/white/chat-4-xxl.png
I get this correctly:

The Media Control Card uses the theme's primary color for the background.
But if I use this from TMDB:
https://image.tmdb.org/t/p/h100_filter(negate,000,666)/tuomPhY2UtuPTqqFnKMVHvSb724.png
I get this (no picture displayed):

Any idea?

@jaruba regarding your image processing POC, can you try to generate white images on a transparent background?
I'd like to test how they are digested by the Media Control Card.

From my experience pure white images with a transparent background will work to keep the theme's primary color. Pure white is always ignored in selecting a background color. If it can't find any it will stick to theme's primary color. All other colors are picked up as a selection for component background color.
@zappasmart not sure what is wrong with the PNG as https://image.tmdb.org/t/p/h100_filter(negate,000,666)/tuomPhY2UtuPTqqFnKMVHvSb724.png -> I get the same effect, but downloading and saving to local hass folder and showing from there it works fine.
The greyscale as currently produced in some logos will not work the same and the gray will be picked up as background color. Maybe I'm too pessimistic, but sadly I think automatically converting the color logos to pure white will screw up some logos and will make them unreadable or make them unrecognizable (see examples above). I'll leave to @jaruba for further comment if he thinks differently...

Sorry for insisting, I'm just excited about this feature. I made the other branch to commit the logo_paths.json to it, as I wasn't sure we'll be able to integrate it at first, but I pushed that file through the GH web UI to the master branch by mistake. Now that we know it's doable though, there's no problem to use the master branch for this.

OK

Manipulating the images on-the-fly could be troublesome, we'd either need to start a web server in the addon to export them or pay for a server that will manipulate them for all users. I think the way we do it right now is the most efficient way, and I can update the list and logos in minutes by just running the bot again. When it comes to streaming services and tv channels, it's a rare event for a new one to appear, so updating the list isn't something that needs to be done frequently.

True, it's not that frequent... but I'm mostly not in favor of manual work. Hass does have the built-in web server (additional files located in 'www' folder are automatically hosted), couldn't we use that? I guess it depends on how much processing power is required for the conversion process and whether it can all be done in Python.

Regarding custom logos, we can (even now) change the logo or add new channel paths by editing logo_paths.json (the bot won't overwrite previous changes to this file, it will only add new entries if available), and adding a new logo to the channel-logos repo I made on GH. I wouldn't give users the possibility of adding logos locally to their setup because I prefer incentivizing the users to collaborate on the component so it leads to better UX for everyone.

Fully recognize that we should be able to benefit of each other's work! So agreed that repository contributions could be the way to go. I was checking and TMDB only accepts networks that are 'original content creators', so many tv channels we are interested in would not be accepted by them. Furthermore there is no automated entries for networks and it would have to be done as a forum post.

Regarding sync vs async, I'll test and give u my results, it's possible that HA gives some warning if we use sync though. (It usually does)

I'll make it work in async to be safe. I'll have a look at that later today, I think it's quite easy to achieve, just need to test if everything works fine still.

@zappasmart @Screwdgeh
So here's what I did, I changed the paths to the previous images (made a commit to use the new paths), and added 4 more image prefixes.
Currently we have:

white background, colored logo: https://jaruba.github.io/channel-logos/export/fff-color
dark background, white logo: https://jaruba.github.io/channel-logos/export/212c39-white
transparent background, colored logo: https://jaruba.github.io/channel-logos/export/transparent-color
transparent background, white logo: https://jaruba.github.io/channel-logos/export/transparent-white
blue background, colored logo: https://jaruba.github.io/channel-logos/export/05a9f4-color
blue background, white logo: https://jaruba.github.io/channel-logos/export/05a9f4-white

I also open sourced the bot, it's available on NPM: https://www.npmjs.com/package/channel-logos
Anyone can use it now to update logo_paths.json and/or create new logo versions based on their own preferences.
@Screwdgeh

Hass does have the built-in web server (additional files located in 'www' folder are automatically hosted), couldn't we use that?

I'm not sure we can, we might not have access to write to that file. Addons are normally sandboxed to prevent them from breaking things. With this said, I haven't tried though..

I guess it depends on how much processing power is required for the conversion process and whether it can all be done in Python.

I used Node.js for the bot as it's the primary programming language I work with, so I work the fastest with it.

I was checking and TMDB only accepts networks that are 'original content creators', so many tv channels we are interested in would not be accepted by them.

This might not be as limited as you think.. Keep in mind that "original content creators" also means creators of Talk Shows, Game Shows, News Shows, Reality Shows, Sports, etc. etc. All channels that I know of have some original content..

We can now set show_logos to any of these values: 'blue-color', 'blue-white', 'dark-white', 'white-color', 'transparent-color', 'transparent-white'. Default is 'white-color'.

@zappasmart @Screwdgeh
Please test it when u have some time, if everything goes well I'd release an initial version with this feature. If we find better ways to do this we can change it in the future.

Let me try and get a bunch of logo updates for Dutch television (a few missing and a few outdated logos) to TMDB and see how far I get, confidence level not very high.
The new color options are also sexy! The blue-color combo actually works very well too, I'm not sure which one to pick now! I think I still like the 'white-color' the best.
The transparent option sadly does not work very well as explained earlier (nor for color as the logo becomes invisible against the background, nor for white as most of them are grayscale rather than true white, making again at least parts of the logo blend against the background color).
Made 2 commits with some small changes:

option for 'none' to disable logos completely (documentation update included)
run the media update (title and image) as an async job instead which should prevent the warnings the update takes long. This way it does take a bit longer when a new logo needs to be looked up, but it still updates the title and picture at the same time so impact is minimal I'd say.

Good to go from my point of view!

A few comments from my side:

I'd change URL fff-color to white-color for naming consistency with the other options
The 05a9f4 options work well only if the user has the default theme. I don't use the default theme, I installed a theme I like more, so using those options will look visually inconsistent because of different blue colors
for the same reason, do you think option 212c39 should be changed to a dark neutral color instead of dark blue?
do you think it could be worth testing with other media player cards like the Mini Media Player? https://github.com/kalkih/mini-media-player

@zappasmart

the URL patterns are generated from the HEX color code and logo preference, i'd rather not change this, the setting in the config is white-color for it though
the blue option is obviously for a specific case, but many may use the default theme so there may be a purpose for it, as the transparent ones aren't working as one would expect
i've changed the color of dark-white to a neutral color, and renamed the previous one to darkblue-white
i'm not sure what you're asking regarding Mini Media Player, i never used it

I released v1.5.4

I've tested a bit with the Italian TV channels and looks good so far. Thank you for your efforts in implementing it!

Further update on my efforts at TMDB. Seems I was sorely mistaken on ways to edit the database! The good news is that anyone is able to edit existing networks including logos via TMDB UI and also add any that are missing. Although not super straight forward it is definitely possible! I added documentation to the component on how the information can be updated.
Since this can really be done properly and it also helps TMDB to have an updated list of TV networks, my suggestion/request is that we also proactively pick up any changes we get from TMDB. This also makes me wonder - I saw networks with multiple logos... do you know what logic you use to pick one? Just the first or last or?
Next to TMDB we should have a separate source of app icons. I went through the list of most popular Samsung Smart TV apps and though of some popular other sources used (Chromecast, Xbox, Playstation, Nvidia SHIELD, Nintendo switch, Wii). Have a set of 40 logos ready. But I see we have no way of adding logos for these yet. Propose we make an 'app_logo_import' folder in the repository or create a seperate repository for app logos? What do you think @jaruba?
I've started looking into the component and given I have a bit of NodeJS experience as well, happy to start contributing there too.

@Screwdgeh

my suggestion/request is that we also proactively pick up any changes we get from TMDB

Sure, we can make such a change easily, but it will take significantly more time to update. I can improve on the logic a lot though, I can add a folder to download both versions of the logo first, add the folder to .gitignore, then start processing all the versions we're using from the local files. If I want to go full psycho on it, I could check the headers before downloading the logos, if the content-length didn't change, it won't update it at all. 😛

This also makes me wonder - I saw networks with multiple logos... do you know what logic you use to pick one? Just the first or last or?

I just use the first in the logo array: https://github.com/jaruba/channel-logos/blob/main/pull.js#L50

I went through the list of most popular Samsung Smart TV apps and though of some popular other sources used (Chromecast, Xbox, Playstation, Nvidia SHIELD, Nintendo switch, Wii).

If we add Nvidia Shield, we'd better add other popular smart boxes too: Fire TV, Fire Stick, Apple TV, etc.

But I see we have no way of adding logos for these yet.

There is no reason to not use the channel-logos repo for extra logos too, as those are not just channel logos anyway, TMDB includes: Netflix, Disney+, etc. also.
My thinking is this:

we can edit the logo_paths.json from the channel-logos repo directly, when you pull a new version of the file it will not remove the non-TMDB entries from the file
add a other folder in the root of the web branch, we can also make 2 folders in other, a color folder and a white folder, and put the logos there accordingly, if we have only color versions that should not be an issue, the logic will just fail for all "white" versions of that logo and it shouldn't show in HA

Let's presume a new entry is "xbox": "/xbox.png", it will first try to get /xbox.png from the ./other/color/ folder, if it fails, then it tries to get it from TMDB.

I've started looking into the component and given I have a bit of NodeJS experience as well, happy to start contributing there too.

I added you as a contributor to channel_logos.
You'll probably notice the funny way I used to edit the images. I use puppeteer which in turn uses Chrome. I'm creating a 200x200 headless browser window and use CSS for styling the page, then I take a screenshot of that browser window, that's our new image. 😅

@Screwdgeh I updated the channel-logos repo to support custom images. Just upload them to this folder: https://github.com/jaruba/channel-logos/tree/main/custom
You only need to add the colored versions of the logos, as it can make the white versions from them itself now.
There is a readme in that folder explaining how it should be used. Ping me once it's done so I can update all the logo images again.

Hey @jaruba - great!
I did not have write access to the repository, so created a fork and pull request for now: https://github.com/jaruba/channel-logos/pull/1
Have a look through the logos and let me know if we have missed any crucial ones...
I have 1 other request - I have submitted several logo changes and (additions) at TMDB. Is it possible to implement update scenario for TMDB as well? At the same time I figured out that it would be best to: - take highest vote logo from the multiple logos presented by TMDB - if multiple logos have the highest vote, take the last from the array as it seems the last in the array is the one actually added last, so most recent. It seems TMDB applies the same logic to show network logo on their website.
Would be great to see them show up after your update!

@Screwdgeh
So I redid all the images, I didn't add the logic you proposed yet though (but the logic seems fair so I'll look into adding it), you can test the master branch of this repo if you want, I'll hold back on making a new release for the moment. There seems to be an issue in puppeteer where the screenshots I make are of a poor quality. All tickets regarding this seem to suggest using deviceScaleFactor: 2 which makes our 200x200 images become 400x400. (same as just making them 400x400 in the first place tbh, should probably look into using deviceScaleFactor: 1.5)
This makes the images be of a better quality because it literally resizes them in the browser for our needs, but it also makes what was previously a 14kb image be a 45kb image..
Let's see how it goes..
Another thing to note is that instead of hardcoding logo_paths.json, we could download it from https://jaruba.github.io/channel-logos/logo_paths.json, which would make the HA component update in real time instead of updating whenever we update the component.. But it's a 74kb file, which would need to be downloaded every time HA restarts.. Not sure what would be best in this case.. (given that a single image is 45kb in the current case..)
Opinions on all this are welcomed.
Future planned tasks regarding this:

use @Screwdgeh 's logo choosing logic
download all images first to a /temp folder before reprocessing all the images (would increase speed a lot)
check the content-length header of TMDB remote images before downloading to ensure that we're not redownloading the same images (would increase speed a lot)
run the bot on an interval
bot should auto-update the github repository periodically

Sounds great @jaruba! I can make the changes to download the logo_paths.json periodically. We can just write to file and check modification date of the file every so often to see if it requires updating, that would avoid downloading every time at an HA restart. Need to check if we can do same thing using content-length to avoid downloading the file if it is the same.

@jaruba I have added the functionality as described above. It even checks the file modification date online via http header (HTTP HEAD to only fetch the headers) first and only downloads it if that is more recent than the file stored locally. You could use this same mechanism for the images from TMDB API if it provides the same functionality of the 'Last-Modified' header in the header response.
I did run into 1 issue for my own installation: I copied the original component over with a non-hass user account, meaning the hass account running home assistant did not automatically have write access to the paths file to edit it and resulted in a permissions issue. If we want to be safe it is best to leave the logo_paths.json out and download it first time the component starts. Then there should not be any permission issues. Do you agree to remove the logo_paths.json file from the files to prevent this issue from happening?
Out of interest - did you update the logo_paths file with new TV channels from TMDB or not yet? Because I did not see the logos appearing yet for the new TV channels I added there (e.g. 'Ziggo Sport' and 'RTL Z')... If you do update it let me know, immediately a good test!

@Screwdgeh

You could use this same mechanism for the images from TMDB API if it provides the same functionality of the 'Last-Modified' header in the header response.

I was actually overthinking this, I can just check against the filenames to know if it's the same image or not.. lol..

I did run into 1 issue for my own installation: I copied the original component over with a non-hass user account, meaning the hass account running home assistant did not automatically have write access to the paths file to edit it and resulted in a permissions issue.

I was actually thinking that it might be a good idea to still leave the logo_paths.json in the component (and rename it to logo_paths_local.json), we could then use it as a backup if there's any error (whatsoever) during downloading / writing the remote file. The one in the component would just update when we update the component.
The thing is that there are many issues that can arise from attempting to download / write the file.. we should not fully rely on downloading for this feature.

Out of interest - did you update the logo_paths file

I don't think I updated the logo_paths file, only the images. I'll do another update as soon as I change the logic as different logos will be chosen.

I was actually overthinking this, I can just check against the filenames to know if it's the same image or not.. lol..

I guess indeed you can't edit existing images on TMDB, so they will always get a new URL indeed lol

I was actually thinking that it might be a good idea to still leave the logo_paths.json in the component (and rename it to logo_paths_local.json), we could then use it as a workaround if there's any error (whatsoever) during downloading / writing the remote file. The one in the component would just update when we update the component.
The thing is that there are many issues that can arise from attempting to download / write the file.. we should not fully rely on downloading for this feature.

Done in a new commit. Small change, guess you're right this is safest and given the logo_paths don't update often this still has value to the user! We can keep logo_paths.json as is... I download a new file under logo_paths_download.json and keep it as recent as possible checking against the online repository.

I don't think I updated the logo_paths file, only the images. I'll do another update as soon as I change the logic as different logos will be chosen.

Clear - let me know when you do - curious to see the autoupdate do its magic!

@Screwdgeh
I updated logo_paths.json in channel-logos, and also updated all the logos based on your proposed logic (top score first, alternatively the last logo in the array)
Test it out.
If everything works correctly, all that's left is to make a script to run the bot monthly.

Works as expected while testing today, updated file was downloaded automatically and I can see all the added and updated logos while browsing through the related tv channels!
Awesome stuff!


Repository history off ???

Somehow the git history seems to be totally off for this repo. According to repo page on GitHub, the last commit to master was made 3 years ago.

However, a click on commit shows all recent merged commits.
A local checkout of the repo contains the same head from three yeas ago 😕
What is going on here?

That’s totally normal; git history is not chronological. GitHub just shows the commit order in a confusing way.


Funnel Entry Date
<YYYY>-<MM>
Idea Owner
@<Github-Handle>
Idea Description
<An elevator pitch (value statement) that describes the idea in a clear and concise way.>
For <BFH member>
who <do something>
the <solution>
is a <something – the ‘how’>
that <provides this value>
unlike <current solution or non-existing solution>
our solution <does something better — the ‘why’>
Business Outcomes
<The measurable benefits that the business can anticipate if the idea hypothesis is proven to be correct.>
Leading Indicators
<The early measures that will help predict the business outcome hypothesis.>
Nonfunctional Requirements (NFRs) e.g. via ISO-25010
<Nonfunctional requirements (NFRs) associated with the idea.>
--
derived from SAFe Epic Hypothesis Statement Template

Template has already been created via https://github.com/bfh/open-source/commit/4b38a4cd4d383266ee2b3713b9bdc2660f8ca3c9 - thanks @l00mi !


FLEX-1693 New devicefilter

SonarQube analysis reported 40 issues

 3 critical
 20 major
 17 minor

Watch the comments in this conversation to review them.
Top 10 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 Device.java#L166: Make "authorizations" private or transient.
 FirmwareManagementService.java#L209: Use isEmpty() to check whether the collection is empty or not.
 FirmwareManagementService.java#L352: Method has 12 parameters, which is greater than 7 authorized.
 FirmwareManagementService.java#L352: The Cyclomatic Complexity of this method "addDeviceModelFirmware" is 11 which is greater than 10 authorized.
 FirmwareManagementService.java#L390: Use isEmpty() to check whether the collection is empty or not.
 FirmwareManagementService.java#L422: The Cyclomatic Complexity of this method "changeDeviceModelFirmware" is 11 which is greater than 10 authorized.
 FirmwareManagementService.java#L422: Method has 12 parameters, which is greater than 7 authorized.
 FirmwareManagementService.java#L551: Define and throw a dedicated exception instead of using a generic one.
 DeviceManagementEndpoint.java#L111: Remove this unused method parameter "request".
 Device.java#L226: Constructor has 9 parameters, which is greater than 7 authorized.

All is well.

SonarQube analysis reported 40 issues

 3 critical
 20 major
 17 minor

Watch the comments in this conversation to review them.
Top 10 extra issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 Device.java#L166: Make "authorizations" private or transient.
 FirmwareManagementService.java#L209: Use isEmpty() to check whether the collection is empty or not.
 FirmwareManagementService.java#L352: The Cyclomatic Complexity of this method "addDeviceModelFirmware" is 11 which is greater than 10 authorized.
 FirmwareManagementService.java#L352: Method has 12 parameters, which is greater than 7 authorized.
 FirmwareManagementService.java#L390: Use isEmpty() to check whether the collection is empty or not.
 FirmwareManagementService.java#L422: The Cyclomatic Complexity of this method "changeDeviceModelFirmware" is 11 which is greater than 10 authorized.
 FirmwareManagementService.java#L422: Method has 12 parameters, which is greater than 7 authorized.
 FirmwareManagementService.java#L551: Define and throw a dedicated exception instead of using a generic one.
 DeviceManagementEndpoint.java#L111: Remove this unused method parameter "request".
 Device.java#L226: Constructor has 9 parameters, which is greater than 7 authorized.

All is well.


Missing Data Aggregate By Deployment/Controller/Controller Kind

Hi, I am installing kubecost with a helm chart using version 1.92.0.
This kubecost use an external data source which is VictoriaMetrics installed by victoria-metrics-k8s-stack
you can check the full helm chart kubecost value here, https://github.com/zufardhiyaulhaq/community-ops/blob/master/clusters/home-lab-kubernetes-01/releases/kubecost-system/cost-analyzer.yaml
The issue is when I open the kubecost dashboard, I cannot aggregate with Deployment/Controller/Controller Kind. It's always shows __unallocated__ & __idle__

But when I am aggregate by pod, it's working fine

I try to open the diagnostic page, and there are 2 warnings but I am not sure that these warning is causing aggregate with Deployment/Controller/Controller Kind fail.

gz#2032(related to Zendesk ticket #2032)

Kubernetes version: 1.23
Kubecost version: 1.92.0
VictoriaMetrics version: v1.78.0


Registry Settings Options

Is there a registry source for these policies?  Chrome, for example, documents its @ https://www.chromium.org/administrators/policy-list-3#ExtensionInstallWhitelist ;  I attempted to use the same via Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge\ExtensionInstallWhitelist (and corresponding ExtensionInstallSources), but I still get this error attempting to install the extension:

This extension is not from any known source, and may have been added without your knowledge.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d1cd3ad5-bb4a-54ff-84e7-18ca0b11368f
Version Independent ID: 984abd8f-45eb-737b-ad74-07ec8261f9eb
Content: Microsoft Edge Browser Policy Documentation
Content Source: edgeenterprise/microsoft-edge-policies.md
Product: microsoft-edge
GitHub Login: @Brianalt-Msft
Microsoft Alias: stmoody

Hello @zastrowm
I was having the same issue, it seems that for Edge contrary to the documentation, you need to have the value in both ExtensionInstallWhitelist and ExtensionInstallAllowList
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge\ExtensionInstallWhitelist]
"1"="myvalue"
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge\ExtensionInstallAllowlist]
"1"="myvalue"
With these keys I was able to get my extension to work.

Has this changed recently? Setting both or either of those keys doesn't seem to be working (i.e. allowing the extension) on a Mac even though the edge://policy page reflects the policy changes.

Hello @zastrowm
I was having the same issue, it seems that for Edge contrary to the documentation, you need to have the value in both ExtensionInstallWhitelist and ExtensionInstallAllowList
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge\ExtensionInstallWhitelist]
"1"="myvalue"
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Edge\ExtensionInstallAllowlist]
"1"="myvalue"
With these keys I was able to get my extension to work.

It should work with just one policy ExtensionInstallAllowlist - as that has been the only one used in Microsoft Edge since inception. Can you try using only this policy and confirm?

Edge says the settings are "in error" but has allowed me to go with ExtensionInstallForcelist
See https://i.imgur.com/YrwEmiC.png - The old ones are listed as error but the ForceList is up and running; The registry key is ExtensionInstallForcelist (REG_SZ 1 = )

Hi All, if you are still facing an issue setting up ExtensionInstallAllowList policy pls share a screenshot of the edge://policy page. Make sure you expand the Error details so that the error is visible.

I was able to use ExtensionInstallAllowlist, although it didn't seem to take at first. I had to also reload policies from edge://policy and hard refresh edge://extensions, and then the warning disappeared.

Clean up of old issues.  Appears the question was previously answered.
If you have general questions regarding Microsoft Edge please reach out on the Microsoft Edge Tech Community or for Enterprise specific questions please visit the Enterprise section of the Tech Community. Thanks!

Hi, I have the same problem but for Edge Dev. Could someone point out what the registry key for Edge Dev is? Thanks so much!


Retry operator issue

Check to see if we have a disposable before trying to dispose

@davidwdan Can you add a test that exposes where it previously went wrong?

@asm89 added a test


Google Fonts Documentation

I was setting up some fonts and noticed the docs are a bit outdated. (Sorry if this is not the right place to report this)
Since the scss files moved it's not necessary to adjust the path to ../../fonts and the default ../fonts/ from google-webfonts-helper would work. Also the snippet include the wrong path.

Nice catch! Google fonts docs are for the official 5.1.3.x releases. You are using the latest main (<IP_ADDRESS>) branch where the scss structure has been changed. But is not official released yet.
Docs for 5.2.0 are already rewritten, but not published yet. There are some more points in the docs that have changed.

Closed by <IP_ADDRESS> release https://bootscore.me/documentation/google-fonts/ 👍


About the result of "EEGClip"

Thank you for your excellent work and patient answers before.  I tried many times but I failed, so I came again. In the "EEGClip", I have two questions.
The first is about the result. I have run the "main.py" for 2785 epochs but the result is still lower than 0.05.

The second is about evaluating. I could only find "main.py" to train the model but I did not find "evaluate.py" to get the result of testing data. Perhaps I overlooked something?
Thank you very much for your attention and patience!

Hi @xuxiran , check this issue [ link ].  If it doesn't work for you, please let us know.
For evaluation this is code we have used [ topkrecall.py ]

Thank you very much for your timely response. I have read the closed issue #2 and #7. I made sure that the dataset is "5-95Hz". However, I could not get satisfactory answer.
I doubt that maybe I miss some pre-programs? Do I need to prepare any other files before running "main. py" in "EEGClip"? Just like before running "train. py" for "EEGStyleGAN-ADA-CVPR40", I need to prepare eegbestckpt and imageckpt.
Or, as I understand it, "EEGClip" should be an end-to-end architecture, and I should simply run "main. py"?
In addition, I evaluated this code by running "topkrecall.py", but I still could not get a good result. After modifying some problems, I successfully ran the "topkrecall.py", but the result was still lower than I expected.

When I try to use the pre-trained parameters, it seems that the model of pre-trained parameters is not the model we got in "EEGClip".

Thank you very much for your attention and patience!

Hi @xuxiran , thank you for raising the issue. I am adding Dwip in the loop @dwipddalal in the loop. He will address the problem soon.

Thank you very much! You are the best repository owner I have seen!

I have finished running main.py and are running classification.py now. It seems works. Thank you very much!

Congratulations! You seem to be almost successful.
Do you use filtered EEG data? I'm still running main.py. Can you share the clip_2048.pth file?

Hi @zihang-xu , we are also running the experiments in parallel. We will upload the clip_2048.pth once it is done. We made a few updates to the code after making the final updates on Git Hub. I will let you know.
For filtered EEG data, we have to check again. We have used and reported the results in the paper.
P.S. The initial experiment code got deleted due to some confusion while cleaning the drive, so we have to re-run and check which one worked earlier 😅

Thanks for your patience and explanation! I look forward to your updates.
Experiment results can be deleted by mistake. I've done the same thing 🤣

If you have new results, look forward to sharing and updating them. :)
Thanks again！

Is EEGCLIP also a zero sample recognition task?

Hi @weipipione , if you are asking about zero-shot classification. We have performed that experiment with a triplet-loss-based feature extraction method. We haven't explored it for EEGCLIP.

@prajwalsingh Hello, may I ask where the original data was obtained? I did not find any relevant connections, like EEGCVPR40 Dataset

Hi @Applicant-233 , the data is obtained from here [link]

Hi @Applicant-233 , the data is obtained from here [link]

Thank you! The link offers raw data, but the preprocessed dataset used in the paper is not included.

@Applicant-233 Both raw and preprocess data are there with name 5_95...

@Applicant-233 Both raw and preprocess data are there with name 5_95...

Thanks!

Thank you for your work, so the checkpoint for the model present in EEGStyleGAN-ADA/EEGClip/classification_eeg.py is not uploaded yet? I cant find it on the repo

这个代码从哪里运行呢


Why the standard modules can be overriden?

There is a proof:
---- MODULE testnat ----
EXTENDS Naturals
THEOREM test == 0 \in Nat
OBVIOUS
====

It is proven if no additional paths are given:
$ opam exec -- tlapm /home/karolis/CODE/pub/tlapm-lsp/deps/zenon/zenon-src/regression/backend/testnat.tla
...
[INFO]: All 1 obligation proved.

But after I added support for loading modules from jars/zips I found that this doesn't work:
$ opam exec -- tlapm /home/karolis/CODE/pub/tlapm-lsp/deps/zenon/zenon-src/regression/backend/testnat.tla \
  -I 'jar:/home/karolis/.vscode/extensions/alygin.vscode-tlaplus-nightly-2024.2.606/tools/tla2tools.jar!tla2sany/StandardModules'

(* created new ".tlacache/Naturals.tlaps/Naturals.thy" *)
(* fingerprints written in ".tlacache/Naturals.tlaps/fingerprints" *)
File "Naturals.tla", line 1, character 1 to line 36, character 77:
[INFO]: All 0 obligation proved.
(* loading fingerprints in ".tlacache/testnat.tlaps/fingerprints" *)
(* created new ".tlacache/testnat.tlaps/testnat.thy" *)
(* fingerprints written in ".tlacache/testnat.tlaps/fingerprints" *)
File "/home/karolis/CODE/pub/tlapm-lsp/deps/zenon/zenon-src/regression/backend/testnat.tla", line 6, characters 1-7:
[ERROR]: Could not prove or check:
           0 \in Nat
File "/home/karolis/CODE/pub/tlapm-lsp/deps/zenon/zenon-src/regression/backend/testnat.tla", line 1, character 1 to line 8, character 4:
[ERROR]: 1/1 obligation failed.
There were backend errors processing module `"testnat"`.
 tlapm ending abnormally with Failure("backend errors: there are unproved obligations")
Raised at Stdlib.failwith in file "stdlib.ml", line 29, characters 17-33
Called from Tlapm_lib.process_module in file "src/tlapm_lib.ml", line 446, characters 12-77
Called from Tlapm_lib.main.f in file "src/tlapm_lib.ml", line 573, characters 23-43
Called from Stdlib__List.fold_left in file "list.ml", line 123, characters 24-34
Called from Tlapm_lib.main in file "src/tlapm_lib.ml", line 576, characters 13-40
Called from Tlapm_lib.init in file "src/tlapm_lib.ml", line 588, characters 8-33

If I understand correctly, the standard/built-in module Naturals is now overridden by a module from the tla2tools.jar. This probably breaks some internal encodings for the backend provers.
@muenchnerkindl, maybe you know why there is a possibility of overriding the built-in modules? Is that intended, or a bug?

I don't remember how built-in modules are recognized and what's the story with module overrides but @damiendoligez may know.

There is even a test checking if the standard modules can be overridden. I understand why that could be useful for the TLC module, but overriding for Naturals and others looks strange.
https://github.com/tlaplus/tlapm/blob/b9b9a92b187b523cff0c70dfa3ab8d9cabf546dc/src/module/m_save.ml#L343

I'll add an option for the tlapm to prefer StdLib instead of modules with the same name and will use it in the LSP server.  Probably --prefer-stdlib. The default behavior will remain the same.

Option --prefer-stdlib is added in https://github.com/tlaplus/tlapm/pull/93.


Config to lower level of heuristics EXTRACT.28 and EXTRACT.13

Is your feature request related to a problem? Please describe.
When analysing Python packages, I often came across false positives from those two heuristics:

EXTRACT.28 (Extractable _RDATA section found)
EXTRACT.13 (Single Executable Inside Archive File)

Apparently, it's not an uncommon situation for Python packages compiled for Windows to have executable parts in _RDATA, I also came across multiple DLLs triggering this heuristic (e.g. fe27c4c07c0cfbb2ee28c8409e5a8db89d86c6c2d76c6e3b79ab31979138b215).
The second isn't uncommon as well, and in addition - looks like it has some issues with properly handling binary files from which an executable was extracted. I've noticed it's sometimes triggered when an exe is extracted from another exe, or when it manages to decompile code from a PYC file (but not always):

(__decompiled_source.py comes from my service, Extractor didn't see it)
Describe the solution you'd like

A config option to make EXTRACT.28 and EXTRACT.13 informative.
Improvements to EXTRACT.13 to be triggered only when an executable is extracted from an archive, not another executable or decompiled.

Describe alternatives you've considered

Option to disable heuristics at all - but it would just remove information that could be useful.
Hardcoded lowering score of those heuristics - definitely not, it would break use cases apart from mine.
Generic option to override the default/maximum score of a heuristic by an administrator - this could be interesting and allow adjusting an AL instance to the local use case, but requires much more work.

Additional context
Extract service already has multiple options for adjusting some heuristics, but not those.

This is another issue that I missed a while ago.
A couple months ago we modified the score of EXTRACT.28 to reduce it to 200 points instead of 300. That should stop it from causing a suspicious result section.
Regarding EXTRACT.13 (and possibly EXTRACT.16), I just updated the Extract service to not flag any python script coming out of executables. I also added a new structure for false positive detection for file extension, just in case our identification for python script is not perfect... 😅
You should be able to test v4.5.0.stable48 and see if that solves your issues! Thanks for the report!

Thank you! I think it should almost entirely solve my issue. In exchange, I've uploaded new misidentified files :innocent: #284


Discrepancy in Notification format

I am trying to hook up a notification service to hipchat/slack that alerts a group on push from our CI server
as per https://github.com/docker/distribution/blob/master/notifications/event_test.go#L28 the notification event should contain the tag that has been pushed
but when I see the notification body I do not see the tag instead I see the SHA digest, for example: https://gist.github.com/shreyu86/ecded0f6b26bf20c8911#file-gistfile1-json-L12
did the format for the notification change? I am using the following registry config: https://gist.github.com/shreyu86/334fd5f92245e1d0191b

@shreyu86 If you are looking for the tag of manifest, it is in the manifest itself. Use the URL to download it and you'll be able to access it.
The tests are simply testing that the format is not tampered with. The format of the urls is not part of that particular unit test. Originally, pre-2.0 release, we had included the tag, since this was the only supported way to fetch manifests. With the advent support for pull by digest, we changed to a digest reference for the notification URL (again, pre release). These urls are the only way to uniquely fetch the exact resource that was pushed. If using a tag url, by the time the event is received, a different manifest could be the value of that tag.
Note that we don't send a notification separately for the tag since tag objects aren't yet represented as unique entities.

@stevvooe thanks for the clarification, this makes a lot of sense, I was confused and then looked at the tests to see how canonical format looks like which led to my curiosity, definitely manifest contains the source of truth and shall use it for creating notifications.

@shreyu86 Great, I'll go ahead and close.


Center modal vertically

Description
Could you please explain, how I can center modal vertically?
Screenshot, black rectangle is the area where I want the modal to be:

And here is where modal be if I set "ending_top: 50%":

Repro Steps
Sorry, I don't know, what I have to write here :(
Screenshots / Codepen
http://codepen.io/anon/pen/dNvaQy
(Thank you for materializecss!)

You would have to add transform: translateY(-50%); to the modal. This shifts it up by half of the modal's height. You may run into some problems of that style being overwritted by JS though. Maybe we will look into making this a bit easier.

@Dogfalo, thank you for answer!
I added style="transform: translateY(-50%) !important;" to modal div but it didn't work. Also I tried to do this from javascript: $('#modal2').css('transform', 'translateY(-50%)');, but this also didn't work. So as you say there is some problem with overwritting by JS. Could you please check that I did (http://codepen.io/anon/pen/WRjRxM) and maybe there is another solution? Thank you!

@dima74 right now, until there is some sort of option in the modal configuration for it, there are two options:

You should put that $('#modal2').css('transform', 'translateY(-50%)'); into the ready callback of the modal to avoid the css overwrite.

  $('#modal2').modal({ending_top: '50%', ready: function(modal) {modal.css('transform', 'translateY(-50%)'); }});
But this option creates a non appealing effect as you will see the modal moving to it's position after being shown.

On your css (not on javascript).
#modal2 { transform: translateY(-50%) scaleX(1) !important; }

@Nohinn, thank you a lot!
Your solution works.
Also I noticed, that if I only put #modal2 { transform: translateY(-50%) !important; } to css (not inline as I tried before) then It works too (http://codepen.io/anon/pen/PWjwmB).
Thanks again!


Support for multiple private registries in 2.6 (secretmigrator code)

#37658
It will fix the logic of the function CreateOrUpdatePrivateRegistrySecret to store secrets for multiple registries.
https://github.com/rancher/rancher/blob/dfe44512d12d5674607be1848e53ce83f5b67e6c/pkg/controllers/management/secretmigrator/clusters.go#L661
The relevant secrets in the namespace cattle-global-data will be created correctly.

We can merge this now, but it looks like you ran a git pull on this branch and added a merge commit to the PR. Can you remove the merge commit with
git fetch origin
git rebase origin/release/v2.6

Thanks!

@cmurphy
My bad, should be fixed now.


Addition of sub- & superscript text styling plugins

Concerning #404: the addition of sub- & superscript text styling plugins (visualised in the editor toolbar).
These plugins are not added to the basic styles set but to a new extended styles set. The option to show these buttons by default will be set to false.
What still needs to be done:

[ ] Request new sub- & superscript icons within ember-appuniversum to use within the toolbar button (@brenner-company).
[ ] Move sub- & superscript buttons next to the existing buttons on the left of the toolbar (@abeforgit).
[ ] Fix bug where added buttons are only toggle-able two times (@abeforgit).
[ ] Addition of new toolbar option. Something like showExtendedTextStyleButtons (@abeforgit)?

Superseded by https://github.com/lblod/ember-rdfa-editor/pull/531


Add deployment file for the 2nd test case and update README.md

Both the manifest and deployment files look good.  However, we recently renamed "location" key under the Action entity to be "function".   This was noted on a recent call, but perhaps went un-noticed.  The goal was to align the name for the functional code artifact (semantics) to other Serverless offerings, but perhaps we should change back to "location" for now that way all entities can have the same keyword.  Let us leave "location" for now and discuss on next specification call with Paul.

Please note that the project was renamed wskdeploy and your branch is wsktool still

So I will keep "location" for now, till we decided to change to "function".


Link hash table primes externally to prevent data duplication in binary

I've started analyzing Godot binaries on Windows using SizeBench and on top of the duplicate data results were hash_table_size_primes and hash_table_size_primes_inv.

As visible above the tool shows around 168KB of wasted memory (tested on commit 36e943b6b20cb7a8a89bc30489c4a81c3e149d74 scons target=template_release debug_symbols=yes)
This PR reduces build size by 547KB ( scons target=template_release).
File scoped const arrays are linked internally which means each time hashfuncs.h are included, those two arrays are not merged into single data point (which would be fine and expected for const, read only data). ~I've moved their initialization to hashfuncs.cpp and marked extern in hashfuncs.h.~ Thanks to @Malcolmnixon suggestion I've made it simpler by using inline constexpr from C++17 standard.
I've not benchmarked it yet. In theory it can improve the CPU cache, because all of the hash_maps and hash_sets reuse the data from the same address in memory (meaning - there is higher chance that it will be still present in cache when needed).

That's a good catch.
Since C++17 is now supported and the guidelines promote using constexpr, you might want to see if keeping the table in the header and marking them constexpr (and possibly inline) works as well. In theory this should also merge multiple instances of the data across different translation units, but also expose the values as constants to the compiler allowing constant-folding if accessed discretely.

I think I tried constexpr already and it didn't change anything in this case, but  I'll try inline constexpr. As you said, in theory it should work the same as the solution with extern.

Thanks!


Update Makefile

As some of our customer may edit wrong in "make edit-cluster" step or they need to update their cluster, so they need rolling update options. Also, if they choose use makefile to doing their kops operation, we my better list all operations so that they have no need to maintain another set of Environment variable.

thanks @jansony1


Check correct use of getNodeType methods

Check that the node to parse the typestring is attached to a SourceUnit node, using getContainingSourceUnit method, which already has an assertion to check the result is not undefined
Replace every use of getNodeType and getNodeTypeInCtx with the new functions

Sem tests running.

Sem tests pass.


Update unit tests

Codecov Report

Merging #18 (d2807bd) into master (961af94) will decrease coverage by 0.55%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master      #18      +/-   ##
============================================
- Coverage     29.44%   28.89%   -0.56%
+ Complexity       98       96       -2
============================================
  Files            13       13
  Lines           309      308       -1
============================================
- Hits             91       89       -2
- Misses          218      219       +1

Impacted Files
Coverage Δ
Complexity Δ

src/Storage/StorageFactory.php
92.30% <0.00%> (-7.70%)
8.00% <0.00%> (-1.00%)

src/Twig/Node/ToggleNode.php
96.29% <0.00%> (-0.14%)
8.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 961af94...d2807bd. Read the comment docs.


enhance github repo create action

Description
could we add github repo and use our own template instead of default template?
also could we setup the repo as internal options?
Describe the Proposal
A clear and concise description of what you would like.
Has the Feature been Requested Before?
Please link related issues here, if any.
Describe the Alternatives You Have Considered
A clear and concise description of any alternative solution or feature you've considered.
Additional Context
Add any other helpful context or screenshots about the feature request here.

Thanks for your feedback. Regarding the problem of custom templates, it is impossible for DevStream to identify an unknown template. However, we provide a plugin mechanism, you can develop your own plugin according to the DevStream specification, and implement custom template parsing in this plugin.
See https://github.com/merico-dev/stream/blob/main/docs/creating_a_plugin.md for more info.

@JaneLiuL We use Golang to render the templates. For example, this is the template we have been using.
When defining the templates, you need to set some "placeholders", and when rendering the template, you need to know what variables to pass to those placeholders as values.
Because of this nature, this plugin currently can't support a user-defined template.
However, this might be possible. @daniel-hutao For example, the users could pass a link to the template location, and the users could set the values in the options section of the config.
@JaneLiuL Thanks again for this valuable suggestion; we will have a discussion about it and let you know the result as soon as possible!
There is another way, which is as Daniel said, if you are interested, you can create a custom plugin for it, but bear in mind that the plugin should be not too specific to your own project because the plugins are supposed to work for most of the communities.