Setting the CPU affinity of worker threads(iou-wrk-xxx and iou-mgr-xxx)

From the kernel source code of IO uring, it is found that the worker thread is forced to set CPU affinity on NUMA node, and its affinity cannot be modified. My program (Application Based on dpdk) requires exclusive CPU and does not allow other threads to run on some CPUs. I hope axboe can open the interface for setting CPU affinity or use the CPU affinity property of the parent thread. Thank you!!
set_cpus_allowed_ptr(tsk, cpumask_of_node(wqe->node));
tsk->flags |= PF_NO_SETAFFINITY;
	tsk = create_io_thread(io_wqe_worker, worker, wqe->node);
	if (IS_ERR(tsk)) {
		if (atomic_dec_and_test(&wq->worker_refs))
			complete(&wq->worker_done);
		kfree(worker);
		return false;
	}

	tsk->pf_io_worker = worker;
	worker->task = tsk;
	set_cpus_allowed_ptr(tsk, cpumask_of_node(wqe->node));
	tsk->flags |= PF_NO_SETAFFINITY;

	raw_spin_lock_irq(&wqe->lock);

Would be reasonable to use affinity of a task it's bound to, need to look into it

Would be reasonable to use affinity of a task it's bound to, need to look into it

Here I put my solution. Get the cpumask of process 2 (kthreadadd) and the cpumask of the current NUMA node, and do the and operation to get the final cpumask(see get_cpumask_by_numa). In addition, I modified the affinity of kswapd and kcompactd in the same way.
The kernel version (5.12.6)
static bool get_cpumask_by_numa(struct cpumask* mask, int node){
	struct cpumask kthread_mask;
	if (0 != sched_getaffinity(2, &kthread_mask)){
		return false;
	}

	if(node == -1){
		cpumask_copy(mask, &kthread_mask);
		return true;
	}

	if(!cpumask_and(mask, &kthread_mask, cpumask_of_node(node))){
		return false;
	}
	return true;
}

static bool create_io_worker(struct io_wq *wq, struct io_wqe *wqe, int index)
{
	struct io_wqe_acct *acct = &wqe->acct[index];
	struct io_worker *worker;
	struct task_struct *tsk;
	struct cpumask mask;

	__set_current_state(TASK_RUNNING);

	worker = kzalloc_node(sizeof(*worker), GFP_KERNEL, wqe->node);
	if (!worker)
		return false;

	refcount_set(&worker->ref, 1);
	worker->nulls_node.pprev = NULL;
	worker->wqe = wqe;
	spin_lock_init(&worker->lock);
	init_completion(&worker->ref_done);

	atomic_inc(&wq->worker_refs);

	tsk = create_io_thread(io_wqe_worker, worker, wqe->node);
	get_cpumask_by_numa(&mask, wqe->node);
	if (IS_ERR(tsk) || cpumask_empty(&mask)) {
		if (atomic_dec_and_test(&wq->worker_refs))
			complete(&wq->worker_done);
		kfree(worker);
		return false;
	}

	tsk->pf_io_worker = worker;
	worker->task = tsk;
	set_cpus_allowed_ptr(tsk, &mask);
	tsk->flags |= PF_NO_SETAFFINITY;

	raw_spin_lock_irq(&wqe->lock);
	hlist_nulls_add_head_rcu(&worker->nulls_node, &wqe->free_list);
	list_add_tail_rcu(&worker->all_list, &wqe->all_list);
	worker->flags |= IO_WORKER_F_FREE;
	if (index == IO_WQ_ACCT_BOUND)
		worker->flags |= IO_WORKER_F_BOUND;
	if (!acct->nr_workers && (worker->flags & IO_WORKER_F_BOUND))
		worker->flags |= IO_WORKER_F_FIXED;
	acct->nr_workers++;
	raw_spin_unlock_irq(&wqe->lock);
	wake_up_new_task(tsk);
	return true;
}

static int io_wq_fork_manager(struct io_wq *wq)
{
	struct task_struct *tsk;
	struct cpumask mask;

	if (wq->manager)
		return 0;

	WARN_ON_ONCE(test_bit(IO_WQ_BIT_EXIT, &wq->state));

	init_completion(&wq->worker_done);
	atomic_set(&wq->worker_refs, 1);
	tsk = create_io_thread(io_wq_manager, wq, NUMA_NO_NODE);
	get_cpumask_by_numa(&mask, -1);

	if (!IS_ERR(tsk)) {
		wq->manager = get_task_struct(tsk);
		set_cpus_allowed_ptr(tsk, &mask);
		wake_up_new_task(tsk);
		return 0;
	}

	if (atomic_dec_and_test(&wq->worker_refs))
		complete(&wq->worker_done);

	return PTR_ERR(tsk);
}

static bool io_wq_worker_affinity(struct io_worker *worker, void *data)
{
	struct task_struct *task = worker->task;
	struct rq_flags rf;
	struct rq *rq;
	struct cpumask mask;

	rq = task_rq_lock(task, &rf);
	get_cpumask_by_numa(&mask, worker->wqe->node);
	do_set_cpus_allowed(task, &mask);
	task->flags |= PF_NO_SETAFFINITY;
	task_rq_unlock(rq, task, &rf);
	return false;
}

The necessary bits have long ago been merged.

@axboe @isilence  Could you explain why you simply did not clear PF_NO_SETAFFINITY in the flags rather than 1) keep  PF_NO_SETAFFINITY 2) adding a specific register call to change the affinity? I am sure there is a good reason but I have not been able to figure it out. Is it for backwards compatibility? Is it because the pid of these threads is not immediately clear to userspace? Thanks in advance


on-off-button

Fixes Issue
Closes #1400

Changes proposed

Screenshots

Checklist

[ ] You've linked this PR to the correct issue.
[ ] You have checked that the code is working correctly.
[ ] You ⭐ the repository!

Note to reviewers

Please check #1346

@adity2911  please review this PR


Updated repl docs to include brew install instructions

I think preferable on macOS, and seems pretty well maintained.

This has fallen behind due to my slowness in looking at it but if you bring it up to date I'll merge it

Closing due to inactivity...


Redo VALIDHACK

goal is to improve openpilot IMAGE=2 perf.
GPU=1 DEBUG=4 FORWARD_ONLY=1 IMAGE=2 python3 test/test_ops.py TestOps.test_simple_padding_conv2d generates
  int lidx0 = get_local_id(0); /* 4 */
  float4 cast0 = (float4)(0.0f,0.0f,0.0f,0.0f);
  int alu0 = (lidx0+1);
  float4 val0 = ((((lidx0*(-1))<0)&(lidx0<3))?read_imagef(data1, smp, (int2)((alu0%2),((alu0/2)+(-1)))):cast0);

which can be simplified into float4 val0 = read_imagef(data1, smp, (int2)((lidx0-1),0)); because out of bound image returns 0, and thus reading (1, -1), (0, 0), (1, 0), (0, 1) from a 2x1 image is equivalent to reading (-1, 0), (0, 0), (1, 0), (2, 0)

most of the gain are coming from this
  int idx0 = get_global_id(0); /* 8 */
  int idx1 = get_global_id(1); /* 32 */
  int idx2 = get_global_id(2); /* 64 */
  float4 cast0 = (float4)(0.0f,0.0f,0.0f,0.0f);
  int alu0 = (idx1*24);
  int alu1 = ((idx1*32)+idx0);
  float4 val0 = *((__global float4*)(data3+idx0*4));
  float4 acc0 = cast0;
  float4 acc1 = cast0;
  float4 acc2 = cast0;
  float4 acc3 = cast0;
  for (int ridx0 = 0; ridx0 < 3; ridx0++) {
    for (int ridx1 = 0; ridx1 < 3; ridx1++) {
      int alu2 = ((idx2*2)+ridx1);
      int alu3 = (alu2+(-1));
      bool alu4 = (((idx2*(-2))+(ridx1*(-1)))<0);
      for (int ridx2 = 0; ridx2 < 3; ridx2++) {
        int alu5 = (ridx2*3);
        int alu6 = (alu0+alu5+ridx0);
        float4 val1 = (alu4?read_imagef(data1, smp, (int2)((alu6+3),alu3)):cast0);
        float4 val2 = (alu4?read_imagef(data1, smp, (int2)((alu6+9),alu3)):cast0);
        float4 val3 = (alu4?read_imagef(data1, smp, (int2)((alu6+15),alu3)):cast0);

the valid is -A < 0 and the idxy is (something, A-1). and it can be dropped because A-1 go out of bound if A <= 0

counter example in python test/external/external_test_valid_remove.py that valid cannot be dropped
  # int gidx0 = get_group_id(1); /* 64 */
  # int gidx1 = get_group_id(0); /* 2 */
  # int lidx2 = get_local_id(1); /* 16 */
  # int lidx3 = get_local_id(0); /* 8 */
  # float4 acc0 = (float4)(0.0f,0.0f,0.0f,0.0f);
  # float4 acc1 = (float4)(0.0f,0.0f,0.0f,0.0f);
  # float4 acc2 = (float4)(0.0f,0.0f,0.0f,0.0f);
  # float4 acc3 = (float4)(0.0f,0.0f,0.0f,0.0f);
  # for (int ridx0 = 0; ridx0 <= 2; ++ridx0) {
  #   for (int ridx1 = 0; ridx1 <= 2; ++ridx1) {
  #     for (int ridx2 = 0; ridx2 <= 2; ++ridx2) {
  #       bool alu0 = (((gidx0*(-2))+(ridx1*(-1)))<0);
  #       float4 val0 = (((((gidx1*(-128))+(lidx2*(-8))+(ridx2*(-1)))<0)*alu0))?(read_imagef(data1, smp, (int2)((((gidx1*384)+(lidx2*24)+(ridx2*3)+ridx0+765)%768),((gidx0*2)+ridx1+((gidx1+((lidx2+((ridx2+7)/8)+15)/16)+1)/2)+(-2))))):(float4)(0.0f,0.0f,0.0f,0.0f);

  # data1 has shape (128, 768, 4)

gidx0 = 2
gidx1 = 0
lidx2 = 0
lidx3 = 0
ridx0 = 0
ridx1 = 0
ridx2 = 0

alu0 = (((gidx0*(-2))+(ridx1*(-1)))<0)
print(f"{alu0=}")  # True
valid = (((((gidx1*(-128))+(lidx2*(-8))+(ridx2*(-1)))<0)*alu0))
print(f"{valid=}")  # 0
idxy = ((((gidx1*384)+(lidx2*24)+(ridx2*3)+ridx0+765)%768),((gidx0*2)+ridx1+((gidx1+((lidx2+((ridx2+7)//8)+15)//16)+1)//2)+(-2)))
print(f"{idxy=}")  # (765, 2)

so valid is 0, but idxy is not out of bound so valid cannot be dropped. not sure why in #1832 the test passed even if this is dropped (it's kernel 3 now and kernel 1 in compile.py in that pr)

oh the trick is to also rewrite the idx, so idx became (int2)((ridx0+(gidx1*384)+(lidx2*24)+(ridx2*3)+(-3)),((gidx0*2)+ridx1+((gidx1+((lidx2+((ridx2+7)/8)+15)/16)+1)/2)+(-2))))

NOTE: dropping valid only seems to make openpilot slower, consider only rewrite the idx and keep valid at the end of this to see if it's faster

Good to close?

sure, one more broad case with https://github.com/tinygrad/tinygrad/pull/6716/files but a lot slower / complex. leave as reference and close this one


ECU not respond after automation reboot

Hi, after using the newest version and automation (thanks) 96 after a few days the integration hungs.
what i see:

switch ecu is on
binary cached is on
sensors remain last state
no fault in log

what i tried:

use shell command, no result
use power cycle ecu, no result
Use restart Home Assistant and it works again

Are you sure you left it as is for a longer period than the interval? Waiting long enough to recover.

Check the picture, it is overnight to the next morning 😙

If it occurs next time, see what happens when you flip the ECU Query Device off/on and wait a while. Are you using the suggested automation from the documentation?

I’ll try that. I use the automation from the doc. Any new about the rest command?

Haven't spend time on that REST option yet. I am very interested in the part where the automatic reboot takes place. Maybe I'm overlooking something. I've modified the subject title. If someone else doesn't have a problem with the automation, that's also valuable information to help find the problem.

I could make things easier by integrating the reboot in the integration, the automation is not needed then. I can work on that this weekend. To test if this would be possible, I've updated https://github.com/HAEdwin/homeassistant-apsystems_ecur/blob/main/custom_components/apsystems_ecur/switch.py
If you would like to try and replace switch.py then turn the query switch off and back on, the ECU should reboot (ECU-R pro and ECU-C only). If this works I can then add it on the right place in the integration. Replace localhost on line 84 by your ECU's IP-Address before restart of HA.

same problem here, integration keeps hanging

My plan is to reset the ECU after 5 tries without flipping the switch because flipping the switch it is a unnecessary action. But I really need feedback on my previous message.

I could make things easier by integrating the reboot in the integration, the automation is not needed then. I can work on that this weekend. To test if this would be possible, I've updated https://github.com/HAEdwin/homeassistant-apsystems_ecur/blob/main/custom_components/apsystems_ecur/switch.py If anyone would like to try and replace switch.py then turn the query switch off and back on, the ECU should reboot (ECU-R pro and ECU-C only). If this works I can then add it on the right place in the integration. Replace localhost on line 84 by your ECU's IP-Address before restart of HA. Please check response in home-assistant.log also.
The URL need some cleaning and reformatting though - can't test it myself but spaces are not allowed for example, so someone needs to find the correct formatted URL to get the ECU to reboot. This might work but I can't test: Line 84: get_url= urllib.request.urlopen("http://localhost/index.php/management/set_wlan_ap%20-H%20X-Requested-With:%20XMLHttpRequest%20--data-raw%20SSID=ECU-WIFI_local&channel=0&method=0&psk_wep=&psk_wpa=%20--compressed%20--insecure")

Switch won’t go on, got this error on screen
Failed to call service switch/turn_on. HTTP Error 400: Bad Request
nothing in debug log

Thanks for testing, the switch works fine and triggers the URL to restart the ECU. Exception is the way the URL is formatted and if it needs to be a get or put method I'm not sure. HTTP Error 400 means that the ECU cannot process the request so the URL has to be formatted or called correctly. I take it that you've replaced localhost for the IP-Address from your ECU. The response should be: "Response Status: 200" which means the ECU was able to process the request. I might have made a mistake, hope this works and returns 200:
Replace line 2 for:
import requests
At line 84:
get_url= requests.get("http:// < IP > /index.php/management/set_wlan_ap%20-H%20X-Requested-With:%20XMLHttpRequest%20--data-raw%20SSID=ECU-WIFI_local&channel=0&method=0&psk_wep=&psk_wpa=%20--compressed%20--insecure")
At line 85:
_LOGGER.warning("Response Status: "+ str(get_url.status_code))

WARNING (SyncWorker_4) [custom_components.apsystems_ecur.switch] Response Status: 400
Sorry, still 400, not 200. ECU don't reset after changing the code in you message (incl ip).

Í tried to reproduce the issue by powering down the ECU. This is my log:
2022-10-08 13:13:39.139 WARNING (SyncWorker_3) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable
2022-10-08 13:18:42.099 WARNING (SyncWorker_4) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable
2022-10-08 13:23:45.143 WARNING (SyncWorker_3) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable
2022-10-08 13:28:48.098 WARNING (SyncWorker_2) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable
2022-10-08 13:28:48.102 WARNING (SyncWorker_2) [custom_components.apsystems_ecur] Communication with the ECU fails after 5 repeated attempts. Try power cycling or reset the ECU. Querying is stopped automatically.
2022-10-08 13:29:48.456 WARNING (SyncWorker_1) [custom_components.apsystems_ecur.switch] Response Status: 200

So the fifth try the ECU query switch is turned off and the automation calls the reboot (in my case just http://google.com) which is successfull returning 200. I then power up the ECU within the minute then after that minute the automation turns the ECU Query switch back on. After five minutes the cache is turned off after a succesfull ECU query. All goes well here and I'm not able to reproduce the issue.
I checked the states of entities but they're all good (complies to the results above). I'm not sure what is going on but it would take some trouble shooting on your part to see where it goes wrong. Did the ECU indeed restart? What is the state of the ECU entities when looking using the developer tools?

The ECU didn't restart (checked with terminal and ping the ip). I see no other state change when switching the ecu switch

So the first thing to focus on is getting the right URL to get the ECU to restart and return 200 instead of 400:
How about leaving the double quotes out?
shell_command:
  reboot_ecu: '/usr/bin/curl "http:// < IP > /index.php/management/set_wlan_ap -H X-Requested-With: XMLHttpRequest --data-raw SSID=ECU-WIFI_local&channel=0&method=0&psk_wep=&psk_wpa="

I think I've found a solution, I've done 2 commits (only getting ip to the url string has to be done).  After some testing of the command I came to the conclusion that it needs a custom header and at least the SSID value.
url = 'http://<ip>/index.php/management/set_wlan_ap'
headers = {'X-Requested-With': 'XMLHttpRequest',}
data = {'SSID': 'ECU-WIFI_local'}
get_url = requests.post(url, headers=headers, data=data)

Thank you for trouble shooting, I've added the code to the integration with additional error handling. I hope this works out fine. Just Released 1.2.20 beta for download. Do not forget to disable the old automation. I'm happy to receive any feedback on results.

Let’s try it the next couple of days

If the restart works I'll add a binairy sensor to indicate when the ECU restarted, log entries will be removed. With this sensor you'll be able to raise a notify (to your phone for example) to monitor if all is good.
When any results on the restart are there please post the home-assistant.log part so I can determen what to do with the response returned.

today I had the following error, integration stops working. Restarting the ECU won't help, even a power cycle. Only restarting Home Assistant helps.
Logger: custom_components.apsystems_ecur
Source: custom_components/apsystems_ecur/__init__.py:117
Integration: APSystems PV solar ECU (documentation, issues)
First occurred: 4:23:43 PM (1 occurrences)
Last logged: 4:23:43 PM
Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment

Must have been a glitch receiving data, I'll have to add some code to catch these exceptions in APSystemsSocket.py to enable looking at the received data.

So the first thing to focus on is getting the URL right to get the ECU to restart and return 200 instead of 400: How about leaving the double quotes out?
shell_command:
  reboot_ecu: '/usr/bin/curl "http:// < IP > /index.php/management/set_wlan_ap -H X-Requested-With: XMLHttpRequest --data-raw SSID=ECU-WIFI_local&channel=0&method=0&psk_wep=&psk_wpa=" '

Or simplifying the URL so that the ECU restarts anyway, for example leaving out: X-Requested-With: XMLHttpRequest --data-raw

i'm having a similar problem on the last few days:
"[custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable"
and tried the command:
'/usr/bin/curl "http://<IP_ADDRESS>/index.php/management/set_wlan_ap -H X-Requested-With: XMLHttpRequest --data-raw SSID=ECU-WIFI_local&channel=0&method=0&psk_wep=&psk_wpa=" '
my ecu did not reboot, and the wifi network name for local access (side button) the ecu changed to "ECU-WIFI_local" instead of
old id.
ecu is new: ECU-R-BR

I included the restart procedure in release 1.2.21
If the hotspot changed to ECU-WIFI_local and is temporary available, you could assume the ECU did restart. You will have to wait at least 5 minutes (default interval=300 seconds) before data reappears in HA.
What are the first four digits of your ECU-R-BR ID?

Yesterday again no proper restart of the ECU, had to restart ECU and Home Assistant and reinstall the integration to get it working.
 Exception error: 'NoneType' object does not support item assignment
2022-10-16 15:56:31.027 DEBUG (SyncWorker_9) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [30, 58, 30, 58], 'voltage': [225], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 82, 'frequency': 50.0, 'temperature': 25, 'model': 'QS1', 'channel_qty': 4, 'power': [30, 58, 30, 58], 'voltage': [225]}}, 'ecu_id': '216200070324', 'today_energy': 3.0, 'lifetime_energy': 135.2, 'current_power': 176, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': True, 'querying': True}
2022-10-16 15:56:31.028 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.027 seconds (success: True)
2022-10-16 16:01:31.006 DEBUG (SyncWorker_2) [custom_components.apsystems_ecur] Querying ECU

It looks like the integration receives invalid data and uses the cache (that can happen once in a while). I can't see what is happening the next run 5 minute later at 16:01:31 Querying ECU. Does the log end there and no further errors are logged? I would expect that either it got data (Got data from ECU) or again had to use the cache (Using cached data from last successful...) if the log does not show either of them I might have to direct you to add some extra debug lines so that I narrow the search where it goes wrong.
The restart procedure is initiated after 5 times using the cache but in your case it doesn't seem to get that far.
Reinstall of the integration does not sound logical unless the storage medium is corrupt. The software won't change before and after reinstall only the initial query is performed immediately. In that case you can initiate an immediate query by reconfiguring the integration. Results should be available directly after that.

Reinstall doesn't make sense to me either. The log stops after the last message. Is there another way to debug?

Of course I tested all scenarios as best as possible and it works like a charm here but I might have found something that was introduced at some point in time. Best thing I can do is add some extra debug lines correct the possible cause and release a beta after doing some testing again... thanks for your patience.

I've added some debugging lines and did one change. If you first install 1.2.21 and then overwrite it with my file, you will see more info. Hopefully thinks don't go wrong now and the reset procedure is going to work.
https://github.com/HAEdwin/homeassistant-apsystems_ecur/blob/main/custom_components/apsystems_ecur/APSystemsSocket.py

thanks, I'll try it and let you know.

I included the restart procedure in release 1.2.21 If the hotspot changed to ECU-WIFI_local and is temporary available, you could assume the ECU did restart. You will have to wait at least 5 minutes (default interval=300 seconds) before data reappears in HA. What are the first four digits of your ECU-R-BR ID?

ECU-R-BR id is 2162000xxxxx
i tested again while pinging the ip on another window, confirmed that it stopped replying for a few seconds.
HA log still repeating "Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable" on v1.2.21
the "host is unreachable" message makes no sense to me (i'm not a python guy), the curl and ping works properly from "docker exec -it homeassistant /bin/bash", i reconfigured the integration with the ip found on "nmap -p 8899 <IP_ADDRESS>/24 --open" (still the same ip, still accepted.), and powercycling doesn't change anything.

Thanks for the info, ICMP ping packets are connectionless and therefor different from TCP packets. You could well have a ping response but cannot make a connection. The temporary no response might indicate a restart though, the home-assistant.log should reflect the succes of the post method to the ECU. When the integration was still using the cache did you try to using Packet Sender to confirm it is not possible to connect?

Thanks for the info, ICMP ping packets are connectionless and therefor different from TCP packets. You could well have a ping response but cannot make a connection. The temporary no response might indicate a restart though, the home-assistant.log should reflect the succes of the post method to the ECU. When the integration was still using the cache did you try to using Packet Sender to confirm it is not possible to connect?

i don't have the 'packet sender' here, but to make sure i'm testing the same network as docker is runing:
on: 'docker exec -it homeassistant /bin/bash'
did: "echo APS1100160001END | nc <IP_ADDRESS> 8899"
got: "APS120121000121620005633901C��
� "310016ECU_R_PRO_2.0.7B020America/Campo_Grande��Ũz�A00END
"
on "tail -f homeassistant.log" still: "2022-10-17 16:21:41.232 WARNING (SyncWorker_3) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable"
(no other response on the last few days, tried other integration versions too)

Thanks for the info, ICMP ping packets are connectionless and therefor different from TCP packets. You could well have a ping response but cannot make a connection. The temporary no response might indicate a restart though, the home-assistant.log should reflect the succes of the post method to the ECU. When the integration was still using the cache did you try to using Packet Sender to confirm it is not possible to connect?

i don't have the 'packet sender' here, but to make sure i'm testing the same network as docker is runing: on: 'docker exec -it homeassistant /bin/bash' did: "echo APS1100160001END | nc <IP_ADDRESS> 8899" got: "APS120121000121620005633901C�� � "310016ECU_R_PRO_2.0.7B020America/Campo_Grande��Ũz�A00END " on "tail -f homeassistant.log" still: "2022-10-17 16:21:41.232 WARNING (SyncWorker_3) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Invalid data error: [Errno 113] Host is unreachable" (no other response on the last few days, tried other integration versions too)

also:

the right before the period in green, i possibly had HA and/or ECU offline (pluged off), but i remember that the moment i turned all on i had data on HA home

Why would HA not be able to reach the ECU ([Errno 113] Host is unreachable), all seems to go well including the ECU restart. Is there a firewall blocking HA?

Why would HA not be able to reach the ECU ([Errno 113] Host is unreachable), all seems to go well including the ECU restart. Is there a firewall blocking HA? What happens if you reconfigure the integration by going to [Settings] > [Devices & Services] and choose [Configure] in the integration box?

on HA page shows the popup "success"
on tail -f homeassistant.log shows the "host unreachable" message (might be coincidence, it appears every few seconds anyways.
there's no firewall, and docker configs are on defaults (the "echo|nc" test confirms), there's someway of increase the details on what is the "host" and related variables its failing to reach on the log? (i'm not python savvy but can paste code if needed).

Why would HA not be able to reach the ECU ([Errno 113] Host is unreachable), all seems to go well including the ECU restart. Is there a firewall blocking HA? What happens if you reconfigure the integration by going to [Settings] > [Devices & Services] and choose [Configure] in the integration box?

on HA page shows the popup "success" on tail -f homeassistant.log shows the "host unreachable" message (might be coincidence, it appears every few seconds anyways.
there's no firewall, and docker configs are on defaults (the "echo|nc" test confirms), there's some way to increase the details on the log to see what is the "host" and related variables ? (i'm not python savvy but can paste code if needed).

something is messing up some config somewhere:
i rebooted and got this on .log:
"2022-10-17 21:29:17.137 DEBUG (MainThread) [custom_components.apsystems_ecur] config={'host': '<IP_ADDRESS>', 'scan_interval': 300}
"
the ip is wrong, changed again on HA:

still the same wrong ip on .log
tried including ip's that don't exists on HA, got errors:

for some reason he is trying to get data from an ip that is not possible to include on config, while ignoring the ip i wrote on config.

@doug-last This seems to be a new issue somewhere in the config flow. I'm also not that skilled at Python but glad if I can help - for me this integration works like a charm since long-time ago :) Let's continue this issue here #125

Today i got the same problem, see log below. Integration is stopping at query step 2
2022-10-22 07:56:54.006 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur] Querying ECU
2022-10-22 07:56:54.007 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query ECU step 1: open socket
2022-10-22 07:56:54.012 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query ECU step 2: send & receive data
2022-10-22 07:56:59.017 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query ECU step 3: close socket
2022-10-22 07:56:59.019 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query ECU step 4: process data
2022-10-22 07:56:59.019 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query Inverters step 1: open socket
2022-10-22 07:56:59.021 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query Inverters step 2: send & receive data
2022-10-22 07:57:04.027 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query Inverters step 3: close socket
2022-10-22 07:57:04.028 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query Inverters Signal step 1: open socket
2022-10-22 07:57:04.030 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query Inverters Signal step 2: send & receive data
2022-10-22 07:57:09.035 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query Inverters Signal step 3: open socket
2022-10-22 07:57:09.036 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Process inverter data
2022-10-22 07:57:09.036 WARNING (SyncWorker_27) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-22 07:57:09.037 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [0, 0, 0, 0], 'voltage': [0], 'inverters': {'[806000038042](tel:806000038042)': {'uid': '[806000038042](tel:806000038042)', 'online': False, 'signal': 81, 'frequency': 0.0, 'model': 'QS1', 'channel_qty': 4, 'power': [0, 0, 0, 0], 'voltage': [0]}}, 'ecu_id': '[216200070324](tel:216200070324)', 'today_energy': 0.0, 'lifetime_energy': 144.9, 'current_power': 0, 'qty_of_inverters': 1, 'qty_of_online_inverters': 0, 'data_from_cache': True, 'querying': True, 'restart_ecu': False}
2022-10-22 07:57:09.038 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.033 seconds (success: True)
2022-10-22 08:02:09.005 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur] Querying ECU
2022-10-22 08:02:09.005 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query ECU step 1: open socket
2022-10-22 08:02:09.012 DEBUG (SyncWorker_27) [custom_components.apsystems_ecur.APSystemsSocket] Query ECU step 2: send & receive data

Hi,
I have ECU-R with SunSpec logo ECU-ID 21620002xxxx
I had the same problem today, at evening I saw that integration (v1.2.22) had hanged at 13:31:
2022-10-25 13:31:06.022 WARNING (SyncWorker_4) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
I didn't have any Debug messages - should I enable it somwhere?
Below what I had in HA at 20:25, before any activities.

At 20:25 I restarted ECU with curl query by manually activation of disabled automation (from previous version); ping was lost for some time.

After 1 hour nothing changed.
Reboot of HA at 21:20 - integration started working as normal.
Waiting for next hangs. :)
P.S. whole integration is a great piece of work, and need only cherry on the top - working workaround for new ECU-R. I keep finger crossed.

Most obvious case is that the integration gets stuck in a loop in APSystemsSocket.py

I would expect a received datastring would always contain END\n but it might sometimes not be the case because data is not being send when the ECU got stuck.
You have to change lines 76,77,78 to this.

If this works I'll release a new version asap.

Hello,
After the change you suggested, it looks like the reboot of the ECU has been successfully triggered.
Great!

2022-10-26 23:39:42.723 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration apsystems_ecur which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2022-10-29 14:19:17.021 WARNING (SyncWorker_9) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-29 14:24:32.021 WARNING (SyncWorker_2) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-29 14:29:47.022 WARNING (SyncWorker_9) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-29 14:35:02.021 WARNING (SyncWorker_3) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-29 14:40:17.019 WARNING (SyncWorker_9) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-29 14:40:17.021 WARNING (SyncWorker_9) [custom_components.apsystems_ecur] Communication with the ECU failed after 5 repeated attempts.

@kuzabi Thank you for the detailed report on the changes. Glad it is solved...

i can confirm that restart if working fine now
2022-10-30 09:07:47.003 DEBUG (SyncWorker_11) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:08:02.022 DEBUG (SyncWorker_11) [custom_components.apsystems_ecur] Got data from ECU
2022-10-30 09:08:02.022 DEBUG (SyncWorker_11) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 72, 'frequency': 50.0, 'temperature': 15, 'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.06, 'lifetime_energy': 156.7, 'current_power': 83, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': False, 'querying': True, 'restart_ecu': False}
2022-10-30 09:08:02.023 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.022 seconds (success: True)
2022-10-30 09:13:02.006 DEBUG (SyncWorker_16) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:13:17.036 WARNING (SyncWorker_16) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-30 09:13:17.059 DEBUG (SyncWorker_16) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 72, 'frequency': 50.0, 'temperature': 15, 'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.06, 'lifetime_energy': 156.7, 'current_power': 83, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': True, 'querying': True, 'restart_ecu': False}
2022-10-30 09:13:17.061 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.060 seconds (success: True)
2022-10-30 09:18:17.005 DEBUG (SyncWorker_6) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:18:32.024 WARNING (SyncWorker_6) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-30 09:18:32.026 DEBUG (SyncWorker_6) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 72, 'frequency': 50.0, 'temperature': 15, 'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.06, 'lifetime_energy': 156.7, 'current_power': 83, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': True, 'querying': True, 'restart_ecu': False}
2022-10-30 09:18:32.027 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.026 seconds (success: True)
2022-10-30 09:23:32.006 DEBUG (SyncWorker_1) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:23:47.026 WARNING (SyncWorker_1) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-30 09:23:47.028 DEBUG (SyncWorker_1) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 72, 'frequency': 50.0, 'temperature': 15, 'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.06, 'lifetime_energy': 156.7, 'current_power': 83, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': True, 'querying': True, 'restart_ecu': False}
2022-10-30 09:23:47.029 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.028 seconds (success: True)
2022-10-30 09:28:47.002 DEBUG (SyncWorker_6) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:29:02.037 WARNING (SyncWorker_6) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-30 09:29:02.038 DEBUG (SyncWorker_6) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 72, 'frequency': 50.0, 'temperature': 15, 'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.06, 'lifetime_energy': 156.7, 'current_power': 83, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': True, 'querying': True, 'restart_ecu': False}
2022-10-30 09:29:02.040 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 15.039 seconds (success: True)
2022-10-30 09:34:02.004 DEBUG (SyncWorker_6) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:34:17.036 WARNING (SyncWorker_6) [custom_components.apsystems_ecur] Using cached data from last successful communication from ECU. Exception error: 'NoneType' object does not support item assignment
2022-10-30 09:34:17.037 WARNING (SyncWorker_6) [custom_components.apsystems_ecur] Communication with the ECU failed after 5 repeated attempts.
2022-10-30 09:34:18.431 DEBUG (SyncWorker_6) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 72, 'frequency': 50.0, 'temperature': 15, 'model': 'QS1', 'channel_qty': 4, 'power': [22, 19, 21, 21], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.06, 'lifetime_energy': 156.7, 'current_power': 83, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': True, 'querying': True, 'restart_ecu': True}
2022-10-30 09:34:18.432 DEBUG (MainThread) [custom_components.apsystems_ecur] Finished fetching apsystems_ecur data in 16.429 seconds (success: True)
2022-10-30 09:39:18.005 DEBUG (SyncWorker_5) [custom_components.apsystems_ecur] Querying ECU
2022-10-30 09:39:33.030 DEBUG (SyncWorker_5) [custom_components.apsystems_ecur] Got data from ECU
2022-10-30 09:39:33.030 DEBUG (SyncWorker_5) [custom_components.apsystems_ecur] Returning {'model': 'QS1', 'channel_qty': 4, 'power': [77, 62, 82, 66], 'voltage': [227], 'inverters': {'806000038042': {'uid': '806000038042', 'online': True, 'signal': 74, 'frequency': 50.0, 'temperature': 18, 'model': 'QS1', 'channel_qty': 4, 'power': [77, 62, 82, 66], 'voltage': [227]}}, 'ecu_id': '216200070324', 'today_energy': 0.16, 'lifetime_energy': 156.8, 'current_power': 287, 'qty_of_inverters': 1, 'qty_of_online_inverters': 1, 'data_from_cache': False, 'querying': True, 'restart_ecu': False}

Thanx for the feedback, this is now solved in v1.2.23 where an infinite loop caused the integration to get non-responsive and the ECU reset was never called due to that. Disadvantage: the check if all data was received is not there anymore, I would expect no larger data lengths than 1024 being returned but with large solararrays you never know...

Hi Edwin, it's working great, the reset of the AP. But I was thinking, it reset the AP with a default name and no security. After this the AP is available for 1 hour. Maybe better to set a WPA password. Is it possible to make a config in the integration to set an own SSID and WPA password?
data for requests.post would be:
data= {'ssid': 'ECU-WIFI_local', 'channel'; 0, 'method': 2, 'psk_wep': 'homeassistent', 'psk_wpa': 'homeassistent'})
ssid = ssid
channel = 0-13 (0=auto)
method = 0,1,2 (no/wep/wpa)
psk_wep = password wep
psk_wpa = password wpa


Update PyPIRepository::resolve_reqs() for pip>=22.1.1

pip 22.1.1 splits RequirementSet::add_requirement() to RequirementSet::add_named_requirement() and RequirementSet::add_unnamed_requirement(). Here, we update PyPIRepository::resolve_reqs() to call the right method.
Contributor checklist

[ ] Provided the tests for the changes.
[X] Assure PR title is short, clear, and good to be included in the user-oriented changelog

Maintainer checklist

[x] Assure one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
[x] Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

Fixes #1623

I just saw the discussion in #1623. If I get the gist of it right, since pip internals are used by pip-tools, we have to either add conditionals like the one here, or we need to vendor pip. So until a decision is made on vendoring, this should be an ok solution.

Only Ubuntu 3.11-dev is failing and it doesn't seem related

I'll merge and prepare the release shortly.

@atugushev That was my plan. Merge whatever is ready and safe to merge and tag a new release to fix the serious bug with newer pip.

Thanks for the quick release!


Respond with JavaScript function result

Hello,
I am sadly stuck again. I have to give access to quite a lot of data to the code in Python. So it is made available via special methods. It seems to work fine in Python. If I execute the code below, it will print in Python correctly { b: 123 }.
When I however directly return it, then I get an error. For some reason I get another error when I follow what the first one recommends
Thanks a lot for your help!
Here is the code to easily reproduce the issue I am facing:
const { python, builtins } = require('pythonia');

async function main() {
	const testData = {
		a: () => {
			return {
				b: 123,
			};
		},
	};

	let response = undefined;

	const callback = (data) => {
		response = data;
	};

	await builtins.exec(
		`def main():
	print(testData['a']());
	return testData['a']()
callback(main())
`,
		{
			callback,
			testData,
		},
	);

	console.log('test1');
	console.log(response);
	// Expected:
	//    { b: 123 }
	// Actual response:
	//    (Some Python object) Use `await object.toString()` to get this object's repr().

	console.log('test2');
	console.log(await response.toString());
	// ERROR:
	// Python Error  Call to '' failed:
}
main().then(() => python.exit());

JSPyBridge is largely intended for importing and calling libraries between the two languages. Single files can also be imported with the require or python.import functions. What you're doing with exec here is about the same as writing the Python script to a file and then python.importing it. I don't advise using it as an arbitrary code execution environment as it's not much different than using eval() from in Node.js, except with the added overhead of bridging to Python.
The issue here with your code seems to be related to garbage collection handling. Converting the JS proxy object to a Python native one should fix,
await builtins.exec('cb(testData["a"]().valueOf())', { cb, testData })

And in the JS side callback,
var cb = async data => { let a = await data.valueOf(); console.log('cb', a) }

Amazing, thanks a lot for your help. Works perfectly.


CSS: support for aside.sidebar

... which is produced by docutils 0.17+
This should be merged once #40 has been reverted.

See https://github.com/sphinx-doc/sphinx/pull/9125, which will be in Sphinx 4.0.


Make querying more robust

Does this PR close any open issues?
Closes #335
Closes #282
Give a longer description of what this PR addresses and why it's needed
This PR will make it so that a user can conduct advanced querying of the connectivity network.
At the moment we can only query for one attribute per node or edge, and we'd like to query for more than one attribute.
Provide pictures/videos of the behavior before and after these changes (optional)
Are there any additional TODOs before this PR is ready to go?
TODOs:

[x] Create UI for expanding + collapsing querying
[x] Update AQL queries to work with multi-attribute querying

I've updated the TODOs above, but we should chat about how the NOT logic should work, and if we should allow for paths that go through the start node again (e.g. the path goes out and back and then to a target)

The test query for the marclab crashes the browser for the RC1 dataset, but works for RPC1 (which is smaller):
(CBb*) -[BC Conventional Synapse] AND NOT [Ribbon Synapse]-> (*)

@JackWilb the UI is 🗑🔥 right now and the new data structure might be too. But it's a start, let's chat sometime this week to see what improvements can be made

For context, Jack and I met to discuss how there are two different types of edge queries that happen and how the first iteration of the UI could support both independently but not together.
The two types of edge queries are:

Querying along 1 edge but using that edge's attributes as filters
Querying along multiple edges between nodes

At the moment, given the use case with the marclab we are only allowing the user to search along the first hop and only two different edges. We figured this is a good compromise given that we don't have other user cases and it will increase the complexity of the AQL query.
Here is our sketch for the querying:
=
@JackWilb please add to this if you think I am missing anything


Ubuntu 18.04 syncplay crashes on open, snap install

Kernel version fwiw 4.15.0-96-generic.
I installed 1.6.4 via snap and got this
$sudo snap install syncplay --classic
syncplay 1.6.4 from Syncplay installed
$ syncplay
Segmentation fault (core dumped)

The AppImage 1.6.4a-x86_64 works though

Is https://github.com/Syncplay/syncplay/issues/260#event-3232513393 of any use?

I have the same issue, also on Ubuntu 18.04, but with kernel 4.15.0-99-generic.
 I can reproduce the long output by removing the $HOME/snapd/syncplay directory.

@MPvHarmelen I don't use Linux so I can only be of limited help, but for now it seems like one solution is to not use the Snap for now. If you can fix the Snap then that would be great.

I updated to Ubuntu 20.04 and the problem seems solved. I still get the gtk warnings on first run, but no segfault:
$ snap install syncplay --classic
syncplay 1.6.4 from Syncplay installed
$ syncplay
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/Adwaita
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/DMZ-Black
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/DMZ-White
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/hicolor
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/Humanity
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/Humanity-Dark
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/LoginIcons
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/ubuntu-mono-dark
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/ubuntu-mono-light
/snap/syncplay/3/usr/sbin/update-icon-caches: 16: gtk-update-icon-cache-3.0: not found
WARNING: icon cache generation failed for /home/<user>/snap/syncplay/3/.local/share/icons/default
QGtkStyle could not resolve GTK. Make sure you have installed the proper libraries.
Qt: Session management error: None of the authentication protocols specified are supported


MacOS JDK/JRE 8 build with SDK older than 10.9

Platform:
MacOS
Architecture:
x64
The JDK/JRE is build with an SDK older than 10.9 which prohibits it from being notarised by Apple.
We have a product which uses the current JRE8 released by adopt. When notarising the App we receive the error from Apple for every binary (executable or dylib): "The binary uses an SDK older than the 10.9 SDK."

Duplicate of https://github.com/AdoptOpenJDK/openjdk-build/issues/1130. Read  https://github.com/AdoptOpenJDK/openjdk-build/issues/1130#issuecomment-611728413 for details. Should arrive soon.


Add initial support for the Trace Overview view

Users of the trace extension would like to have an overview of the trace that they are analyzing so they can select a range
that’s interesting, then filter, zoom, and get more details. It also helps users maintain the context.
This commit adds initial support for the Trace Overview view as part of the Histogram view. It puts the common code between the trace-overview-component and xy-output-component in an abstract class. Then, the trace-overview-component extends this class to define custom behaviors. In the future, the xy-output-component should also extend this abstract class. The behavior of the trace-overview-component should mimic the trace overview of the Histogram view in the Trace Compass desktop application.
This PR is for Issue #664.

Issue created per @MatthewKhouzam comment: https://github.com/eclipse-cdt-cloud/theia-trace-extension/issues/781

I think that all histogram series should be selected by default. Especially since there is currently no way for the user to select all elements in the left tree, as far as I know. [1]

Yes, Bernd mentioned this too.

Is it intentionally not resizable? I would have expected a thinner view, but the height cannot be changed. [2]

It is made intentionally not resizable. But now to think of it, maybe it is better to have the height resizable since each monitor may have different resolution.

Finally, maybe just me, but I find that for an overview, the left tree and y-axis are distracting, I wonder if they could be hidden, or togglable. Maybe a future improvement. [3]

Yes, Bernd said that the left tree will need to be hidden in the future.
I personally think point [1] and [3] can be a task on its own, mean while [2] can be quickly changed in this PR. What do you think?

Created issue on making the filter tree togglable/hidden: https://github.com/eclipse-cdt-cloud/theia-trace-extension/issues/787


Blazemeter Taurus Package

Added Blazemeter Taurus package.
Microsoft Reviewers: Open in CodeFlow

@KevinLaMS Submitted the file to Microsoft :) Thanks!

Not a malware confirmation.


Commanders Act stream tracking

As SRG SSR GD analyst I want correct playback measurement for apps using Pillarbox.
Acceptance criteria

The Commanders Act SDK is integrated according to the specifications.

Tasks

[ ] Write tests for:

[x] Play at start.
[x] Pause at start.
[x] Pause during playback.
[x] Playback end.
[x] Destroy player during playback.
[x] Destroy player after having played the entire content.
[x] Seek while playing / paused.
[x] Failure at playback start.
[ ] Failure during playback.
[ ] DVR offset for on-demand / live / DVR streams.
[x] Buffer start / buffer stop (we need to stop advancing time when the player is buffering).
[ ] Metadata (and no tracking if no metadata).
[x] Enable / disable tracking on the fly.
[ ] Metadata update.
[ ] Bandwidth.
[ ] Volume.
[ ] Subtitles.
[ ] Audio tracks.
[ ] Rate.
[ ] Heartbeats (on-demand / live / DVR).

[x] Integrate Commanders Act SDK version 5.
[x] Create test helpers to test sent data if needed.
[ ] Implement the corresponding tracker.
[ ] Reuse common code with comScore tracker if meaningful and possible.
[x] Provide media player name and version in streaming events.
[ ] Write an issue to validate the result with the GD.

The failure during playback scenario is difficult to test automatically. Here is how it can be manually verified:

Enable TCDebug event logging.
Build and run the demo application.
Play some video.
Disable the network and seek to an unbuffered location.
Wait ~2 minutes until playback fails and check the Commanders Act event log.

Since we extrapolate some DVR values to lack for player information when disabling the tracker (e.g. if the player is closed during playback) some use case is quite difficult to test automatically as well. Here is how it can be manually verified:

Enable TCDebug event logging.
Build and run the demo application.
Play some DVR stream at the live edge.
Disable the network and let the player enter buffering.
Wait ~30 seconds and close the player. A stop event with media_timeshift having this duration should be observed in the console.

A similar use case for on-demand video:

Enable TCDebug event logging.
Build and run the demo application.
Play some on-demand stream.
Disable the network and wait until the player starts buffering. Check the playback position.
Wait ~30 seconds and close the player. A stop event with media_position having the playback position at which the player is stuck should be observed in the console.

For simplicity we decided not to pause or cancel heartbeats during buffering phases. This is an edge case for which continuing to emit hearbeats is simple and compatible with the current specifications we received.

Unlike SRG Analytics we use indicatedBitrate instead of observedBitRate to better reflect the variant being played, not the average device bitrate.


Cleanup and consolidate .gitignores

This PR consolidates .gitignore rules and removes all the *.iml IDEA projects from the repo, so that people can load the projects and change them without risking unwanted changes to the IDEA project files (which are now removed and ignored).
All the common patterns have been moved to the root .gitignore, and others have been moved to parent folders as needed.
Please double check that the changes to the .gitignores are not affecting any of your internal stuff as it seems there's some traces of non-public data in there, and I cannot check that.

Thanks for the PR!
Couldn't we just consolidate the .gitignores, and leave the .imls intact?   We're concerned that the new user startup will be more complicated than it already is if we don't have the .iml files in the workspace when they start.
We could keep the .iml files in the .gitignore, so that if we update them we'll need to force add them, but at least that way they won't clutter up all our other checkouts when modified.

Also, it would be good to also incorporate all that makes sense from here:
https://github.com/github/gitignore/blob/master/Global/JetBrains.gitignore
Of course, we would also add the .iml files to the .gitignore (which Jetbrains doesn't recommend), but we don't commit them often enough that we can't git add -f them for that case: they are annoying when they change on their own all the time.

I definitely support removing the IML files, but that's because I don't use them. :-)
cc @devoncarew

They are useful for setting up this repo for development w/ IntelliJ. We've definitely seen some unfortunate churn in the iml files however :(
Generally (if the team is using IntelliJ), I think you want to check in the project structure files (*.iml, and some files from .idea/ like the modules.xml) but not check in the files that are per-user specific (settings or window locations) like workspace.xml.
I'm open to removing them if people don't think they're a net value. It will mean some adjustment for people that are currently using the project w/ IntelliJ (when their metadata files disappear out from under them).

Can we or IntelliJ autogenerate these files in some useful manner?

IntelliJ wouldn't be able to w/o knowledge of how the repo is laid out. A user could get it to, by opening / importing each separate module, but that would be a good thing to avoid.
We could generate the files easily, w/ a few hours invested in a script. I don't have plans or time really do to so, but it would be straightforward. The simplest solution is to check in the files defining the project structure - the iml files and such - which is what we have now.

The iml files are generally generated by IntelliJ when importing a module — I did it just for testing, and it works, although it is some additional effort, it's a very low effort in my opinion. I understand though that this may cause some issues. I removed them out of habit, since as an Android dev I usually have the Gradle build.gradle files that are enough for IDEA to re-generate its project and module settings. In fact, I normally ignore them and remove them from the repo.
I see that this may not be the best idea in this case, though, since there's no parallel mechanism to inform the project structure. It's a semi-manual task (mostly consisting in a couple clicks in IDEA) to have it load the Dart SDK, and it is adding friction for new contributors.
As a reference, this is what I generally have in my Android projects' .gitignores: https://github.com/squanchy-dev/squanchy-android/blob/26ebffb3e2f123d309f831c5237cc40814731401/.gitignore#L11-L46 — noteworthy difference from github's own IDEA .gitignore is that it includes the .idea folders, excluding some high-churn and low utility files; it is tailored to Android projects, though, so there may be something else we might want to keep in VCS, even though I wouldn't really know what.
I'm happy to revert f1db6db but at that point I'm thinking it may not be worth continuing with this PR at all; if someone changes the project structure and forgets to force add the iml files, we might end up in a tough corner where the project "works on my machine" but won't for others. Your call, team :)

FYI, I'm going to make another PR that includes just the .gitignore parts of this (thanks again!), and Devon and I have another plan to fix the whole .iml problem.

Sounds great! Thanks for looking at this folks ❤


[Tree Taxonomy - Design] Dormancy Badge Design

Currently there is a pill badge design for evergreen trees. I have not seen if other types have different colors, but to make it responsive to other screen sizes and situations I have started with 6 types. I have to still check the accessibility and readability of the color. I have to also check the sizing per badge. All have a consistent design, but I want all summer tags to be warm and winter tags to be cold. And the rest be consistent in that part.
There is already a design

The new design

Will update with comments below.


Is Flutter framework compiled with the latest (non-vulnerable) version of Zlib?

Environment:
I am currently running on the latest stable version of Flutter framework for my mobile application development.
Flutter 3.0.2 • channel stable • https://github.com/flutter/flutter.git
Framework • revision cd41fdd495 (2 weeks ago) • 2022-06-08 09:52:13 -0700
Engine • revision f15f824b57
Tools • Dart 2.17.3 • DevTools 2.12.2

pubspec.yaml
environment:
  sdk: '>=2.17.1 <3.0.0'

Problem:
During a recent Vulnerability Assessment and Penetration Testing (VAPT) on my mobile application, the security team has found a vulnerable version of Zlib (1.2.8) in the following directories of my app:
zlib 1.2.8     : Found zlib in file lib/mips64/libmodpdfium.so
zlib 1.2.8     : Found zlib in file lib/arm64-v8a/libmodpdfium.so
zlib 1.2.8     : Found zlib in file lib/x86_64/libmodpdfium.so
zlib 1.2.8     : Found zlib in file lib/x86/libmodpdfium.so
zlib 1.2.8     : Found zlib in file lib/armeabi-v7a/libmodpdfium.so
zlib 1.2.8     : Found zlib in file lib/mips/libmodpdfium.so

Question:
May I know whether Flutter has already updated the referenced Zlib to the latest version of 1.2.12 (https://zlib.net/ChangeLog.txt)? Otherwise, may I know how can I get my mobile app to reference the updated version?
Related issues:

Zlib v1.2.8 is affected by Zlib Vulnerability - CVE-2018-25032
https://github.com/dart-lang/sdk/issues/48725
https://github.com/flutter/flutter/issues/101709

To view the dependencies used by the Flutter Engine at any version, you can navigate to https://github.com/flutter/engine/blob/<engine revision>/DEPS. Flutter at this engine revision uses zlib 1.2.11.
I believe (@zanderso can correct me if I'm mistaken), the dependencies are rolled automatically to keep in sync with the version specified by the Dart SDK.
@keithhie The VATP flag for 1.2.8 is not from the Flutter engine though. That seems to be from a libmodpdfium which is not a Flutter dependency. You will have to followup with that project separately on how to update its usage of zlib.


Languages Service loads project and dependencies in Notebooks

This updates the compilation logic for the language service when the active document is a notebook to include any specificed project root folder such that dependencies are correctly resolved. This avoids spurious error squiggles on items that come from dependencies and even enables go to definition and hover for those items.

Tests added, and good thing too, caught a bug reporting errors from dependencies of dependencies.


Pojavlauncher doesn’t play

Describe the issue
When trying to run in pojavlauncher 3.0-development (main/af88821). The container refuses to launch.
Instructions to reproduce
Install latest pojavlauncher and download 1.20 and click play while logged in. I’m on a iPhone 8 Plus on iOS 16.5.
What version of LiveContainer are you using?
1.0
Other
No response

I think it's because it works in my environment (Version 1.0-release (main/94bb6fd)), so I think it's because of your environment.
If you still have a problem, it might be a good idea to attach an error message or an ips file.


custom distDir generates 404 error and hot reload doesn't work properly anymore

Hi,
If I use a custom distDir:

the following error appears in the console: http://myUrl/_next/-/page/_error / 404 not found
hot reloading does not work properly anymore

If I go back to the default, it goes away.
my next.config.js file:
// see source file server/config.js
module.exports = {
  webpack: null,
  poweredByHeader: false,
  distDir: '.build',
  assetPrefix: ''
}

Thanks,
Paul

Duplicate of #1803 ?
What version are you using? This seems to be working for me on the newly released 2.3.0

Hi, it works well with 2.3.0. I was working with 2.2.0. Thanks!

Closing this 👍


Quicktest Docker read-only Error

Hello,
I'm trying to install FINN but I get this error about docker which are shown in pics. I changed ownership of folders but it is still the same. I could not change ownership of "xinstall".  What should I do ? I haven't run any FINN code yet.
Additionally, When I run "0_how_to_work_with_onnx", I get "ModuleNotFoundError: No module named 'finn.util.visualization'" error.
I'm following the instructions from the getting started page. I have done first 3 steps. My Ubuntu version is 22.04. Is it about that? FINN version is V0.8.1 and Vivado version is 2022.1.

Bests,
Batuhan,

Hey, I don't know if it is too late but you should add yourself into the docker group so that you do not have to run docker with sudo if you haven't done it already: https://docs.docker.com/engine/install/linux-postinstall/
Otherwise, what fixed it for me, when I had to restart due to some issues with jupyter notebook, is to basically clear docker.
Documentation for pruning: https://docs.docker.com/config/pruning/
docker system prune
Please be mindful that everything will be wiped, so if you have something else with docker you should back it up.

Thanks a lot @anguyen9630 for answering to the issue! @Ba1tu3han did that solve your problem? I will leave this issue open for a few days but then will close it as answered.

I am closing this due to inactivity. Please feel free to reopen or create a new issue if the problem persists!


Ensure runtime context on a per-device basis

Close #3320.
If we need a test for this, we'll need to run it as a subprocess, because somewhere else in the test suite we'll establish contexts for all devices, and this bug won't show up (which is why we didn't see it until now). Let me know if a test is desired.
I think the overhead of extra runtime API calls like getDevice() and getDeviceCount() are very low, so performance impact, if any, can only come from the usage of a Python list. I could try to do some quick benchmarks if desired, but I think this bug definitely needs to be fixed one way or another.

I think the overhead of extra runtime API calls like getDevice() and getDeviceCount() are very low, so performance impact, if any, can only come from the usage of a Python list. I could try to do some quick benchmarks if desired, but I think this bug definitely needs to be fixed one way or another.

Added a snippet
cpdef test_ensure_context():
    _ensure_context()

to cupy/cuda/runtime.pyx and tested in ipython with %timeit magic, I got:
Before:
135 ns ± 0.386 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

This PR (with Python list):
193 ns ± 0.152 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

Replace Python list by std::vector of bools:
179 ns ± 0.255 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

If desired I can switch to the vector version.
(Note the timings are obtained after warm-up, since the first shot is much longer.)

Jenkins, test this please

Jenkins CI test (for commit 641c3a7092d4dc29a63a3e9e526f3a0dc6ab2c16, target branch master) succeeded!

Thanks, Emilio!


If the value in a check definition includes invalid characters in the source field, sensu-api crashes.

If the value in a check definition includes invalid characters (spaces) in the source field, sensu-api crashes instead of just marking the check as invalid and moving on.
Sensu version 0.20.3.
{
"_index": "logstash-2015.12.07",
"_type": "json",
"_id": "xxxsomeid",
"_score": null,
"_source": {
"timestamp": "2015-12-07T14:05:59.350300-0700",
"level": "fatal",
"message": "check source cannot contain spaces or special characters",
"object": {
"handlers": [
"notify_slack"
],
"command": "/etc/sensu/conf.d/team/plugins/check-http.rb -s -h xxsomehost -u 'https://xxsomehostapi/v0.1/healthcheck/' -q '"message": "OK",'",
"interval": 60,
"subscribers": [
"xxsome-poller-phx"
],
"source": "foo.host.com PHX",
"name": "check_url_api_phx"
},
"@version": "1",
"@timestamp": "2015-12-07T21:05:59.651Z",
"type": "json",
"host": "xxxsomehost",
"path": "/var/log/sensu/sensu-api.log",
"sensu_team": "xxxsometeam"
},
"sort": [
1449522359651,
1449522359651
]
}

Sensu's configuration validation is strict and this applies to the Sensu API, as it too interacts with Sensu check definitions, e.g. check request publishing.


A strange phenomenon in the training process

Hello, I would like to ask a question, that is, will the phenomenon of "Train Loss: nan f1: 0.000; Val Loss: nan f1 = 0.000" suddenly appear during the network training process, which causes the test f1 to only reach a maximum of 0.55 Magnitude.

Hmmm, I don't recall seeing this, did you check the gradient? Might be gradient explosion.

This phenomenon occurs when I add the regular term.
In addition, please allow me to ask another question. During the training process on the DisGeNet task, the "train loss" will be far greater than the "val loss". "Val loss" will first decrease and then increase, but f1 is still increasing. situation? Does the GisGeNet dataset have other division ratios, such as 7:2:1 or 8:1:1.
Sorry to bother you, thank you very much.

What do you mean by "add the regular term"? The regularization term? Would you mind pointing to which file which line?
DisGeNet has the same division ratio in the experiment.

Yes, the above situation will appear after adding Hypergraph regularization. Have you tried other division ratios of DisGeNet? The best model results are trained with a dataset ratio of 6:2:2?

I haven't tried other division ratios, just 6:2:2, and the split was fixed from the beginning, so that the model and I won't accidentally look into test data.
You can try reducing nratio (say 0.1) to see if it helps.


Trigger ASIM tester

Required items, please complete
Change(s):

See guidance below

Reason for Change(s):

See guidance below

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

See guidance below

Checked that the validations are passing and have addressed any issues that are present:

See guidance below

Guidance <- remove section before submitting

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR.  Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process.  Providing associated Issues that this resolves also easily connects the reason.

Change(s):

Updated syntax for XYZ.yaml

Reason for Change(s):

New schema used for XYZ.yaml
Resolves ISSUE #1234

Version updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.  If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

Guidance for Detection checks
General contribution guidance
PR validation troubleshooting

@sivanguetta : can we close this one? Other old PRs?

Hi @sivanguetta
We wanted to check on the status of PR #5440. PR is pending from more than 90 days. Let us know if any assistance is required for this PR. As per our standard operating procedures if no response is received in the next 7 business days, we will close this PR. Thank you for your cooperation.

Hi @sivanguetta
Could you please provide your approval to close

Hi @sivanguetta
Could you please provide your approval to close

Hi @sivanguetta
Could you please provide your approval to close

@sivanguetta, since we have not received response from many days, we are closing your PR per our standard operating procedures. If you still need support for this issue, you can re-open the PR at any time.
If you do re-open, we simply request that you ensure the PR has response to the last request. Thank you for your cooperation.


Add support for java records

Fixes #26926
Reviewing cheatsheet
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

@mlopatkin No problem, thanks! Indeed, the previous version of the code wasn't up to the task.

The merge queue build has started. Click here to see all failures if any.

The merge queue build has started. Click here to see all failures if any.


[Triton-MLIR][DOCS] Correct spelling

This PR corrects spelling like #664 for Triton-MLIR. It should not break anything.

I see. So even if it says "First-time contributors ...", actually any outsiders cannot run workflows. I thought this repository uses Require approval for first-time contributors, not Require approval for all outside collaborators, but the message on GitHub was just confusing.
Thanks for your comment @ptillet!

Regardless of this, anyway this PR wouldn't hurt Triton-MLIR.


paranthesis missing

Hi,
whenever I publish or subscribe we are missing "{" in the JSON. so finally it says invalid JSON payload to server. So app is unable to consume that JSON.

for a sample json like this {
d =     {
text = "siva this is venkata"
}
}
after receiving in atleast once case its missing "{" and receiving like below.
I solved this as well. I am ready to share the changes . you are taking payload from 2nd character after dividing from header and payload combination.
d =     {
    text = "siva this is venkata"
}

}

Sorry, but I didn't face this issue till now. It would be great if you could replicate this issue with a failing test case.

I have corrected in my code and the change is in MQTTPublishPacket.swift`  if qos != .atLeastOnce {
messageID = 256 * UInt16(payload[0]) + UInt16(payload[1])
payload = payload.subdata(in: 0..<payload.endIndex)
        let datastring = NSString(data: payload, encoding: String.Encoding.utf8.rawValue)
        print(datastring)

    } else {
        messageID = 0
    }`

Now if you notice your old code the payload.subdata line you are not starting from 0. You are starting from 2...which was the cause of that problem.


Problems with the CLA

Today I read the blog post "A quick stroll around our code repositories" on the MathJax blog and got interested in contributing to MathJax. For example with the Grunt/Gulp stuff you mentioned in the blog post. (In fact, I already contributed to MathJax by translating some stuff over at the translatewiki.)
Then I saw that MathJax requires contributors to sign a CLA. So it read what you want me to sign. But from reading the CLA the following questions remained unanswered:

Why do you need my phone number/mailing address/real name?

Under what circumstances would you contact me on my phone? (The answer is probably "never". So why do you need it?)

Under what circumstances would you write me a letter/pay me a visit/do something else with my mailing address?

What is with people who don't have a phone? What is with people like Vitaly Friedman (a famous (homeless) web developer and editor-in-chief of Smashing Magazine), who don't have a mailing address? Are they not allowed to contribute?

Who has access to these information? I mean, I'm getting enough spam these days. I don't need more. (I'm not implying that any one of you is sending spam.)

How are these information stored (e.g. encrypted or not)?

What happens if my phone number/mailing address changes? Do I have to provide the new information to you?

How you verify if the information I gave you are correct? (Or, still correct after some time? See previous problem.)

I haven't seen a requirement in the CLA that my information have to be correct. (Ok, ok you could say this is ridiculous, but you started with this legal stuff.)

So, do you write me a letter, and call me, to check if they are correct?

Or, well if you don't actually care what I write into that form, here you go:
I agree to the CLA.

Name: John Doe
Phone:<PHONE_NUMBER>
Address:
1600 Pennsylvania Avenue Northwest
Washington
DC 20500

Furthermore: while my employer encourages open source work, we have no written "Open Source Contribution Policy". And I don't think I should bother my boss creating one, because some random CLA requires one.

I hope that you don't think of this issue as a random person trolling around, but see how ridiculous at least some parts of your CLA are. Good open source should encourage people to contribute and not scare them away with weird CLAs.
But I'm not here to only complain. I have a few improvement suggestions.
Dropping the CLA
I have absolutely no problem with the content of the CLA (if I understood the officialese correctly). I am also not against having a CLA. My problem is the meta stuff around the CLA.
But let me suggest to drop the CLA. What was the reason to require it in the first place? Does this reason outweigh the fact, that many possible contributors will not contribute to MathJax because of the CLA?
MathJax wouldn't be the first project to do so. For example, in June 2014 Joyent removed the node.js CLA with the goal of broadening contributions.
They are even calling CLAs an open source anti-pattern in that blog post.
Furthermore, the fact, that the Site for the MathJax CLA is copied verbatim from the mentioned former node.js CLA in many places, together with the many problems I mentioned above, raises the feeling, that the MathJax CLA is not completely thought through in many places...
(I am assuming here, MathJax copied from node.js not the other way round.)
Improving the CLA
I understand, if dropping the CLA might not be an option for you. But in this case let me give some improvement suggestions.
Take a look at Yahoo's CLA, which is (at least in my opinion) a more or less ideal CLA.

It starts with Human-readable summary of the license
The only thing it requires is the Github user name. No phone number, mailing address or real name. They don't even want my email address!
They do not ask for my Employer, Employer Contact, Employer Open Source Contribution Policy.
It requires you to log in through Github. So they can be sure that the person signing the CLA and the person making the contributions are the same. (This is also something your CLA is missing.)

Adapting the points 2 and 3 would remove all of the problems I mentioned above.
If the MathJax CLA is like the Yahoo CLA I will happily sign it immediately. So yes, I want to give back and contribute to MathJax, but I will not sign the CLA in the current form. And I am pretty sure, that I am not the only person having this opinion.
A happy new year to everyone from a (hopefully) future contributor
P.S.:
Yes, I know this issue is not a software bug in MathJax. But I want to make this discussion public, so mailing you at<EMAIL_ADDRESS>wasn't an option for me. So please do not close this issue as "invalid".

Thanks for sharing your thoughts on the CLA. This discussion would probably fit better on the mathjax-dev mailing list but let me try to respond here. (It's actually the first time somebody has been concerned with the CLA so this might not be a concise as it could be.)
First off, please keep in mind that MathJax team members do not make decisions about legal matters like CLAs or licensing. Also, we are not lawyers and cannot provide legal advice or identify all relevant scenarios that might have affected the decisions.
Second, because of this, the only answer we can give to some questions raised is "because this was the legal advice MathJax received". Revisiting the decisions regarding licensing is currently not planned but is obviously not impossible (it would, however, incur significant cost).
Third, please note that only contributions to the core of MathJax require a CLA (primarily, the jax'es and the core extensions). So, for example, contributions to the dev tools, the docs, and MathJax-node or in the form of non-essential files (like bower.json) do not require a CLA. This decision was made to simplify contributions; it's where almost all contributions come in and where Apache-compatible licensing was considered sufficient protection.
To answer some of your specific questions.

Re: necessary personal details. I don't think it's set in stone what's necessary but, theoretically, if a contributor cannot provide sufficient information then, yes, we (the team members) are not permitted to merge the code.
Re: storage of personal information. As is probably obvious, the form feeds into a Google spreadsheet that can be accessed by the project management (myself and some people at the AMS, MathJax managing partner). After confirming the contributor's information, a copy of the finalized CLA is stored with the AMS. The initial confirmation is done by email but alternatives are possible. I'm not sure how frequently it is checked but I think it's quite rarely. As I'm not a lawyer, I cannot tell you what might be legally required here.
Re: contacting contributors. The main reason to contact a contributor would be to inquire about the contributed code or to check if the information on file is up to date.
Re: corporate contributions. Again I'm not a lawyer but I think this is only relevant if the contributor acts as an employee (e.g., they contribute company code not owned by themselves, or contribute as part of their work for that company), that is, it serves to protect the employee from personal liability.
Re: wording. Save for the non-binding opening (mostly taken, as you point out, from Nodejs's page), the CLA is the standard Apache CLA. MathJax itself is licensed under Apache license and this is the natural CLA to go along with that.

I hope this clarifies things a little and I hope you'll consider contributing. I think, ultimately, this comes down to a question of trust towards the MathJax Consortium, i.e., AMS and SIAM. Naturally, we (the MathJax team) are very biased here :-)

Thanks for sharing your thoughts on the CLA. This discussion would probably fit better on the mathjax-dev mailing list but let me try to respond here.

I though about the mailing list, too. But I decided for Github, since I think it is more likely that programmers which are not closely connected to MathJax will see it, too.

(It's actually the first time somebody has been concerned with the CLA so this might not be a concise as it could be.)

My text wasn't concise either :-)

[...] Revisiting the decisions regarding licensing is currently not planned but is obviously not impossible (it would, however, incur significant cost).

Not changing the CLA, does not make me too happy apparently. But since there is the next point, this shouldn't matter to much. Therefore I am closing the issue. (I'm just surprised, that dropping some fields in the CLA incurs significant cost. But probably you need to contact a lawyer and that makes it expensive.)

Third, please note that only contributions to the core of MathJax require a CLA (primarily, the jax'es and the core extensions). So, for example, contributions to the dev tools, the docs, and MathJax-node or in the form of non-essential files (like bower.json) do not require a CLA. This decision was made to simplify contributions; it's where almost all contributions come in and where Apache-compatible licensing was considered sufficient protection.

This is interesting and basically solves my problem. You should also add these information to the CLA page, too.
(I am surprised, that not having to sign the CLA for specific files does not make things overly complicated.)

To answer some of your specific questions. [...]

So everything is basically as I thought it would be. I just want to pick out a specific point:

Re: contacting contributors. The main reason to contact a contributor would be to inquire about the contributed code or to check if the information on file is up to date.

Hmm, since there is paragraph 6 in the CLA: "You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. [...]" you shouldn't be calling former contributors and ask them about their code. And I would be surprised, if this has ever happened. The only place to discuss code related things should be Github (and perhaps the mailing list). Just create a new issue, @mention the former contributor and perhaps he will contribute again. If he is not contributing again, there is nothing you can do about it.
And storing personal information to check if the personal information is up to date is just weird.

I hope this clarifies things a little and I hope you'll consider contributing. I think, ultimately, this comes down to a question of trust towards the MathJax Consortium, i.e., AMS and SIAM. Naturally, we (the MathJax team) are very biased here :-)

Yes, it clarifies things. It is not, that I don't trust the MathJax consortium. I am just not too fond of spreading my personal information around the internet. And sometimes I question why I should give them away.
So the grunt/gulp stuff should go on the https://github.com/mathjax/MathJax-dev/issues list, right?

You should also add these information to the CLA page, too.

This is documented at https://github.com/mathjax/MathJax/wiki/Contributor-license-agreement-etc

(I am surprised, that not having to sign the CLA for specific files does not make things overly complicated.)

It does but we think it's worth it.

But probably you need to contact a lawyer and that makes it expensive.

Correct.

you shouldn't be calling former contributors and ask them about their code.

The scenario I had in mind was somebody questioning the legality of the contribution; in that case, I would expect to reach out to the contributor (if only to hear both sides).
If somebody really does not want to be in contact with MathJax, then perhaps they should not contribute (and perhaps MathJax should not be accepting such contributions to save both sides the trouble; trust has to go both ways after all). Fortunately, being interested in core development and objecting to the CLA has not coincided so far.

So the grunt/gulp stuff should go on the https://github.com/mathjax/MathJax-dev/issues list, right?

Yes but you could also post this on this tracker (for greater visibility). This might be interesting.


Consider how the visualiser handles a beneficial ownership network changing over a period of time

Current situation
At the moment, all data is rendered according to the specification here.
No account is taken of fields like Interest.endDate, replacesStatements or field values like Interest.share.exact: 0 which indicate that information has changed over time, or that an interest is NOT present at a given time.
This means that when data represents change over time, it will either not be parsed (see this OO Register/visualiser issue) or it may be visualised in a confusing way: it may look like a snapshot in time, but actually historical information is layered over current information.
With BODS 0.4 there will be clearer requirements regarding how to represent changing beneficial ownership over time.
What to do
We need to consider how the visualiser should handle data that represents change over time, both in BODS 0.4 and prior versions of the standard.
The minimal requirements is probably: have the visualiser explicitly represent information at a given point of time.

Actually, I'm not sure that my assumptions above about the current situation are right. This code is using information from the replacesStatements field.
@codemacabre - When you have some time to investigate... could you let us know how the visualiser currently handles changing information about the same node/edge?
(This will help inform the decision regarding OO Register data not rendering in the visualiser.)

@codemacabre - I think we need to write out a description here of how to extract 'active' network information at date X from a BODS 0.4 dataset. To act as a specification. Do you want to have a crack at that? Or I can do it this week sometime.

The specification is on page 2 here

Beyond what's in the specification, we need to decide how the user of the tool inputs a time, T. The minimum and maximum possible values of T will be a function of the data inputed. I think we can simply say that for BODS 0.4-like data, we use the minimum and maximum values of statementDate.
I'm not sure how a date-picker would handle minimum and maximum possible values. It may be more elegant and user-friendly to use a slider. (And poss an 'apply' button, depending on how redraw works and how fast it happens.) The 'resolution' of the slider steps needs to be considered too.
(We also need to consider what happens with pre-0.4 data GUI-wise. For example we could just grey out the slider, with the slider handle to the far right.)

@codemacabre, in the spec we say "...the data presented to the tool [...] is presumed to have a time dimension (that is to show the properties of people, entities and relationships changing over a period of time) if: ...". We haven't discussed the expected behaviour of the tool and UI in cases where the data is found NOT to have a time dimension.
How about this: there are three possible UI states:

In cases where the data is found NOT to have a time dimension (whether the data is 0.2/0.3-like or 0.4-like), the UI has no time-control toolbar (as now).

In cases where the data is 0.2/0.3-like and has a time dimension, the UI has a time-control toolbar, consisting of:

'Date:' label
A date value (latest statementDate of an object in the BODS array), timepoint T
A disabled slider with the handle of the slider at the right end. (Or - if it is likely to be annoying for users to have an unusable slider, we could replace the disabled slider with a message in italics like: "(Latest state of the network.)"

The idea is to indicate that: there is a time dimension in the data; that the BO network visualised is that of the final state of the data; that the final state of the network existed at timepoint T.

In cases where the data is 0.4-like and has a time dimension, the UI has a time-control toolbar, consisting of:

A 'Date:' label
A date value (latest statementDate of an object in the BODS array), timepoint T
A slider which can be used to alter the date value (between min and max values of T as specced above)
(Possibly an 'Apply' button if necessary.)

In working on this, I've encountered a weird bug with the arrowheads:

I believe it's because of the Bezier curves from bezier-js, so in disabling the Bezier library entirely and using CSS transform to offset the lines, it improves, but the arrowheads are still misaligned:

There's still a fair bit of work to do on the changes-over-time, so ideally I'd leave this bug as a separate issue (which I'm aware we might not have time to get to) rather than try to solve now.
@kd-ods Are you happy for this approach, or should I continue to try to resolve it now, potentially extending the changes-over-time work?

Yes - I'm very happy for you to file this as a separate issue and leave it for now, @codemacabre.


Simplify ssl/tls/dtls version checks

This PR aims to simplify version checks throughout the codebase by using the version_cmp() function of statem_lib.c.

I've reported the unrelated CI failure as https://github.com/openssl/openssl/issues/23176

Pushed to master. Thanks!


Default profile ( #122 )

As discussed in the ticket, unlinked characters will now be linked to a single default profile instead of having a new one created for every character. This profile is currently always shown first in the list (could be put in last position, I just feel like a fixed position is better than it being in the alphabetical sorting) and its basic data is automatically updated depending on the character.
Actions have been hidden for this profile, as well as tab buttons in the profile details window. In the latter, if the default profile is selected, a message warns to create a new profile (with a button for this directly underneath) or link to an existing one. Creating a new profile there will automatically select it (I'm unsure if this behaviour should also be replicated in the Profiles list window, I think it should but importing a profile cannot currently be done on the selected profile - could also be something to look into) and move to the details of the new profile.
As notes are currently a tab in the profile details, their dedicated button has been disabled in the player's target frame if the default profile is selected, but I'll think it over. Extended features (inventory, bags, quest log) will work as with any other profile.
Points still to consider: Do we want to show more on the details tab? Would we use this opportunity to look at #179 (This might be tricky with new characters all tied to the default one)?
Last point: Will have to take this PR into account for #415 so that the default profile is hidden during a search.

Ready for review.


nonsense change.

just to check if the test do not fails.

ok error are on my site ... sry for this pull request


can't find how to set the proxy

$ elan default stable
returns
info: syncing channel updates for 'stable'
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { description: "Failure when receiving data from the peer", code: 56, extra: Some("Received HTTP code 407 from proxy after CONNECT") }', src/elan-utils/src/utils.rs:519:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

just like ruby's gem has  --http-proxy, elan should have a way to pass the proxy

This would be great. I tried proxychains on elan and it does not work.

elan use curl to do downloading, so we can use env variable to set proxies, like this:
$env:ALL_PROXY="socks5h://<IP_ADDRESS>:1080"

elan use curl to do downloading, so we can use env variable to set proxies, like this:
$env:ALL_PROXY="socks5h://<IP_ADDRESS>:1080"

valid solution, thank you.


Make offlineExchange import respect useTypeImports

Description
Change import of offlineExchange from @urql/exchange-graphcache to respect config.useTypeImports since only types are used in the generated code.
Related # (issue)
#445
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as
expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Have tested that typescript configured with importsNotUsedAsValues = error no longer throws errors during tsc after the change has been made.
Checklist:

[x] I have followed the
CONTRIBUTING doc and the
style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose
the solution you did and what alternatives you considered, etc...

Resolved the two failing tests

hey @HalfdanJ can you please sync the branch with main and also run yarn changeset  and explain the change  so we can have it part of release notes. Read about changeset here

I think this got resolved in this pr: https://github.com/dotansimha/graphql-code-generator-community/pull/476. After i updated to latest version of the lib i was able remove my patch I had in place.


[2.7] Expand Kubernetes Upgrade Validation Tests for Local and Imported Clusters

Issue: https://github.com/rancher/qa-tasks/issues/611

Problem

Currently imported clusters and local cluster upgrades tests happen manually.
Solution

Add local upgrade test case and expand existing upgrade Kubernetes validation test to enable testing on downstream imported clusters.
Engineering Testing
Automated Testing

Test types added/modified:

Validation (Go Framework)

Existing / newly added automated tests that provide evidence there are no regressions: Runs will be provided offline.
Todos:

[x] Add missing godocs

Just a general question Ugur.
Do these tests have the same scope than what's done with the downstream clusters upgrade tests?
Are there any constrains or differences?

Do these tests have the same scope than what's done with the downstream clusters upgrade tests?
Are there any constrains or differences?

@izaac Yes that's right, these tests would have the same scope with the downstream clusters. The test we're calling won't change but the input for a cluster ID can be local (the same input just like downstream clusters), which will point out to this new subtest. By separating only the subtests we should have the vertical flexibility between local and downstream going further, and add validations respectively when needed.
I'd say the biggest difference is going to be RKE, as for imported or local clusters now we will need RKE cli to be installed on the environment. I am planning to add the RKE install step to the Jenkinsfile when I get to the Jenkins part of it.

Needs rebase

Fixed the linter issues and resolved the configuration generation part after the file package changes, should be ready for another round

Forward port: https://github.com/rancher/rancher/pull/43327

There was a conflict with the client package for this PR after this: https://github.com/rancher/rancher/pull/43319

Conflict is resolved, and should be ready for another round


Rate predicted labels: Add analytics and use template for email notification

The feedbacks are now being recorded. The frontend remains to be updated

Email is now using template


Add Material 3 support for TabBar

fixes https://github.com/flutter/flutter/issues/111962 https://github.com/flutter/flutter/issues/103642
Description

This PR adds support Material 3 (except preferredSize and indicatorWeight, this will be done in a separate issue).
Added support for creating a rounded tab indicator for Material 3.
Added support selected state for M3 overlay.
Added the indicatorColor  and new  dividerColor property for the theme.
Also  _LerpColors is removed, it's no longer needed.

Preview

Close up of M3 indicator and divider

Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Also, it looks like some gold images need to be approved.

Goldens are approved. 👍