when request with error(like 404),  the script(which created by dynamic import) fires a timeout instead of the onerror

Bug report

What is the current behavior?
About the dynamic import, when the script's(like some js resource in CDN) get an error(like 404 or abort), the webpack made it trigger the timeout logic instead of the onerror logic. So, it will throw an error after 120 seconds rather than right now.
eg.
when I block the url, the error throw after 120s.

If the current behavior is a bug, please provide the steps to reproduce.
you can clone the demo project form here: https://github.com/lanmch/webpack-dynamic-error-demo
run the script
npm install
npm run build
then open the dist/index.html in browser

may there are some speciel code in our project.
Like  the dynamic module's name is src_dynamic-module_js, so that the file js name is src_dynamic-module_js.bundle.js.
To make my js resource preloaded, I insert the src_dynamic-module_js with script into the final html by HtmlWebpackPlugin.
So that I find the code of webpack_require.l's definition may have some error about this case.

What is the expected behavior?
I think the error maybe throw right now when it have problem with network, or in few minutes rather than 120s.

Other relevant information:
webpack version:  5.72.1
Node.js version: 14.16.0
Operating System: macOS

you have timeout error...

Since webpack inserts script it can't control http flow. So webpack has 3 options => onError ( loading error ), onLoad and setting custom timeout. In your case it fires timeout, not sure why onError doesn't triggered..

because in webpack_require.l, the same src with dynamic import not generate(so the onerror can bind the function onCompletet), but it trigger the timeout.

I think there should have to judge the dynamic import script is be load before in html or not

I don't understand what you mean.. e.g. in your example if just delete src_dynamic_import.js chunk onError will be triggered..

you mean delete here?

no, remove this script from dist folder and you will get onError error

the dist is generate by webpack, how can I remove it? It one of the dynamic import bundle js file

aha?

I don't understand then.. in your example I have error because of
<script src="file:///Users/bytedance/Desktop/demo-project/webpack-dynamic-demo/dist/src_dynamic-module_js.bundle.js"></script>

that doesn't exists on my disk...

You getting timeout error because of your network.. ( thats why it is fired after 120s ) With "normal" chunk loading error it will be fired immediately.

I don't understand then.. in your example I have error because of
<script src="file:///Users/bytedance/Desktop/demo-project/webpack-dynamic-demo/dist/src_dynamic-module_js.bundle.js"></script>

that doesn't exists on my disk...
You getting timeout error because of your network.. ( thats why it is fired after 120s ) With "normal" chunk loading error it will be fired immediately.

sorry, you can change the script in webpack-dynamic-error-demo/index.html to simulate it
<script src="{your project path}/webpack-dynamic-demo/dist/src_dynamic-module_js.bundle.js"></script>

In our project, we use a plugin like preload-webpack-plugin for webpack5, to make the main purpose here is to make some scripts on the main link as early as possible to avoid the insertion of some asynchronous tasks. Because to show the minimal reproducible demo, I dont show you the plugin. So I try to use the hard code to let you know the final build result in my project

I don't understand then.. in your example I have error because of
<script src="file:///Users/bytedance/Desktop/demo-project/webpack-dynamic-demo/dist/src_dynamic-module_js.bundle.js"></script>

that doesn't exists on my disk...
You getting timeout error because of your network.. ( thats why it is fired after 120s ) With "normal" chunk loading error it will be fired immediately.

sorry, you can change the script in webpack-dynamic-error-demo/index.html to simulate it
<script src="{your project path}/webpack-dynamic-demo/dist/src_dynamic-module_js.bundle.js"></script>

In our project, we use a plugin like preload-webpack-plugin for webpack5, to make the main purpose here is to make some scripts on the main link as early as possible to avoid the insertion of some asynchronous tasks. Because to show the minimal reproducible demo, I dont show you the plugin. So I try to use the hard code to let you know the final build result in my project

In final result, the script maybe like
<script src="https://www.cdnpath.com/js/src_dynamic-module_js.bundle.js" ></script>

So, there is no file path about 'file://' in the code actually

ok, you can preload this way.. but how it is related to timeout error?

sorry, you can change the script in webpack-dynamic-error-demo/index.html to simulate it

this way all works fine for me..

like this in your dist/index.html
<script src="{your project path}/webpack-dynamic-demo/dist/src_dynamic-module_js.bundle.js"></script>

when the website run with index.html, the webpack_require__.l will try to append the dynamic import script to html.

But at first, it will judge if the html has the script same tag, and set the 'needAttach' value.
eg. in the demo, when the index.html run, it will try to append the script with the src of 'src_dynamic-module_js.bundle.js', but at first in the judge function, it already exists, so it will not append to html.

Then, in the code behind, it will define a timeout about this js resource, but not append it to html dom
So when we have an error with network of 'src_dynamic-module_js.bundle.js', the script of it will load with error in Chrome. And as the code executes to load the dynamic import js, it judge and define timeout but not append script.
The final phenomenon is the onerror with the onScriptComplete will throw after the timeout code is executed（120000ms)

ok, but this is possible only with non-blocking js + I'm not sure is it possible to kind of miss error event...

ah... ok I see, need to debug first..

not sure that it is possible to solve this.. we can't track loading state for script tag..

not sure that it is possible to solve this.. we can't track loading state for script tag..

yes, I also can not find a very good way to solve this problem till now.
In my project, I just set the timeout value from 120000 to 10000 when the needAttach is false.

not sure that it is possible to solve this.. we can't track loading state for script tag..

for another, could you please tell me why the default timeout time is 120 seconds rather than other value?


Merge Hotfix/Style guidelines v0.4.2

What is new?
Updated

Reworked header component to match portal
Reworked footer component to match portal
Reworked search input to match portal

Added

New SCSS structure, allowing a optimized scalability from the application
Updated frontend dependencies file.
Matched portal styles
Refactored responsiveness from components

Ok, tested and ready for merging


Update README.md

Update path to eos-s3 toolchain release

This is probably not relevant any more, closing. Thanks for the help though!


[Bug][RayCluster] Fix RAY_REDIS_ADDRESS parsing with redis scheme and…

Why are these changes needed?
The current parsing of RAY_REDIS_ADDRESS in a redis cleanup job is not correct and may result in the error reported in the slack channel: https://ray-distributed.slack.com/archives/C02GFQ82JPM/p1697556705340069
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError: too many values to unpack (expected 2)

This is because the current parsing:
host, port = os.getenv('RAY_REDIS_ADDRESS').rsplit(':')

may result in more than two values in the following cases:

RAY_REDIS_ADDRESS contains multiple addresses.
RAY_REDIS_ADDRESS contains IPv6 addresses.
RAY_REDIS_ADDRESS contains a username and a password.

Related issues
#1557
A Potential Solution
We could use the private method get_address, which actually parses a redis address, from the upstream ray. This would let us stay consistent with the upstream. However, I think importing the get_address is not suitable right now for the following reasons:

The naming of get_address is too vague. I believe it will be changed to something like get_redis_address soon.
The get_address doesn't handle multiple addresses which are allowed in the RAY_REDIS_ADDRESS according to here.
The get_address implementation doesn't seem to be quite right to me. It doesn't consider most properties allowed in the redis uri spec, such as password, and database number. A valid redis uri looks like this redis://user:secret@localhost:6379/0 and it will not be parsed corrected by the get_address.
The get_address was introduced only 2 months ago. I am afraid that it doesn't exist in most ray releases.

Proposed Solution
I propose using the standard urlparse to handle the parsing. It will help us extract the scheme, hostname, port, and even username and password more correctly. I also recommend contributing this method back to the upstream ray.
In this PR, I did the following

To handle multiple addresses according to here, I also use the .split(',') and only pick the first part.
If the address doesn't have a scheme, then I add a redis:// scheme prefix to it. This is required by the urlparse.
Use the urlparse to extract scheme, hostname, port, and password from the address and ignore other properties that aren't supported by the cleanup_redis_storage. Note that I also enable the use_ssl flag if the scheme is rediss.

Testing
To test this change, I added a new test, duplicated from the original ray-operator/config/samples/ray-cluster.external-redis.yaml, and added a redis scheme redis:// to its RAY_REDIS_ADDRESS.
Checks

[x] I've made sure the tests are passing.
Testing Strategy

[x] Unit tests
[x] Manual tests
[ ] This PR is not tested :(

Hi @kevin85421,
Here is the result of running python3 tests/test_sample_raycluster_yamls.py. All 14 tests including the new ray-cluster.external-redis-uri.yaml are passed.
▶ RAY_IMAGE=rayproject/ray:2.7.0 OPERATOR_IMAGE=kuberay/operator:nightly python3 tests/test_sample_raycluster_yamls.py
2023-10-28:12:55:01,755 INFO     [test_sample_raycluster_yamls.py:62] {'ray-image': 'rayproject/ray:2.7.0', 'kuberay-operator-image': 'kuberay/operator:nightly'}
2023-10-28:12:55:01,755 INFO     [test_sample_raycluster_yamls.py:64] Build a test plan ...
2023-10-28:12:55:01,756 INFO     [test_sample_raycluster_yamls.py:68] [SKIP TEST 0] ray-cluster.autoscaler.large.yaml: Skip this test because it requires a lot of resources.
2023-10-28:12:55:01,756 INFO     [test_sample_raycluster_yamls.py:68] [SKIP TEST 1] ray-cluster.gke-bucket.yaml: Skip this test because it requires GKE and k8s service accounts.
2023-10-28:12:55:01,756 INFO     [test_sample_raycluster_yamls.py:70] [TEST 2]: ray-cluster.py-spy.yaml
2023-10-28:12:55:01,771 INFO     [test_sample_raycluster_yamls.py:70] [TEST 3]: ray-cluster.tls.yaml
2023-10-28:12:55:01,779 INFO     [test_sample_raycluster_yamls.py:68] [SKIP TEST 4] ray-cluster-tpu.yaml: Skip this test because it requires TPU resources.
2023-10-28:12:55:01,780 INFO     [test_sample_raycluster_yamls.py:68] [SKIP TEST 5] ray-cluster.complete.large.yaml: Skip this test because it requires a lot of resources.
2023-10-28:12:55:01,780 INFO     [test_sample_raycluster_yamls.py:70] [TEST 6]: ray-cluster.heterogeneous.yaml
2023-10-28:12:55:01,788 INFO     [test_sample_raycluster_yamls.py:70] [TEST 7]: ray-cluster.mini.yaml
2023-10-28:12:55:01,796 INFO     [test_sample_raycluster_yamls.py:70] [TEST 8]: ray-cluster.autoscaler.yaml
2023-10-28:12:55:01,804 INFO     [test_sample_raycluster_yamls.py:70] [TEST 9]: ray-cluster.external-redis-uri.yaml
2023-10-28:12:55:01,811 INFO     [test_sample_raycluster_yamls.py:70] [TEST 10]: ray-cluster.embed-grafana.yaml
2023-10-28:12:55:01,819 INFO     [test_sample_raycluster_yamls.py:70] [TEST 11]: ray-cluster.external-redis.yaml
2023-10-28:12:55:01,827 INFO     [test_sample_raycluster_yamls.py:70] [TEST 12]: ray-cluster.head-command.yaml
2023-10-28:12:55:01,835 INFO     [test_sample_raycluster_yamls.py:70] [TEST 13]: ray-cluster.volcano-scheduler-queue.yaml
2023-10-28:12:55:01,843 INFO     [test_sample_raycluster_yamls.py:70] [TEST 14]: ray-cluster.custom-head-service.yaml
2023-10-28:12:55:01,852 INFO     [test_sample_raycluster_yamls.py:70] [TEST 15]: ray-cluster.separate-ingress.yaml
2023-10-28:12:55:01,860 INFO     [test_sample_raycluster_yamls.py:70] [TEST 16]: ray-cluster.volcano-scheduler.yaml
2023-10-28:12:55:01,869 INFO     [test_sample_raycluster_yamls.py:70] [TEST 17]: ray-cluster.complete.yaml
2023-10-28:12:55:01,877 INFO     [utils.py:320] Execute command: kind delete cluster
Deleting cluster "kind" ...
Deleted nodes: ["kind-control-plane"]
2023-10-28:12:55:07,58 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
error: context "kind-kind" does not exist
2023-10-28:12:55:07,452 INFO     [utils.py:320] Execute command: kind create cluster --wait 900s --config /Users/ruian/Code/go/kuberay/tests/framework/config/kind-config.yaml
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.27.3) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
 ✓ Waiting ≤ 15m0s for control-plane = Ready ⏳
 • Ready after 16s 💚
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! 😊
2023-10-28:12:55:53,836 INFO     [utils.py:259] Download Docker images: {'ray-image': 'rayproject/ray:2.7.0', 'kuberay-operator-image': 'kuberay/operator:nightly'}
2023-10-28:12:55:53,836 INFO     [utils.py:320] Execute command: docker image inspect rayproject/ray:2.7.0 > /dev/null
2023-10-28:12:55:53,925 INFO     [utils.py:272] Image rayproject/ray:2.7.0 exists
2023-10-28:12:55:53,926 INFO     [utils.py:320] Execute command: docker image inspect kuberay/operator:nightly > /dev/null
2023-10-28:12:55:54,14 INFO     [utils.py:272] Image kuberay/operator:nightly exists
2023-10-28:12:55:54,15 INFO     [utils.py:275] Load images into KinD cluster
2023-10-28:12:55:54,15 INFO     [utils.py:320] Execute command: kind load docker-image rayproject/ray:2.7.0
Image: "rayproject/ray:2.7.0" with ID "sha256:c9419e64c139a08bc5afe5beb982addc9dcceaa92124ec4b58c1776911ebadb8" not yet present on node "kind-control-plane", loading...
2023-10-28:12:58:14,49 INFO     [utils.py:320] Execute command: kind load docker-image kuberay/operator:nightly
Image: "kuberay/operator:nightly" with ID "sha256:0059dd9fa648ecd04f9761a2d0c1ceae66f209caee75a9478354ebc9bdc36c4b" not yet present on node "kind-control-plane", loading...
2023-10-28:12:58:17,600 INFO     [utils.py:300] Install both CRD and KubeRay operator by kuberay-operator chart
2023-10-28:12:58:17,600 INFO     [utils.py:320] Execute command: helm install -n default -f /var/folders/83/kqwzb_5j7t729zvyjhbfhzjr0000gn/T/tmp3rsmdi3f_values.yaml kuberay-operator /Users/ruian/Code/go/kuberay/helm-chart/kuberay-operator/ --set image.repository=kuberay/operator,image.tag=nightly
NAME: kuberay-operator
LAST DEPLOYED: Sat Oct 28 12:58:23 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
2023-10-28:12:58:25,35 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.py-spy.yaml
raycluster.ray.io/raycluster-py-spy created
configmap/ray-example created
2023-10-28:12:59:56,619 INFO     [prototype.py:379] --- RayClusterAddCREvent 91.10193181037903 seconds ---
2023-10-28:12:59:56,640 INFO     [utils.py:320] Execute command: kubectl exec raycluster-py-spy-head-jjmfh -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 21:59:57,414	INFO worker.py:1335 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 21:59:57,414	INFO worker.py:1452 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
[2023-10-27 21:59:59,425 W 40 40] global_state_accessor.cc:389: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
[2023-10-27 22:00:00,427 W 40 40] global_state_accessor.cc:389: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
[2023-10-27 22:00:01,428 W 40 40] global_state_accessor.cc:389: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
2023-10-27 22:00:02,551	INFO worker.py:1636 -- Connected to Ray cluster.
{'memory':<PHONE_NUMBER>.0, 'object_store_memory': 558419558.0, 'CPU': 1.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:00:35,688 INFO     [prototype.py:409] --- Cleanup RayCluster 30.74703598022461 seconds ---
2023-10-28:13:00:35,688 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.py-spy.yaml --ignore-not-found=true
configmap "ray-example" deleted
.2023-10-28:13:00:35,849 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:00:35,993 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.tls.yaml
raycluster.ray.io/raycluster-tls created
secret/ca-tls created
configmap/tls created
2023-10-28:13:00:48,728 INFO     [prototype.py:379] --- RayClusterAddCREvent 12.374605655670166 seconds ---
2023-10-28:13:00:48,752 INFO     [utils.py:320] Execute command: kubectl exec raycluster-tls-head-dxqrf -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:00:49,439	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:00:49,439	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:00:49,466	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'node:<IP_ADDRESS>': 1.0, 'CPU': 1.0, 'object_store_memory': 491856691.0, 'memory':<PHONE_NUMBER>.0, 'node:__internal_head__': 1.0}
2023-10-28:13:01:26,17 INFO     [prototype.py:409] --- Cleanup RayCluster 35.22021770477295 seconds ---
2023-10-28:13:01:26,18 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.tls.yaml --ignore-not-found=true
secret "ca-tls" deleted
configmap "tls" deleted
.2023-10-28:13:01:26,191 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:01:26,346 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.heterogeneous.yaml
configmap/ray-code created
raycluster.ray.io/raycluster-heterogeneous created
2023-10-28:13:01:39,252 INFO     [prototype.py:379] --- RayClusterAddCREvent 12.588768005371094 seconds ---
2023-10-28:13:01:39,278 INFO     [utils.py:320] Execute command: kubectl exec raycluster-heterogeneous-head-n6tbp -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:01:40,274	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:01:40,274	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:01:40,307	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'node:<IP_ADDRESS>': 1.0, 'CPU': 6.0, 'object_store_memory':<PHONE_NUMBER>5.0, 'memory':<PHONE_NUMBER>2.0, 'node:<IP_ADDRESS>': 1.0, 'node:<IP_ADDRESS>': 1.0, 'node:__internal_head__': 1.0, 'node:<IP_ADDRESS>': 1.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:02:13,526 INFO     [prototype.py:409] --- Cleanup RayCluster 31.43476891517639 seconds ---
2023-10-28:13:02:13,526 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.heterogeneous.yaml --ignore-not-found=true
configmap "ray-code" deleted
.2023-10-28:13:02:13,682 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:02:13,824 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.mini.yaml
raycluster.ray.io/raycluster-mini created
2023-10-28:13:02:15,178 INFO     [prototype.py:379] --- RayClusterAddCREvent 1.032729148864746 seconds ---
2023-10-28:13:02:15,198 INFO     [utils.py:320] Execute command: kubectl exec raycluster-mini-head-msccf -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:02:16,337	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:02:16,338	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
[2023-10-27 22:02:19,409 W 13 13] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
[2023-10-27 22:02:20,410 W 13 13] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
2023-10-27 22:02:21,417	INFO worker.py:1642 -- Connected to Ray cluster.
{'node:__internal_head__': 1.0, 'object_store_memory': 559873228.0, 'memory':<PHONE_NUMBER>.0, 'node:<IP_ADDRESS>': 1.0, 'CPU': 1.0}
2023-10-28:13:02:55,583 INFO     [prototype.py:409] --- Cleanup RayCluster 31.684444904327393 seconds ---
2023-10-28:13:02:55,584 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.mini.yaml --ignore-not-found=true
.2023-10-28:13:02:55,724 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:02:55,860 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.autoscaler.yaml
raycluster.ray.io/raycluster-autoscaler created
configmap/ray-example created
2023-10-28:13:02:58,233 INFO     [prototype.py:379] --- RayClusterAddCREvent 2.045454978942871 seconds ---
2023-10-28:13:02:58,258 INFO     [utils.py:320] Execute command: kubectl exec raycluster-autoscaler-head-m942r -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
Defaulting container name to ray-head.
Use 'kubectl describe pod/raycluster-autoscaler-head-m942r -n default' to see all of the containers in this pod.
2023-10-27 22:02:58,929	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:02:58,929	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
[2023-10-27 22:03:00,942 W 19 19] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
2023-10-27 22:03:02,010	INFO worker.py:1642 -- Connected to Ray cluster.
{'node:<IP_ADDRESS>': 1.0, 'object_store_memory': 528251289.0, 'memory':<PHONE_NUMBER>.0, 'node:__internal_head__': 1.0}
2023-10-28:13:03:38,850 INFO     [prototype.py:409] --- Cleanup RayCluster 35.88992714881897 seconds ---
2023-10-28:13:03:38,850 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.autoscaler.yaml --ignore-not-found=true
configmap "ray-example" deleted
.2023-10-28:13:03:39,1 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:03:39,137 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.external-redis-uri.yaml
configmap/redis-config created
service/redis created
deployment.apps/redis created
secret/redis-password-secret created
raycluster.ray.io/raycluster-external-redis-uri created
configmap/ray-example-uri created
2023-10-28:13:04:15,320 INFO     [prototype.py:379] --- RayClusterAddCREvent 35.82306885719299 seconds ---
2023-10-28:13:04:15,342 INFO     [utils.py:320] Execute command: kubectl exec raycluster-external-redis-uri-head-vx9vf -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:04:15,937	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:04:15,937	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:04:15,951	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'node:<IP_ADDRESS>': 1.0, 'CPU': 1.0, 'object_store_memory':<PHONE_NUMBER>.0, 'memory':<PHONE_NUMBER>4.0, 'node:__internal_head__': 1.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:04:53,819 INFO     [prototype.py:409] --- Cleanup RayCluster 37.06717300415039 seconds ---
2023-10-28:13:04:53,819 INFO     [utils.py:320] Execute command: test $(kubectl exec deploy/redis -- redis-cli --no-auth-warning -a $(kubectl get secret redis-password-secret -o jsonpath='{.data.password}' | base64 --decode) DBSIZE) = '0'
2023-10-28:13:04:54,186 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.external-redis-uri.yaml --ignore-not-found=true
configmap "redis-config" deleted
service "redis" deleted
deployment.apps "redis" deleted
secret "redis-password-secret" deleted
configmap "ray-example-uri" deleted
.2023-10-28:13:04:54,383 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:04:54,517 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.embed-grafana.yaml
raycluster.ray.io/raycluster-embed-grafana created
2023-10-28:13:05:06,94 INFO     [prototype.py:379] --- RayClusterAddCREvent 11.267714023590088 seconds ---
2023-10-28:13:05:06,114 INFO     [utils.py:320] Execute command: kubectl exec raycluster-embed-grafana-head-nv4lx -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:05:06,681	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:05:06,682	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:05:06,696	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'node:__internal_head__': 1.0, 'CPU': 2.0, 'node:<IP_ADDRESS>': 1.0, 'memory':<PHONE_NUMBER>.0, 'object_store_memory': 799749733.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:05:43,906 INFO     [prototype.py:409] --- Cleanup RayCluster 36.00530767440796 seconds ---
2023-10-28:13:05:43,907 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.embed-grafana.yaml --ignore-not-found=true
.2023-10-28:13:05:44,46 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:05:44,178 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.external-redis.yaml
configmap/redis-config created
service/redis created
deployment.apps/redis created
secret/redis-password-secret created
raycluster.ray.io/raycluster-external-redis created
configmap/ray-example created
2023-10-28:13:06:20,435 INFO     [prototype.py:379] --- RayClusterAddCREvent 35.870128870010376 seconds ---
2023-10-28:13:06:20,457 INFO     [utils.py:320] Execute command: kubectl exec raycluster-external-redis-head-cxtmq -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:06:21,116	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:06:21,116	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:06:21,130	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'node:__internal_head__': 1.0, 'memory':<PHONE_NUMBER>1.0, 'object_store_memory':<PHONE_NUMBER>.0, 'node:<IP_ADDRESS>': 1.0, 'CPU': 1.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:06:59,72 INFO     [prototype.py:409] --- Cleanup RayCluster 37.13770318031311 seconds ---
2023-10-28:13:06:59,72 INFO     [utils.py:320] Execute command: test $(kubectl exec deploy/redis -- redis-cli --no-auth-warning -a $(kubectl get secret redis-password-secret -o jsonpath='{.data.password}' | base64 --decode) DBSIZE) = '0'
2023-10-28:13:06:59,622 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.external-redis.yaml --ignore-not-found=true
configmap "redis-config" deleted
service "redis" deleted
deployment.apps "redis" deleted
secret "redis-password-secret" deleted
configmap "ray-example" deleted
.2023-10-28:13:06:59,832 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:06:59,973 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.head-command.yaml
raycluster.ray.io/raycluster-mini created
configmap/ray-example created
2023-10-28:13:07:14,578 INFO     [prototype.py:379] --- RayClusterAddCREvent 14.281078815460205 seconds ---
2023-10-28:13:07:14,622 INFO     [utils.py:320] Execute command: kubectl exec raycluster-mini-head-zcctn -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:07:15,213	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:07:15,213	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:07:15,229	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'object_store_memory': 557763379.0, 'memory':<PHONE_NUMBER>.0, 'node:<IP_ADDRESS>': 1.0, 'node:__internal_head__': 1.0, 'CPU': 1.0}
2023-10-28:13:07:47,819 INFO     [prototype.py:409] --- Cleanup RayCluster 31.789110898971558 seconds ---
2023-10-28:13:07:47,820 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.head-command.yaml --ignore-not-found=true
configmap "ray-example" deleted
.2023-10-28:13:07:47,972 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:07:48,114 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.volcano-scheduler-queue.yaml
raycluster.ray.io/test-cluster-0 created
2023-10-28:13:08:00,791 INFO     [prototype.py:379] --- RayClusterAddCREvent 12.372737884521484 seconds ---
2023-10-28:13:08:00,811 INFO     [utils.py:320] Execute command: kubectl exec test-cluster-0-head-x4jdq -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:08:01,463	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:08:01,463	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:08:01,477	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'CPU': 3.0, 'node:<IP_ADDRESS>': 1.0, 'memory':<PHONE_NUMBER>.0, 'object_store_memory':<PHONE_NUMBER>.0, 'node:<IP_ADDRESS>': 1.0, 'node:__internal_head__': 1.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:08:33,836 INFO     [prototype.py:409] --- Cleanup RayCluster 31.040663957595825 seconds ---
2023-10-28:13:08:33,836 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.volcano-scheduler-queue.yaml --ignore-not-found=true
.2023-10-28:13:08:35,302 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:08:35,446 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.custom-head-service.yaml
raycluster.ray.io/raycluster-custom-head-service.yaml created
2023-10-28:13:08:37,795 INFO     [prototype.py:379] --- RayClusterAddCREvent 2.0454230308532715 seconds ---
2023-10-28:13:08:37,814 INFO     [utils.py:320] Execute command: kubectl exec raycluster-custom-head-service.yaml-head-86zvl -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:08:38,392	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:08:38,393	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
[2023-10-27 22:08:40,434 W 47 47] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
2023-10-27 22:08:41,441	INFO worker.py:1642 -- Connected to Ray cluster.
{'node:<IP_ADDRESS>': 1.0, 'node:__internal_head__': 1.0, 'CPU': 1.0, 'memory':<PHONE_NUMBER>.0, 'object_store_memory': 561701683.0}
2023-10-28:13:09:16,209 INFO     [prototype.py:409] --- Cleanup RayCluster 31.784589052200317 seconds ---
2023-10-28:13:09:16,209 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.custom-head-service.yaml --ignore-not-found=true
.2023-10-28:13:09:16,475 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:09:16,616 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.separate-ingress.yaml
raycluster.ray.io/raycluster-ingress created
ingress.networking.k8s.io/raycluster-ingress-head-ingress created
2023-10-28:13:09:18,998 INFO     [prototype.py:379] --- RayClusterAddCREvent 2.054396152496338 seconds ---
2023-10-28:13:09:19,17 INFO     [utils.py:320] Execute command: kubectl exec raycluster-ingress-head-jfv2x -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:09:19,627	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:09:19,628	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
[2023-10-27 22:09:21,641 W 47 47] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
[2023-10-27 22:09:22,642 W 47 47] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
2023-10-27 22:09:23,650	INFO worker.py:1642 -- Connected to Ray cluster.
{'CPU': 1.0, 'node:<IP_ADDRESS>': 1.0, 'node:__internal_head__': 1.0, 'memory':<PHONE_NUMBER>.0, 'object_store_memory':<PHONE_NUMBER>.0}
2023-10-28:13:09:56,771 INFO     [prototype.py:409] --- Cleanup RayCluster 30.745362758636475 seconds ---
2023-10-28:13:09:56,771 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.separate-ingress.yaml --ignore-not-found=true
ingress.networking.k8s.io "raycluster-ingress-head-ingress" deleted
.2023-10-28:13:09:56,935 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:09:57,72 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.volcano-scheduler.yaml
raycluster.ray.io/test-cluster-0 created
2023-10-28:13:09:58,423 INFO     [prototype.py:379] --- RayClusterAddCREvent 1.029797077178955 seconds ---
2023-10-28:13:09:58,442 INFO     [utils.py:320] Execute command: kubectl exec test-cluster-0-head-vqtwx -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:09:59,657	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:09:59,657	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
[2023-10-27 22:10:02,734 W 13 13] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
[2023-10-27 22:10:03,735 W 13 13] global_state_accessor.cc:401: Some processes that the driver needs to connect to have not registered with GCS, so retrying. Have you run 'ray start' on this node?
2023-10-27 22:10:04,744	INFO worker.py:1642 -- Connected to Ray cluster.
{'node:<IP_ADDRESS>': 1.0, 'node:__internal_head__': 1.0, 'CPU': 1.0, 'object_store_memory': 561404313.0, 'memory':<PHONE_NUMBER>.0}
2023-10-28:13:10:37,754 INFO     [prototype.py:409] --- Cleanup RayCluster 30.706429958343506 seconds ---
2023-10-28:13:10:37,754 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.volcano-scheduler.yaml --ignore-not-found=true
.2023-10-28:13:10:37,987 INFO     [utils.py:320] Execute command: kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://<IP_ADDRESS>:63998
CoreDNS is running at https://<IP_ADDRESS>:63998/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
2023-10-28:13:10:38,129 INFO     [utils.py:320] Execute command: kubectl apply -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.complete.yaml
raycluster.ray.io/raycluster-complete created
2023-10-28:13:10:49,784 INFO     [prototype.py:379] --- RayClusterAddCREvent 11.286155939102173 seconds ---
2023-10-28:13:10:49,804 INFO     [utils.py:320] Execute command: kubectl exec raycluster-complete-head-552hq -n default -- python -c "import ray; ray.init(); print(ray.cluster_resources())"
2023-10-27 22:10:50,426	INFO worker.py:1329 -- Using address <IP_ADDRESS>:6379 set in the environment variable RAY_ADDRESS
2023-10-27 22:10:50,426	INFO worker.py:1458 -- Connecting to existing Ray cluster at address: <IP_ADDRESS>:6379...
2023-10-27 22:10:50,442	INFO worker.py:1633 -- Connected to Ray cluster. View the dashboard at http://<IP_ADDRESS>:8265
{'CPU': 2.0, 'object_store_memory': 799140249.0, 'memory':<PHONE_NUMBER>.0, 'node:__internal_head__': 1.0, 'node:<IP_ADDRESS>': 1.0, 'node:<IP_ADDRESS>': 1.0}
2023-10-28:13:11:27,742 INFO     [prototype.py:409] --- Cleanup RayCluster 36.10106801986694 seconds ---
2023-10-28:13:11:27,743 INFO     [utils.py:320] Execute command: kubectl delete -n default -f /Users/ruian/Code/go/kuberay/ray-operator/config/samples/ray-cluster.complete.yaml --ignore-not-found=true
.
----------------------------------------------------------------------
Ran 14 tests in 986.095s

OK


The problems about SGD with momentum when learning rate changes

Hi, there.
Currently, SGD(Stochastic Gradient Descent) in MXNet is applied by:
rescaled_grad = lr * (rescale_grad * clip(grad, clip_gradient) + wd * weight)
state = momentum * state + rescaled_grad
weight = weight - state

I found there are two problems in SGD.

Loss of accuracy on float-point number
For the SGD with momentum, the variable state stores the gradients multiplied by learning rate. However, learning rate is usually a small value, such as 1e-3, enabling the state becomes smaller than the gradient. It may loss the accuracy.

The case when learning rate changes.
When learning rate changes, the variable state stores the gradients multiplied by old learning rate. It is wrong.

Solution:
We should update the implement of SGD with momentum, but we should consider the compatibility with old optimizer states.
rescaled_grad = rescale_grad * clip(grad, clip_gradient) + wd * weight
state = momentum * state + rescaled_grad
weight = weight - lr * state

Additional point here is that both TF and pyTorch use that proposed SGD logic, different than MXNet.
I am looking into making that change, I'm not sure if we need to be concerned about the backward compatibility for 2.0.
FYI @shengac since you did a legwork for refactoring optimizers for  2.0, @zhreshold and @eric-haibin-lin  for GluonCV and GluonNLP visibility.

I also agree that the compatibility of such change is not forced in the upcoming 2.0, and it's extremely rare that you would save/load the states using different mxnet versions.

Putting lr inside the momentum is how Momentum SGD or NAG looks like originally. But putting it outside momentum may be more robust to large batch training in warmup stage.


Automatically expand components in Object Inspector

I feel like this is a personal issue, but I'll suggest a solution in case anyone agrees with me.
I get annoyed when I load up my Duality project, and I have to expand all the components from the selected GameObject in the Object Inspector.
Perhaps there should be an option to auto expand the components, or just save their state, so if I expand one component, save, close and reopen the editor, the component is still expanded.

I'm backing this. It would be a nice little feature to have.

I'd like to give it a go, but I can't find the code that tracks the state of the components. @ilexp mind showing the line where it defines the state?

You mean whether or not a Component PropertyEditor is expanded? That's actually happening in the GroupedPropertyEditor base class, which is part of AdamsLair.WinForms. Probably the easiest way to approach this would be to modify the Object Inspector to load and save its gridExpandState as part of its SaveUserData / LoadUserData method plus some tweaking and potentially debugging.

Probably the easiest way to approach this would be to modify the Object Inspector to load and save its gridExpandState as part of its SaveUserData / LoadUserData method plus some tweaking and potentially debugging.

It's going to be a little bit more involved. Selections don't persist through a reload of the editor and so whatever SaveUserData saves will not be valid the next time the editor opens. We could delay actually setting the expand states on load until the first selection is made, but then it would have to be the same selection as before the editor closed in order for us to expand the relevant components.
As part of implementing this we could look into saving the current object selection whenever the editor closes - at which point we could then address this issue pretty simply.
Another option would be to save the expand state based on the component type somehow - so if I expand the Transform on one object, then the Transform component of any object I select will be expanded. I feel like this may a better general solution as it will in essence store the users preference for component expansion without tying it to a specific object selection and can be used throughout their use of the editor whereas this issue just handles re-opening the editor.

Another option would be to save the expand state based on the component type somehow - so if I expand the Transform on one object, then the Transform component of any object I select will be expanded. I feel like this may a better general solution as it will in essence store the users preference for component expansion without tying it to a specific object selection and can be used throughout their use of the editor whereas this issue just handles re-opening the editor.

Agreed, and assuming I understand you correctly, this is also how ExpandState already works 🙂
It's completely independent of selection and instead preserves the expanded paths in "editor hierarchy", built using editor IDs based on property name and type. No need to restore selection, and it will even remember expand states for editors that don't currently exist, until they exist again. What's missing is that this state is actually persisted, rather than being gone when restarting the editor or closing and re-opening the inspector window.

I have three ideas on how to make this work - which do you prefer?
...
I prefer option 2 as it has a precedent within Duality already and it fairly explicit about what it's doing.

I think both option 2 and 3 would be decent, no strong preference here. If you're up for it, option 2 it is 👍

Done by @deanljohnson. Closing this.


ERROR: could not insert 'v4l2loopback': Operation not permitted

Ubuntu 16.04 server with xfce-desktop
ERROR: could not insert 'v4l2loopback': Operation not permitted
when i use
sudo modprobe 'v4l2loopback'
I have privileged container
Host: Ubuntu 16.04 64 bits
Guest: Ubuntu 16.04 64 bits

Yep, that's normal, containers can't load kernel modules. That's kinda the point of containers, you're sharing your kernel with the host, so you can't have a container load kernel code.
You should do:
lxc config set CONTAINER-NAME linux.kernel_modules v4l2loopback

Which will have LXD load the kernel module for you at the host level.

with your code does't work too

I'm not familiar with v4l2loopback but some googling suggests that it's an out of tree kernel module which you need to build and load yourself. That part needs to be done as root on the host.
Once the module is loaded, it creates /dev/video* devices which you can then pass into the container using for example:
lxc config device add CONTAINER-NAME video0 unix-char path=/dev/video0


Adding if function (like the js version)

if (a,b,c) equivalent to a?b:c in js,, and to b if a else c in python

Thank you

Could this please make its way into a released version?

@laurence-hudson-mindfoundry Done: https://pypi.org/project/py_expression_eval/0.3.5/

Perfect, thanks!


As a Labs developer I'd like access to a CI environment...

...so that (CI) the Authorization module can be automatically tested every time there is a commit to the master branch
Part of Epic #29

Part of this will be implemented in https://github.com/nearform/labs-authorization/issues/285
I'm not sure if we need to provide "deployment scripting" for it. I fear it may end up too generic or too specific. Thoughts? @nherment @paolochiodi @mihaidma @dberesford

@p16 The initial discussion was that we'll set a private hand made CI system. Now that the project is public, tools like Travis are doing most of this work.
We should still wait for feedback before closing it.

maybe @dgonzalez knows more on this. /cc @p16

I think we can close this. We do have a CI in place


Add support for root config and TYPST_ROOT env

[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] I have searched the discussions and believe that my question is not already covered.

Feature Request

Typst has added support for TYPST_ROOT, which can be used as the root dir when using absolute path import, e.g. when I set it to $HOME/local/typst/, all #import "/my/absolute/path" will be translated into #import $HOME/local/typst/my/absolute/path. Optionally this can also be passed in as an editor config entry.
Detail:
https://github.com/typst/typst/blob/f16ac4d258bd0981506d01456bd5f43079e00fa5/cli/src/args.rs#L14-L15

I also encountered this bug and posted this issue, which is the last thing that blocks my migration to typst. Any chance we can fix it?

Even when setting rootPath, typst-lsp ist still reporting an error for me (file not found ...).
The file compiles fine though, with the --root option.
v 0.13.0


 And many software projects do not use it. And someday someone will  add actual support for versions in git since pretty much everyone need it and pretty much everyone need to distinguish those from free-form tags, and lax conventions are not a good way to build robust tooling.
But, the original go.mod proposal decided to enshrine this idiotic needless prefix in its syntax, including in the json outputted by go mod edit -fmt -json, even though this structure (unlike git) actually includes specific version objects, and (unlike git) does not need any v hack to mark that a version field is actually a version field (and that requires custom parsing on top of the json parsing just to pull out the actual semver version human use)
But, go modules can not even use this convention consistently,  we have now a Go object, that contains a something which is obviously a version, using a completely different syntax.
Please make up your mind and use regular syntax that does not require multiple levels of parser quirks

If you're going to make a proposal to change how Modules work, I'd suggest refraining from using words like "stupid" and "idiotic", and sentences like "please make up your mind".

And someday someone will add actual support for versions in git since pretty much everyone need it and pretty much everyone need to distinguish those from free-form tags

Can you back this up with any particular proposal or upstream plan to change git? Also note that Go supports many other source control managers, such as Mercurial and Subversion. So the specific features of Git aren't that important.
I'm also not sure I understand what the problem is with a v prefix. Sure, the prefix is not part of semver, but it clarifies that a string is a version and it's trivial to skip.
In particular, for large breaking changes like this one, you should open a proposal issue, detailing why the benefits of the proposed change greatly outweigh the downsides such as breaking backwards compatibility.

Can you back this up with any particular proposal or upstream plan to change git? Also note that Go supports many other source control managers, such as Mercurial and Subversion. So the specific features of Git aren't that important.

That's somewhat inconsistent with using a v prefix to mark versions, as  v only exists as an example in a git man page (not even as something in the git code) and this git man page example, that a lot of projects do not apply, has spread all over the go module syntax.

I'm also not sure I understand what the problem is with a v prefix. Sure, the prefix is not part of semver, but it clarifies that a string is a version and it's trivial to ski

go mod edit -fmt -json already outputs a field named Version. The field content does not need to clarify it is a version. But anyway if  go mod edit -fmt -json insists in putting versions in a Version field and then tagging a second time the version with a v inside the field, please use the same syntax for the Go version element.

Hi @nim-nim, I think you are making multiple points here, and I just wanted to add a quick comment (and in response to only some portions of what you have said).
When it comes to the v versus no v in VCS tags representing semver versions, I have certainly seen reasonable people have different opinions on what is "best", what is "encouraged", and even what is "permitted" under semver.
That said, I will share my personal understanding, which is that there is a distinction drawn between:

a "semantic version" (where a leading "v" is not part of a "semantic version"), vs.
the mechanism for encoding a "semantic version" into a VCS tag (where a leading "v" is allowed)

In other words, under that interpretation, a leading "v" in a VCS tags is a common and allowed way to encode a semantic version into a VCS tag (although not a universal choice, and not required).
One hint is the master branch for github.com/semver/semver contains this statement as part of an FAQ that was added a few years ago:

prefixing a semantic version with a "v" is a common way (in English) to indicate it is a version number. Abbreviating "version" as "v" is often seen with version control. Example: git tag v1.2.3 -m "Release version 1.2.3", in which case "v1.2.3" is a tag name and the semantic version is "1.2.3".

That certainly suggests to me that a leading v for a semver VCS tag is not only allowed, but also common, and not tied specifically to git.
However, that FAQ is only on the semver master as far as I am aware, and I have also seen reasonable people come to different conclusions even after reading that particular FAQ.
Setting aside for the moment what the best encoding might be, it seems reasonable to me that the go tool has made a choice about a canonical encoding of semantic versions into VCS tags in terms of how they are recorded in a go.mod file. For the go tool, that choice is a leading v for VCS tags representing semver versions.
However, note that tags like "1.2.3" (without a "v") can still be used, although they will not be treated as semantic versions by the 'go' tool. Instead, "go get<EMAIL_ADDRESS>(note no "v") and "go get foo@sometag" are treated as "module queries", and are often recorded as "pseudo-versions". To give a quick concrete example, if you do the following from within a module (note no "v" in "0.0.3"):
  $ go get<EMAIL_ADDRESS>
The result is recorded in the go.mod file as the following pseudo-version, which includes the date and commit hash:
  github.com/jondot/groundcontrol v0.0.0-20130702202810-bf0b23022af7

This behavior is covered in more detail here:
https://golang.org/cmd/go/#hdr-Module_queries
That said, a module should still be tagged in VCS with a leading "v" when released in order for the 'go' tool to be able to properly interpret the tag as a semantic version.
Regarding how that version information should be represented in the JSON returned by go mod edit -json, I guess I can see both sides of the argument, but it also seems to me to be a reasonable choice to have the VCS-encoded representation be the string representation returned in the JSON, including because that is how it is recorded in the go.mod file, and in general go mod edit -json does not "interpret" the information in a go.mod file as much as something like go list -m all.  As the edit in go mod edit -json might suggest, it is more about editing a go.mod file and less about interpreting a go.mod file. This is also suggested by the go mod edit documentation, such as:

go mod edit provides a command-line interface for editing go.mod, for use primarily by tools or scripts. It reads only go.mod; it does not look up information about the modules involved.

and a bit later (under the require discussion):

These flags are mainly for tools that understand the module graph. Users should prefer 'go get path@version' or 'go get path@none', which make other go.mod adjustments as needed to satisfy constraints imposed by other modules.

Separately, I think you might also be commenting on the new go directive in 1.12 that shows the version of the Go language used by the files within that module (e.g., from the 1.12 release notes)?  If so, I believe the go tool chain is explicit that its versioning scheme for Go releases does not follow semver. For example, go1.11 and go1.12 are referred to as "major releases" (which is not what semver would say).  I haven't gone back to double-check the history carefully, but I think Go might either pre-date semver, or they might both have come into existence within a year or two of each other, which says to me that it is not a shock that Go releases have not historically followed semver. There is always room for discussion about changing things going forward, but then there is a balancing act between consistency with what has been done historically, the cost of change, and the benefit of change.
Finally, I am just a member of the broader community myself, so please do not give too much weight to anything I have said here.  ;-)

Hi @thepudds

That certainly suggests to me that a leading v for a semver VCS tag is not only allowed, but also common, and not tied specifically to git.

And the example you quote points directly to the git history of the git project… v before versions didn't exist before git implemented tags without bothering to distinguish versions, and Linus Torvalds just prefixed his version tags with v to workaround this deficiency (and then gave it as example in a git man page).
And, as an aside, there is nothing in git that makes sure a tag starting with v is a version (counter examples are easy to find), or that it is a semver if it is a version (the v-prefixed tags Linus uses are most definitely not semvers since they have weird things like -rcX postfixes)
So the whole house of cards just rests on a misinterpretation of a quick and dirty hack by Linus Torvalds.
But, anyway, one should not need to write pages of scholarly text to justify syntax quirks, go.mod is not a centuries old human language, with entrenched practices, and lots of history behind each wart, it's brand new, the Go element in it has not even hit a stable release so far, there's no reason for it to adopt irregular syntax before the paint is even wet.
(As a human, I find your scholarly explanations interesting and well written. As a go.mod user, I just want the syntax to be regular, easy to use, and be done with it).

You haven't actually mentioned what about this is “inconsistent” — as far as I can see, cmd/go consistently uses the v prefix (on VCS tags, on the command line, and in go.mod files).
Module support has been available since Go 1.11 and there are many go.mod files already out in the world, so at this point any change to the format needs a much more compelling justification than aesthetic preferences. This bike-shed is already painted; we're not repainting it arbitrarily.

The version of the new Go element does not use the version syntax of existing go.mod elements.

That's because it indicates the version of the Go language in use, and versions of the Go language itself have never followed semantic versioning.
(If you believe that the language spec itself should be semantically versioned, that's a separate proposal to file — but, again, you'd need to show a compelling benefit based on more than just aesthetic preferences.)

@bcmills As much as I dislike resurrecting an angry thread, I wanted to ask if you thought it would be possible in the future for the go module solver to detect semantic versions that don't start with a v. I know it's a common convention, but it is unfortunately not a convention that we follow at my company.
The reason I ask is that dep did understand semantic versions without a v, thanks to its use of Masterminds/semver. Now in the process of switching to modules we are finding that we will either have to put up with pseudo-versions or switch our tag convention (which will inevitably break some of our internal processes).

@bvisness, if you have concrete examples of use of tags without the "v" prefix on existing Go projects, please open a new issue to discuss.
(Just to set expectations, though, I still think it's unlikely that we'll support them — what would happen if a repo had both a v1.0.0 tag and a 1.0.0 tag, but at different commits?)


Config

WIP
TODO

[ ] button_mapping
[ ] drag_lock_buttons
[x] scroll_button
[x] use enum instead of string

button_mapping and drag_lock_buttons is not a feature of libinput.
I decided not to implement it in this PR.


changing PICKITEM handling of invalid type

What current issue(s) does this address, or what feature is it adding?
audit of block testnet block 460271 showed a deviation in VMState. The root cause is a difference in handling types that are not of type Array or Map.
How did you solve this problem?
fix the default handling to match C#
How did you make sure your solution works?
block now passes with the same results as neo-cli
Are there any special changes in the code that we should be aware of?
Please check the following, if applicable:

[ ] Did you add any tests?
[X] Did you run make lint?
[ ] Did you run make test?
[X] Are you making a PR to a feature branch or development rather than master?
[X] Did you add an entry to CHANGELOG.rst? (if not, please do)

Coverage increased (+0.009%) to 85.104% when pulling 335d753027476128b40e436c0757b87044a54695 on ixje:fix-pickitem into 60c8f8c3dc5fec654ad3c0d7b36891dea2e208b7 on CityOfZion:development.


umount: can't umount /Users: Invalid argument mount.nfs: an incorrect mount option was specified

Dinghy 4.0.2 - Brew says it's stable branch but i'm pretty sure I reinstalled with latest from prerelease rather than the release trying to address #69
$ dinghy up
Starting the dinghy VM...
Stopping NFS daemon, this will require sudo
Password:
Starting NFS daemon, this will require sudo
Waiting for NFS daemon...
Mounting NFS /Users/lukebarton
umount: can't umount /Users: Invalid argument
mount.nfs: an incorrect mount option was specified
mount: mounting <IP_ADDRESS>:/Users/lukebarton on /Users/lukebarton failed: Device or resource busy
/usr/local/Cellar/dinghy/HEAD/cli/dinghy/ssh.rb:23:in run': Error executing command: ["sudo mount -t nfs <IP_ADDRESS>:/Users/lukebarton /Users/lukebarton -o nfsvers=3,udp,mountport=19321,port=19321,nolock,hard,intr"] (Ssh::CommandFailed)         from /usr/local/Cellar/dinghy/HEAD/cli/dinghy/machine.rb:76:in ssh'
from /usr/local/Cellar/dinghy/HEAD/cli/dinghy/machine.rb:72:in mount'         from /usr/local/Cellar/dinghy/HEAD/cli/cli.rb:181:in start_services'
from /usr/local/Cellar/dinghy/HEAD/cli/cli.rb:78:in up'         from /usr/local/Cellar/dinghy/HEAD/cli/thor/lib/thor/command.rb:27:in run'
from /usr/local/Cellar/dinghy/HEAD/cli/thor/lib/thor/invocation.rb:126:in invoke_command'         from /usr/local/Cellar/dinghy/HEAD/cli/thor/lib/thor.rb:359:in dispatch'
from /usr/local/Cellar/dinghy/HEAD/cli/thor/lib/thor/base.rb:440:in start'         from /usr/local/bin/_dinghy_command:6:in '

Similar to #69 , I would suspect a race condition here. There is boot2docker/docker-machine code to mount /Users using vbox file sharing during VM startup, I bet the dinghy code to unmount /Users so that we can mount the NFS share instead is running too soon.

I modified the command run by the startup script to only umount /Users if it shows up in /proc/mounts:
ssh("if [ $(grep -c '\/Users' /proc/mounts) -gt 0 ]; then sudo umount /Users; fi;")

I hit this issue every once in a while if the VM or my macbook goes to sleep. I usually have to dinghy halt && dinghy up a couple times to get it going again.
This PR would be nice: https://github.com/codekitchen/dinghy/pull/98

I've merged #98 to master, which will remove the umount: can't umount /Users: Invalid argument error, but that error was being ignored. I don't think the fatal error on the next line, mount.nfs: an incorrect mount option was specified, is fixed yet.

I'm going to close this since we haven't seen any reports in a long time. Hopefully it's fixed, if not we can open a new issue.


Bump console to<EMAIL_ADDRESS>

$subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6692178408

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6692178408
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6692514048

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6692514048
Status: failure

Will be onboarded with https://github.com/wso2/product-is/pull/17417


[FEATURE] Refresh ended lives on machine's idle state change

Is your feature request related to a problem? Please describe.
My laptop often runs into sleep mode if I lock it and take it out home. When I restore it, I'll find some lives are missing from the ended lives list. I must restart the browser to fetch the latest list.
Describe the solution you'd like
Chances are that holo-schedule detects the machine's idle state change through idle API and refresh, or clean up, the list of ended lives.
Describe alternatives you've considered
This problem is caused by browsers' pausing working when the machine runs into a 'locked' state. I've not found an alternative.
Additional context
This API requests an additional permission idle according to the docs and its compatibility status can be found here. For current users, this change may request them to permit the additional permission in order to use the latest version. Maybe we can build it into an option which requests permissions only when users toggle it on, but this issue may not fit into this situation.

Firefox seems to fail to play well with idle API and I cannot toggle the 'locked' state in it.

Maybe we can record the timestamp of every successful network request and perform a cleanup if there is some time since the last successful one. This alternative covers both cases where laptops are locked and where there is a bad network condition.
Cache invalidation is hard to handle and should be performed carefully.


Organizing queries with fragments breaks consuming code

I've been experimenting with apollo-graphql for a while in our Android app, adding additional queries to the app as the schema on our server evolves and grows. Overall it's very cool so far. However I think I may have recently identified a problem that undermines a core advantage of using Apollo over a more basic/naiive approach (e.g. hard-coded query strings and Retrofit) and might have an idea for a solution.
(Note: this issue appears to have some similarities to https://github.com/apollographql/apollo-android/issues/547)
Here's a SWAPI example to illustrate. Say you start with the following query to fetch a Person:
query personById {
  person(id: "cGVvcGxlOjE=") {
    id
    name
    gender
    height
  }
}

Which returns this response:
{
  "data": {
    "person": {
      "id": "cGVvcGxlOjE=",
      "name": "Luke Skywalker",
      "gender": "male",
      "height": 172
    }
  }
}

Apollo will generate a class like this for the response data:
public static class Person {
    final @Nonnull String __typename;
    final @Nonnull String id;
    final @Nullable String name;
    final @Nullable String gender;
    final @Nullable Integer height;
    // ...
}

So far, so good.
Now say you add another query, this time to fetch a list of people:
# New query!
query people {
  allPeople {
    people {
      id
      name
      gender
      height
    }
  }
}

query personById {
  person(id: "cGVvcGxlOjE=") {
    id
    name
    gender
    height
  }
}

Now, it turns out that the app is always interested in the same fields of a Person, regardless of whether they came from a list or individually.
Luckily GraphQL provides a nice way of factoring out duplicate fields like this - Fragments:
query personById {
  person(id: "cGVvcGxlOjE=") {
    ... PersonDetails
  }
}

query people {
  allPeople {
    people {
      ... PersonDetails
    }
  }
}

fragment PersonDetails on Person {
  id
  name
  gender
  height
}

This can be considered a refactoring of the queries. Their readability/maintainability has been improved without any effect on meaning or server response:
{
  "data": {
    "person": {
      "id": "cGVvcGxlOjE=",
      "name": "Luke Skywalker",
      "gender": "male",
      "height": 172
    }
  }
}

However, the picture isn't so pretty in the Android code. The introduction of fragments to the query ends up propagating to the generated response models in the form of a new .fragments() field. This breaks all of the code and tests that deal with mapping Person responses from Apollo:
public static class Person {
    final @Nonnull String __typename;
    private final @Nonnull Fragments fragments;
    // ...
    public static class Fragments {
        final @Nonnull PersonDetails personDetails;
        // ...
    }
}

public class PersonDetails implements GraphqlFragment {
  final @Nonnull String __typename;
  final @Nonnull String id;
  final @Nullable String name;
  final @Nullable String gender;
  final @Nullable Integer height;
  // ...
}

Worse still is the amount of additional boilerplate now required to unwrap fragments() and get to the data of interest, or populate them for creating test objects:
PersonByIdQuery.Person TEST_PERSON =
                  new PersonByIdQuery.Person("Person", "01234", "Luke Skywalker", "male", 172);

becomes:
PersonByIdQuery.Person TEST_PERSON =
                  new PersonByIdQuery.Person("Person",
                      new PersonByIdQuery.Person.Fragments(
                          new PersonDetails("PersonDetails", "01234", "Luke Skywalker", "male", 172)
                      ));

It seems to me that this will ultimately end up discouraging developers from actively maintaining and organising GraphQL queries in the project, just to avoid the breakages that will result. A basic, pre-Apollo approach using manually-created response models would not have this particular problem because the structure of the response is identical whether using fragments or not (obviously the manual approach has its own set of problems...)
Now I understand that mapping fragments onto separate objects was a conscious design choice and that there are cases where it's beneficial to parse the response into the original query structure, fragments included. For example, this idea of "data masking" from Relay, where different UI components might contribute their own fragments to a parent query. In that case it'd be convenient to have the response already parsed into its constituent Fragments to be passed back to the relevant UI components.
However, at present it's not possible to use fragments in their most basic sense - simply eliminating duplication in queries by grouping fields - without also requiring code changes.
Possible Solutions/Workarounds
Firstly I'm wondering if there's a workaround to this issue or something I haven't considered that anyone's discovered while using apollo-android in a production app?
Failing that, a possible solution I can see is to introduce a new field alongside .fragments() to access some kind of raw/minimal generated model for the query's response (.raw()?). Consuming code would then be robust to any reorganisation of queries that doesn't result in changes to the structure of the response (i.e. "refactors"). Perhaps generation of these raw response classes could also be toggled off via gradle configuration, to reduce method count when they're not needed.
I'm interested to hear any thoughts/discussion on this and can dedicate some time to a proof-of-concept PR if needed. Cheers

Thanks for the reporting and we are acknowledging the issue with fragments. To make it short the idea that we are exploring now is to merge fields defined in fragments to query and make fragment to be an interface, what means it suppose to fix the issue you described. But it will require some quite effort to make it happen, some work was already done in this direction. What we want for now is to release a stable 1.0 version and address this in next 1.1 release.

cc @martijnwalraven @digitalbuddha @BenSchwab

@sav007 thank you for the quick response and explanation. That sounds good.

Any news on this @digitalbuddha @sav007 ? I'm having exactly the same issue as described by @kiwiandroiddev , and I'll just stop using Fragments if this is the only way to use them.

@digitalbuddha Looks like this was closed but the issue still persists. Earlier comment points to that there was some work happening and it was being resolved, Is that no the case anymore?


Update serialization code to use new signatures

Supersedes https://github.com/Azure/autorest.csharp/pull/3434
Fixes https://github.com/Azure/autorest.csharp/issues/3181
Fixes https://github.com/Azure/autorest.csharp/issues/3406
Fixes https://github.com/Azure/autorest.csharp/issues/3353
Fixes https://github.com/Azure/autorest.csharp/issues/2351
Fixes https://github.com/Azure/azure-sdk-for-net/issues/16977
Description
This feature literally changes every model, therefore this PR will have a lot of files changed. Details of features included in this PR are listed below.
In summary, now a model's serialization class will look like this:
public partial class Foo : IUtf8JsonSerializable, IJsonModel<Foo>
{
    void IUtf8JsonSerializable(Utf8JsonWriter writer) => ((IJsonModel<Model>)this).Write(writer, ModelReaderWriterOptions.DefaultWireOptions);

    void IJsonModel<Foo>.Write(Utf8JsonWriter writer, ModelReaderWriterOptions options)
	{
		/* previous serialization implementation with some minor changes around Json and Wire format and raw data field */
	}

    Foo IJsonModel<Foo>.Read(ref Utf8JsonReader reader, ModelReaderWriterOptions options)
	{
		/* validates the format and call DeserializeFoo method to return a model as result */
	}

	internal static Foo DeserializeFoo(JsonElement element, ModelReaderWriterOptions options = null)
	{
		/* previous deserialization implementation with some minor changes around Json and Wire format and raw data field*/
	}

	BinaryData IModel<Foo>.Write(ModelReaderWriterOptions options)
    {
		/* call ModelReaderWriter to write the model into a Binrary data */
        return ModelReaderWriter.Write(this, options);
    }

	Foo IModel<Foo>.Read(BinaryData data, ModelReaderWriterOptions options)
	{
		/* validates the format and call DeserializeFoo method to return a model as result */
	}

	ModelReaderWriterFormat IModel<Foo>.GetWireFormat(ModelReaderWriterOptions options) => ModelReaderWriterFormat.Json; // return json when the model ONLY supports Json, return Xml when the model supports Xml

	/* ToRequestContent and FromResponse method unchanged. */
}

For serialization partial class

[x] All model have to implement IJsonModel<T> where T is itself

[x] If the model is struct, it needs to implement IJsonModel<object> as well, otherwise compilation will fail
[x] If the model is the "unknown" model in a discriminated set, it needs to implement IJsonModel<BaseType>

[x] A few new implementations have to be added to implement the IJsonModel<T> interface.
[x] The main logic of serializing lives in the explicit implementation of IJsonModel<T>, and IUtf8JsonSerializable just calls the implementation
[x] The DeserializeXXX method remains mostly unchanged, a new optional parameter ModelReaderWriterOptions options is added.
[x] When the model is abstract, it should have an attribute [DeserializationProxy(typeof(<the unknown internal type of it>))]
[x] Write readonly properties into JSON when options.Format == ModelReaderWriterFormat.Json
[x] Write the private "additional properties" into JSON when options.Format == ModelReaderWriterFormat.Json.
(question) should we change this to options.Format != ModelReaderWriterFormat.Wire?
[x] Read the private "additional properties" from JSON when options.Format == ModelReaderWriterFormat.Json
[x] every model needs to implement IUtf8JsonSerializable (this might change in the future)

We leave the ToRequestContent method and FromResponse method unchanged, until the corresponding APIs in Azure.Core get into its next stable release.
For the model itself

[x] Every model should have a new private "AdditionallProperties" property:

private Dictionary<string, BinaryData> _serializedAdditionalRawData; // when this model does not have any derived classes
protected internal Dictionary<string, BinaryData> _serializedAdditionalRawData; // when this model does not have any derived classes

[x] This private/internal property automatically becomes AdditionalProperties when the model gets the flag on. Therefore the _serializedAdditionalRawData change does not happen on the models that already have additional properties.
Since _serializedAdditionalRawData is just a private/protected internal version of AdditionalProperties, we will reuse the AdditionalProperties functionalities to implement it.
[x] The internal ctor for deserialization will always accept the additional properties property as a property, no matter it is private or public.
[x] The model factory entries do not add the private additional properties property as a parameter, but the model factory will include the public additional properties as a parameter. This is the same as previous behavior of model factory, no API changes of the existing entries are expected in model factory.
[x] Every model now should have a new internal empty ctor if it does not have public empty ctor (for deserialization).
[x] All models now will have both initialization ctor and serialization ctor (the internal ctor)
[x] Model factory now includes those models without a deserializer previously, meaning new methods will be added to the model factory.

Checklist
To ensure a quick review and merge, please ensure:

[ ] The PR has a understandable title and description explaining the why and what.
[ ] The PR is opened in draft if not ready for review yet.

If opened in draft, please allocate sufficient time (24 hours) after moving out of draft for review

[ ] The branch is recent enough to not have merge conflicts upon creation.

Ready to Land?

[ ] Build is completely green

Submissions with test failures require tracking issue and approval of a CODEOWNER

[ ] At least one +1 review by a CODEOWNER
[ ] All -1 reviews are confirmed resolved by the reviewer

Override/Marking reviews stale must be discussed with CODEOWNERS first

LGTM, just a simple comment. I approved it.
And now, we will always generate serialize/deserialize method for models ignore its usage?


service tree merge error

Coverage decreased (-0.002%) to 17.393% when pulling 91c03289f6357cc478a6e02457e31ff4284b2aea on feature/servicetree into e84f95ac26818add462200ce95a7b2fa497c44b1 on master.


create dedicated tempGuid column & DB migration

No content

Issue created by BlueBubbles Bot in the BlueBubbles commnity on behalf of @zach in #android-requests.

not needed anymore


gamma function: add constants for 128-bit reals.

Is there a platform now where we can actually test D code with 128-bit reals?

I am working on an AArch64 port of LDC. This platform uses 128-bit reals.

Cool.
There has been some discussion by maintainers of std.math about doing, essentially, a re-write to cleanse it of unnecessary C-isms, over-use of inline assembler, etc. I had hoped this could be done before adding 128-bit real and cent support, to avoid duplication of effort.
But, on the other hand, already having a working 128-bit platform to test against would be really helpful...

It seems easy to add support for 128-bit reals. Only this PR, #4036 and an implementation of exp() are required.

At a minimum, there are many other places in std.math where 80-bit constants need to be upgraded to 128-bits (assuming you want the extra precision to actually be useful, anyway).
Also, a number of functions in std.math only really support real: using 128-bit floating-point operations for 32- and 64-bit values is likely to be unacceptably slow. Overloads will need to be created.
I also expect that once we're able to actually test the 128-bit code paths in std.math, problems will be revealed; I have personally fixed at least one such issue (which I only noticed because the code didn't quite work right for 80-bit, either).
Having said all that, a lot of my concern over 128-bit real actually has do with my previous investigation of what it would take to get imbExtended working. That's a far less sane format than the ieeeQuadruple one that you're working on, though, so I don't think you should be too worried.

My first goal is to compile all modules on AArch64. Without it I cannot run the test suite automatically (which will certainly reveal tons of other errors).

My first goal is to compile all modules on AArch64. Without it I cannot run the test suite automatically (which will certainly reveal tons of other errors).

Makes sense. Just remember that the std.math test suite, itself, will likely be somewhat broken until the constants have been expanded for 128-bit some places.

Can't really check the constants.. Anyhow LGTM
As this is related to math - @9il  ?

Can't really check the constants..

We should add sqrt2p constant to std.math, thought (hex representation from wolfram alpha is required).

Ping @redstar

Closing due to author inactivity. @redstar please ping me if you want to reopen.

New pull request: https://github.com/dlang/phobos/pull/5947


offline mode to install

Describe the bug
My environment with unstructrued was unable to access the network， So I need install it offline, and preload model file In the cache
but I found tables_agent(tables.py) load model with hard code "microsoft/table-transformer-structure-recognition" param,

cloud pls use params, so i can use local loaded model
thx
Expected behavior
install offline
Screenshots
If applicable, add screenshots to help explain your problem.
Environment Info
Please run python scripts/collect_env.py and paste the output here.
This will help us understand more about the environment in which the bug occurred.
Additional context
Add any other context about the problem here.

This looks like a nice option to add.

This will work without internet connectivity, but you'll need to move the model files to your HF local cache, such as .cache/huggingface/hub/. You can control the cache location with the HF_HOME or HF_HUB_CACHE environment variables (see here for details). The function in unstructured-inference that downloads the model is below for reference.
https://github.com/Unstructured-IO/unstructured-inference/blob/64cd41c37fe4535702b3be9c6b58f380ca4c7edd/unstructured_inference/utils.py#L188-L195

If you have already obtained the model files locally, but still require online verification with huggingface.co, you can set the environment variables as follows:
export HF_HUB_OFFLINE=1


Problems when closing huge Solution

Hi there
in our team we started using your analyzer for some months now it we really like it. Sadly we noticed that in our solution, everytime we close it, Visual Studio freezes and does not respond for some time. After some investigation we found out that the time it takes to close is related to how often we build, after a full working day it sometimes takes more than 5 minutes till the solution actually is closed.
Therefore we started analyzing what the cause could be, and we managed to reproduce it in a sample project. It looks like the StyleCop.Analyzers have something to do with it.
A sample project can be found here: https://github.com/huserben/AnalyzerPerformanceTest
It's just a solution with 130 empty class libraries and all of them have the StyleCop Analyzer installed. Just rebuild 3 times and then close the solution to recreate the issue.
We also have other analyzers installed (SonarLint, FxCop), and I tested those in the test solution as well, there we did not experience any issue while closing the solution...
Regards and thanks for your help

Hi @huserben,
Thank you so much for taking the time to report this and especially create that sample repository. I'll investigate a bit and see who the best people to fix it are!

@huserben I wasn't able to reproduce the issue as described, but I continued to try various things. I finally reproduced this using the following steps:

Clone the reference project
Open the solution in Visual Studio
Restore NuGet packages
Restart Visual Studio and open the solution again
Fix all SA1633 violations in solution
Fix all SA1028 violations in solution
Rebuild 3 times
Close Visual Studio

@mavasani When profiling the close operation, I found substantial time was spent in Microsoft.CodeAnalysis.Diagnostics.AnalyzerManager.ClearAnalyzerState(valuetype System.Collections.Immutable.ImmutableArray1). Most of this was accessing Keysand callingToImmutableArrayinClearAnalysisScopeState`.

Thanks @sharwell. Let me prepare a PR to clean up that code - avoid unncessary ToImmutableArray invocation and replace concurrent dictionary with a normal dictionary + lock.

Hi,
Cool, I saw the exact same behavior (VS taking a long time before closing & slowing down after using it for a while).
Just wondering - I saw a fix for this has been merged into Roslyn. Is there an (easy) way to use this in an existing installation of Visual Studio, or do we need to wait for the next VS update to incorporate this fix?
Thx!
Frederik.

Unfortunately, you'll just have to wait for an update.


Cannot create event because of bugged field

Please check that this issue hasn't been reported before.

[X] I searched previous Bug Reports and didn't find any similar reports.

Expected Behavior
Creating events works and you select which calendar they're in
Current behaviour
Events can't be created as shown in this image

JavaScript console output
There is no view toggle in settings
Steps to reproduce
Create new event by clicking on date
Calendar information
2 Caldav from same url
1 ICS
Which Operating Systems are you using?

[ ] macOS
[ ] Windows
[X] Linux
[X] Android
[ ] iOS

Obsidian Version
1.5.3
Full Calendar Plugin Version
0.10.7
Checks

[X] I have tried it with all other plugins disabled and the error still occurs.

Possible solution
No response
Comments
No response

Hello, I was also blocked by this. This is currently only working on read for caldav.  You can't add or modify events from caldav calendars. I hope it will be added

That's disappointing, as I can't use a calendar app without the ability to create events. Hopefully that will change.

i added an ical calendar as read only, but added a second calendar that i use for daily notes.
now i can add events to my daily notes but still see the events from my personal calendar. perhaps not what you need but it's the next best thing for me.

Well for my part, I was able to synchronize my notes via ical as well. With those modules:

iCal
APIRequest

So basically, you scan your vault with iCal and it generates a file, for my side .ics; Then you can PUSH your file to a CalDav server such as Radicale. I also use a custom module for triggering https://github.com/Blondwolf/obsidian-file-trigger and finally I use a NodeRED server to create a requestable API.
This is a bit cumbersome and can surely be optimized but at least it update my FullCalendar agenda automatically by pulling then this calendar from my CalDav server. Would be nice to have someone integrate them all together.
Cheers

Version: 1.6.7
OS: Ubuntu 24.04
It does the same. I cannot select my CalDav calendar. And also cannot modify existing events as it gives me Read-only events cannot be modified.

Yes it does the same. Except "just" that you can see your events in calendar module, synchronize it with your phone and any calendar app.
Also somehow with my method, it doesn't make sens to be able to modify your events from the events. Think about it: you're scanning your notes to generate events. You don't create event to generate notes. So even if you were able to modify them, it will be override by the next event generation.
Personnaly I use two calendar, 1 for syncing with my notes, 1 for simple events.


LLVM-11 support

The experimental support for LLVM-11 does not fully work.

Verifying a program (i.e., that defines 'main') seems to work (but has not been tested extensively)
Verifying a function annotated with #[test] does not work: KLEE crashes during its preprocessing phase.
Almost all of the regression tests rely on #[test] and do not work.

The error seen with #[test] is as follows. (This is obtained by using cargo verify -vvvvvv then cutting and pasting the command used to invoke KLEE.)
[Long list of errors that look like the next three lines]
Instruction does not dominate all uses!
  %.i3348 = phi i64 [ <badref>, <badref> ], [ %.i3682, %2236 ], [ %.i3344, %2227 ]
  %2378 = insertelement <4 x i64> %.upto24216, i64 %.i3348, i32 3

PHI nodes not grouped at top of basic block!
  %.i0345 = phi i64 [ <badref>, <badref> ], [ %.i0679, %2236 ], [ %.i0341, %2227 ]
label %2370
in function _ZN4test9run_tests17h3c40b1ee8455d4dbE

LLVM ERROR: Broken function found, compilation aborted!
/usr/lib/llvm-11/lib/libLLVM-11.so.1(_ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x1f)[0x7fae470b242f]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(_ZN4llvm3sys17RunSignalHandlersEv+0x22)[0x7fae470b0762]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(+0xaa6905)[0x7fae470b2905]
/lib/x86_64-linux-gnu/libc.so.6(+0x46210)[0x7fae4612d210]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7fae4612d18b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7fae4610c859]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(+0x9f9c28)[0x7fae47005c28]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(+0x9f9a48)[0x7fae47005a48]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(+0xc2295f)[0x7fae4722e95f]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE+0x3b9)[0x7fae471c1579]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE+0x33)[0x7fae471c6b23]
/usr/lib/llvm-11/lib/libLLVM-11.so.1(_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE+0x3e0)[0x7fae471c1b90]
klee(+0xa34bf)[0x564c44ede4bf]
klee(+0x49e4c)[0x564c44e84e4c]
klee(+0x2cc9c)[0x564c44e67c9c]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fae4610e0b3]
klee(+0x3c16e)[0x564c44e7716e]
Aborted

It is not obvious what is going wrong here.

Note: @fshaked pointed out that, instead of using -vvvvvv and then grovelling through the output produced, it is better to use the flag '--script commands.sh' to generate a file containing exactly the shell commands used by cargo-verify. You can then execute the commands in commands.sh one by one, modify the commands, etc. to try to debug.

Note that rustc is now based on LLVM12 and that KLEE has a LLVM12 pull request: 1389

Fyi, I've tried integrating LLVM12 by also cherry-picking the changes, you've applied to inkwell in this fork on top of the latest commit from inkwell master branch...
The docker images building process terminates well, including rvt-patch-llvm compilation but so far I can only see stack traces when running klee against generated bitcodes.
By checking out the aforementioned branch supporting LLVM12, I'll get more promising results hopefully.


Refactor template for AudioContext and OfflineAudioContext

Now that OfflineAudioContext does not really some Option workaround (cf. #44), it should be more simple to share BaseAudioContext stuff and reduce duplicated code between AudioContext and OfflineAudioContext templates.

fixed with #46


Next 33734/blue green pipeline2

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Added Trapping Rain Water.cpp

Description
Related Issue: #2436
Added Trapping Rain Water.cpp in Leetcode Problems
I am a participant of Hacktoberfest 2021. Kindly review and merge this branch or please label it as 'hacktoberfest-accepted' . Thank You !!
Question : Given n non-negative integers representing an elevation map where the width of each bar is 1, compute how much water it can trap after raining.
Solution : The solution is in C++ language.
Type of change

[X] Addition of a new file

Checklist:

[X] My code follows the style guidelines(Clean Code) of this project
[X] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works

I have raised a new issue (#2436). Kindly review and merge this branch or please label it as 'hacktoberfest-accepted' . Thank You !!

DS ALGO PROJECTS ARE NOT COUNTED FOR HACKTOBERFEST21 so i will close this PR


"01.pem" created when not wanted.

When running OpenSSL ca, an "01.pem" (or ".pem") is being created in "outdir" even though "out" (the name of the desired output file) is being specified.  This appears redundant.

Version used:
OpenSSL 1.1.1k  25 Mar 2021, from Git client for Windows x64.

Marking as inactive, to be closed at the end of 3.4 dev, barring further input


djs/Scale first point by 2 before fft

Before scaling

After scaling

@deepanshs I approved PR, but I'm not sure why the 3.9 and 3.10 builds are failing. This should probably be addressed in another PR. I can look into why this week.


Screen Pump

Describe your problem here.
Like, how can I make the freeplay screen pump to the song bpm like in IC
https://user-images.githubusercontent.com/101379721/179360113-89bb233f-2f5b-40b2-af56-7a76421fcf83.mp4
?
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
FreeplayState.hx

BUMP NOT PUMP!!


Best practices in regex matching and editing the entire component content

I have an anti-caps feature in my plugin and I am looking to know what is the best practice to replace excessive caps in an Adventure component.
Current code (chatMessage is a legacy string):
final String[] words = chatMessage.split(" ");

boolean capsAllowed = false;
boolean whitelisted = false;

for (int i = 0; i < words.length; i++) {
	final String word = words[i];

       // Some logic code here

	if (!whitelisted) {
		if (!capsAllowed) {
			final char firstChar = word.charAt(0);

			words[i] = firstChar + word.toLowerCase().substring(1);

		} else
			words[i] = word.toLowerCase();

		capsAllowed = !words[i].endsWith(".") && !words[i].endsWith("!") && !words[i].endsWith("?");
	}

	whitelisted = false;
}

// I need to set the component here.
String.join(" ", words);

My current solution involves converting the component to legacy and then back which destroys hover/click events and to my knowledge has a unnecessarily performance hit.
Any advice would be greatly appreciated!
Matej

A couple of rewrites and you can get there


Contest Overall All Submission 데이터 테이블 구현

Describe the problem and solution
Contest Overall 페이지의
All Submission 데이터 테이블을 구현합니다
Notion task
TAS-
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines
[X] Check that there isn't already an issue

Connected Notion task
 Contest Overall All Submission 데이터 테이블 구현


git-extras completion does not work [probably a documentation issue]

I have git-extras installed and the commands work, e.g. git summary prints the summary. git-extras plugin is enabled on oh-my-zsh but git summ<TAB> does not autocomplete, neither do other commands from git-extras. In the plugin notes there is a phrase:

If you have both zsh and git installed, you need to make sure that the zsh-provided _git takes precedence.

I believe this is exactly my problem, unfortunately I could not find any way of ensuring the precedence. So I do not believe this is a bug on OMZ part, but instead—a documentation issue. The docs should either say how to make sure or link to the relevant docs.
What I tried (without success):
$ rpm -q --filesbypkg git-core| grep comp

---
git-core                  /etc/bash_completion.d/git-prompt.sh
git-core                  /etc/bash_completion.d/git.sh
git-core                  /etc/zsh_completion.d
git-core                  /etc/zsh_completion.d/_git
git-core                  /usr/lib/git/mergetools/codecompare
git-core                  /usr/share/tcsh/git.complete

---
$ sudo rm  /etc/bash_completion.d/git-prompt.sh /etc/bash_completion.d/git.sh /etc/zsh_completion.d/_git

Fixed in be65adc6c364e842307c7b4e8da30f0162b629f0


Manifest thumbnail is read from manifest if available.

This could be improved so that if the thumbnail URI is an IIIF image service, then a thumbnail of the required size should be requested; but it seems sensible this way too.

:+1: to this PR from me! Thanks Hashimmm

Thanks. I only meant to include one commit. :$ The second one adds description tooltips to the manifest titles in the manifest list panel.

I would like to merge this, but not with the tooltips. However, I understand why you would want them. In fact, there's a whole issue about it at #287.

Sure, I made a separate branch and another pull request for this ( #533 ).
Thanks.


CI failure: Test Panicked

Copy-paste the link of the CI build where that test failure has happen.
https://github.com/spidernet-io/spiderpool/runs/7719298483?check_suite_focus=true#step:17:2634

#515