No changes written to file

I'm running ksync in a DIND local cluster scenario.
I've configured everything, ran ksync init then ksync create --name coordinator -l app=mlbench,component=coordinator --context local --reload=false /home/zenon/DEV/epfl/mlbench/mlbench/coordinator/ /app/code/
(I tried with reload=true as well, but the application should reload by itself anyways)
sync watch works as well , debug output:

ksync watch --log-level debug
DEBU[0000] initializing kubernetes client                context=local
DEBU[0000] kubernetes client created                     host="http://localhost:8080"
DEBU[0000] watching for updates                          ContainerName= LocalPath=/home/zenon/DEV/epfl/mlbench/mlbench/coordinator/ LocalReadOnly=false Name=coordinator Namespace=default Pod= Reload=false RemotePath=/app/code/ RemoteReadOnly=false Selector="app=mlbench,component=coordinator"
DEBU[0000] cleaning background daemon
DEBU[0000] starting syncthing                            args="[/home/zenon/.ksync/bin/syncthing -gui-address localhost:8384 -gui-apikey ksync -home /home/zenon/.ksync/syncthing -no-browser]" cmd=/home/zenon/.ksync/bin/syncthing
INFO[0000] listening                                     bind=<IP_ADDRESS> port=40322
DEBU[0000] RESTY 2018/07/06 16:14:26 ERROR [Get http://localhost:8384/rest/system/config: dial tcp [::1]:8384: connect: connection refused] Attempt [1]
DEBU[0000] new event                                     deleted=false name=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb status=Running type=ADDED
INFO[0000] new pod detected                              pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
DEBU[0000] added service                                 ID=9b01da22760a145e12ecc4d03c02d64f4fdcb0804f81c36c96cc7be10612e07d Name=mlbench-coordinator NodeName=kube-node-2 PodName=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb
DEBU[0000] checking to see if radar is ready             nodeName=kube-node-2
DEBU[0000] found pod name                                Namespace=kube-system RadarPort=40321 SyncthingAPI=8384 SyncthingListener=22000 nodeName=kube-node-2 podName=ksync-dmp8h
DEBU[0000] found pod                                     nodeName=kube-node-2 podName=ksync-dmp8h status=Running
DEBU[0000] starting tunnel                               LocalPort=42425 Namespace=kube-system Out= PodName=ksync-dmp8h RemotePort=40321 url="http://localhost:8080/api/v1/namespaces/kube-system/pods/ksync-dmp8h/portforward"
DEBU[0000] tunnel running                                LocalPort=42425 Namespace=kube-system Out="Forwarding from <IP_ADDRESS>:42425 -> 40321\nForwarding from [::1]:42425 -> 40321\n" PodName=ksync-dmp8h RemotePort=40321
DEBU[0000] [monitor] 16:14:26 INFO: Starting syncthing   name=syncthing
DEBU[0000] [start] 16:14:26 INFO: Generating ECDSA key and certificate for syncthing...  name=syncthing
DEBU[0000] [CV3OM] 16:14:26 INFO: syncthing v0.14.48 "Dysprosium Dragonfly" (go1.10.2 linux-amd64<EMAIL_ADDRESS>2018-05-14 06:53:06 UTC  name=syncthing
DEBU[0000] [CV3OM] 16:14:26 INFO: My ID: CV3OM2R-UEGMGXC-DZXVZTE-AZTPJMX-ELTLMGM-7G47SQH-LQHAYYL-FWLOPQH  name=syncthing
DEBU[0000] checking to see if radar is ready             nodeName=kube-node-2
DEBU[0000] found pod name                                Namespace=kube-system RadarPort=40321 SyncthingAPI=8384 SyncthingListener=22000 nodeName=kube-node-2 podName=ksync-dmp8h
DEBU[0000] found pod                                     nodeName=kube-node-2 podName=ksync-dmp8h status=Running
DEBU[0000] starting tunnel                               LocalPort=39651 Namespace=kube-system Out= PodName=ksync-dmp8h RemotePort=8384 url="http://localhost:8080/api/v1/namespaces/kube-system/pods/ksync-dmp8h/portforward"
DEBU[0000] tunnel running                                LocalPort=39651 Namespace=kube-system Out="Forwarding from <IP_ADDRESS>:39651 -> 8384\nForwarding from [::1]:39651 -> 8384\n" PodName=ksync-dmp8h RemotePort=8384
DEBU[0000] checking to see if radar is ready             nodeName=kube-node-2
DEBU[0000] found pod name                                Namespace=kube-system RadarPort=40321 SyncthingAPI=8384 SyncthingListener=22000 nodeName=kube-node-2 podName=ksync-dmp8h
DEBU[0000] found pod                                     nodeName=kube-node-2 podName=ksync-dmp8h status=Running
DEBU[0000] starting tunnel                               LocalPort=35347 Namespace=kube-system Out= PodName=ksync-dmp8h RemotePort=22000 url="http://localhost:8080/api/v1/namespaces/kube-system/pods/ksync-dmp8h/portforward"
DEBU[0000] tunnel running                                LocalPort=35347 Namespace=kube-system Out="Forwarding from <IP_ADDRESS>:35347 -> 22000\nForwarding from [::1]:35347 -> 22000\n" PodName=ksync-dmp8h RemotePort=22000
DEBU[0000] RESTY 2018/07/06 16:14:26 ERROR [Get http://localhost:8384/rest/system/config: dial tcp [::1]:8384: connect: connection refused] Attempt [1]
DEBU[0001] RESTY 2018/07/06 16:14:27 ERROR [Get http://localhost:8384/rest/system/config: dial tcp [::1]:8384: connect: connection refused] Attempt [2]
DEBU[0001] [CV3OM] 16:14:27 INFO: Single thread SHA256 performance is 235 MB/s using minio/sha256-simd (174 MB/s using crypto/sha256).  name=syncthing
DEBU[0001] [CV3OM] 16:14:27 INFO: Archiving a copy of old config file format at: /home/zenon/.ksync/syncthing/config.xml.v26  name=syncthing
DEBU[0001] RESTY 2018/07/06 16:14:27 ERROR [Get http://localhost:8384/rest/system/config: dial tcp [::1]:8384: connect: connection refused] Attempt [2]
DEBU[0002] [CV3OM] 16:14:28 INFO: Hashing performance is 206.12 MB/s  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Updating database schema version from 0 to 2...  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Updated symlink type for 0 index entries and added 0 invalid files to global list  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Finished updating database schema version from 0 to 2  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Overall send rate is unlimited, receive rate is unlimited  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Rate limits do not apply to LAN connections  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: TCP listener ([::]:22000) starting  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Device CV3OM2R-UEGMGXC-DZXVZTE-AZTPJMX-ELTLMGM-7G47SQH-LQHAYYL-FWLOPQH is "ZENON-PC" at [dynamic]  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Loading HTTPS certificate: open /home/zenon/.ksync/syncthing/https-cert.pem: no such file or directory  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Creating new HTTPS certificate  name=syncthing
DEBU[0002] RESTY 2018/07/06 16:14:28 ERROR [Get http://localhost:8384/rest/system/config: dial tcp [::1]:8384: connect: connection refused] Attempt [3]
DEBU[0002] [CV3OM] 16:14:28 INFO: GUI and API listening on <IP_ADDRESS>:8384  name=syncthing
DEBU[0002] [CV3OM] 16:14:28 INFO: Access the GUI via the following URL: http://localhost:8384/  name=syncthing
DEBU[0003] [CV3OM] 16:14:29 INFO: Adding folder "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" (coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb)  name=syncthing
DEBU[0003] [CV3OM] 16:14:29 INFO: No stored folder metadata for "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb": recalculating  name=syncthing
DEBU[0003] [CV3OM] 16:14:29 INFO: Ready to synchronize "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" (coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb) (readwrite)  name=syncthing
DEBU[0003] [CV3OM] 16:14:29 INFO: Completed initial scan of readwrite folder "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" (coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb)  name=syncthing
INFO[0003] updating                                      pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
INFO[0003] updating                                      pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
WARN[0003] {"alloc":9305640,"connectionServiceStatus":{"tcp://<IP_ADDRESS>:22000":{"lanAddresses":["tcp://<IP_ADDRESS>:22000"],"wanAddresses":["tcp://<IP_ADDRESS>:22000"]}},"cpuPercent":0,"goroutines":42,"myID":"CV3OM2R-UEGMGXC-DZXVZTE-AZTPJMX-ELTLMGM-7G47SQH-LQHAYYL-FWLOPQH","pathSeparator":"/","startTime":"2018-07-06T16:14:26.610132871+02:00","sys":40863992,"tilde":"/home/zenon","uptime":2,"urVersionMax":3}
INFO[0003] syncthing listening                           port=8384 syncthing=localhost
WARN[0003] {"alloc":9332440,"connectionServiceStatus":{"tcp://<IP_ADDRESS>:22000":{"lanAddresses":["tcp://<IP_ADDRESS>:22000"],"wanAddresses":["tcp://<IP_ADDRESS>:22000"]}},"cpuPercent":0,"goroutines":42,"myID":"CV3OM2R-UEGMGXC-DZXVZTE-AZTPJMX-ELTLMGM-7G47SQH-LQHAYYL-FWLOPQH","pathSeparator":"/","startTime":"2018-07-06T16:14:26.610132871+02:00","sys":40863992,"tilde":"/home/zenon","uptime":2,"urVersionMax":3}
DEBU[0003] restarting local syncthing
DEBU[0003] [CV3OM] 16:14:29 INFO: Restarting             name=syncthing
INFO[0003] finished unary call with code OK              grpc.code=OK grpc.method=RestartSyncthing grpc.service=proto.ksync.Ksync grpc.start_time="2018-07-06T16:14:29+02:00" grpc.time_ns=718597 peer.address="<IP_ADDRESS>:40044" span.kind=server system=grpc
DEBU[0003] [CV3OM] 16:14:29 INFO: Exiting                name=syncthing
INFO[0003] folder sync running                           pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
DEBU[0003] RESTY 2018/07/06 16:14:29 ERROR [unexpected EOF] Attempt [1]
DEBU[0003] [monitor] 16:14:29 INFO: Syncthing exited: exit status 3  name=syncthing
DEBU[0004] [monitor] 16:14:30 INFO: Starting syncthing   name=syncthing
DEBU[0004] RESTY 2018/07/06 16:14:30 ERROR [Get http://localhost:8384/rest/events?since=14: dial tcp [::1]:8384: connect: connection refused] Attempt [2]
DEBU[0004] [CV3OM] 16:14:30 INFO: syncthing v0.14.48 "Dysprosium Dragonfly" (go1.10.2 linux-amd64<EMAIL_ADDRESS>2018-05-14 06:53:06 UTC  name=syncthing
DEBU[0004] [CV3OM] 16:14:30 INFO: My ID: CV3OM2R-UEGMGXC-DZXVZTE-AZTPJMX-ELTLMGM-7G47SQH-LQHAYYL-FWLOPQH  name=syncthing
DEBU[0005] [CV3OM] 16:14:31 INFO: Single thread SHA256 performance is 235 MB/s using minio/sha256-simd (177 MB/s using crypto/sha256).  name=syncthing
DEBU[0006] RESTY 2018/07/06 16:14:32 ERROR [Get http://localhost:8384/rest/events?since=14: dial tcp [::1]:8384: connect: connection refused] Attempt [3]
DEBU[0006] [CV3OM] 16:14:32 INFO: Hashing performance is 198.40 MB/s  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Ready to synchronize "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" (coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb) (readwrite)  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Overall send rate is unlimited, receive rate is unlimited  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Rate limits do not apply to LAN connections  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: TCP listener ([::]:22000) starting  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Device CV3OM2R-UEGMGXC-DZXVZTE-AZTPJMX-ELTLMGM-7G47SQH-LQHAYYL-FWLOPQH is "ZENON-PC" at [dynamic]  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Device HCF2NSO-ALBGN5R-4E7LGV2-CNNTDYA-EKWTBCK-44IA6QK-2IAXMHI-LK2KVQL is "virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" at [tcp://<IP_ADDRESS>:35347]  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Completed initial scan of readwrite folder "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" (coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb)  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: GUI and API listening on <IP_ADDRESS>:8384  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Access the GUI via the following URL: http://localhost:8384/  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Established secure connection to HCF2NSO-ALBGN5R-4E7LGV2-CNNTDYA-EKWTBCK-44IA6QK-2IAXMHI-LK2KVQL at <IP_ADDRESS>:36380-<IP_ADDRESS>:35347/tcp-client (TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305)  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Device HCF2NSO-ALBGN5R-4E7LGV2-CNNTDYA-EKWTBCK-44IA6QK-2IAXMHI-LK2KVQL client is "syncthing v0.14.48" named "ksync-dmp8h" at <IP_ADDRESS>:36380-<IP_ADDRESS>:35347/tcp-client  name=syncthing
DEBU[0006] [CV3OM] 16:14:32 INFO: Device HCF2NSO-ALBGN5R-4E7LGV2-CNNTDYA-EKWTBCK-44IA6QK-2IAXMHI-LK2KVQL folder "coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb" (coordinator-virtuous-fly-mlbench-coordinator-699747c74b-2gnmb) has a new index ID (0x84FED7B664DF945D)  name=syncthing
INFO[0007] updating                                      pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
INFO[0007] update complete                               pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
INFO[0018] updating                                      pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
INFO[0018] update complete                               pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
INFO[0018] updating                                      pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator
INFO[0018] update complete                               pod=virtuous-fly-mlbench-coordinator-699747c74b-2gnmb spec=coordinator

so everything seems to work ok.
ksync get outputs:
   NAME              LOCAL            REMOTE      STATUS                           POD                          CONTAINER   +-------------+---------------------+------------+----------+---------------------------------------------------+-----------+   coordinator   mlbench/coordinator   /app/code/                                                                 watching   virtuous-fly-mlbench-coordinator-699747c74b-2gnmb
but if I connect to the pod and look at the content of a file I changed locally, it still has the original content. I.e. changes don't take effect even though ksync registers the change and writes "update complete"
Changing the remote file (kubectl exec -it virtuous-fly-mlbench-coordinator-699747c74b-2gnmb -- /bin/sh and editing it) doesn't get registered in ksync watch or syncthing.
Since everything seems to be working, except it's not working, I'm a bit at a loss and any help would be appreciated.

@Panaetius it can be confusing to debug at this point, so let me clear a few pieces up.
I'm not entirely sure that the -l some=something,somethingelse=somewhere (i.e. with the comma separator) works in ksync, but admittedly I haven't tried it. Usually I specify multiple selectors with multiple -l flags. I only mention it because I suspect the problem may be that the entire selector comment was ingested and processed as a single string, meaning nothing would match. Try recreating your ksync create and see what happens.
The "pod reload" triggering/complete messages are actually local, generated from events emitted by the remote syncthing event stream. Since the selectors aren't applied when looking for a pod (after it is first found) that would explain why the reloads were triggered when things changed locally, but the file was not changed.
As a side note, reload just determines whether the remote container is rolled after any file change.
See if that change fixes things for you and we can work from there. If not, you might also try specifying a pod without the selectors and see if that works (as a test).

@timfallmk Regarding the -l some=something,somethingelse=somewhere, that indeed does not work, it just picks the first selector specified and ignores the rest. I already tried it with multiple -l flags yesterday, that didn't work either.
I just tried
ksync create -p jumpy-rodent-mlbench-coordinator-6bc59dd765-5hglw --local-read-only --name coordinator --context=local /home/zenon/DEV/epfl/mlbench/mlbench/coordinator/ /app/code/
ksync create -p jumpy-rodent-mlbench-coordinator-6bc59dd765-5hglw --local-read-only --name coordinator /home/zenon/DEV/epfl/mlbench/mlbench/coordinator/ /app/code/
ksync create -p jumpy-rodent-mlbench-coordinator-6bc59dd765-5hglw --local-read-only --name coordinator --context=local --reload=false /home/zenon/DEV/epfl/mlbench/mlbench/coordinator/ /app/code/
with the same results (not working).

$ kubectl get pods
NAME                                                READY     STATUS    RESTARTS   AGE
jumpy-rodent-mlbench-coordinator-6bc59dd765-5hglw   1/1       Running   0          20h
jumpy-rodent-mlbench-experiment-94ff54498-rjjwt     1/1       Running   0          20h
jumpy-rodent-mlbench-experiment-94ff54498-t8pcw     1/1       Running   0          20h

I checked all 3 containers to see if there's files there, just to make sure it didn't pick the wrong folder. None of them contain my changed file (it's called views.py and I did find / -iname "views.py" on all three containers. Only the one on coordinator (the one I'm trying to use ksync with) exists and that one still has the original content.

If you visit the syncthing web UI (I believe it defaults to 8384), does it show the correct target destinations?

It looks fine, the paths are correct and it lists modifications to the file correctly.

Are there any additional logs I could provide to help solve the issue?
And thank you for the support!

That's very odd. The only thing I can think of, other than the above, is a file permissions issue. Even that should trigger an error though.
@grampelberg ideas?

ksync makes some assumptions about how the FS is laid out under the covers. What FS driver are you using and where is it putting files? I'm wondering if DIND is the issue.

I use https://github.com/kubernetes-sigs/kubeadm-dind-cluster if that helps, v1.10.
It's all Overlay 2 and stored in /var/lib/docker and /dind/docker on the host and in the kubernetes nodes, respectively.
 We're only mounting /var/lib/docker in (https://github.com/vapor-ware/ksync/blob/master/pkg/ksync/cluster/daemon_set.go#L78).
The solution to this should be adding a configuration option to install that lets you configure that path. @timfallmk want to take a look?

It would probably make more sense to detect what the docker daemon is pointed at, but I'll look into it.

Great, looking forward to it being configurable.
For now I'll try just getting the dind cluster to use the regular path, I think changing the /dind/docker path on line 89 of https://github.com/kubernetes-sigs/kubeadm-dind-cluster/blob/5e48a6377148b1e0d1b3d8c4609a140c7b00b57b/image/rundocker should do. Haven't had a chance to test it yet. I'll report back once I tried it, in case there's some other issues with kubeadm DIND.

@Panaetius It might take some time before I get to it. In the meantime, changing the configured volume mount @grampelberg listed and recompiling should let you use it.

Good call @timfallmk, that's just a daemonset. Just use kubectl to edit the daemonset to the right path and it should (tm) work (hopefully).

That would also work.

@timfallmk
I just tried again with the newest version and sadly it still doesn't seem to work.
I connected to the syncthing service running on the node and enabled (most) debug log settings. The path is now correct. E.g.:

http: GET "/rest/system/log?since=2018-07-12T09%3A20%3A28.292799137Z": status 200, 716 bytes in 0.09 ms
2018-07-12 09:20:32 read IndexUpdate message
2018-07-12 09:20:32 queueing IndexUpdate(2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2, concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx, 1 files)
2018-07-12 09:20:32 Index update (in): 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 / "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx": 1 files
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx Update(2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2, [1])
2018-07-12 09:20:32 insert; folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" device=2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 File{Name:"main/views.py", Sequence:57, Permissions:0644, ModTime:2018-07-12 09:20:31.590656547 +0000 UTC, Version:{[{J7HTQSK 9} {2AMD5RW 6}]}, Length:900, Deleted:false, Invalid:false, NoPermissions:false, BlockSize:131072, Blocks:[Block{0/900/1563634915/51e3f54be12434506e24c60afc79af3f73349cb2e3ed24131c08cffb9fe22e54}]}
2018-07-12 09:20:32 update global; folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" device=2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 file="main/views.py" version={[{J7HTQSK 9} {2AMD5RW 6}]} invalid=false
2018-07-12 09:20:32 new global after update: {{{[{J7HTQSK 9} {2AMD5RW 6}]}, 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2}, {{[{J7HTQSK 9} {2AMD5RW 5}]}, 7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4}}
2018-07-12 09:20:32 log 87 RemoteIndexUpdated map[items:1 version:57 device:2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx]
2018-07-12 09:20:32 Device 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 sent an index update for "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" with 1 items
2018-07-12 09:20:32 http: GET "/rest/events?since=81": status 200, 282 bytes in 58212.87 ms
2018-07-12 09:20:32 http: GET "/rest/events?since=81": status 200, 282 bytes in 31767.09 ms
2018-07-12 09:20:32 size.go:85 basic /var/syncthing/config/ Usage . {90801340416<PHONE_NUMBER>60}
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 pulling (ignoresChanged=false)
2018-07-12 09:20:32 log 88 StateChanged map[to:syncing from:idle duration:58.237464443 folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx]
2018-07-12 09:20:32 Folder "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" is now syncing
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 copiers: 2 pullerPendingKiB: 32768
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx WithNeed(7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4)
2018-07-12 09:20:32 need folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" device=7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4 name="main/views.py" need=true have=true invalid=false haveV={[{J7HTQSK 9} {2AMD5RW 5}]} globalV={[{J7HTQSK 9} {2AMD5RW 6}]} globalDev=2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2
2018-07-12 09:20:32 stats.DeviceStatisticsReference.WasSeen: 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2
2018-07-12 09:20:32 open: open /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/main/.syncthing.views.py.tmp: no such file or directory
2018-07-12 09:20:32 log 89 ItemStarted map[type:file action:update folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx item:main/views.py]
2018-07-12 09:20:32 Started syncing "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" / "main/views.py" (update file)
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 need file main/views.py; copy 1, reused 0
2018-07-12 09:20:32 progress emitter: registering concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx main/views.py
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 sharedPullerState concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx main/views.py pullNeeded start -> 1
2018-07-12 09:20:32 model@0xc420096fc0 REQ(out): 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2: "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" / "main/views.py" o=0 s=900 h=51e3f54be12434506e24c60afc79af3f73349cb2e3ed24131c08cffb9fe22e54 wh=5d332ce3 ft=false
2018-07-12 09:20:32 wrote 148 bytes on the wire (2 bytes length, 4 bytes header, 4 bytes message length, 138 bytes message (132 uncompressed)), err=
2018-07-12 09:20:32 read Response message
2018-07-12 09:20:32 sharedPullerState concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx main/views.py pullNeeded done -> 0
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 http: GET "/rest/events?since=82": status 200, 461 bytes in 0.15 ms
2018-07-12 09:20:32 http: GET "/assets/img/favicon-sync.png": status 304, 0 bytes in 0.05 ms
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 closing main/views.py
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Sending main/views.py non-remove
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Sending main/views.py non-remove
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Ignoring main/.syncthing.views.py.tmp
2018-07-12 09:20:32 log 90 ItemFinished map[action:update folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx item:main/views.py error: type:file]
2018-07-12 09:20:32 progress emitter: deregistering concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx main/views.py
2018-07-12 09:20:32 aggregator/"coordinator-s-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" (coordinator-s-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx): Skipping path we modified: main/views.py
2018-07-12 09:20:32 aggregator/"coordinator-s-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" (coordinator-s-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx): Skipping path we modified: main/views.py
2018-07-12 09:20:32 Finished syncing "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" / "main/views.py" (update file): Success
2018-07-12 09:20:32 http: GET "/rest/events?since=84": status 200, 246 bytes in 45.53 ms
2018-07-12 09:20:32 aggregator/"concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" (concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx): Skipping path we modified: main/views.py
2018-07-12 09:20:32 aggregator/"concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" (concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx): Creating eventDir at: main
2018-07-12 09:20:32 aggregator/"concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" (concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx): Tracking (type non-remove): main/views.py
2018-07-12 09:20:32 aggregator/"concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" (concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx): Resetting notifyTimer to 1s
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Sending main/views.py non-remove
2018-07-12 09:20:32 basic /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/ Watch: Sending main/views.py non-remove
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx Update(7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4, [1])
2018-07-12 09:20:32 removing sequence; folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" sequence=39 main/views.py
2018-07-12 09:20:32 adding sequence; folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" sequence=40 main/views.py
2018-07-12 09:20:32 insert; folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" device=7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4 File{Name:"main/views.py", Sequence:40, Permissions:0644, ModTime:2018-07-12 09:20:31.590656547 +0000 UTC, Version:{[{J7HTQSK 9} {2AMD5RW 6}]}, Length:900, Deleted:false, Invalid:false, NoPermissions:false, BlockSize:131072, Blocks:[Block{0/900/1563634915/51e3f54be12434506e24c60afc79af3f73349cb2e3ed24131c08cffb9fe22e54}]}
2018-07-12 09:20:32 update global; folder="concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" device=7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4 file="main/views.py" version={[{J7HTQSK 9} {2AMD5RW 6}]} invalid=false
2018-07-12 09:20:32 new global after update: {{{[{J7HTQSK 9} {2AMD5RW 6}]}, 7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4}, {{[{J7HTQSK 9} {2AMD5RW 6}]}, 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2}}
2018-07-12 09:20:32 log 91 LocalIndexUpdated map[version:40 folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx items:1 filenames:[main/views.py]]
2018-07-12 09:20:32 log 92 RemoteChangeDetected map[path:main/views.py modifiedBy:2AMD5RW folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx folderID:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx label:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx action:modified type:file]
2018-07-12 09:20:32 stats.FolderStatisticsReference.ReceivedFile: concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx main/views.py
2018-07-12 09:20:32 http: GET "/rest/events?since=85": status 200, 234 bytes in 36.62 ms
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx WithHaveSequence(40)
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 changed 1
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 copiers: 2 pullerPendingKiB: 32768
2018-07-12 09:20:32 Remote change detected in folder "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx": modified file main/views.py
2018-07-12 09:20:32 Sending indexes for concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx to 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 at <IP_ADDRESS>:22000-<IP_ADDRESS>:36182/tcp-server: 1 files (<127 bytes) (last batch)
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx WithNeed(7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4)
2018-07-12 09:20:32 wrote 211 bytes on the wire (2 bytes length, 4 bytes header, 4 bytes message length, 201 bytes message (201 uncompressed)), err=
2018-07-12 09:20:32 sendReceiveFolder/concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx@0xc4200b9680 changed 0
2018-07-12 09:20:32 log 93 StateChanged map[folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx to:idle from:syncing duration:0.103977409]
2018-07-12 09:20:32 Folder "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" is now idle
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx WithNeedTruncated(7777777-777777N-7777777-777777N-7777777-777777N-7777777-77777Q4)
2018-07-12 09:20:32 progress emitter: bytes completed for concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx: 0
2018-07-12 09:20:32 model@0xc420096fc0 NeedSize("concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx"): {0 0 0 0 0 0 []}
2018-07-12 09:20:32 log 94 FolderSummary map[summary:map[localDirectories:5 localDeleted:0 localBytes:143268 needDirectories:0 needDeletes:0 needBytes:0 invalid: globalSymlinks:0 globalDeleted:0 needFiles:0 inSyncBytes:143268 stateChanged:2018-07-12 09:20:32.701780007 +0000 UTC m=+202.825836477 version:97 pullErrors:0 globalDirectories:5 needSymlinks:0 localFiles:21 localSymlinks:0 inSyncFiles:21 state:idle sequence:97 ignorePatterns:false globalFiles:21 globalBytes:143268] folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx]
2018-07-12 09:20:32 stats.DeviceStatisticsReference.WasSeen: 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2
2018-07-12 09:20:32 Summary for folder "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" is map[localDirectories:5 inSyncBytes:143268 globalFiles:21 globalSymlinks:0 globalDirectories:5 localDeleted:0 pullErrors:0 needDirectories:0 globalDeleted:0 needFiles:0 localSymlinks:0 inSyncFiles:21 sequence:97 localBytes:143268 needDeletes:0 needBytes:0 version:97 localFiles:21 needSymlinks:0 globalBytes:143268 state:idle]
2018-07-12 09:20:32 concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx WithNeedTruncated(2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2)
2018-07-12 09:20:32 model@0xc420096fc0 Completion(2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2, "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx"): 100.000000 (0 / 143268 = 0.000000)
2018-07-12 09:20:32 log 95 FolderCompletion map[device:2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 completion:100 needBytes:0 needItems:0 globalBytes:143268 needDeletes:0 folder:concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx]
2018-07-12 09:20:32 Completion for folder "concrete-termite-eponymous-turtle-mlbench-coordinator-555447fdc7-q2ncx" on device 2AMD5RW-GVNA4P3-CGBIJH2-G6OECKY-73OEILD-LDEN5TX-PDJS7AQ-2DBL7Q2 is 100%

We can see it saying that it's updating the file /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/main/views.py so the new docker path of "/dind/docker" is working correctly.
But if I connect to the kube node that the pod is running on, while the path is correct, the file is still unchanged (last modified yesterday, content is unchanged as well):

ls -alh /dind/docker/overlay2/2c73c728b130d6eb4d923838a400ad6b51df9947946a0a7cd4c0e882efcd05c3/merged/app/code/main/
total 48K
drwxr-xr-x 1 root root 4.0K Jul 11 10:45 .
drwxr-xr-x 1 root root 4.0K Jul 11 20:11 ..
-rw-r--r-- 1 root root    0 Jul  6 12:02 init.py
drwxr-xr-x 1 root root 4.0K Jul 12 08:37 pycache
-rw-r--r-- 1 root root   63 Jul  6 12:02 admin.py
-rw-r--r-- 1 root root   83 Jul  6 12:02 apps.py
drwxr-xr-x 2 root root 4.0K Jul  6 12:02 migrations
-rw-r--r-- 1 root root   57 Jul  6 12:02 models.py
-rw-r--r-- 1 root root   60 Jul  6 12:02 tests.py
-rw-r--r-- 1 root root  169 Jul 11 10:45 urls.py
-rwxrwxrwx 1 root root  795 Jul 11 20:11 views.py

I can edit the file manually in the shell on the node, which does change the application code, i.e. the change persists on the node.

This has to be something with how they're handling volume mounts. What happens if you roll the container after making a change (and seeing that change go through in the logs)?

What happens if you roll the container

Do you mean rollback? Or what do you mean with "roll"?
I can try it first thing tomorrow morning.

Sorry, "restart". It's possible the filesystem changes are not being picked up after the file is changed on disk. Other than that, I'm kinda out of ideas.

Nothing in particular happens when I restart the container.
But I did some more digging and think I figures it out (well, the problem, not a solution).
I just changed the code file locally with one change (Adding test_source), waited for it to propagate, then connected to a shell inside the container and changed the same file, but with a different string (Test2), then I just ran grep for those strings in the /var/lib/docker/ folder of my host system.
Results:
[zenon@ZENON-PC docker]$ sudo grep -r --include views.py 'Welcome!test_source!' overlay2/18e66711fe3374c4a29dcc1564692ae28bfb47e4daed4288714a39b350fab71b/diff/var/lib/docker/overlay2/e700956c8a5208553f4c03555f240cf992910f684bc063ac65333e6c3dbf48e6/merged/app/code/main/views.py:    return HttpResponse("Welcome!test_source!") overlay2/18e66711fe3374c4a29dcc1564692ae28bfb47e4daed4288714a39b350fab71b/merged/var/lib/docker/overlay2/e700956c8a5208553f4c03555f240cf992910f684bc063ac65333e6c3dbf48e6/merged/app/code/main/views.py:    return HttpResponse("Welcome!test_source!")  [zenon@ZENON-PC docker]$ sudo grep -r --include views.py 'Welcome!Test2' volumes/kubeadm-dind-kube-node-1/_data/docker/overlay2/e700956c8a5208553f4c03555f240cf992910f684bc063ac65333e6c3dbf48e6/diff/app/code/main/views.py:    return HttpResponse("Welcome!Test2")
So syncthing changes the respective entry in the overlay2 folder of the docker contents in the host overlay 2 folder, whereas changing the content directly changes the  content of the overlay2 folder in the kubeadm-dind-kube-node-1 volume.
Hope this helps solve the problem.

In that case it seems like something else is using the default graph driver path. I'll root around and see if I can find it, unless @grampelberg knows already.

Hello. I'm having this exact issue. Local files trigger "reloads" but nothing is change in the remote. And if I modify a file in the remote, that does not trigger nothing and I don't get the file locally.
I have a default installation of kubernetes and my "poc" is using nginx with a simple html website.


openstack completer

Command
openstack
Description
I would love to see support for the openstack cli to be added. It is used by many private and clouds. I know it's a lot, but theres no support for anything other than bash, which makes it incredibly hard to use in nushell.
Homepage
openstack.org
Documentation
https://docs.openstack.org/python-openstackclient/latest/
Repository
https://github.com/openstack/python-openstackclient
Complexity
XL
Anything else?
https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/complete.html
In addition to the python-openstackclient the python-neutronclient would be very much needed as well, as the OS client is very modular and neutron is very important for the use of the cli.

Polar

related:  https://github.com/openstack/cliff/blob/master/cliff/complete.py


Refactoring ocean temperature - plus other new ocean diagnostics and preprocessors.

After several excellent contributions from various members of the community, an initial draft of the ocean diagnostic section is now available for review.
This pull request contains the following new features:

A fix for latitudes values greater than 90 degrees in HadGEM2-ES and HadGEM2-CC, described in issue #378
A new folder called ocean in the esmvaltool/diag_scripts. This folder contains four new diagnostics and one toolkit. The four new diagnostics are:

diagnostic_timeseries.py: Produces a simple time series plot
diagnostic_maps.py: Produces a 2D map of the region
diagnostic_transects.py: Produces an image of the transects
diagnostic_profiles.py: Produces a profile plot

The toolkit is diagnostic_tools.py, and it contains several useful functions used by the four ocean diagnostics.

Two new name list files are also included in this pull request:

esmvaltool/namelists/namelist_OceanBGC.yml:
esmvaltool/namelists/namelist_OceanPhysics.yml
These two name lists are not meant to be fully functioning nameliests to evaluate all facets of ocean models.  Instead, they should serve as an example of how to produce various kinds of figure of the ocean. They also make use of several preprocessors. Note that these diagnostics are not a direct replacement for the previous marine metrics (namelist_GlobalOcean.yml), described in issue #350.

In addition, several new preprocessors were needed in order to evaluate of CMIP5 models marine components.

average_volume
time_average
extract_volume
extract_trajectory
extract_slice
depth_integration

Finally, two new vertical interpolation schemes were added:

nearest_extrap
linear_extrap

@ledm Could you please push your REFACTORING_OceanTemperature to the ESMValTool repository and open a PR from there?
This way we can use the automatic tests from circle-ci.
Thanks!

Re-opened another pull request here: https://github.com/ESMValGroup/ESMValTool/pull/438


Deprecated Atom APIs will be removed June 1st

Atom will no longer load this package after June 1st without changes. There are a couple deprecations on the latest version (0.5.1) of this package:

Use activationCommands instead of activationEvents in your package.json Commands should be grouped by selector as follows: json "activationCommands": { "atom-workspace": ["foo:bar", "foo:baz"], "atom-text-editor": ["foo:quux"] }
This method is deprecated on the model layer. Use TextEditorElement::pixelPositionForBufferPosition instead
Use TextEditor::moveRight() instead

Visit https://gist.github.com/benogle/6d09e295c84b717ef9b4 and search for your package name to see up-to-date deprecations.
If this package has been replaced by another package or functionality in core, please reply with this information.
See https://github.com/atom/atom/issues/6867 for more info. Thanks!

Thanks for letting us know. Merged https://github.com/avendael/atomic-emacs/pull/59 and released 0.6.0.

I think we've solved everything here, so closing.


PDS: Add restore lib and test basic restore functionality.

Summary: Add restore lib and test basic restore functionality.
Logs - https://aetos.pwx.purestorage.com/resultSet/testSetID/273020/testCaseID/1495367
Latest log - https://aetos.pwx.purestorage.com/resultSet/testSetID/273199

As part of next PR, will address the error logging covering wider scope.
cc: @lsrinivas-pure


Fix undefined assetOverwrites variable

Closes #48.
The assumption about the usage of assetOvewrites might be incorrect. Please correct the pull request if this variable is expected to be defined at this point.

Thanks for fixing this :)


support proxy authentication

I'm working behind a proxy which requires authentication and Mapbox Studio isn't working in that context.
(I'd love to just change the network settings but I'm working on a project for the Dept of Education in New South Wales, Australia and don't have immediate control over the network.)
In issue #860 (general proxy support), there is a comment saying "no proxy authentication yet." Since it seems this may be a sufficiently different case than the bare minimum of getting Mapbox Studio working through a non-authentication-required proxy, I'll make a different issue for it here, but feel free to close this if you think it is too close to being a duplicate.
Thanks!
cc: @Rustuma @Lilith-Palmer

@techieshark can you share some details about the proxy?
Do you know if it's squid or another software?
Are there PAC scripts being used?
How do you authenticate with other programs?

@BergWerkGIS according to whatismyproxy.com, "The header: HTTP_X_BLUECOAT_VIA is present", which seems to indicate we're behind a Blue Coat system. We don't use any proxy auto config / PAC scripts. I'm on a Mac, and have the proxy address + my authentication info in the Network Preferences.
Chrome uses the system's proxy config automatically. Command line programs I use like git have their own proxy configuration settings in their config files.

@techieshark thanks for providing additional information.
I'm planning on looking into the proxy issue again as soon as I've set up one for testing (hopefully next week).
Although I'm not confident that Blue Coat will be supported any time soon.
Details about Blue Coat were already discussed in #860.
Short version: it seems to have an internal white list of user agents.
If a software doesn't have a proper user agent set, that software gets blocked.
Not all node modules we use, set proper user agents.
Once I've identified all modules that don't set proper user agents I will contact the maintainers and try to get the changes upstream.
Might take some time.

@BergWerkGIS Interesting! Thank you for the info and for looking into this. Let me know should you need anything else from my end.

Bluecoat NTLM here. Mapbox Studio is logging a 500 error. Chrome/Chromium should have no issues authenticating with our proxy (I'm posting from a Chrome session now).
Notes:
Users here do not know the proxy address, username or password. The proxy address is done via DHCP/PAC/Group Policy and credentials are grabbed using SSPI. Again, Chrome/Chromium should have no issues with this as it queries the Windows proxy config.

@voltagex sorry I missed your post, could you try to just start Mabox Studio (don't do anything there) and then go to Chrome http://localhost:3000 ?
Any success?

@voltagex
Would be great if you could try this proxy-enabled version of Mapbox Studio linked here:
https://github.com/mapbox/mapbox-studio/issues/860#issuecomment-65508867
and tell me, if it works.
But beware, it is quite old (Dec 2014).
At the moment, I don't see any chance to support Bluecoat in the near future as one of its features seems to be blocking programs with uncommon user agents.
Mapbox Studio relies on a lot of node modules that are out of our control and don't provide any or invalid (as seen by Bluecoat) user agent strings.

Will try it tomorrow. It's not even connecting to the proxy at this point
but I don't believe user agent filtering is in effect where I am.

@BergWerkGIS this "works" - as in it hits my local chaining proxy (which is handling NTLM auth) and gets to the OAuth screen. Is it possible to provide a newer build? It goes back to the first Connect screen after I hit "Allow".

@BergWerkGIS bummer, but thanks for looking into this.
Bluecoat's strategy seems better at preventing people from using legitimate (but non standard) tools like Mapbox Studio rather than enforcing real security - it sounds like if a malicious program wanted to get through the firewall, it'd just change its user agent from 'Terminator Virus' or whatever to 'Internet Explorer'.

My solution is to use Fiddler to chain to the corporate proxy. It can
rewrite user agents if need be.
On 14 May 2015 11:11 am, "Peter W"<EMAIL_ADDRESS>wrote:

@BergWerkGIS https://github.com/BergWerkGIS bummer, but thanks for
looking into this.
Bluecoat's strategy seems better at preventing people from using
legitimate (but non standard) tools like Mapbox Studio rather than
enforcing real security - it sounds like if a malicious program wanted to
get through the firewall, it'd just change its user agent from 'Terminator
Virus' or whatever to 'Internet Explorer'.
—
Reply to this email directly or view it on GitHub
https://github.com/mapbox/mapbox-studio/issues/1315#issuecomment-101871240
.

Is it possible to provide a newer build?

Yes, I'm planning on another deep dive into the proxy problem during the coming weeks.
But, at the moment I'm not able to specify an exact time frame.

@techieshark @voltagex
Would be great, if you could give the latest proxy test version a spin and let me know how it goes.
Downloads and instructions https://github.com/mapbox/mapbox-studio/issues/860#issuecomment-112001682

@BergWerkGIS Thanks for your work on this! I've been bike touring in New Zealand and don't have access to the system behind the proxy, but I'll talk to @Lilith-Palmer to see if anyone else can test this out over there.

I pasted the information above to the command line and received the response below. I am operating behind a proxy (and use a pac) and cannot get mapbox studio to open. I have connected offline and then tried to connect but still no luck. Usually an etimedout error.
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
C:\Users\mkjxh>SET HTTP_PROXY=http://<IP_ADDRESS>:8080
C:\Users\mkjxh>SET HTTPS_PROXY=http://<IP_ADDRESS>:8080
C:\Users\mkjxh>cd C:\Program Files\mapbox-studio
C:\Program Files\mapbox-studio>mapbox-studio.exe
C:\Program Files\mapbox-studio>[1456:1210/155026:INFO:renderer_main.cc(211)] Ren
derer process started
[10180:1210/155029:INFO:CONSOLE(0)] <Thu, 10 Dec 2015 20:50:29 GMT> Mapbox Studi
o @ http://localhost:3000/
[10288:1210/155050:INFO:renderer_main.cc(211)] Renderer process started
[7464:1210/155150:INFO:renderer_main.cc(211)] Renderer process started
[10180:1210/155151:INFO:CONSOLE(5)] "Uncaught ReferenceError: $ is not defined",
source: https://assets-cdn.github.com/assets/frameworks-b7c9523d93e6cd0235a0a8b
7d1a691f7909d3834cd9c9caa7b1674ba508b644c.js (5)
[10180:1210/155151:INFO:CONSOLE(1)] "Uncaught ReferenceError: jQuery is not defi
ned", source: https://assets-cdn.github.com/assets/github-b42430c121e963c7370a99
d029c78f3c2fdc9b75b73adf817f07802e33988e81.js (1)

@jahoover proxy configuration via PAC is not supported and I don't think I can offer any help here as it seems your configuration lets some requests pass and others not: some assets are loaded but not the jquery module.
You might want to try the new Mapbox Studio that runs in your browser and doesn't need a local install:
https://www.mapbox.com/mapbox-studio/

Wilhelm,
Thank you for your suggestion. My understanding however is that to add shape files and integrate with a base map for use in Tableau I need to use Mapbox Studio Classic. If that is not the case, do you have an article that describes providing this functionality via Mapbox studio?
jah
From: Wilhelm Berg<EMAIL_ADDRESS>Sent: Monday, December 14, 2015 2:32 AM
To: mapbox/mapbox-studio-classic
Cc: Hoover, John A.
Subject: [EXTERNAL] Re: [mapbox-studio-classic] support proxy authentication (#1315)
@jahooverhttps://github.com/jahoover proxy configuration via PAC is not supported and I don't think I can offer any help here as it seems your configuration lets some requests pass and others not: some assets are loaded but not the jquery module.
You might want to try the new Mapbox Studio that runs in your browser and doesn't need a local install:
https://www.mapbox.com/mapbox-studio/
—
Reply to this email directly or view it on GitHubhttps://github.com/mapbox/mapbox-studio-classic/issues/1315#issuecomment-164368594.

Hey guys, I am working with cordova phonegap. I want to use mapbox in my app. when I added the cordova plugin for mapbox it gives me the error in console i.e Error: exec proxy not found for :: Mapbox :: show. have anyone idea about this. please help me.

@Anuj-786 That error is not related to this issue.


Spigot build from source

I tried to run a Spigot server with
docker run -d \
-v /home/hannes/dataSpigot:/data \
-p 25565:25565 \
--restart unless-stopped \
--name spigot \
-e BUILD_FROM_SOURCE=true \
-e TYPE=SPIGOT \
-e EULA=TRUE itzg/minecraft-server:multiarch

but I run into
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 2 1000 1000 4096 Dec 13 09:20 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:22 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:22 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:22 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:22 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:22 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:22 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:23 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:23 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory
[init] Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x 3 1000 1000 4096 Dec 13 09:23 /data'
[init] Resolved version given LATEST into 1.16.4
[init] Resolving type given SPIGOT
[init] Building Spigot 1.16.4 from source, might take a while, get some coffee
[init] ....[init] done
mv: cannot stat 'spigot-*.jar': No such file or directory
[init] ERR failed to build Spigot
Loading BuildTools version: git-BuildTools-7fe9375-122 (#122)
Java Version: Java 8
Current Path: /data/temp/.
Could not successfully run git. Please ensure it is installed and functioning. Cannot run program "git" (in directory "."): error=2, No such file or directory

Isn't build from source supported anymore, or how to make it work ?

I haven't used that feature in a while and it apparently it has some missing tool requirements, like git. I'm curious, is there a particular reason you want to build from source?

Currently I've no hard reason to do so, it's just to be more "secure" and mostly because this switch is there.
I want only point out that there is something missing, and maybe you keep this in mind, when you build a new image

git does seem to be installed, so it'll take some more troubleshooting. Sounds like I can keep this pretty low priority.

As of 1.19.2 this issue is still active. As far as I can tell it's related to invalid JVM opts when running BuildTools.jar. If you provide values for the env vars INIT_MEMORY and MAX_MEMORY the build seems to complete just fine

Good find @ryantate13 . Those variables fall back to MEMORY, but MEMORY is no longer defaulted before that point. That will be easy to fix.


PandaPickAndPlace-v3 Training and Hyperparameters

Hi @qgallouedec,
I have been trying to reproduce the results of some of the experiments, in particular for the PandaPickAndPlace task. However, I was only able to find hyperparameters for v1. Should results be reproducible for v3?
I tried using both the DDPG and TQC. However, I mostly focused on TQC since it is clearly documented in two places: https://huggingface.co/qgallouedec/tqc-PandaPickAndPlace-v1-3157870761 and https://wandb.ai/openrlbenchmark/sb3.
I can't get anywhere near the results presented in these two sources. I also tried to train the same agent in a dense environment as a sort of sanity check. The results were quite good, the success rate goes above 90% without any issues.
To Reproduce
Here is an example of the code I have been trying to run. For your convenience, I removed all callbacks and checkpoints. Also, I am using the bleeding edge version for all the packages, as presented in the docs.
import gymnasium as gym
import panda_gym
from stable_baselines3 import HerReplayBuffer
from sb3_contrib import TQC

env = gym.make("PandaPickAndPlace-v3")

model = TQC(
    "MultiInputPolicy",
    env,
    batch_size=2048,
    buffer_size=1_000_000,
    gamma=0.95,
    learning_rate=0.001,
    policy_kwargs=dict(net_arch=[512, 512, 512], n_critics=2),
    replay_buffer_class=HerReplayBuffer,
    replay_buffer_kwargs=dict(goal_selection_strategy='future', n_sampled_goal=4),
    tau=0.05,
    seed=3157870761,
    verbose=1
)

model.learn(
    total_timesteps=1500000.0,
    progress_bar=True
)

This is very surprising. There are no big changes between v1 and v3. The friction is better managed and that's it.
I'll take a look on my end, I'll get back to you.

This is very surprising. There are no big changes between v1 and v3. The friction is better managed and that's it. I'll take a look on my end, I'll get back to you.

Sounds good. Let me know if there is anything I can do to help out.

I just realized that the way I was loading the model from a checkpoint isn't correct and does not work properly. This might be the issue. Please give me some time to investigate, I will keep you updated.
Sorry for wasting your time.

Hi, I am getting back to you with more information. I was not able to replicate results even when training continuously. I am attaching code, plots and environment-related information. Please let me know if you need anything else or if you find a bug in my code.
Local Environment
Plots
Success Rate

Reward

Episode Length

Code (in its entirety)
import gymnasium as gym
import panda_gym
import numpy as np
import datetime

from stable_baselines3 import HerReplayBuffer
from stable_baselines3.common.callbacks import CheckpointCallback
from sb3_contrib import TQC

env = gym.make("PandaPickAndPlace-v3")

# Create TQC agent:
model = TQC(
    "MultiInputPolicy",
    env,
    batch_size=2048,
    buffer_size=1_000_000,
    gamma=0.95,
    learning_rate=0.001,
    policy_kwargs=dict(net_arch=[512, 512, 512], n_critics=2),
    replay_buffer_class=HerReplayBuffer,
    replay_buffer_kwargs=dict(goal_selection_strategy='future', n_sampled_goal=4),
    tau=0.05,
    seed=3157870761,
    verbose=1,
    tensorboard_log='./tensorboard/TQC/',
)

stringified_time = datetime.datetime.now().strftime("%Y%m%d%H%M%S")
checkpoint_callback = CheckpointCallback(
    save_freq=100_000,
    save_path=f"./models/{stringified_time}/",
    name_prefix="tqc_panda_pick_and_place"
)  # Create checkpoint callback.

# Model training:
model.learn(
    total_timesteps=1_100_000,
    callback=checkpoint_callback,
    progress_bar=True
)
model.save("tqc_panda_pick_and_place_final")  # Save final model.

System Information

OS: Windows-10-10.0.18363-SP0 10.0.18363
Python: 3.9.16
Stable-Baselines3: 2.0.0a6
PyTorch: 2.0.0
GPU Enabled: True
Numpy: 1.24.3
Cloudpickle: 2.2.1
Gymnasium: 0.28.1

Colab Experiment
I tried training it in Colab as well, the environment timed out at around 400k steps, I am also attaching the same information for that experiment. The results do not look the same to me, but I could not find the difference in the code. I can share the notebook as well. :)
Plots
Success Rate

Code
!pip install panda-gym
!pip install git+https://github.com/DLR-RM/stable-baselines3
!pip install git+https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/
!pip install tqdm
!pip install rich

import gymnasium as gym
import panda_gym
import numpy as np
import datetime
from stable_baselines3 import HerReplayBuffer
from stable_baselines3.common.callbacks import CheckpointCallback
from sb3_contrib import TQC

base_path = '<user-specific-after-mounting-drive>'

env = gym.make("PandaPickAndPlace-v3")

model = TQC(
    "MultiInputPolicy",
    env,
    batch_size=2048,
    buffer_size=1000000,
    gamma=0.95,
    learning_rate=0.001,
    policy_kwargs=dict(net_arch=[512, 512, 512], n_critics=2),
    replay_buffer_class=HerReplayBuffer,
    replay_buffer_kwargs=dict(goal_selection_strategy='future', n_sampled_goal=4),
    tau=0.05,
    seed=3157870761,
    verbose=1,
    tensorboard_log=f'{base_path}/tensorboard/',
)

stringified_time = datetime.datetime.now().strftime("%Y%m%d%H%M%S")
checkpoint_callback = CheckpointCallback(
    save_freq=10_000,
    save_path=f"{base_path}/models/{stringified_time}/",
    name_prefix="tqc_panda_pick_and_place"
)  # Callback for saving the model

# Model training:
model.learn(
    total_timesteps=1_000_000.0,
    callback=checkpoint_callback,
    progress_bar=True
)
model.save(f"{base_path}/tqc_panda_pick_and_place_final")

System Information

OS: Linux-5.10.147+-x86_64-with-glibc2.31 # 1 SMP Sat Dec 10 16:00:40 UTC 2022
Python: 3.10.11
Stable-Baselines3: 2.0.0a6
PyTorch: 2.0.0+cu118
GPU Enabled: True
Numpy: 1.22.4
Cloudpickle: 2.2.1
Gymnasium: 0.28.1
OpenAI Gym: 0.25.2

Thanks, I'll take a look. I'm going back to you soon.

Hi, is did anyone figure out the problem at the end? I can't reproduce PIckAndPlace results using TQC or SAC with either huggingface hyperparameters or hyperparameters from the panda-gym paper.

Hi, I made no progress. I got inconsistent results and never managed to replicate the ones documented.

Ok, the error was actually on my side. While I still can't reproduce SAC results, TQC works with huggingface hyperparameters after fixing the bug in my code.


Unset properly the buffer to release the file descriptor

Without this line a file descriptor was kept open after the call of $socket->close()

Great to hear you found a fix. Just looking over the code and noticed you unset buffer to clear any left over open connections. But shouldn't unreferencing SocketWrapper instances achieve the same? Or am I missing something?

@WyriHaximus unset the SocketWrapper instance don't close the buffer, I also tried it. :)

@edhelas ah thanks for clearing that up :+1:

Thank you for keeping up with this @edhelas, the changes LGTM :+1:  Can you squash this into a reasonable number of commits?
Also note that this is technically a BC break, @WyriHaximus I'll leave this up to you, can you assign this to the proper release milestone?

I made a mistake when squashing my commits into one, sorry for that. I made a new PR here https://github.com/reactphp/zmq/pull/49.

No worries, these things tend to go wrong occasionally. Please ask for assistance next time, restoring this is generally preferred over opening a new PR and losing discussion history :+1:


Remove unnecessary variable 'state' in LexerImpl by using Stack

In my opinion, the 'state' variable in LexerImpl seems unnecessary. You can remove them by using Stack data structure to check the current state. (Replaced LinkedList to Stack to make it clear that the 'states' is managed by Stack)
All the test cases have passed.

java.util.Stack is effectively deprecated in favor of Deque, here is what the javadoc has to say:

A more complete and consistent set of LIFO stack operations is provided by the Deque interface and its implementations, which should be used in preference to this class. For example:
Deque<Integer> stack = new ArrayDeque<Integer>();

@zimmi
Now I understand that Pebble is using LinkedList instead of Stack. Thanks for letting me know.
However, the 'state' variable still seems unnecessary for me because you can use the peek () method to check the current state. Is it okay to create another PR to remove the 'state' variable from the LexerImpl?

I'm not a maintainer of this repo, just a happy user of pebble. :)
That said, your change looks fine to me. Just replace the Stack with ArrayDeque, the previous LinkedList is probably also less efficient. Most methods of Stack are needlessly synchronized, that's one of the reasons it shouldn't be used anymore. The javadoc of ArrayDequesays:

This class is likely to be faster than Stack when used as a stack, and faster than LinkedList when used as a queue.

I'd just update this PR, but you might want to wait for one of the maintainers to make the call.

@zimmi
All right then I'll wait until the maintainer confirms this PR. Thanks for your feedback. It helped me to understand Java better.

@zimmi
Okay then I'll close this and change the Stack to ArrayDequeue to create another PR. Thank you for your feedback. It helped me to better understand Java :)

@zimmi
Thank you for your feedback. As you said, I will wait for the maintainer to confirm this PR :)

Instances of LexerImp are never called from multiple threads, each thread creates its own instance. So the synchronization is useless. That aside, just using thread safe data structures is not enough to achieve thread safety of a whole program. See this SO question for the concrete case of Vector / Stack.

Should you use Deque instead of Stack ?

This could be debatable. Deque could be better if you're looking at performance, but Stack is better in terms of semantics.
Deque can be used like Stack at times, and sometimes like Queue but I think this is a double edged sword. Stack, on the other hand, is just Stack. A Stack cannot be used just like a Queue even if it extends Vector.
I think it's a matter of choice.

IMHO, no, choice is not debatable: Stack is obsolete and Dequeue’s performance is better.

Alright then is it okay to close this PR and make another PR to change all the Stack to Deque?

Why not edit this one?

No need to close this PR. Just commit and push to your branch and the PR will bu updated

It's done. Thanks.


Cüzdandan atma problemi

Describe the bug
Sürekli hesabımdan atıyor ve puanlar gözükmüyor
Reproduction
Error mesage
System Info
System

Logs
go-ibax

Validations

[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Retweet tasks hit the Twitter API's monthly limit for our developer accounts, so some users won't be able to earn IBAX points


Pre-trained weights for SDXL version

Thank you for sharing your codebase. The omni library looks really great!
I would like to use DreamLLM with SDXL. Will the pre-trained weights be released on HuggingFace?

Hi @ktrk115,
Thank you for your interest! I currently have no resources to do it. However, I have provided the implementation and training scripts for DreamLLM-SDXL. You can train a DreamLLM-SDXL easily with the provided script. Please note the tips I wrote before you train the model.
Best,
Runpei

Hi Runpei, thanks for your reply! Your training tips will definitely help. I would like to train it by myself but am concerned about the GPU requirements. Do you have an estimate of how long it would take to train DreamLLM-SDXL with mid-scale GPU resources, say 8 A100 80GB GPUs available?

Hi @ktrk115,
To get a result quickly, I suggest you run stage-I on LAION-400m using approximately 20M images. You should get a reasonable result before the training is finished. For example, you can try to test the model every 500 or 1k steps. If you train the model on an 8xA100 machine, the training should not be very time-consuming. But note that since SDXL requires a larger resolution and the batch size would be smaller, making the training a bit slower.

Thank you for your kind advice! I'll try that once I'm familiar with the omni library.

You are welcome.

Hi @ktrk115,
We recently trained a new T2I and personalization DreamLLM-SDXL model for another benchmark project. I will publish this model, and I will ping you then.

Sounds great, looking forward to it!


Refactoring of Styles management and editing

In view of the revised UX of the master client for GeoNode, the management and editing of Datasets and Maps styles will be refactored as described below.
Dataset styles
The model for dataset styles will be very similar to the current one:

A dataset can reference multiple styles
A dataset will have a default style assigned
Admins, the dataset owner, and any user with edit grants on the dataset, can add new styles to the dataset and change the default style

Map styles
The management of styles for datasets embeeded in a map change significantly.

A MapLayer (according to the semantics and model being implemented in https://github.com/geosolutions-it/nexus-geonode/issues/468) can be assigned any style available from the underlying Dataset styles BUT their source cannot be viewed (and consequently edited). This lets users share styles from their dataset keeping their source private.
New styles can be created for the MapLayer, either from scratch ror from basic templates provided by the Styled editor. These styles will be connected to the MapLayer but not the underlying Dataset.
The styles specifically created for a MapLayer will be stored directly in Geoserver.
GeoNode will be able to retrieve the currently selected style for a MapLayer throught the MapLayer.current_style modiel field.

The client will adopt the extended Map API to store and retrieve all the information about the MapLayers and The MapLayer styles in particular.

moved to https://github.com/GeoNode/geonode-mapstore-client/issues/575


intercept_request not defined in Ferrum 0.4

Using url_whitelist and calls to visit would result in an NameError
 NameError:
            undefined local variable or method `intercept_request' for #<Ferrum::Page:0x000056434d93ab48>
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/forwardable.rb:224:in `page'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/forwardable.rb:224:in `goto'
          # ./spec/features/embedding/login_page_spec.rb:508:in `visit_page'
          # ./spec/features/embedding/login_page_spec.rb:47:in `block (6 levels) in <top (required)>'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/bin/rspec:23:in `load'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/bin/rspec:23:in `<top (required)>'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/cli/exec.rb:74:in `load'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/cli/exec.rb:74:in `kernel_load'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/cli/exec.rb:28:in `run'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/cli.rb:463:in `exec'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/cli.rb:27:in `dispatch'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/cli.rb:18:in `start'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/lib/ruby/2.6.0/bundler/friendly_errors.rb:124:in `with_friendly_errors'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/bin/bundle:23:in `load'
          # /home/zkhan/.asdf/installs/ruby/2.6.2/bin/bundle:23:in `<main>'

Locking to Ferrum 0.3 resolved the issue for me.

Do you use cuprite from master?

Nope? Just from rubygems.org

OK, if it's been addressed I'll try again without locking, perhaps it was cuprite that needed an bundle update.

@zkhan yep that should be it! Try to update both. Thanks

Double checked and 0.7.0 cuprite does not work with as described ferrum 0.4

Thanks I'll take a look!

@zkhan you are right I didn't release cuprite for the last changes made in ferrum. Doing it now, will be fixed in 0.7.1


[FEATURE] Add hover info text for detector "Active" or "Disabled" status

Is your feature request related to a problem?
Detector status (Active/Inactive) on Detector Details page is marked in red text color.

This should be accompanied with hover information.
What solution would you like?
Adding a Tooltip around the Detector status label. Is this what is expected? @amsiglan @kmleung
Also, what is the expected text?
What alternatives have you considered?
Is there some special behavior, similar to autotrigger using EuiTour?
Do you have any additional context?

@kamingleung  There are no intermediate states, but while the status is being adjusted, it can take a couple of seconds. While we work on improving the perf, do we want to show some loading indicator here?


added the backtotop button component to pages

Issue Number
fixes #117

Describe the changes you've made

added the backtotop button component to pages
onClick takes to the top/where the anchor of teh page has been set

Checklist

[x] My code follows the code style of this project.

sure.. thanks @Abhijay007 will do so

@Devesh21700Kumar It looks good and seems to work fine but one thing I want to suggest is that please try to align the back to top button at the extreme bottom of the page and slightly upper than the bottom navbar as it is somewhere in the middle right now which is not look that good in the UI.

Thanks for the review @Abhijay007 ..
I've set it now at a fixed distance of 160px from the last chapter listitem
It looks like this now


[Question] last_login cannot be null

I'm getting this error for new users on my prod graphite setup:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 486, in get_or_create
    return self.get(**lookup), False
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 399, in get
    self.model._meta.object_name
django.contrib.auth.models.User.DoesNotExist: User matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 148, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 310, in _query
    conn.query(q)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 548, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 775, in _read_query_result
    result.read()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1156, in read
    first_packet = self.connection._read_packet()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 725, in _read_packet
    packet.raise_for_error()
  File "/usr/local/lib/python3.6/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.6/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.IntegrityError: (1048, "Column 'last_login' cannot be null")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.6/site-packages/django/utils/deprecation.py", line 90, in __call__
    response = self.process_request(request)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/middleware.py", line 78, in process_request
    user = auth.authenticate(request, remote_user=username)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 73, in authenticate
    user = backend.authenticate(request, **credentials)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/backends.py", line 143, in authenticate
    UserModel.USERNAME_FIELD: username
  File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 488, in get_or_create
    return self._create_object_from_params(lookup, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 527, in _create_object_from_params
    raise e
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 520, in _create_object_from_params
    obj = self.create(**params)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 413, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/base_user.py", line 73, in save
    super().save(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 717, in save
    force_update=force_update, update_fields=update_fields)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 747, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 830, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 868, in _do_insert
    using=using, raw=raw)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1133, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1285, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 148, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.6/site-packages/pymysql/cursors.py", line 310, in _query
    conn.query(q)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 548, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 775, in _read_query_result
    result.read()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 1156, in read
    first_packet = self.connection._read_packet()
  File "/usr/local/lib/python3.6/site-packages/pymysql/connections.py", line 725, in _read_packet
    packet.raise_for_error()
  File "/usr/local/lib/python3.6/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.6/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
django.db.utils.IntegrityError: (1048, "Column 'last_login' cannot be null")

Oddly, this isn't happening in my dev setup which is the exact same version of django, graphite-web, etc.
Googling for an answer I found some posts saying a migration of django was necessary. Is there a migration option to set the last login field to accept "null"?
Version info:
asgiref==3.4.1
attrs==21.4.0
Automat==20.2.0
cairocffi==1.2.0
cffi==1.15.0
charset-normalizer==2.0.12
constantly==15.1.0
cryptography==37.0.2
Django==2.1
django-tagging==0.4.6
hyperlink==21.0.0
idna==3.3
incremental==21.3.0
pycparser==2.21
PyMySQL==1.0.2
pyparsing==3.0.9
python-memcached==1.59
pytz==2022.1
six==1.16.0
sqlparse==0.4.2
Twisted==22.4.0
txAMQP==0.8.0
typing_extensions==4.1.1
urllib3==1.26.9
whisper==1.2.0
whitenoise==4.1.4
zope.interface==5.4.0

This database was originally on an older version of django. When I ran the migrations the first time, they did not upgrade the database and permit the last_login column to permit null. I couldn't find a way to fix this with Django so I had to update the database directly with: ALTER TABLE auth_user modify last_login datetime null;

Sorry, Django migrations are complicated and unfortunately, out of scope, of Graphite itself. But if fixing outside of Django worked - that's also an solution.


Handle sudden subject hash change

#1034 introduced a hash comparison in addition to equality comparison, and it broke the following examples (using rspec-rails):
it { expect { 'noop' }.not_to change { country.reload.cities } }

a workaround to that was to use the .to_a on the relation:
it { expect { 'noop' }.not_to change { country.reload.cities.to_a } }

and the underlying problem is with the objects that are changing their hash during the comparison.
Notable example:
def ==(other)
  load_target == other
end

The problem is originally described here, and the proposed solution was to raise the question against Rails team to fix the CollectionProxy#hash behaviour, but I believe this wouldn't work for the following reasons:

The issue is specific to RSpec, while Rails comes with its own testing framework that does the assertions using ==, and is not affected by the problem in question.

Fixing the issue would require to either:

stick to the original hash that the object was assigned that breaks the 'dynamic hashcode value depending on the contents of the object itself' that mutable Ruby objects seem to adhere to
pre-load the relation that may be a huge performance hit

The change swaps conditions in a way that the hashcode of the objects being compared is not yet updated by the other sneaky condition.
This might also fix the issue described in #1070

Awesome. Thanks for your patience and accepting this blazingly fast, Jon!

When is this change scheduled to be released? I'm experiencing the same issue when using the change matcher on associations.

We don't have a release schedule, things tend to go out as warranted. However I'm happy to announce I've just shipped 3.8.3 containing this patch.


cannot import name event_pb2

get an import error.
---> 23 from .src import event_pb2
     24 from .src import summary_pb2
     25 from .event_file_writer import EventFileWriter

ImportError: cannot import name event_pb2

Did you compiled the protos and installed the Python package successfully? Please provide more info for me to reproduce.

check around, found that files in tensorboard/python/tensorboard/src isn't copied to the site-packages/tensorboard. Manually copy these files, it works.

@kernel8liang thanks for the feedback, let me check.

could you tell me your system?

do you install the package directly under the default python path?

Please try the latest PyPI installation.

FYI: Met the same problem with custom built tensorboard wheel(since StringIO issue(#31) is still unfixed in 1.0.0a6). Fixed the same problem with https://github.com/dmlc/tensorboard/issues/15#issuecomment-277595345 .

@JiamingSuen Please build from source, as we haven't finished the latest PyPI currently.

Meet the same issue at the latest master branch.
envs:
Ubuntu16.04
Python2.7
Bazel 0.6


Feat/orderbook snapshot

updates proto files for the exchange API
Provide example on how to load an orderbook snapshot and keep it updated using the new streaming updates API for both spot and derivative markets

LGTM


User in plugin.conf not used

Is user used in plugin.conf ? Seems that this was added because of enabling the plugin per user level but this is obsolete since issue #3

Nope, ins't used anymore.

Okay, I will remove it. Thank you!

Okay, I will remove it. Thank you!


Multifile option? *.png / *.jpg

I have a directory full of logos and I'd like to make the whole process as automatic as possible...
would it be possible to set the sourcePath ("/source/path/") and destinationPath ("/destination/path") or at least to define multiple sources dynamically like src: "/source/path/*.png" ?
Cheers

Same issue here. This plugin is great, other than globbing pattern configuration. Is there a way we can add support for all images in a folder? Thanks!

I second that.


Reduce lock contention in Worker

..., by caching the metric.
Benchmark will come soon.
@hicqu @UncP PTAL

Waiting for the benchmark result.

LGTM.

PTAL @zhangjinpeng1987


Kafka: Support for AWS IAM role auth

Description
Follow up of the https://github.com/dapr/components-contrib/pull/3240
This is to add an AWS IAM role authentication aside from AWS credentials.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: https://github.com/dapr/components-contrib/issues/3239 (Already closed but related issue)
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@eunicecompra can you please fix the DCO?
https://docs.dapr.io/contributing/contributing-overview/#developer-certificate-of-origin-signing-your-work

@eunicecompra can you please fix the DCO? https://docs.dapr.io/contributing/contributing-overview/#developer-certificate-of-origin-signing-your-work

@ItalyPaleAle Missed this one. Done signing off my commits.


dbeaver/pro#2469 Improve outline and highlighting support

Outline is correctly showing joined tables (closes dbeaver/pro#2469)

Alias for nested queries is optional now (closes #22715)

ORDER BY i.* is now supported (closes dbeaver/pro#2470)

verified


fix: expand globs in npm scripts correctly

Why?
Originally the npm scripts were using the glob /**.js which is no different than using /*.js, expanding to all the files inside that directory but not recursively through all subdirectories.
Fix
A proper glob to expand recursively to all subdirectories would be /**/*.js if the globstar option is set (shopt -s globstar).
Even within environments with an up-to-date version of Bash (i.e. TravisCI) the proper configuration for expanding a glob such as **/*.js will not load when running non-interactive shells (i.e. npm scripts).
I'm not saying this solution is the best solution out there but adding a feature to beautify-lint that expands the globs correctly will not solve the underlying problem with globs and npm scripts.
Before the patch:
$ npm run beautify

><EMAIL_ADDRESS>beautify /Users/oviedoga/Documents/dev/webpack
> beautify-rewrite lib/**.js hot/**.js bin/**.js benchmark/*.js test/*.js

Done. (145 files checked)

After the patch:
 npm run beautify

><EMAIL_ADDRESS>beautify /Users/oviedoga/Documents/dev/webpack
> beautify-rewrite `glob 'lib/**/*.js' 'hot/**/*.js' 'bin/**/*.js'` benchmark/*.js test/*.js

- lib/dependencies/HarmonyExportImportedSpecifierDependency.js
- lib/dependencies/HarmonyModulesHelpers.js
- lib/optimize/CommonsChunkPlugin.js
- lib/optimize/RemoveParentModulesPlugin.js
- lib/optimize/UglifyJsPlugin.js
Done. (5 files updated, 252 files checked)

What about using the find command instead?
beautify-rewrite $(find lib hot bin benchmark test -name '*.js')

Take care of windows compatibility too.

fixed


Add discord notifications for voting

What does this PR do?
This PR adds a GitHub Workflow that sends notifications via a Discord Webhook when the "vote now" label is added or removed to/from issues or PRs. The following screenshot shows how these notifications can look on Discord (The screenshot shows data from a testing repo).

Motivation
I brought up the idea of automatic notifications in the discord and there seemed to be some interest in a feature like this.
More
In order for this to work two additional steps are required:

A discord admin must register a webhook for the #song-format channel on Discord @bohning
An admin of this repo must add the webhook URL as a CI secret with the name DISCORD_WEBHOOK. I don't currently know who that would be.

I added both the webhook on Discord and the DISCORD_WEBHOOK secret to the repository secrets.

Updating the branch and going to merge it, thanks guys!

In which Discord server is this channel?

@basisbit Sorry for the late reply. It should be the #song-format channel in the UltraStar* Creators Community Discord]


Patch Stratos to be compatible with new Cloud Foundry log streaming protocol

Background
Cloudfoundry recently changes its architecture for clients to receive streaming logs from applications.
Previously, logs were accessed via the Loggregator Traffic Controller component as seen in this document: https://docs.pivotal.io/application-service/2-8/loggregator/architecture.html
However, logs are now access via the Log Cache VMs: https://docs.pivotal.io/application-service/3-0/loggregator/architecture.html
Consequently, CloudFoundry components are being updated to retrieve logs from the Log Cache service. In the process, Stratos broke, because it is unmaintained.
This is the code that Stratos is using to retrieve logs: https://github.com/cloudfoundry/stratos/blob/3de685ceeb01a751e84e6287576f950863d074e3/src/jetstream/plugins/cloudfoundry/cf_websocket_streams.go#L127
You can see that it relies on code which was recently deprecated: https://github.com/cloudfoundry/noaa/commit/f0749146decfd357a5583c29bd3ad2b15b322c85#diff-6062a9244038ef69f131734bb106b690368b624472c63597d883eb687376fd17L23
Even if that code in the noaa library weren't deprecated, the ability to retrieve logs that way was deprecated in Loggregator as of version 107, which we are now running: https://github.com/cloudfoundry/loggregator-release/releases/tag/v107.0.0
So the only option is to update Stratos to retrieve logs directly from Log Cache
Acceptance criteria

Stream of app logs are viewable in the dashboard

Implementation
One idea is to see if we can copy the code from the cf CLI, also written in Go, and re-use it in Stratos to fetch the logs:
https://github.com/cloudfoundry/cli/blob/be7b0fd5de6ec7ddcdb9f055ef8563cc6738c639/cf/commands/application/logs.go#L101
It might also be helpful to run CF_TRACE=1 cf logs <app> to observe how it fetches logs
Tasks

[ ] Update Stratos fork to fetch logs from log cache service
[ ] Deploy updated Stratos to dev and test whether the log stream is visible

Someone patched their own fork of Stratos: https://github.com/freudl/stratos/pull/1. Hopefully we can just adopt their changes

2/28/23 Update:
In build/bk-build.sh, I changed go get<EMAIL_ADDRESS>to go install<EMAIL_ADDRESS>I was able to get the build to compile and run locally. When I pushed it to development via a merge into main, I saw what appears to be a database migration failure. Further investigation revealed that the migrations in the cnsis table were out of order according to their primary key.
Possible next steps:
a. Continue and look at the staging database to see what migrations were applied and bring the development database to the same state manually (not preferred)
b. Roll back the development database to its prior state and rerun the deployment to see if it fixes itself (semi-preferred)
c. Roll back development to main to a point where it compiles and deploys per what is running in production, and roll back the dev database to the same point. (semi-preferred)
b or c are most likely the next step
If we choose c, we lose all of the changes we made to main after the merge from the upstream cloudfoundry master branch.
We are putting this aside for now until we have more time this week to come back to it.

Update: I have the issues with Go binary compilation for Stratos (binary name is jetstream) fixed, but now there are issues with the build failing when trying to apply the database migrations that were merged in from the upstream repo.
To test whether this is an issue with us not doing the upgrade path correctly, I'm attaching a new test database service to stratos in dev,  stratos-db-mboyd-test, which is empty to see if the database migrations will cleanly.
WIP is on this branch: https://github.com/cloud-gov/cg-deploy-stratos/tree/fix-stratos-build

I take it back. The suggested fix for the Go binary compilation from Cloudfoundry slack was to use CGO_ENABLED=0, but that is causing the binary compilation to fail with:
# bitbucket.org/liamstask/goose/lib/goose
../../../../go/pkg/mod/bitbucket.org/liamstask/goose@v0.0.0-20150115234039-8488cc47d90c/lib/goose/dialect.go:119:15: undefined: sqlite3.Error


Critical Bug fix in the XML Syntax when Defining Default Values for A Global Tile Port

The XML syntax to define a default value for a global tile port was default_value="<int>" while in architecture example default_val="<int>" was used.
This has caused the default value definition in these examples to be dropped by parser.
This PR aims to fix this critical bug by setting the expected syntax as default_val="<int>", being consistent with circuit model port definition.

Seems fine, is there any worry that any current uses would be broken?

Now the default_val in example architectures become effective in testbench generation.
There could be some test cases to fail. But I just need to set a correct value to the default_val .


Fix remaining occurrences of config.pl in all.sh

Description
The .pl version is now a compat wrapper around the .py script. Better call the .py script directly.
This PR is intended to remove a few simple merge conflicts when merging crypto back, by making sure both sides use .py everywhere.
Status
READY
Requires Backporting
NO - the TLS branches don't have config.py

CI only fails due to an unrelated preexisting issue on Mbed OS.