Storage in Image Management

Picking up from https://github.com/kubernetes-incubator/cri-o/pull/28, this PR builds on #210 to pull images using containers/image, and to put images, pod sandboxes, and containers in space managed by containers/storage.
Close #298

I'm wondering why Travis disappeared, @mrunalp do you know anything?

Needs rebase.

Ready.  Recently-landed changes meant that some of the functionality that was implemented in the storage cri branch was no longer needed, so it's been pruned out.

There is a lint failure:
checking lint
cmd/server/main.go:88:3:warning: should use for range instead of for { select {} } (gosimple)
make: *** [Makefile:35: lint] Error 1

From quick manual testing it works fine :) Now running tests locally and reviewing.

Rebased to master to resolve conflicts, fixed the lint error, and introduced a proposed default_transport configuration option which, while it's not really an accurate name, hopefully points people in the right direction: if an image name in a pull request can't be parsed as a pull source, we'll prepend the value and try again.
There's some additional logic to convert references to images in other transports into references suitable for use with the local storage when creating images and containers, so some of the self-tests still refer to images using the docker:// prefix, and some don't, hopefully to better exercise the logic.
Reporting of image sizes is kind of broken - if you get an answer at all, it is probably wrong if any of the layers were compressed.

When I run make install on this PR, I see a bunch of SELinux related errors:
[root@dhcp-16-129 cri-o]# make install
install -D -m 755 ocid /usr/bin/ocid
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
install -D -m 755 ocic /usr/bin/ocic
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
install -D -m 755 conmon/conmon /usr/libexec/ocid/conmon
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
install -D -m 755 pause/pause /usr/libexec/ocid/pause
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
install -d -m 755 /usr/share/man/man{8,5}
install -m 644 docs/ocid.8 -t /usr/share/man/man8
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
install -m 644 docs/ocid.conf.5 -t /usr/share/man/man5
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
install -D -m 644 ocid.conf /etc/ocid.conf
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found

cc: @rhatdan

We have a lot of lint errors in container/storage vendoring.

Looks like you got garbage in
/etc/selinux/targeted/contexts/files/file_contexts.bin:  line 1 error due to: Non-ASCII characters found
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin:  line 1 error due to: Non-ASCII characters found
Just remove these files, they should get recreated.

@rhatdan That fixed it for me. Thanks!

@mrunalp Yeah, it looks like the linters are having trouble with cgo code.

@runcom Do you have any thoughts on how to deal with the lint issues. Are they fixable by modifying the code.

Can we skip with grep -v the whole vendored code?

@runcom @nalind Yeah, makes sense to skip vendored code in this repo. We can fixup what we can in containers/storage separately.

@mrunalp @nalind are we holding this one a little more so Nalin could figure out how to remove the containers/image dependency or can I go ahead and review it?

@runcom You can wait for the removal of containers/image dependency.

@nalind something should be wrong in the cleanup stuff, basically if I "ctrl-c" ocid, I'm left with:
/var/lib/ocid/devicemapper
/var/lib/ocid/devicemapper/mnt/798a44ab34fa878b2d3ff9934cf61a0a235ebf4ea3f19dd933caa6b60e987f29
/var/lib/ocid/devicemapper/mnt/7bc934fa2568dc5504c216876b47a8abd73ded0d7aa5bfb2127f12e38a1a6d59

mounted

Tried running this patch with kubelet, a simple nginx pod start failed (kubelet log):
I1213 10:38:05.982481   18019 server.go:674] Event(v1.ObjectReference{Kind:"Pod", Namespace:"default", Name:"nginx-272207814-gmg98", UID:"3b230c4e-c120-11e6-a418-fa163e66591a", APIVersion:"v1", ResourceVersion:"362", FieldPath:""}): type: 'Normal' reason: 'SandboxReceived' Pod sandbox received, it will be created.
E1213 10:38:10.348822   18019 remote_runtime.go:78] RunPodSandbox from runtime service failed: rpc error: code = 2 desc = error creating pod sandbox with name "default-nginx-272207814-gmg98-0": Error writing blob: ApplyLayer wait: no child processes stdout: {"layerSize":239840}
 stderr:
E1213 10:38:10.348891   18019 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "nginx-272207814-gmg98_default(3b230c4e-c120-11e6-a418-fa163e66591a)" failed: rpc error: code = 2 desc = error creating pod sandbox with name "default-nginx-272207814-gmg98-0": Error writing blob: ApplyLayer wait: no child processes stdout: {"layerSize":239840}
 stderr:
E1213 10:38:10.348911   18019 kuberuntime_manager.go:643] createPodSandbox for pod "nginx-272207814-gmg98_default(3b230c4e-c120-11e6-a418-fa163e66591a)" failed: rpc error: code = 2 desc = error creating pod sandbox with name "default-nginx-272207814-gmg98-0": Error writing blob: ApplyLayer wait: no child processes stdout: {"layerSize":239840}
 stderr:

And there is nothing help in ocid logs.
@nalind Do you know what's the possible cause?

@nalind Do you know what's the possible cause?

@feiskyer please follow my instruction in https://gist.github.com/runcom/ba58bf2f64e38d9f5d376d587751a0f9 (for Centos 7) on how to test this pull request.

something is going wrong when ocid is restarted and we request to pull an image we already pulled before the restart:
13:03:16 [github.com/kubernetes-incubator/cri-o] ‹demo› sudo ./ocic image list
ID: c3aa2798e0100e1f0090cabd56aff8476e7e0bd2d211452e6c1dabd86cbbddaf
Tag: docker.io/kubernetes/pause:latest
Tag: kubernetes/pause
Tag: docker.io/kubernetes/pause
ID: c7c264a871489ed302de80067fb561e8b88c90d3acf910b234bf643bb09a3655
ID: 22bae1e7997fdddd6e9a2ad7d8c771eb9bef76cfd5cdebb8e52d80443ab001b4
Tag: docker.io/library/redis:latest
Tag: redis:latest
Tag: docker.io/library/redis
ID: 150c7a727048577299507032145f9e4bee22135c69abb9dc2172276a6324d42f
Tag: docker.io/library/nginx:latest
Tag: nginx:latest
Tag: docker.io/library/nginx
ID: 6cee24dae6d1292d905f07a293f811affa79287e0820d65b8d2217aa06024d2c
Tag: docker.io/library/redis:latest
Tag: redis:latest
Tag: docker.io/library/redis
``

BTW, my https://github.com/runcom/ocid-1/tree/demo branch is in good shape building from this PR :+1: (stopping ocid and restarting it now seems to work also and containers get re-synced with kubelet also!)

Updated to vendor containers/image's master branch.

Tests hang on my system with this PR (while I can run tests just fine with #210)

@runcom I'm not entirely sure it's all about the pulling of images (I think, but haven't verified, that the integration tests were faster in this branch as recently as a few days ago), but I'm working on a test helper which will import the locally-built "pause" binary into storage as an image.

Using a bin2img helper seems to be helping.  I suppose it also makes the code easier to test on other architectures, which is a bonus I hadn't thought about.

@nalind this could be rebased/reworked based on #210 now

Okay, rebased.  The updated version of containers/image needed to have pkg/error added to the dependencies list.

Great thanks @nalind! I'll test this our by the end of the day

@nalind Looks like we can merge the first 2 commits separately as they are fixing issues outside of the integration. Could you create separate PRs for those?

Will do.

Rebase is ready.  Thanks for merging #291 so quickly.

Tests are still super slow for me :cry: not sure it's something with devicemapper since I see a bunch of:
Dec 20 16:34:48 runcom.usersys.redhat.com kernel: XFS (dm-6): Mounting V5 Filesystem
Dec 20 16:34:48 runcom.usersys.redhat.com kernel: XFS (dm-6): Ending clean mount
Dec 20 16:35:32 runcom.usersys.redhat.com kernel: XFS (dm-6): Unmounting Filesystem
Dec 20 16:35:35 runcom.usersys.redhat.com kernel: XFS (dm-6): Mounting V5 Filesystem
Dec 20 16:35:35 runcom.usersys.redhat.com kernel: XFS (dm-6): Ending clean mount
Dec 20 16:35:35 runcom.usersys.redhat.com kernel: XFS (dm-6): Unmounting Filesystem
Dec 20 16:35:37 runcom.usersys.redhat.com kernel: XFS (dm-6): Mounting V5 Filesystem
Dec 20 16:35:37 runcom.usersys.redhat.com kernel: XFS (dm-6): Ending clean mount

But just one test takes around 1m. @nalind which graphdriver are you using?

By default, I'm also using device mapper.  The localintegration suite finishes in around 5 to 7 minutes in my VM.

By default, I'm also using device mapper. The localintegration suite finishes in around 5 to 7 minutes in my VM.

I test on my host and here is the time it takes for me (laptop is pretty fast):
16:32:54 [github.com/kubernetes-incubator/cri-o] ‹pr/origin/189› sudo make localintegration
go build --tags "selinux seccomp  " -o ocid ./cmd/server/
go build -o ocic ./cmd/client/
go build -o kpod ./cmd/kpod/
make -C conmon
make[1]: Entering directory '/home/amurdaca/go/src/github.com/kubernetes-incubator/cri-o/conmon'
make[1]: 'conmon' is up to date.
make[1]: Leaving directory '/home/amurdaca/go/src/github.com/kubernetes-incubator/cri-o/conmon'
make -C pause
make[1]: Entering directory '/home/amurdaca/go/src/github.com/kubernetes-incubator/cri-o/pause'
make[1]: 'pause' is up to date.
make[1]: Leaving directory '/home/amurdaca/go/src/github.com/kubernetes-incubator/cri-o/pause'
make -C test/bin2img
make[1]: Entering directory '/home/amurdaca/go/src/github.com/kubernetes-incubator/cri-o/test/bin2img'
go build -o bin2img
make[1]: Leaving directory '/home/amurdaca/go/src/github.com/kubernetes-incubator/cri-o/test/bin2img'
./test/test_runner.sh
++ time bats --tap .
1..39
ok 1 # skip (skip this test since apparmor is not enabled.) load default apparmor profile and run a container with it
ok 2 # skip (skip this test since apparmor is not enabled.) load a specific apparmor profile as default apparmor and run a container with it
ok 3 # skip (skip this test since apparmor is not enabled.) load default apparmor profile and run a container with another apparmor profile
ok 4 # skip (skip this test since apparmor is not enabled.) run a container with wrong apparmor profile name
ok 5 # skip (skip this test since apparmor is not enabled.) run a container after unloading default apparmor profile
ok 6 ctr remove
ok 7 ctr lifecycle
ok 8 ctrs status for a pod
ok 9 ctr list filtering
ok 10 ctr list label filtering
ok 11 ctr metadata in list & status
ok 12 ctr execsync
ok 13 ctr execsync failure
ok 14 ctr stop idempotent
ok 15 image pull
ok 16 image list with filter
ok 17 image list/remove
ok 18 image status/remove
ok 19 pod release name on remove
ok 20 pod remove
ok 21 pod list filtering
ok 22 pod metadata in list & status
ok 23 pass pod sysctls to runtime
ok 24 pod stop idempotent
ok 25 pod remove idempotent
ok 26 pod stop idempotent with ctrs already stopped
ok 27 restart ocid and still get pod status
ok 28 ocid restore
ok 29 ocic runtimeversion
ok 30 ctr seccomp profiles unconfined
ok 31 ctr seccomp profiles runtime/default
ok 32 ctr seccomp profiles wrong profile name
ok 33 # skip (need https://issues.k8s.io/36997) ctr seccomp profiles localhost/profile_name
ok 34 ctr seccomp profiles falls back to pod profile
ok 35 ctr seccomp profiles falls back to unconfined
ok 36 pod seccomp profiles unconfined
ok 37 pod seccomp profiles runtime/default
ok 38 pod seccomp profiles wrong profile name
ok 39 # skip (need https://issues.k8s.io/36997) pod seccomp profiles localhost/profile_name

real    28m52.778s
user    1m37.085s
sys     0m49.643s

On the topic of integration tests, the current system of integration testing leaves an incredible amount of crap on my host. You've got nsfs bindmounts that haven't been cleaned up, runc containers that clash with the test configurations, shm mounts that cause further retests to fail, etc.
We really need to fix this, because it's making it hard for people (at least for me) to actually test that their code works...

 You've got nsfs bindmounts that haven't been cleaned up, runc containers that clash with the test configurations, shm mounts that cause further retests to fail, etc.
We really need to fix this, because it's making it hard for people (at least for me) to actually test that their code works...

you're right @cyphar - after this is merged (hopefully) we can fully run the integration tests in a docker container (with the Dockerfile). Hopefully that helps, or not?

I am seeing the same slow tests that @runcom reported. Maybe we should cache the images for the duration of the test suite. 30 min tests aren't going to help with development :/

How about dropping image pull from container create? That way we follow the k8s model of first having to pull the image? (We can add enhancements to ocic to pull first if required).

@mrunalp This revision includes some of the changes discussed above, but my attempts to work around requiring each test run to pull down its own copy of the redis image, if it used it, turned up some sync problems in the storage library's devicemapper driver, so it still implicitly pulls images that it doesn't have when it is asked to create a container.
While those get tended to, the integration tests run to completion for me when I adjust helpers.bash to either set $STORAGE_DRIVER to a different driver name, or have it invoke both ocid and bin2img with their --storage-driver options (the options we add to ocid now use - instead of _ in their names, to match the style of other option names that are already provided by ocid) to have the same effect.

@nalind I am giving it a try. Thanks!

The tests passed for me but took 23 minutes. @nalind Do you think overlay will work if we run all tests on a single ocid instance without teardown for each test? We can try to make that change and rename a pod per test so there are no clashes and then we could clobber everything at the end of all the tests.
cc: @runcom

The tests passed for me but took 23 minutes. @nalind Do you think overlay will work if we run all tests on a single ocid instance without teardown for each test? We can try to make that change and rename a pod per test so there are no clashes and then we could clobber everything at the end of all the tests.

I think we could have a single ocid running, each test makes sure to cleanup pods and containers without removing the ocid root. This would allow us to keep stored images across tests I believe.

@runcom Could you run the tests on this PR with overlay driver?

@runcom Could you run the tests on this PR with overlay driver?

yup, testing right now

@nalind could you add test/bin2img/bin2img to the .gitignore also?

@mrunalp tests run fine, the issue is that they're still slow even with overlay (and overlay2)

@runcom Yeah, let's figure out how to fix that in bats my not tearing down per test.

@runcom We will wait to fix those issues until after break, so we will not merge, but we can carry a patch so people can start experimenting with a packaged version of OCID on Rawhide.
For example I can make sure it works with SELinux in enforcing mode and fix container-selinux to make it happen.  We also need to start working on docker-storage-setup and convirt it to container-storage-setup so it will work with both container runtimes.

@runcom We will wait to fix those issues until after break, so we will not merge, but we can carry a patch so people can start experimenting with a packaged version of OCID on Rawhide.
For example I can make sure it works with SELinux in enforcing mode and fix container-selinux to make it happen. We also need to start working on docker-storage-setup and convirt it to container-storage-setup so it will work with both container runtimes.

SGTM

@nalind can you cherry-pick here the top most 3 commits from https://github.com/runcom/ocid-1/commits/alpha? So this PR works with basic k8s pod functionalities

@nalind when starting ocid with overlay2 and docker started already with overlay2 I see:
$ mount
[...]
/dev/mapper/fedora-root on /var/lib/docker/overlay2 type ext4 (rw,relatime,seclabel,data=ordered)
/dev/mapper/fedora-root on /var/lib/ocid/overlay2 type ext4 (rw,relatime,seclabel,data=ordered)

is this expected?

@mrunalp
Direct pulling this PR and trying to build it caused this error:
✔ ~/go/src/github.com/kubernetes-incubator/cri-o [storage|…9]
07:18 $ make
mkdir -p /home/vagrant/go/src/github.com/kubernetes-incubator/cri-o/vendor/src/github.com/kubernetes-incubator
ln -sfn /home/vagrant/go/src/github.com/kubernetes-incubator/cri-o /home/vagrant/go/src/github.com/kubernetes-incubator/cri-o/vendor/src/github.com/kubernetes-incubator/cri-o
go build --tags "selinux seccomp  " -o ocid ./cmd/server/
# github.com/mtrmac/gpgme
vendor/src/github.com/mtrmac/gpgme/data.go:4:20: fatal error: gpgme.h: No such file or directory
 // #include <gpgme.h>
                    ^
compilation terminated.
Makefile:58: recipe for target 'ocid' failed
make: *** [ocid] Error 2

So installed package: gpgme-devel and libassuan-devel and then build successful.
But this did not fix https://github.com/kubernetes-incubator/cri-o/issues/299 either.

@runcom okay, I'm assuming that you also want the ones that you added after https://github.com/kubernetes-incubator/cri-o/pull/189#issuecomment-268641503.  The graph drivers would make whichever location they're using as their top level private, so if you're using both docker and ocid, what you're seeing in https://github.com/kubernetes-incubator/cri-o/pull/189#issuecomment-268642176 makes sense.

@surajssd When you say this doesn't fix #299, which method are you using to list images?

To remove the dependency on having a /etc/containers/policy.json in place that lets us run the tests, this revision adds a --signature-policy switch to ocid, with comments in the default configuration file which encourage it to not be set, and uses the one from the test directory when running tests.  The default config also used - instead of _ for some of the settings which this PR was adding, which didn't match the tags that indicated which settings it needed to read in, but I think I've straightened that out.

Updated with another stab at the underscores/hyphens correction, and to remove redundant calls to the storage StartContainer (mount) and StopContainer (unmount) methods, neither of which should be attempted when the runtime monitor is running and using the container's rootfs.  We mount a container's rootfs after creating it, and unmount it right before destroying it.

Updated the README per conversation in #298.

Per out-of-band conversation, I've added a copyimg tool which we use to pull down the redis image at the start of a test run (if it's been removed by make clean), stash it in test/testdata/redis-image, and import it from there before starting ocid.  It and the bin2img tool now use the github.com/urfave/cli package for parsing command line options to be more in line with ocid itself.  I'm hoping this'll speed up integration test runs.
Since we no longer depend on the implicit pulling of images, and I'm told that's not expected by CRI, that logic is no longer there.  I'm not entirely sure I like that, though, since it only adds a repetitive step for people who are trying to drive it with ocic.
Tests for image pulling now pass a flag to the start_ocid function to tell it to not build an image for pause, so that if ocic image pull fails, it's more obvious to the test.
Lastly, I added a check that I forgot to add earlier, skipping the updating of the pod and container lists when the storage library tells us that the set of Containers it keeps hasn't been changed since the last time we changed it ourselves or checked for changes made by others.  This should save some time.

okay, tests run ~4 minutes now :+1: I will review the new bits.
@runcom Can you also review/test this?

Alright, tests run in reasonable time now :+1: Thanks @nalind
Since the other commits are mine, I'm gonna just test the integration with k8s before acking on this PR for the last time hopefully :) @jjlakis tried this PR and he was successful when running with k8s though :)

storage is busted for me :( I cannot run with overlay2:
DEBU[2017-01-12 17:08:36.249872332+01:00] [graphdriver] trying provided driver "overlay2"

ocid is stuck at that in the log file and doesn't proceed. I've cleaned up everything so the state on my laptop should be good (tried more than twice...)
Plus, when testing this PR with devicemapper AND k8s there's some kind of deadlock when k8s calls ListImage:
DEBU[2017-01-12 17:06:23.132102522+01:00] ListImages:

the call logs that in ocid logs, and k8s repeats the call some time after but with the same (stuck) result. Notice that when calling ListImages with ocic you get:
DEBU[2017-01-12 17:06:09.078923152+01:00] ListImages: filter:<image:<image:"" > >

and it seems to work..at least it won't get stuck.
I'm running with k8s master (which requires also to re-vendor k8s here in cri-o) but re-vendoring doesn't solve anything unfortunately .

That "ListImages: " debug message suggests that the API client isn't setting the Filter field in its ListImagesRequest.

@nalind it seems ListImages, ListPodSandbox and ListContainers timeout when called from k8s (note that k8s does really fast calls one after the other). Snippet from kubelet log:
E0112 17:30:23.317131   17460 remote_runtime.go:228] ListContainers with filter "label_selector:<key:\"io.kubernetes.managed\" value:\"true\" > " from runtime service failed: rpc error: code = 4 desc = context deadline exceeded
E0112 17:30:23.317236   17460 kuberuntime_container.go:304] getKubeletContainers failed: rpc error: code = 4 desc = context deadline exceeded
E0112 17:30:23.317248   17460 kubelet.go:1131] Container garbage collection failed: rpc error: code = 4 desc = context deadline exceeded
I0112 17:30:23.317344   17460 server.go:701] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"<IP_ADDRESS>", UID:"<IP_ADDRESS>", APIVersion:"", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'ContainerGCFailed' rpc error: code = 4 desc = context deadline exceeded
E0112 17:30:23.321132   17460 remote_image.go:58] ListImages with filter "<nil>" from image service failed: rpc error: code = 4 desc = context deadline exceeded
E0112 17:30:23.322148   17460 kuberuntime_image.go:106] ListImages failed: rpc error: code = 4 desc = context deadline exceeded
W0112 17:30:23.323381   17460 image_gc_manager.go:164] [imageGCManager] Failed to monitor images: rpc error: code = 4 desc = context deadline exceeded
E0112 17:30:23.322742   17460 remote_image.go:58] ListImages with filter "<nil>" from image service failed: rpc error: code = 4 desc = context deadline exceeded
E0112 17:30:23.323482   17460 kuberuntime_image.go:106] ListImages failed: rpc error: code = 4 desc = context deadline exceeded

This is what we have in ocid log:
[ omitted ]
DEBU[2017-01-12 17:28:02.829617463+01:00] devmapper: deactivateDevice END()
DEBU[2017-01-12 17:28:02.837024792+01:00] sandboxes: map[]
DEBU[2017-01-12 17:28:02.837058315+01:00] containers: &{map[] {{0 0} 0 0 0 0}}
DEBU[2017-01-12 17:28:23.319179651+01:00] ListContainersRequest filter:<label_selector:<key:"io.kubernetes.managed" value:"true" > >
DEBU[2017-01-12 17:28:23.319330072+01:00] updating sandbox and container information
DEBU[2017-01-12 17:28:23.321837658+01:00] ListImages:

Can it be a deadlock somewhere since the calls coming from k8s are really fast? not sure what has introduced this behavior though :/ cause I was able to use k8s just fine ~1month ago.

That "ListImages: " debug message suggests that the API client isn't setting the Filter field in its ListImagesRequest.

yeah, right, but why should ocid/c/storage just wait?

Whoops, that looks like yesterday's change to Update() breaking it.  Reverting that.

Re-pushed with the changes that made Server.update() block itself removed.

Unfortunately this is still failing when trying to run an nginx pod, I got two kind of errors:
DEBU[2017-01-12 22:43:43.695579003+01:00] devmapper: AddDevice(hash=8369603abb954aa8c9076570cc7853aa0e5967d52f3f1e980740c7c5970f1d8f basehash=)
DEBU[2017-01-12 22:43:43.695614405+01:00] devmapper: Failed to read /var/lib/containers/devicemapper/metadata/8369603abb954aa8c9076570cc7853aa0e5967d52f3f1e980740c7c5970f1d8f with err: open /var/lib/containers/devicemapper/metadata/8369603abb954aa8c9076570cc7853aa0e5967d52f3f1e980740c7c5970f1d8f: no such file or directory

and just after running the nginx pod (I had to pull the pause image with ocic before restarting ocid):
DEBU[2017-01-12 22:41:36.270567467+01:00] parsed reference into "[devicemapper@/var/lib/containers]docker.io/kubernetes/pause:latest"
2017/01/12 22:41:36 grpc: Server failed to encode response proto: Marshal called with nil

In this last case we're probably doing something where the response returned to grpc is nil (I didn't look in the code though)

The second error from my previous comment is easily (100% for me) reproducible by just trying to start a pod (nginx in my case).

@runcom Do we have heavier "conformance" type tests that would check common apps like nginx, like you're doing? Just wondering if there's some automation we could add to help with that, even if they're only run by CI or only run occasionally (due to being too heavy for normal day-to-day use)

@runcom We were mistakenly clearing err at sandbox_run.go:74 and sandbox_run.go:214, and returning both a nil result and no error isn't something the rpc layer expects.  That should be fixed now.

@runcom We were mistakenly clearing err at sandbox_run.go:74 and sandbox_run.go:214, and returning both a nil result and no error isn't something the rpc layer expects. That should be fixed now.

awesome, @sameo was also seeing issues while integrating this PR to work with k8s

A CRI client can't be expected to know that it needs to tell us to pull the pause image, so changed the patch that disables implicit image pulling to still pull pause if we need it.

Rebased.

Added detail to the error message that we return when we don't have a given image, and didn't pull it.

My image pull problem on kubernetes cluster:
  FirstSeen	LastSeen	Count	From			SubObjectPath		Type		Reason		Message
  ---------	--------	-----	----			-------------		--------	------		-------
  18m		18m		1	{default-scheduler }				Normal		Scheduled	Successfully assigned nginx-1978115907-qqxfg to gklab-97a-012
  18m		18m		1	{kubelet gklab-97a-012}				Normal		SandboxReceived	Pod sandbox received, it will be created.
  18m		2m		6	{kubelet gklab-97a-012}	spec.containers{nginx}	Normal		Pulling		pulling image "nginx"
  16m		17s		6	{kubelet gklab-97a-012}	spec.containers{nginx}	Warning		Failed		Failed to pull image "nginx": rpc error: code = 4 desc = context deadline exceeded
  16m		17s		6	{kubelet gklab-97a-012}				Warning		FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "nginx" with ErrImagePull: "rpc error: code = 4 desc = context deadline exceeded"

  16m	4s	22	{kubelet gklab-97a-012}	spec.containers{nginx}	Normal	BackOff		Back-off pulling image "nginx"
  16m	4s	22	{kubelet gklab-97a-012}				Warning	FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "nginx" with ImagePullBackOff: "Back-off pulling image \"nginx\""

@jjlakis could you provide logs from ocid and the kubelet?

For some reason you're getting a timeout while pulling the image :confused: could a race somewhere?

mmm this branch isn't working with kubernetes master anymore. I'll update k8s API in master and provide a fix for this PR once @nalind rebase.

@nalind once https://github.com/kubernetes-incubator/cri-o/pull/321 is merged and you rebase here, could you cherry-pick https://github.com/runcom/ocid-1/commit/080949a842325ace76251eb2aada3b0d3465cce6 to fix containers creation?

So, with latest k8s master+this PR I'm getting:
Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath           Type            Reason          Message
  ---------     --------        -----   ----                    -------------           --------        ------          -------
  5m            5m              1       {default-scheduler }                            Normal          Scheduled       Successfully assigned nginx-2932712833-czxrb to <IP_ADDRESS>
  5m            5m              1       {kubelet <IP_ADDRESS>}                             Normal          SandboxReceived Pod sandbox received, it will be created.
  3m            3m              1       {kubelet <IP_ADDRESS>}     spec.containers{nginx}  Warning         Failed          Failed to pull image "nginx": rpc error: code = 4 desc = context deadline exceeded
  3m            3m              1       {kubelet <IP_ADDRESS>}                             Warning         FailedSync      Error syncing pod, skipping: failed to "StartContainer" for "nginx" with ErrImagePull: "rpc error: code = 4 desc = context deadline exceeded"

  5m    2m      2       {kubelet <IP_ADDRESS>}     spec.containers{nginx}  Normal  Pulling pulling image "nginx"
  53s   53s     1       {kubelet <IP_ADDRESS>}     spec.containers{nginx}  Normal  Pulled  Successfully pulled image "nginx"
  53s   53s     1       {kubelet <IP_ADDRESS>}     spec.containers{nginx}  Normal  Created Created container with id 2b0d2e527f26207ab1e9ec7c88ff1604cc54cfa5a8744a4d293f115042972867
  53s   53s     1       {kubelet <IP_ADDRESS>}     spec.containers{nginx}  Normal  Started Started container with id 2b0d2e527f26207ab1e9ec7c88ff1604cc54cfa5a8744a4d293f115042972867

which means the container came up eventually (and it works fine!). Still getting that weird context deadline exceeded though which is likely the cause of @jjlakis's ImagePullBackOff. I'll investigate it further.

@mrunalp @nalind @sameo @jjlakis figured out the ImagePullBackOff issue. It's just a matter of starting the kubelet with --runtime-request-timeout=5m (or whatever timeout duration you deem appropriate).

@runcom I saw my first k8s pod on top of CRI-O with the timeout addition :-D
CRI-O master + this PR, k8s@dca0b2a73d6daf89924c1aa7469bd3ea63cf1f5f + kubelete started with ca0b2a73d6daf89924c1aa7469bd3ea63cf1f5f

Rebased.

Ok, idle cluster (1 master, 2 nodes, one of them with cri-o, v1.5.1) works for:

this PR with my networking fix #316
@runcom's fix-189 branch with newest k8s + my networking fix

by "works" I mean that all pod microservices (kube-proxy, flannel, etc.) are running together with its infra-containers what I can check by runc list.
I added --runtime-request-timeout=5m to kubelet and ImagePullBackOff does not appear, but there's another problem...  Image is pulled, container is created, started, then it crashes and... image is pulled again.
  FirstSeen	LastSeen	Count	From			SubObjectPath			Type		Reason		Message
  ---------	--------	-----	----			-------------			--------	------		-------
  3m		3m		1	{default-scheduler }					Normal		Scheduled	Successfully assigned nginxtest-3772885699-cbcnb to gklab-97a-012
  3m		3m		1	{kubelet gklab-97a-012}					Normal		SandboxReceived	Pod sandbox received, it will be created.
  21s		21s		1	{kubelet gklab-97a-012}	spec.containers{nginxtest}	Normal		Pulled		Successfully pulled image "nginx"
  21s		21s		1	{kubelet gklab-97a-012}	spec.containers{nginxtest}	Normal		Created		Created container with id 0d534d33772b573d2c5ff655b9057d1f4ac58c7f32ccf212836efbec9bb9990f
  21s		21s		1	{kubelet gklab-97a-012}	spec.containers{nginxtest}	Normal		Started		Started container with id 0d534d33772b573d2c5ff655b9057d1f4ac58c7f32ccf212836efbec9bb9990f
  3m		20s		2	{kubelet gklab-97a-012}	spec.containers{nginxtest}	Normal		Pulling		pulling image "nginx"

Crashed container the node (runc list):
default-nginxtest-3772885699-cbcnb-0-infra                         122515      running     /run/containers/aufs-containers/ccb203f2ff14df363259529e7ce83f6d32eb02fe7f40ea17bbe9ec52d81d7247/userdata   2017-01-17T15:08:33.31557929Z
default-nginxtest-3772885699-cbcnb-0-nginxtest-0                   0           stopped     /run/containers/aufs-containers/0d534d33772b573d2c5ff655b9057d1f4ac58c7f32ccf212836efbec9bb9990f/userdata   2017-01-17T15:12:08.282083761Z

Assuming that I did something wrong and container crashed because of trivial reason and kubelet simply tries to run it again - why image is pulled again? And what is pretty weird - kubelet decides to send PullImage to runtime again (maybe kubelet doesn't know that image is already pulled?). Working on this right now

Assuming that I did something wrong and container crashed because of trivial reason and kubelet simply tries to run it again - why image is pulled again? And what is pretty weird - kubelet decides to send PullImage to runtime again (maybe kubelet doesn't know that image is already pulled?). Working on this right now

very likely something wrong with ImageStatus reporting the image doesn't exist and k8s tries to pull it again.

@jjlakis after it crashes, is ocid logging another PullImageRequest?  If it isn't, then I'm not sure why it's being re-pulled.

@runcom Got the same feeling, adding debug logs to ImageStatusResponse right now
@nalind Yes, as I wrote before, kubelet sends PullImageRequest

Did you'll check the image pull policy?

On Jan 17, 2017, at 7:22 AM, JJ Łakis<EMAIL_ADDRESS>wrote:
@runcom Got the same feeling, adding debug logs to ImageStatusResponse right now
@nalind Yes, as I wrote before, kubelet sends PullImageRequest
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@runcom cherry-picked your patch, thanks!

I saw this error while testing with k8s:
Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath   Type            Reason          Message
  ---------     --------        -----   ----                    -------------   --------        ------          -------
  13s           13s             1       {default-scheduler }                    Normal          Scheduled       Successfully assigned mynginxtest-2-1578605517-q1bls to <IP_ADDRESS>
  13s           13s             1       {kubelet <IP_ADDRESS>}                     Normal          SandboxReceived Pod sandbox received, it will be created.
  10s           10s             1       {kubelet <IP_ADDRESS>}                     Warning         FailedSync      Error syncing pod, skipping: failed to "CreatePodSandbox" for "mynginxtest-2-1578605517-q1bls_default(048ae8e3-dd12-11e6-acc0-52540003a09a)" with CreatePodSandboxError: "CreatePodSandbox for pod \"mynginxtest-2-1578605517-q1bls_default(048ae8e3-dd12-11e6-acc0-52540003a09a)\" failed: rpc error: code = 2 desc = error creating pod sandbox with name \"default-mynginxtest-2-1578605517-q1bls-0\": Error writing blob: devmapper: Error saving transaction metadata: devmapper: Error creating metadata file: open /var/lib/ocid/devicemapper/metadata/.tmp827313610: no such file or directory"

Worked for me after I cleaned up /var/lib/ocid and restarted everything. Needs rebase :)

@nalind Sorry about all the rebase hell :cry: One of my changes renamed vendor/src to vendor, so if you run into trouble, it may be easiest to remove the commit containing the new dependencies, and re-create them with the new build system. Since it's my fault, I'd be happy to do the rebasing for you if you'd like, and otherwise I'm here if you have questions about what I did in #320

Rebased, adding debug statements in the handlers for image-related requests that log the results.  Updated vendoring results now also pull in github.com/pkg/errors and github.com/opencontainers/go-digest, and the updated github.com/opencontainers/runtime-tools generator's AddProcessEnv method now takes two arguments.

Hmm, I'm not that familiar with travis integration, but it doesn't seem to have run for this last rebase.  https://www.traviscistatus.com/ says everything should be working, though.  Any advice?

Hmm, I'm not that familiar with travis integration, but it doesn't seem to have run for this last rebase. https://www.traviscistatus.com/ says everything should be working, though. Any advice?

mmm https://travis-ci.org/kubernetes-incubator/cri-o/pull_requests seems like anything has been triggered from this PR. Could you amend the last commit and re-push? If that doesn't work, it should be something on their end definitely.

Sure.

@nalind CI triggered, you can remove that test commit now.

Okay, CI seems to be back.

LGTM

LGTM

LGTM
I'm merging this one right now - there are still issues around devicemapper though and maybe c/storage, we'll tackle these in follow up since I don't believe there's anything left to do as part of this PR (it's better to iterate on simple changes from now on) - for instance, I had to increase the timeout to 15m to be able to run  2 pods spawned at the same time, there seems to be some kind of weird locking which prevents even k8s itself to ping the CRI-O, as said, let's tackle this kind of stuff in follow up. Thanks a lot @nalind and everyone who helped with this :)

:tada: :tada: :tada: :tada: :tada: :tada: :tada: :tada: :tada:
:tada: :tada: :tada: :tada: :tada: :tada: :tada: :tada: :tada:
:tada: :tada: :tada: :tada: :tada: :tada: :tada: :tada: :tada:
Awesome! Great to see this, looking forward to giving it a spin :smile:


Update .stickler.yml

Do not merge !!!
This is a work-in-progress PR to investigate stickler-ci configuration

@markstory I'm having trouble configuring stickler-ci. Could I call on your kind offer of help?
I'm expecting compliance failures (but I'm not getting any) and I'm attempting to restrict stickler-ci to check PRs against the REFACTORING/backend branch (only for now) and to only analyse and check the esmvaltool/backend directory ...
As a test, I created a separate dummy repo and copied the esmvaltool/backend files into it, enabled stickler-ci and bingo, it does a grand job at identifying loads of issues, as I expected, see https://github.com/bjlittle/stickler-test/pull/1. So stickler-ci is working on that repo, but not this one ... :confused:
What am I doing wrong here? Any help would be greatly appreciated.

@bjlittle Stickler only reviews changes in pull requests. Until the .stickler.yml is in master, no pull requests in this repository will be reviewed.

@markstory Thanks. Makes sense. Only new or changed files in the PR are reviewed, not the existing files in the repo that aren't part of the PR.
So the .stickler.yml must be on the master branch, not a feature branch, right? So does that mean when stickler-ci ran against this PR it was pretty much a NOP? i.e. it ran but reviewed nothing?

So the .stickler.yml must be on the master branch, not a feature branch, right?

The config file will be read from the pull request HEAD commit, not the default branch.

So does that mean when stickler-ci ran against this PR it was pretty much a NOP? i.e. it ran but reviewed nothing?

Yes, because this pull request has no modified python files.

Okay, so after some testing it appears that stickler-ci isn't working with branch names that include a / character.
I've touched base with @markstory the sticker-ci core dev, who has provided some awesome support :+1: He will investigate further and hopefully resolve this issue for us on the stickler-ci side.
I'm going to close this PR and wait for a fix.


graphql: Populate ImageSummary missing fields:

Description, Labels, Licenses, Title, Documentation, Source
closes #786
Signed-off-by: Petu Eusebiu<EMAIL_ADDRESS>
What type of PR is this?

Which issue does this PR fix:
What does this PR do / Why do we need it:
If an issue # is not available please add repro steps and logs showing the issue:
Testing done on this change:

Automation added to e2e:

Will this break upgrades or downgrades?
Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

LayerSummary is populated only globalSearch, is that ok?

eg:     "RepoListWithNewestImage": [
{
"Name": "alpine2",
"NewestImage": {
"Tag": "latest",
"Digest": "09adf9e2597f9b589a13fba3a056058f9209ba10eedfc5bcd8caa0816b90ffec",
"LastUpdated": "2022-07-18T21:00:15.800417218Z",
"ConfigDigest": "b72857f16e642ea7fbb8d6febebd00278f366fbf2d15294d784b8edd473dbde0",
"IsSigned": false,
"Size": "2799739",
"Platform": {
"Os": "linux",
"Arch": "amd64"
},
"Vendor": "",
"Score": 0,
"Description": "",
"Title": "",
"Documentation": "",
"Licenses": "",
"Labels": "",
"Source": ""
}
},


getUUID not available for cassandraSql

when i use spark sql , I am not able to get UUID from cassandra .
If i use getString then it throws a cast exception.
Is there a work around for this.

This issue has been quiet for a while now, so I will close it out.
If you still need help on this, please raise your question on the Spark Connector User's list here:
https://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user


Cannot convert undefined or null to object

Hi,
I am trying to use the api bug strange error is coming: Cannot convert undefined or null to object
https://runkit.com/npm/katal

Could you please post the full code snippet which caused this bug?


Swagger Doc incorrect for Convert functionality

Describe the bug
The swagger doc for the convert api has an incorrect url. The doc shows this:
/api/convert/{template}
The actual url is this:
/api/convert/hl7v2/{template}
To Reproduce
Steps to reproduce the behavior:

Navigate to converterurl/api-docs/#/
Scroll to /api/convert/{template}
Notice it is incorrect
Attempt to send a post request with an hl7v2 message to that endpiont
Notice http 502 error
Send a post request to updated uri: /api/convert/hl7v2/{template}
Notice http 200 success

Expected behavior
Swagger doc should accurately reflect api
Screenshots
Problem:

Does not match:

Browser:
n/a
Additional context
Recent update to the converter changed the URI from hl7 to hl7v2, and swagger documentation did not reflect this.

The {template} names itself contains  'hl7v2/' or 'cda/' prefixes.  For example, 'cda/ccd.hbs' and 'hl7v2/ADT_A01.hbs'. You can get the list of template names using the GET /api/templates.


Partially revert #18070 (Add support for Windows CA certificate store)

As discussed at OTC. Blocker for 3.2. I kept and tweaked the documentation and the store itself.

Updated.

Merged.


More space for dispensation fields. Remove eSaude name and logo

Coverage remained the same at 62.718% when pulling b226f2c8db9eb45a399b35dbe07b304300e02b86 on valvijo:dispensation-fixes into c0130549cf5db33c6c4b0cc7b72630c1149354a2 on esaude:master.


v1.1.1

Fix:

change from blockDefaults to blockDefaultProps and stubBlockProps

Updates:

"@lowdefy/block-tools": 1.0.1-alpha.13

Codecov Report

Merging #21 into main will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #21   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines           35        40    +5
  Branches        11        11
=========================================
+ Hits            35        40    +5

Impacted Files
Coverage Δ

src/blocks/ContainerBlock/ContainerBlock.js
100.00% <100.00%> (ø)

src/blocks/ContextBlock/ContextBlock.js
100.00% <100.00%> (ø)

src/blocks/DisplayBlock/DisplayBlock.js
100.00% <100.00%> (ø)

src/blocks/InputBlock/InputBlock.js
100.00% <100.00%> (ø)

src/blocks/ListBlock/ListBlock.js
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54cc40a...13a630f. Read the comment docs.


getColumnSummary cleanup

Description
only runs the expensive bits if it's actually needed now (which is often, since most queries have a number column)
Checklist

[ ] For UI or styling changes, I have added a screenshot or gif showing before & after
[ ] I have added a changeset

Approving for now; for phase 2 we should sub this for the SQL statements


Add logs to VMware instance configuration process

Description
The virtual machine configuration process is used in a lot of VMware workflows, such as instance creation, start and scale. However, there are no logs in this process, which difficults the troubleshooting of errors, specially of those that are returned by VMware.
This PR adds logs to this process in order to make the troubleshooting of errors easier.
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[X] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[X] Minor

Screenshots (if appropriate):
How Has This Been Tested?
In a local lab:

I created a VM using only IDE controllers;
I stopped the VM;
I changed the controllers of the VM from IDE to SCSI;
I started the VM, and verified that the logs presented the reconfigurations of the VM in order to remove the disks and NICs, to add the SCSI controllers, and the general VM configuration.

ubuntu@lab-vmware-mgmt-01:~$ sudo grep 'Reconfiguring virtual machine' $mgmt_log
2024-05-14T11:17:15,122 DEBUG [c.c.h.v.m.VirtualMachineMO] (DirectAgent-147:[ctx-9fd2c09e, <IP_ADDRESS>, job-783/job-784, cmd: StartCommand]) (logid:64cbc567) Reconfiguring virtual machine [{"internalCSName":"i-2-217-VM"}] using spec [{"deviceChange":[{"operation":"REMOVE","device":{"capacityInKB":51200,"capacityInBytes":52428800,"shares":{"shares":1000,"level":"NORMAL"},"storageIOAllocation":{"limit":-1,"shares":{"shares":1000,"level":"NORMAL"},"reservation":0},"diskObjectId":"518-3001","key":3001,"deviceInfo":{"label":"Hard disk 1","summary":"51,200 KB"},"backing":{"diskMode":"persistent","split":false,"writeThrough":false,"thinProvisioned":true,"uuid":"6000C29b-5522-5947-c391-e29f44c11a73","contentId":"fb640b892cf0174da714e57339203aa5","digestEnabled":false,"sharing":"sharingNone","fileName":"[c2b40d03a7b132fc928f7f9f39cf4297] i-2-217-VM/ROOT-217.vmdk","datastore":{"value":"datastore-22","type":"Datastore"},"backingObjectId":""},"controllerKey":200,"unitNumber":1}},{"operation":"REMOVE","device":{"addressType":"manual","macAddress":"02:01:00:d0:00:04","wakeOnLanEnabled":true,"resourceAllocation":{"reservation":0,"share":{"shares":50,"level":"NORMAL"},"limit":-1},"uptCompatibilityEnabled":false,"key":4000,"deviceInfo":{"label":"Network adapter 1","summary":"cloud.guest.615.200.1-vSwitch0"},"backing":{"network":{"value":"network-380","type":"Network"},"deviceName":"cloud.guest.615.200.1-vSwitch0","useAutoDetect":false},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":false,"status":"untried"},"slotInfo":{"pciSlotNumber":32},"controllerKey":100,"unitNumber":7}}]}].
2024-05-14T11:17:16,964 DEBUG [c.c.h.v.m.VirtualMachineMO] (DirectAgent-147:[ctx-9fd2c09e, <IP_ADDRESS>, job-783/job-784, cmd: StartCommand]) (logid:64cbc567) Reconfiguring virtual machine [{"internalCSName":"i-2-217-VM"}] using spec [{"deviceChange":[{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":0,"key":-4}},{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":1,"key":-3}},{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":2,"key":-2}},{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":3,"key":-1}}]}].
2024-05-14T11:17:29,603 DEBUG [c.c.h.v.m.VirtualMachineMO] (DirectAgent-149:[ctx-a1985c08, <IP_ADDRESS>, job-783/job-785, cmd: StartCommand]) (logid:64cbc567) Reconfiguring virtual machine [{"internalCSName":"i-2-217-VM"}] using spec [{"guestId":"otherLinux64Guest","numCPUs":1,"memoryMB":512,"memoryHotAddEnabled":false,"cpuHotAddEnabled":false,"deviceChange":[{"operation":"EDIT","device":{"key":3000,"deviceInfo":{"label":"CD/DVD drive 1","summary":"Remote device"},"backing":{"exclusive":false,"deviceName":""},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":200,"unitNumber":0}},{"operation":"ADD","device":{"capacityInKB":0,"key":-2,"backing":{"diskMode":"persistent","fileName":"[c2b40d03a7b132fc928f7f9f39cf4297] i-2-217-VM/ROOT-217.vmdk","datastore":{"value":"datastore-22","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":0}},{"operation":"ADD","device":{"addressType":"Manual","macAddress":"02:01:00:d0:00:04","key":-3,"backing":{"network":{"value":"network-380","type":"Network"},"deviceName":"cloud.guest.615.200.1-vSwitch0"},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":true}}}],"cpuAllocation":{"reservation":0,"limit":-1},"memoryAllocation":{"reservation":0,"limit":512},"extraConfig":[{"key":"machine.id","value":""},{"key":"devices.hotplug","value":"true"},{"key":"nvp.vm-uuid","value":"ee781edb-bb7b-4d15-8dfe-88b582bc1419"},{"key":"nvp.iface-id.0","value":"47973c05-5956-4c89-a0d8-b7ef218cb7de"},{"key":"dataDiskController","value":"scsi"},{"key":"Message.ReservedCapacityFreed.Flag","value":"false"},{"key":"nestedVirtualizationFlag","value":"false"},{"key":"vmware.reserve.mem","value":"false"},{"key":"vmware.reserve.cpu","value":"false"},{"key":"nicAdapter","value":"E1000"},{"key":"rootDiskController","value":"scsi"},{"key":"RemoteDisplay.vnc.enabled","value":"true"},{"key":"RemoteDisplay.vnc.password","value":"xDys93MKuuAJTnvoPya1tw"},{"key":"RemoteDisplay.vnc.port","value":"5918"}]}].

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9613

@blueorangutan package

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9732

Did you test @hsato03 ?

clgtm in principal. Did you check if this object is not toString()d anywhere else, @winterhazel ?

@DaanHoogland yes, its toString() is not used anywhere, as its not even implemented. Whenever we need to log this object, we're converting it to JSON.

ok, so no default or inherrited toString used anywhere? great.

@blueorangutan package

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 10454

@DaanHoogland @rohityadavcloud can we run the CI?

@DaanHoogland a [SL] Trillian-Jenkins test job (alma8 mgmt + vmware-70u3) has been kicked to run smoke tests

[SF] Trillian test result (tid-11425)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server a8
Total time taken: 51027 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9080-t11425-vmware-70u3.zip
Smoke tests completed. 135 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_06_purge_expunged_vm_background_task
Failure
348.12
test_purge_expunged_vms.py

test_02_restore_vm_with_disk_offering
Error
63.32
test_restore_vm.py

test_03_restore_vm_with_disk_offering_custom_size
Error
58.23
test_restore_vm.py


[1.20.1] Connector 19 Crash

Describe the bug
crashes on startup process exited with code 2, attached is latest.log
Steps to reproduce
1.Install Connector 19 and Connector Extras 1.4.0
2.Crash
3.
...
Logs
https://gist.github.com/HaizUber/fa3742a0bc949620eb65f260b1bcaa0c
Additional context
can provide more info like modlist and other stuff related with the log itself when needed, no crash-report is generated

its exordium

Use the Forge version of exordium, which is compatible


Engage filter logic by vintage year- allowance sub types

Definition of Ready

[x] Does this meet the INVEST criteria?
[x] Does this have a clearly defined user?
[x] Does this have an estimate?
[x] Does this have acceptance criteria?
[x] Does this have sufficient context for the specific functionality being developed (design, business rules, etc.)
[x] Does this have a dev link been provided in the design?
[x] Is this traceable to a feature, epic, or PI?

User Story
As a custom data download user I want the filter logic to engage when I select vintage year(s) So that I can avoid producing an allowance query with no results
Acceptance Criteria
Given that a user has applied account info or allowance holdings as a data sub type
When the filter panel functionality is displayed
Then the filter logic is engaged
Given that a user has selected an allowance sub type (allowance holdings)
When filter criteria is applied for vintage year
Then all other filters are narrowed based on the applied vintage year criteria
Given that the filter logic is engaged
When new filter criteria is applied
Then all previously selected criteria that are still valid are maintained
Given that the filter logic is engaged
When the filter panel functionality is displayed
Then non applicable selections are removed from the combo box filter list
AND grayed out from the check lists
Given that the filter logic is engaged
When vintage criteria is removed (ex. filter tags, filters)
Then the filters are narrowed based on the applied criteria
Given that a user has selected vintage year
When the the filter panel functionality is displayed
Then the system displays a combo box with the list of vintage years
Given that the filter logic is engaged
When the "select all" functionality is selected on the checklist component
Then the system selects criteria that is not greyed out
Given that the filter logic is engaged
When the checklist functionality is displayed
Then the select all functionality is disabled for groupings that only contain greyed out criteria
Development Notes
The scope of this ticket excludes account type
Does this Story require 508 Testing?

[ ] Implementing a new authoring tool that will produce code / content for the application
[ ] Implementing a 3rd party widget that will be part of the application
[x] Adding new / updating custom functionality
[ ] Adding new / updating a page template
[ ] Adding new / replacing existing functionality (unless the replacement has already passed 508 testing)
[ ] Full Site Review prior to a scheduled EPA 508 review
[ ] Planning a video that will appear on the application
[ ] Producing a video that will appear on the application
[ ] Producing Training materials
[ ] Producing documents

Context
(User journeys, design mockups, architectural diagrams, business rules, data element definitions)
Testing
Test Case: TC469
Tester: @JanellC
Definition of Done

[x] Have all Acceptance Criteria been agreed to and validated by the scrum team?
[x] Has the code been unit tested and peer reviewed?
[x] Have the functional tests been executed?
[x] Have all defects been reviewed and dispositioned, resolved, or deferred?
[x] Is the user story ready to be deployed to the test/staging environments?

TC469 passed @tlageman-epa . This ticket is ready for 508 testing


Make hprose aware of CommonJS

I am planning on writing an RPC layer that is capable of running on "almost" all frameworks. In order to properly support this, hprose-js needs to learn to export. This is simple. As you know NodeJS, this will look very familiar to you:
module.exports = hprose;

The idea is, to have hprose initialize on a local variable first (hprose in my case). Then we can do something like jQuery does here: https://github.com/jquery/jquery/blob/2.1.4/dist/jquery.js#L15-L38
Do you think this is possible?
Kind regards,
Ingwie

CommonJS is very useful on:

Server-side JavaScript applications
Command line tools
Desktop GUI-based applications
Hybrid applications (Titanium, Adobe AIR)

But hprose-js can only work on browser-applications, it can't work on server-side.
hprose-nodejs is a node.js (CommonJS) module, it can work on everywhere except browser.
hprose-html5 is another hprose implementation for browser-applications, and it also can work on Hybrid applications (Titanium, Adobe AIR, cocos2d-js jsb applications). I added requirejs and seajs support for this version.
I think to add requirejs and seajs support for  hprose-js is more useful than making hprose-js aware of CommonJS. Do you think so?
Best regards,
Ma Bingyao

RequireJS - which basically is CommonJS - is becoming very, very popular these days. In fact, jQuery, for instance, handles both styles, AMD and CommonJS at once, and allows to be ran inside NodeJS if a window-like object can be specified, so they say.
What I would suggest for hprose-js is that it implements at least CommonJS. It gives developers the option to smoothly move their code to another platform without losing much features or having a big burden of converting their code.
Actually I don't know what the difference between hprose-js and hprose-html5 is. It looks very similar from a short glance.
So my opinion is, that you should add RequireJS support, since that is pretty much CommonJS.
Here is a little template that can maybe help you with getting this going:
(function(root, construct){
    if(typeof module != "undefined" && typeof exports == "object") {
        module.exports = construct();
    } else {
        root.hprose = construct();
    }
})(this, function(){

    // hprose code goes here. Here is an example:
    var hprose = {};
    hprose.foo = function() {}

    return hprose;
    // A module that require()'d this file now gets the hprose object.

});

Kind regards, Ingwie.

features
hprose-js
hprose-html5
hprose-nodejs

binary data
❌
✅
✅

browser applications
✅
✅
❌

old browser
✅
❌
❌

cross domain
flash & cors
cors
-

http client
✅
✅
✅

websocket client
❌
✅
todo

tcp client
❌
❌
✅

Server-side applications
❌
maybe
✅

Command line tools
❌
maybe
✅

Desktop GUI-based applications
maybe
✅
✅

Hybrid applications (Titanium, Adobe AIR)
maybe
✅
✅

http service/server
❌
❌
✅

tcp service/server
❌
❌
✅

unix socket service/server
❌
❌
todo

websocket service/server
❌
❌
todo

Completer/Future async
todo
✅
todo

RequireJS(AMD)
todo or not
✅
❌

SeaJS(CMD)
todo or not
✅
❌

CommonJS
❌
todo
✅

global HproseXXX (such as HproseClient)
✅
❌
✅

hprose.XXX (such as hprose.Client)
❌
✅
✅

hprose-js and hprose-html5 have different API interface, because we want they can work together without conflict.
The significance of the existence of hprose-js is only in order to be compatible with old browsers.
hprose-nodejs is compatible with the above two kinds of API interface. so if the developers want to move the hprose-js or hprose-html5 application to another platform, they don't need to converting any code, only to change the hprose-js or hprose-html5 to hprose-nodejs. and there is no feature losing, instead, they will get more features.
jQuery need to handle AMD and CommonJS at once, because it only have one implementation.
Hprose don't need to handle AMD and CommonJS at once, because hprose have three implementation.
But I can add CommonJS support to hprose-html5, because it is easy to do and more useful than hprose-js.

Oh I see! So if hprose-js is ment for older browsers, then I am likely better off when using hprose-html5.
In the long run, you should consider if it maybe isnt easier to merge the two together instead of maintaining two similar, but not similar, projects at the same time.
For now, I'd suggest adding the CommonJS/AMD support since many people seem to use it. One great usage is WebPack which I am successfuly using in my BIRD3 project - it makes many things much easier.

I will added CommonJS support for hprose-html5, tonight.

Awesome! I can then work with hprose in my app also in client side. :)
By the way, what is this Future-async stuff about?

Completer/Future async is come from dart language.
But I simplify its implementation. I only reserved the following methods:

class
method

Completer
constructor

Completer
complete

Completer
completeError

Future
then

Future
catchError

for example:
    var client = new hprose.Client.create('http://hprose.com/example/');
    client.then(function(stub) {
        stub.hello('World')
        .then(function(result) {
            console.info(result);
        })
        .catchError(function(e) {
            console.error(e);
        })
    })
    .catchError(function(e) {
        console.error(e);
    })

and I will add this feature in PHP, too, with this feature, we can publish an async service.

I already have added CommonJS support for hprose-html5, just now.

hprose for javascript is update.

features
hprose-js
hprose-html5
hprose-nodejs

binary data
:white_check_mark:
:white_check_mark:
:white_check_mark:

browser applications
:white_check_mark:
:white_check_mark:
:x:

old browser
:white_check_mark:
:x:
:x:

cross domain
flash & cors
cors
:white_check_mark:

http client
:white_check_mark:
:white_check_mark:
:white_check_mark:

tcp client
:white_check_mark:
:white_check_mark:
:white_check_mark:

unix socket client
:x:
:x:
:white_check_mark:

websocket client
:white_check_mark:
:white_check_mark:
:white_check_mark:

Server-side applications
maybe
maybe
:white_check_mark:

Command line tools
maybe
maybe
:white_check_mark:

Desktop GUI-based applications
:white_check_mark:
:white_check_mark:
:white_check_mark:

Hybrid applications (Titanium, Adobe AIR)
:white_check_mark:
:white_check_mark:
:white_check_mark:

http service/server
:x:
:x:
:white_check_mark:

tcp service/server
:x:
:x:
:white_check_mark:

unix socket service/server
:x:
:x:
:white_check_mark:

websocket service/server
:x:
:x:
:white_check_mark:

Completer/Future async
:white_check_mark:
:white_check_mark:
:white_check_mark:

RequireJS(AMD)
:white_check_mark:
:white_check_mark:
:x:

SeaJS(CMD)
:white_check_mark:
:white_check_mark:
:x:

CommonJS
:white_check_mark:
:white_check_mark:
:white_check_mark:

global HproseXXX (such as HproseClient)
:white_check_mark:
:x:
:white_check_mark:

hprose.XXX (such as hprose.Client)
:white_check_mark:
:white_check_mark:
:white_check_mark:

Awesome, thanks for the update! :)


sharing provider states?

I'm guessing someone already thought of this, but I couldn't find any documentation on it.
Are there any plans to support Pact.provider_state_for  so that providers with multiple consumers can dry out provider states by asking their consumers to use something that already exists?

I figured out how to do this
['some','values'].each do |consumer|
Pact.provider_states_for consumer do

There are global provider states. Use Pact.provider_state.
On 8 Oct 2016 9:16 AM, "johngluckmdsol"<EMAIL_ADDRESS>wrote:

I figured out how to do this
['some','values'].each do |consumer|
Pact.provider_states_for consumer do
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/realestate-com-au/pact/issues/114#issuecomment-252373950,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAbPFM3KDGeaoJdJv8byrovgfbKtvNSpks5qxsSEgaJpZM4KPjOV
.


Contextual axes

addresses #11

[x] creates axes contextual component
[x] ensure that axes are properly scoped (d3-select is only being applied to the g in its context)
[x] supports having a label
[x] transitions work (axes smoothly transition)
[x] responsivity
[x] adds tests

closes #11


Ensure own password is also hidden when querying information_schema.user_mapping_options

CrateDB version
5.7.1
CrateDB setup information
No response
Problem description
When querying information_schema.user_mapping_options one owns password is shown. passwords and usernames for other users are hidden.

[ ] t.b.d. We might want to show the username values for other users and also display their password as *********, considering only an AL user can query those tables / views

Steps to Reproduce

Create Foreign Server
Create User Mapping for own user
Query SELECT * FROM information_schema.user_mapping_options;

Actual Result
Actual users password is display.
Other users passwords are blank / hidden.
Expected Result
Password is shown as ********* as in sys.users

Btw, this is consistent to Postgres.

option_value character_data
Value of the option. This column will show as null unless the current user is the user being mapped, or the mapping is for PUBLIC and the current user is the server owner, or the current user is a superuser. The intent is to protect password information stored as user mapping option.
https://www.postgresql.org/docs/16/infoschema-user-mapping-options.html

This was also a point of discussion in the initial PR:  https://github.com/crate/crate/pull/15591#discussion_r1506169482
I'd be inclined to keep it as is for now, but we can keep this open to gather some more feedback.


Create the entities with jphister:entity subgenerator

read: https://jhipster.github.io/creating_an_entity.html
Try out in the app, use git as your tool to check differences, revert, etc.
Only commit when 100% sure of the model and the entities

Depends on #3

some things:
I don't see the created_at and updated_at in any entity.
Enrolment
Status should be an ENUM not a open string.
Course
Please explain the line of thinking to decide course points from 0 to 10

Can a course worth 0 points?
The max of a course is 10 points?

status should be an ENUM (currently can be: DRAFT, PUBLISHED)
ALL
We need pagination.

good!!! =) pull request #33 accepted and merged.


Ready to use Dockerfile

Question:
As i understand it a could use this extension to generate a docker image which can be used as a base for a reverse proxy.
So starting the image it could create a device in tailscale and setup up https certs. Then the caddy plugin could forward all traffic to an remote host.
How would i accomplish that or did i misunderstand something?

You could try the instructions on our official Docker image for "Building your own Caddy-based image": https://hub.docker.com/_/caddy

Good idea... I came up with this:
# Stage 1: Build the custom Caddy server with Tailscale plugin
FROM docker.io/caddy:builder AS builder

WORKDIR /app

ARG CADDY_VERSION v2.6.4
RUN apk add --no-cache nss-tools ca-certificates iptables iproute2 ip6tables

# Build Caddy with Tailscale plugin
RUN xcaddy build $CADDY_VERSION --with github.com/tailscale/caddy-tailscale

# Stage 2: Create the final minimal Caddy image
FROM docker.io/alpine:latest

# Copy the Caddy binary from the builder stage
COPY --from=builder /app/caddy /usr/bin/caddy

# Set environment variables, if needed (you can modify this as required)
ENV TAILSCALE_FROM="tailscale/mydevice"
ENV TAILSCALE_TO="localhost:80"

# Run Caddy with Tailscale proxy command, this does not work yet
# CMD ["caddy", "tailscale-proxy", "--from", "${TAILSCALE_FROM}", "--to", "${TAILSCALE_TO}"]

minimal docker-compose.yml
version: '3.9'

volumes:
  tailscale-proxy-test-state:

services:

  nginx:
    image: docker.io/nginx

  tailscale-vpn:
    image: docker.io/dfalkner/tailscale-proxy
    volumes:
      - tailscale-proxy-test-state:/var/lib
    restart: unless-stopped
    env_file: values.env
    entrypoint: "caddy tailscale-proxy --from tailscale/test --to nginx:80"

values.env
TS_AUTHKEY=<AUTH_KEY>
TAILSCALE_FROM="tailscale/test"
TAILSCALE_TO="nginx:80"

but it does not yet work. Tailscale Admin Console shows the device (it also gets an IP) but the command does not seem to be successful. With a lot of tinkering (of TAILSCALE_FROM) i got http to work but never https.
Any help?

That's good progress...

but the command does not seem to be successful.

What is the error?

This error happens when the command is run manually inside the container after starting
cmd
caddy tailscale-proxy --from tailscale/test --to nginx:80

Error during connecting to https://test.tailscaleNNNN.ts.net (with a browser, i expected this to work)
tailscale-vpn_1  | {"level":"info","ts":1690534804.9615376,"msg":"[unexpected] localbackend: got TCP conn w/o serveConfig; from <IP_ADDRESS>:54736 to port 443"}

At this point, I'm guessing @willnorris or @Xe would have a better clue...

Running
caddy tailscale-proxy --from tailscale+tls/test:443 --to nginx:80

(which according to the docs) is needed result in a parsing error. Is TLS not supported by the tailscale-proxy command?

This is the setup I use and can confirm that it works: https://gist.github.com/clly/567713da815f59b97d50e5f712ad1167
The only suggestion I have is to use a Caddyfile to confirm that works as an experiment.

mhh maybe i should have reported back...
I made my own image which works on a local instance.
Published on dockerhub and github here: https://github.com/falknerdominik/image-tailscale-proxy

@falknerdominik Very interesting, thank you! What I don't quite understand yet: How does Caddy know about the Tailscale instance you're starting in the same container? And how does it generate the TLS certificates for the *.ts.net subdomain? I don't see the Tailscale Caddy plugin getting installed anywhere.


Hi, here is the speaker classifier. It might be slightly different from the one we used in this ariticle due to some code refactoring, but the result should be similar.

Hi, here is the speaker classifier. It might be slightly different from the one we used in this ariticle due to some code refactoring, but the result should be similar.
c_in: The dimension of the content/speaker embedding
c_h: We set it to 128.
c_out: The number of speakers, which is 80 in this paper.
import torch.nn as nn
from einops import rearrange

class Classifier(nn.Module):
    def __init__(self, c_in, c_h, c_out):
        super(Classifier, self).__init__()
        self.in_layer = nn.Linear(c_in, c_h)
        self.conv_relu_block = nn.Sequential(
            nn.Conv1d(c_h, c_h, 3),
            nn.ReLU(),
            nn.Conv1d(c_h, c_h, 3),
            nn.ReLU(),
            nn.Conv1d(c_h, c_h, 3),
            nn.ReLU(),
        )
        self.out_layer = nn.Linear(c_h, c_out)

    def forward(self, x):
        """
        x: (n, c, t)
        """
        x = rearrange(x, 'n c t -> n t c')
        y = self.in_layer(x)
        y = rearrange(y, 'n t c -> n c t')
        y = self.conv_relu_block(y)
        y = y.mean(-1)
        y = self.out_layer(y)
        return y

Originally posted by @KimythAnly in https://github.com/KimythAnly/AGAIN-VC/issues/10#issuecomment-854148318

Hi，Can you provide more details on training the speaker classifier? such as how to set labels


styling 404

fix 404 to show null island buoy
putting 404 styles inside of common styles

no longer necessary because of https://github.com/transitland/feed-registry/commit/3260d576f42f62a16fa1a4668aca26e6d36b1176


Add Krita.app v<IP_ADDRESS>

Krita is a free digital painting and illustration application.
As of this version, the Krita website states that OS X support is "very
experimental right now and unstable," but there are no immediately
noticable problems when running on Yosemite (10.10.1).

Thank you for the contribution. It will be merged in a different pull request due to some simple recommended alterations. Your contribution is still included (and still credited to you), with the appropriate modifications, when applicable.
Changes:

Squashed commits.
Added a small note pointing out the download host is officially different from the project’s home.


NoSuchElementException (when loading change)

Addition information:
This happens after 16 hours without shutting down IntelliJ and trying to fetch new gerrit changes. It first doesn't show changes (empty list) and after changing the filter (selecting different Owner in the list) it throws this exception.
Plugin version: <IP_ADDRESS>-142
IntelliJ version: IntelliJ IDEA 15.0.2 IU-143.1184
OS: Linux 3.13.0-79-generic
Java: Oracle Corporation 1.8.0_74
Exception:
java.util.NoSuchElementException
    at java.util.ArrayList$Itr.next(ArrayList.java:854)
    at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:297)
    at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:289)
    at com.urswolfer.gerrit.client.rest.http.changes.ChangeApiRestClient.get(ChangeApiRestClient.java:158)
    at com.urswolfer.intellij.plugin.gerrit.rest.GerritUtil$12.get(GerritUtil.java:401)
    at com.urswolfer.intellij.plugin.gerrit.rest.GerritUtil$12.get(GerritUtil.java:390)
    at com.urswolfer.intellij.plugin.gerrit.rest.GerritUtil$18$1.run(GerritUtil.java:635)
    at com.intellij.openapi.progress.impl.CoreProgressManager$TaskRunnable.run(CoreProgressManager.java:563)
    at com.intellij.openapi.progress.impl.CoreProgressManager$2.run(CoreProgressManager.java:142)
    at com.intellij.openapi.progress.impl.CoreProgressManager.a(CoreProgressManager.java:446)
    at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:392)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:54)
    at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:127)
    at com.intellij.openapi.progress.impl.ProgressManagerImpl$1.run(ProgressManagerImpl.java:126)
    at com.intellij.openapi.application.impl.ApplicationImpl$8.run(ApplicationImpl.java:365)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jetbrains.ide.PooledThreadExecutor$1$1.run(PooledThreadExecutor.java:55)

(Received as exception report.)

This is most probably because of an expired session.
As a workaround, IntelliJ can be restarted when login expired or Gerrit server config can be changed (web_sessions cache).

Since I'm not able to reproduce this issue, it would be really helpful if somebody who can reproduce it could contact me / post feedback here.

I can reproduce this issue - I believe. Reproduction path:

open IntelliJ with access to internal network where dns name for gerrit server will be resolved
open a change set
suspend machine
change network environment so that there is no way to resolve gerrit server dns anymore
open new project inside IntelliJ - there you are.

For me only solution is to reopen project once I'm in office.

If you can reach your sys admin, you could try to extend session expiration time:
[cache "web_sessions"]
    maxAge = 1 y

https://gerrit-review.googlesource.com/Documentation/config-gerrit.html
If you can change this, please let me know if this fixes your issue. In that case, I need to improve session expiration detection in the plugin.

I'm not sure whether it's related, but it seems there's a bug in Gerrit that makes sessions timeout prematurely, see the discussion at https://groups.google.com/d/topic/repo-discuss/Vk6UeWRHx0A/discussion.

Questions for everyone who can reproduce this issue:

Are you using a HTTP password set in Gerrit web UI settings?
If not, can you please try to do so and use these credentials? Does that help?

Fixed in one of the last releases (fix included in gerrit-rest-java-client).


[W5.11][F11-A4]Zhang Zequn

This is the first enhancement: sort

Good that it sorts the persons recorded in the addressbook instead of just printing out a sorted list.


Adaptively split tiles into smaller ones when there aren't enough tiles to render

Closing this ticket as we decided that this feature isn't worth the effort and added complexity. If we do want to do something in this area, the idea from #2752 feels superior anyway.


Add CheckAndSet to Storage interface

@xichen2020

Coverage increased (+0.03%) to 84.216% when pulling c09726dc5dd1fdaf84dca7b3e05ca00987cac9cb on chao-cutover into 56c031689fb080404de0b2acfc385e91627d93d7 on master.

Coverage increased (+0.03%) to 84.21% when pulling 33b08ef49c871e50c375dc207b9cf8e09a50009e on chao-cutover into 56c031689fb080404de0b2acfc385e91627d93d7 on master.


[VS Code] The specified value isn’t a valid Live Share URL. Please check the link provided by the host and try again.

<!--
For Visual Studio problems/feedback, please use the Report

Hi,
Thanks for reporting the issue. Could you please double check with the collaboration host that you got the right URI?
If the issue persists and you are not able to join the collaboration session with the right URI, can you please run the command "Live Share: Export Logs" and drag-and-drop the zip here or send it to<EMAIL_ADDRESS>


Test-DbaLastBackup LSN broken

Bug Report
This may be related to #2182, same error message about LSN broken.
General Troubleshooting steps

Hi,
Can you try again please and see if #2182 did fix the issue or if we have another outstanding issue.
Cheers
Stuart

Closing as no update. Assuming #2182 did fix it.


Connection with prisma-photongo refused.

I created the file prisma.rs like the documentation said:
fn main() {
    prisma_client_rust_cli::run();
}

but when i run the command returns me the following error:
Downloading https://prisma-photongo.s3-eu-west-1.amazonaws.com/prisma-cli-4.8.0-windows-x64.exe.gz to C:\Users\daniloherc\AppData\Local\prisma/binaries\cli\4.8.0\prisma-cli-windows-x64.exe
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("prisma-photongo.s3-eu-west-1.amazonaws.com")), port: None, path: "/prisma-cli-4.8.0-windows-x64.exe.gz", query: None, fragment: None }, source: hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 10061, kind: ConnectionRefused, message: "No connection could be made because the target machine actively refused it." })) }', C:\Users\daniloherc\.cargo\git\checkouts\prisma-client-rust-fa967aa5ad0ec391\895aac7\cli\src\binaries\mod.rs:142:35

 ConnectionRefused, message: "No connection could be made because the target machine actively refused it."

It works on my network, are you behind a proxy or in a restricted network?

No, no. Do you have any idea about what can I do??

Maybe you could download the engine files manually via your browser and put them in the right place? Not much else I can think of sorry.

Okay, thank you!


Uplink warnings for decorator for Python version >= 3.8

Currently using python3.8 causes the following warnings (when running the tests via pytest):
site-packages/uplink/clients/io/asyncio_strategy.py:26: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead

There is a related issue in uplink opne issue that has some issues with using uplink with python3.8. This is another issue.
We should keep an eye on thsi as it will likely cause problems in the future.

Is this fixed? I think it's still present right?

Yeah, maybe got a little too trigger happy tidying up. Will re-open

Solved in #91 🎉


Wasm Windows CI

This adds wasm to windows ci along with few fixes

Yes the compiler was throwing error for non constant array size. I don't have idea about the malloc leaking I am calling free at the end.

Something I found with wasm on windows
1- Unknown array length error
2- func i32_f32_i64_f64_ret_i32_f32_i64_f64 is retuning invalid result in test
3- scripts  are not being loaded properly, e.g. empty_module.wasm is loaded from Debug folder not from script path
4-  wasmtime dll and lib not being copied to build output

Solved.


QuotaExceededError on Safari Mobile while Private browsing

The user get a QuotaExceededError on Safari Mobile while Private mode is enabled and Parse try to use the local storage.
Apparently I found on Stackoverflow that Safari Mobile in Private mode does not allow local storage to be used at all.
Source: http://stackoverflow.com/questions/21159301/quotaexceedederror-dom-exception-22-an-attempt-was-made-to-add-something-to-st
Something needs to be done, because it's broken all the javascript.

Certain behaviors (like caching Config or persisting Installation IDs) depend on local storage. However, we know that this is an issue with Safari on Private Browsing, and so we allow developers to replace the Storage Controller at runtime.
How you do this depends on how you are using the SDK, but you can call Parse.CoreManager.setStorageController() and replace the one being used. If you do not need any sort of persistence features, I recommend either using the StorageController.default.js controller found in the npm module and this repo, or simply implementing a no-op controller yourself.

Thanks for the solution. I think it should be written somewhere that Safari Mobile in Private mode broke the local storage because I don't think it's well known and it's never easy to debug on Safari Mobile (even impossible if you don't have a Mac).
Or you could throw an alert or something like that :
try { localStorage.test = 2; } catch (e) {
    alert('Local storage is disable...');
}

@delux75 I'll wrap the functionality of StorageController.browser.js to catch any exceptions. Even if it won't save anything to disk, it'll keep the SDK functional

That's a great solution imho.


Cannot use multiple sourceDirectories

I have this config on my pom.xml:
..

src/main/groovy,src/main/java
...
However, only *.java files of src/main/groovy source directory are getting formatted

You have to provide me a way to reproduce the issue. A public git repository for example.

On any project with multiple sourceDirectories (src/main/java, src/main/java2)
with only first one configured and second one imported with:

org.codehaus.mojo
build-helper-maven-plugin

initialize

add-source

src/main/java2

Executing hook with "initialize" phase  before "on-pre-commit" goal would fix it because second directory is configured when "build-helper-maven-plugin" is executed . (It is the workaround we use on manual execution formating). However, I don't know how to work with precommit hook in order to ensure "hook" format on both source directories.

@jvicens19 I maintain this library on my free time. If you want me to solve the issue quickly, make my life easy by providing a git repository reproducing it.
If you can’t, don’t expect the issue to be solved soon.

@jvicens19 , as explained in https://github.com/Cosium/git-code-format-maven-plugin/issues/27#issuecomment-550973559, all plugin goals (including on-pre-commit goal) process all *.java files provided by MavenProject#getCompileSourceRoots and MavenProject#getTestCompileSourceRoots.
Comment this issue with a link to a reproduction case if you want me to reopen it.

Hi, I am sorry that the project that reproduces it is not sharable. I will try to reproduce it in a small project and let you see that.
Now I am going to explain better the issue:

MavenProject#getCompileSourceRoots initially includes one source directory defined in ${project.build.sourceDirectory}
Using multiple source directories is possible using a mvn plugin: https://www.mojohaus.org/build-helper-maven-plugin/usage.html
That build-helper-maven-plugin runs on "initialize" maven fase. That means that MavenProject#getCompileSourceRoots is not updated with additional directories after initialize phase
Pre commit hook runs: com.cosium.code:git-code-format-maven-plugin:on-pre-commit.
that goal "on-pre-commit" runs independently and obviously not include "inititalize" phase that would add additional directories.
Running "mvn initialize git-code-format:format-code -Dgcf.globPattern=**/*" solves the issue and do format all directories sources.

It is possible to include initialize phase on the hook *.sh to solve this issue?

@jvicens19 , when I add initialize to the pre commit hook shell, tests fail.


Search interface improvements

Should be tested with https://github.com/bastianjoel/openslides-search-service/tree/all
A search.yml can be found here: https://gist.github.com/bastianjoel/2badee016943ab914af35c37c8fb447c
TODO:

[x] Handling of special characters (search service)
[x] Words infront of html tags not searchable (search service)
[x] Disable fuzzy (search service)
[x] Display submitter names
[x] Add search send button
[x] Remove meeting hyperlinks
[x] Change title color to url color
[x] Rename participant result type to account
[x] Search by tag not working
[x] Search by user in participants, submitters, speakers, candidate (might also require search service change)
[x] Mediafile in meeting not searchable (search service)
[x] Display motion number infront of motion title
[x] Meeting sometimes not displayed (search service)
[x] Always display all filters on all meetings search
[x] Fix dark mode
[x] Fix participant hyperlink

[ ] Under the search field a grey smaller sentence: "To search press Enter or the magnifier"
[ ] There should be a loading bar if it is still searching
[ ] The distance between the search field and the filtern should be bigger (see mock-up)
[ ] Search field should be grey (see mock-up)
[ ] If a the phone view is active (<=700px) then the filters should be each in their own line
[ ] the amount of displayed results should be displayed

@bastianjoel I will merge now, please extract important info and TODOs into a new issue as discussed


add pallas stake snapshots integration

Pre-submit checklist

Branch

[x] Tests are provided (if possible)
[x] Crates versions are updated (if relevant)
[x] Commit sequence broadly makes sense
[x] Key commits have useful messages

PR

[x] No clippy warnings in the CI
[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

Documentation

[ ] Update README file (if relevant)
[ ] Update documentation website (if relevant)
[ ] Add dev blog post (if relevant)

@jpraynaud ready for review

Nothing to say after the review.
Good work.

Hi @falcucci, can you bump the crates versions so that we merge this PR?

@jpraynaud done!


In trying to understand why our HTML to LateX parser sometimes produced incorrect entries (sentence endings that didn't make sense), we found an explanation in the html tokenizer.
In fact, when using CRLF (\r\n) line breaks, we found that calling the Token() method caused some invalid entries when calling Raw() (doubled characters).
The documentation does mention that calling Token() can modify Raw entries, but I can't see why my characters would be doubled.
Here is an exemple of the issue
What did you see happen?
Parsing ABC\r\nDEF\r\nGHI\r\n is returning:
ABC
DEF
GHI
I

What did you expect to see?
ABC
DEF
GHI

Related Issues

x/net/html: Text() corrupts underlying tokenizer buffer #43268 (closed)
x/net/html: html.Parse chomps too much leading newline in <pre> #27807 (closed)
x/net/html: Subsequent Token method calls in Tokenizer don't return same result #22621 (closed)
x/net/html: tokenizer error #34281 (closed)
x/net/html: readRawOrRCDATA() parsing issue when text ends with '<' #20741
x/net/html: not all HTML entities are escaped #38008 (closed)
x/net/html: Tokenizer - incorrect handling of attribute states when U+002F SOLIDUS (/) is encountered #63402 (closed)

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

CC @neild maybe?


When running "PDBClean_MolID_CIF.py" problem reading citation that uses multiple lines

Traceback (most recent call last):
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/Bio/File.py", line 72, in as_handle
with open(handleish, mode, **kwargs) as fp:
TypeError: expected str, bytes or os.PathLike object, not TextIOWrapper
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/fatima/anaconda3/envs/PDBCleanV2/bin/PDBClean_MolID_CIF.py", line 4, in
import('pkg_resources').run_script('PDBClean==0.0.2', 'PDBClean_MolID_CIF.py')
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/pkg_resources/init.py", line 672, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/pkg_resources/init.py", line 1472, in run_script
exec(code, namespace, namespace)
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/PDBClean-0.0.2-py3.10.egg/EGG-INFO/scripts/PDBClean_MolID_CIF.py", line 32, in
master_molID_class_list = molidutils.pdb_to_masterlist(filelist)
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/PDBClean-0.0.2-py3.10.egg/PDBClean/pdbcleanmolidcifutils.py", line 28, in pdb_to_masterlist
this_molID_class = make_MolID_cif(myfile)
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/PDBClean-0.0.2-py3.10.egg/PDBClean/pdbcleanmolidcifutils.py", line 182, in make_MolID_cif
mmcif_dict = MMCIF2Dict(myfile)
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/Bio/PDB/MMCIF2Dict.py", line 37, in init
for token in tokens:
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/Bio/PDB/MMCIF2Dict.py", line 126, in _tokenize
yield from self._splitline(line.strip())
File "/Users/fatima/anaconda3/envs/PDBCleanV2/lib/python3.10/site-packages/Bio/PDB/MMCIF2Dict.py", line 102, in _splitline
raise ValueError("Line ended with quote open: " + line)
ValueError: Line ended with quote open: 'Structural basis of the ribosomal machinery for Peptide bond formation,

This problem has already been addressed before and fixed here: https://github.com/fatipardo/PDBClean-0.0.2/issues/7#issuecomment-1204321310
But somehow we reverted to an old version. So we are using the same fix again.

Notice that just as mentioned in #7 , we see this error when we run  PDBClean_MolID_CIF.py, but to fix, we need to re-run everything starting from the 'clean' step.


Fix diffing of simplified ndarray

This PR fixes a bug in the diff tool, which wasn't able to handle a serialized ndarray in "simplified" form, meaning an array that is serialized like this:
array: !core/ndarray-1.0.0 [0, 1, 2]

Instead of the usual:
array: !core/ndarray-1.0.0
  data: [0, 1, 2]
  datatype: int64
  shape: [3]

Both are permitted by the ndarray-1.0.0 schema.  I noticed this when working on fixing issues with schema defaults -- the frame-1.1.0 schema's defaults are represented in the simplified form, and once they start appearing in files, tests in tests_diff.py start failing.

I'm a bit confused about the code and the expectations of it (I've never looked this part over). It looks like that this is not checking whether two kinds of alternate definitions of arrays will test as equal, but whether it will compare inline arrays as equal. So are you saying that there were no tests for inline arrays previously (there obviously were not), and when you added the test, they failed, and the code was changed to deal with these tests? Should there be tests to test equality of inline and block arrays too?

So are you saying that there were no tests for inline arrays previously (there obviously were not), and when you added the test, they failed, and the code was changed to deal with these tests?

Yup, that's exactly it.

Should there be tests to test equality of inline and block arrays too?

The diff tool is unusual in that it operates on a tagged tree and not a fully deserialized tree with real ndarray values.  Since that's the case, it's sensitive to how objects were actually represented in the YAML.  This weird simplified form of the inline array is unique in that the YAML representation of the ndarray is itself an array instead of an object with an array nested in a data data.  I don't think we need to test inline and block arrays explicitly because they all have the more common YAML representation.


Support for DDSU-666 (1phase?)

Hi,
are there any plans to also support the CHINT DDSU 666 single phase power meter ?
regards
Udo

hi
Yes, i'm planning, but not have enough time currently.

Ok. I’ve simply used the default modbus Integration and created the entities via yaml. Only drawback is that they are not grouped as a device. Otherwise it’s working Fine.


Train WindNet India with MO global data

Train WindNet for India to see the impact of adding MO global NWP data

4 models currently being trained (2 different seeds of the baseline and 2 seeds which include MO global data), results will be synced to W&B once they are done and will be available here: https://wandb.ai/openclimatefix/wind_mo_global

Created more batches sampled from more years of data (by filling in missing bits in some NWPs), now going to start training with these to test the hypothesis that training on data sampled from a larger variety of years will yield more accurate forecasts

This is done, the new model versions are now being validated in dev after some bug fixes


Add support for cuda-device-side print().

This is a new attempt at adding support for CUDA 'print'. Tests now pass even with ENABLE_CUDASIM=1.

I think this looks good - just a quick question about the tests on the diff.


docs(readme) fix typo(s)

Summary
Fixed minor typo(s) in README.md

GitHub was flaky yesterday. Now that I can post a comment: thanks, manually merged to master :)
I am not sure if such changes are in the scope of what Hacktoberfest considers acceptable, but we welcome it nonetheless.


Cannot start the meeting

Hi, I want to try this example by following this article
Deployment went smoothly and I already have 4 JSON files from GenerateEventFunction
When I tried to join as moderator it give me error The meeting you tried to join does not exist

I tried a couple of times but the results are the same. Did I miss something?

@archansel I haven't personally run into this before - in order to be unblocked, maybe you can try deleting your old bucket and stack and re-deploying to a completely new stack and bucket, re-generate your moderator/talent/attendee/broadcast resources and try it again. Please let us know if you run into the same issue.

closing due to inactivity - please reopen if the issue still exists


QueryResponses full path not used in cosmwasm_schema::generate_api!

The full module path (starting from crate root) is not used when specifying the QueryResponses type in the cosmwasm_schema::generate_api! macro.
This can result in the following error when using it without cosmwasm_schema::write_api!:
error[E0433]: failed to resolve: use of undeclared type `QueryResponses`
  --> contracts/mock-core/mock-core.rs:39:5
   |
39 | /     cosmwasm_schema::generate_api! {
40 | |         instantiate: InstantiateMsg,
41 | |         execute: ExecuteMsg,
42 | |         query: QueryMsg,
43 | |     }
   | |_____^ use of undeclared type `QueryResponses`
   |
   = note: this error originates in the macro `cosmwasm_schema::generate_api` (in Nightly builds, run with -Z macro-backtrace for more inf
o)

Thanks a lot for bringing this up and providing the patch. In #1534 I was able to reproduce the issue in a test case and cherry-picked your patch. I'll release that as part of the next 1.1.x, most likely this week.

Fixed and shipped in 1.1.9

Much appreciated! I'll be sure to add a test case next time to save you the hassle - I wasn't sure whether it was warranted, but it does make sense after looking at the one you added.

I made it a habbit for bugs. Whenever there is a bug, I do not ask how do I fix this? but how was it possible to slip through and how can we ensure this never happens again?. Then the fixing part is usually easy.


Bad external font

Hello,
I'm trying to add a font in my mjml.
I added this line :
<mj-font name="miso-bold" href="http://www.ldh-france.org/newsletter/fonts/miso-bold/stylesheet.css" />
But i'm not seeing the good font in the preview (https://mjml.io/try-it-live), but it's good if i open the html file in a browser.
So, if i use a Google font, i haven't this issue.
Do you know why ?
Thanks.

Hi this is probably because font is served through an http server instead of https.
I know chrome can block external resource on different protocol

Ok,
So i change it by <mj-font name="miso-bold" href="https://soutenir.ldh-france.org/fonts/miso-bold/stylesheet.css" /> but i have the same issue :(

Probably a CORS then ?

Hi @chatlumo
Your website has no SSL certificate so your font is blocked by the browser as you can see here :

I'm closing this one because then 👍


Bumping SQL Tools Service to <IP_ADDRESS>

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
8 unchanged lines in 4 files lost coverage.
Overall coverage decreased (-0.01%) to 42.247%

Files with Coverage Reduction
New Missed Lines
%

extensions/sql-database-projects/src/common/httpClient.ts
1
66.18%

src/sql/workbench/services/connection/browser/connectionDialogWidget.ts
1
73.05%

extensions/sql-database-projects/src/dialogs/publishDatabaseDialog.ts
2
52.88%

extensions/notebook/src/jupyter/serverInstance.ts
4
75.5%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
30860

Relevant Lines:
67815

💛  - Coveralls


Metis chain initial tech assessment

Background
LiFi is apparently really bullish on Metis and the Metis team needs a better bridge that can support it.
They have a large treasury, the current bridging UX is quite bad, and LiFi would ask for a grant for us to support.
They asked for a technical assessment before Friday this week in order to get back to Metis team
Definition Of Done
Confirm if Metis has these

[ ] AMB connector -- Assumption is they are a fork of op. Can we use OP connector? https://docs.metis.io/dev/protocol-in-detail/cross-domain-messaging
If they dont support the below, we would need
[ ] High-quality public and private RPCs
[ ] High-quality explorer support
[ ] Relayer support
[ ] Subgraph support

Notes
We will assess impact to growth goals in parallel
We will need governance approval ultimately to luanch this quarter

AMB connector -- Assumption is they are a fork of op.

they are a fork of pre-bedrock optimism. we have a non op-roots version of that connector here.
we will have to make sure these contracts are modified properly to work with optimistic roots.

subgraph = $200/mo * 2
gelato = $25k/yr
most likely to have bugs