Handle MEDIA_CAPTION_TOO_LONG error

Originally reported in #9.
The problem: telegram bot api doesn't allow to add caption to the photo with length more than 200 symbols.
Current solution: message cuts to 200 symbols, warning to stdout, unable to get truncated part of message.
To discuss two ideas:

Split message in two parts: first one is the caption and has 200 symbols length, second one is the next part, sent after the first as plain text without graph.
Separate text and image if length is more than 200 symbols. The first message is graph, the second is plain text (or we can swap them).

What is best? Or any other ideas?

New idea. Today's update:
The Bot API now supports silent notifications. We've added an optional `disable_notification` parameter to all sending-related methods.
https://core.telegram.org/bots/api#available-methods

This means that it will be possible to send text message and graph right after that separately.
My problem is when script sends me a message with graph, I don’t see alert in my push messages, just “Zabbix Bot sent photo”.

Graphs are sending as reply-to messages with images only.


Potential Bug with Input types

Vapor: 4.74.2
Graphiti: 1.10.0
GraphQL: 2.5.0
Pioneer 1.2.1
I am using a Codable struct as an input type for one of my mutations. I defined the Type in the schema itself as such:
Type(CreateLeagueArguments.self) {
        Field("name", at: \.name)
        Field("organizationId", at: \.organizationId)
        Field("level", at: \.level)
}

For the mutation I have:
Field("createOrganization", at: GameOnResolver.createOrganization) {
    Argument("title", at: \.title)
    Argument("info", at: \.info)
    Argument("leagueInfo", at: \.leagueInfo)
}

And the arguments struct for the resolver is:
struct CreateOrganizationArguments: Codable {
    var title: String
    var info: String
    var leagueInfo: CreateLeagueArguments?
}

However, when attempting to run the app, I get:
Fatal error: 'try!' expression unexpectedly raised an error: Cannot use type "Optional<CreateLeagueArguments>" for field "leagueInfo". Mapped GraphQL type is not an input type.

Happy to supply more context, and apologies if this isn't a bug.

Hey @bhaze31, no worries about asking questions here.
In this case I think CreateLeagueArguments should be declared as an Input instead of a Type. Check out the Input Objects section of the Usage Guide for more info.

@NeedleInAJayStack did not even know that Usage Guide existed, and that is exactly what I needed. Really appreciate the speedy response! Will close this out now


on reboot NIC hardware hangs when near line rate packet data on the interface

when rebooting the system under extremely high (near line rate) packet rate on the capture interfaces, the NIC hardware hangs and is no longer responsive.
Power cycling clears the error.
device: fmadio20

tried quick repo, generate line rate on both 20G ports + reboot. could not reproduce the problem
using SF firmware
fmadio@fmadio20-049:~$ sudo sfupdate
Solarstorm firmware update utility [v4.6.0]
Copyright Solarflare Communications 2006-2014, Level 5 Networks 2002-2005
sf0 - MAC: 00-0F-53-20-A8-A0
Firmware version:   v4.6.3
Controller type:    Solarflare SFC9140
Boot ROM version:   v<IP_ADDRESS>2
The Boot ROM firmware is up to date
sf1 - MAC: 00-0F-53-20-A8-A1
Firmware version:   v4.6.3
Controller type:    Solarflare SFC9140
Boot ROM version:   v<IP_ADDRESS>2
The Boot ROM firmware is up to date
sf2 - MAC: 00-0F-53-20-A8-A2
Firmware version:   v4.6.3
Controller type:    Solarflare SFC9140
Boot ROM version:   v<IP_ADDRESS>2
The Boot ROM firmware is up to date
sf3 - MAC: 00-0F-53-20-A8-A3
Firmware version:   v4.6.3
Controller type:    Solarflare SFC9140
Boot ROM version:   v<IP_ADDRESS>2
The Boot ROM firmware is up to date

similar problem on fmadio20-090 interfaces never shown up. happened when issuing an IPMI reboot while capturing at the same time

Probelm occured again during a firmware update. It gets stuck with kernel panic in the sf driver code. Likely infinite loop within the driver some where.


You are manually calling a React.PropTypes validation function...

Here is the stack trace:

Am I missing something?

This is a warning from React itself, not enzyme. Do you perhaps have a custom propType function in your code?

I know it's from React, but I only get it when I test the component with shallow. I'm converting my tests from ReactTestUtils.createRenderer() to enzyme. I did not get this warning before the conversion. That is why I assumed this is related to enzyme. There is no custom propType function involved.

Can you share your test code?

the test (I took out the expectations)
    fit('should work...', ()=> {
      const wrapper = shallow(<Field showLabel={true} entity={entity} schema={schema} label="theLabel" field="a" setValue={()=>null} errors={[]}/>);
    });

I trimmed down the component to this, and I still get the error
export const Field = ({schema, entity, field, setValue, showLabel = true, label, required, readonly, maxlength, errors, input, readMode = false}) =>

;

Field.propTypes = {
  schema: PropTypes.object.isRequired,
  entity: PropTypes.object.isRequired,
  field: PropTypes.string.isRequired,
  showLabel: PropTypes.bool,
  label: PropTypes.string,
  required: PropTypes.bool,
  readonly: PropTypes.bool,
  readMode: PropTypes.bool,
  maxlength: PropTypes.number,
  setValue: PropTypes.func.isRequired,
  errors: PropTypes.array.isRequired,
  input: PropTypes.element
};

It's the second (#820) strange issue I get. Could this be related the fact that I'm using a defective browserify build (see my last comment in #806)?

Can you make sure that you all React packages you have installed are on the same version? (react, react-dom, react-addons-test-utils)

@aweary, what I have is:

<EMAIL_ADDRESS><EMAIL_ADDRESS>

Issues resolved : My project does use react 15. But my tests ran under react 14. My browserify build of enzyme uses react 14 because by default the enzyme project is set to react 14 so when I browserifyed it, it embedded react 14 in the bundle.


Delete User Endpoint

Steps to Reproduce
Try to delete a user
Expected Behavior
This should delete the user and all their data
Actual Behavior
The frontend calls the correct endpoint: DELETE http://<IP_ADDRESS>:8000/api/account/user, but the endpoint no longer exists on the backend
Any additional comments?
No response

@horsburgh @daniel-slaugh Do we want the delete user endpoint to immediately delete the user and their data? It might be safer to have the endpoint set the is_active flag to false for the user, hide all their data, and prevent them from logging in, and then actually delete inactive users some period of time after their last login. From the user's perspective the account would be deleted immediately, but there would be a backdoor for a period of time where an admin user could still recover their account if necessary.

@kjlippold - this is probably a good idea. Maybe have a 30 day delay on actual deletion?

@horsburgh Sounds good, I'll go ahead and do that.

@horsburgh @daniel-slaugh The only problem I can imagine is having to make the backend more complex. If the user deletes their account, I think we want to prevent their sites from getting retrieved when the api calls all sites or a user navigates to the /Browse page. Then we'd need to think about what happens if a site to be deleted has secondary owners.
I think it's a good feature so long as it doesn't add too much complexity.

@daniel-slaugh Generally, we should be able to just check the is_active field when we apply auth rules to our queries and filter the results accordingly.

https://github.com/hydroserver2/hydroserver-webapp-back/pull/166


pls add support for configuring websocket extensions

for example, configuring the "permessage-deflate" extension in the ClientUpgradeRequest object. The object has an addExtension (or similar) method.

@moizsj Pull requests are very welcome! Let us know if you need any help preparing a patch.

Created a PR for the issue I filed.
https://github.com/stylefruits/gniazdo/pull/18
On Wed, Apr 20, 2016 at 12:53 PM, Jan Stępień<EMAIL_ADDRESS>wrote:

@moizsj https://github.com/moizsj Pull requests are very welcome! Let
us know if you need any help preparing a patch.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/stylefruits/gniazdo/issues/17#issuecomment-212297070


fix(doc): update link to selenium promise manager option

The link was pointing to the wrong line.

I signed it!

Could you update this to line 714?

@cnishina rebased and updated line number to 714


General question: How well is FFI supported?

Checklist

[X] I've looked through the issues and pull requests for similar request
[ ] This feature could be solved with a custom docker image (optional)

Describe your request
Let's say a Rust project depends on a C/C++ library.
Assume that the library is written in extremely portable C/C++ code.
Assume that any other C/C++ library sub-dependencies (e.g. RUN apt-get install libcurl, or whatever, on top of a cross base image), are installed in some custom cross Docker images, and Cross.toml is configured to reference the images.
Does cross work well for projects like that?
I am more familiar with FFI in terms of Cgo, where go mod epic fails to even download the C/C++ source code. Do rustc, cargo, and cross have better support for FFI?
Anyone already using cross to generate ports for Rust FFI projects?
Naturally, we want to promote any C/C++ dependencies to pure Rust long term. But as a stopgap measure, it is useful to support FFI when needed.
Describe why this would be a good inclusion for cross
General question about how cross, cargo, and rustc integrate with FFI for cross-compilation purposes.

Yes, this is one of the core features of rust. We actually even have special configuration to install dependencies with cross, see the cross wiki for how to install openssl, which is used by the openssl-sys crate for example.
I'd recommend you check out Ryust by Example: FFI, Nomicon: FFI, A guide for doing FFI using Rust, Youtube: Crust of Rust - Build scripts and FFI and finally The Big Book of Rust Interop
I'd highly suggest you post questions like these on any of the forums available here, the rust community discord or the rust reddit instead 😄


Skip rootfs unmount when no mounts are provided

We (Cloud Foundry Garden) are trying to run containerd as a non-root user for extra security. We have our own component for creating container images so we are not using a snapshotter. While this mostly works we hit the following problem:

Currently when you create a new task, the client will ask the snapshotter (if any) about any mount points that containerd should make and pass them on with the request so that they are performed by the shim over here. If there is no snapshotter the request will contain an empty list of mounts and the shim will not perform any mounts
When you destroy the task, however, the shim will try to unconditionally unmount those mountpoints which were persisted in a filed of the process struct.

As of today this works because containerd is usually run as uid 0 in its respective user namespace, but honestly it feels a bit asymmetric and is breaking our approach to rootless containers which is to run the containerd daemon as a non-root user.
We have no issues with task creation because we are not using a snapshotter, so as mentioned above no mount calls are performed by the shim. However, when we attempt a destroy we hit the unconditional unmount call and as you can imagine this results in an operation not permitted error.
This PR suggests a fix which makes sure that the shim will not do any unmount calls when there is nothing to unmount (no rootfs mounts were passed in to the request, because no snapshotter was used).
Any comments and suggestions are welcome.
Co-authored-by: Julia Nedialkova<EMAIL_ADDRESS>Signed-off-by:  Julia Nedialkova<EMAIL_ADDRESS>

Codecov Report

Merging #3148 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3148   +/-   ##
=======================================
  Coverage   45.16%   45.16%
=======================================
  Files         111      111
  Lines       11962    11962
=======================================
  Hits         5403     5403
  Misses       5727     5727
  Partials      832      832

Flag
Coverage Δ

#linux
49.25% <ø> (ø)
:arrow_up:

#windows
40.49% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7b6fea...ff7e0d4. Read the comment docs.

Codecov Report

Merging #3148 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3148   +/-   ##
=======================================
  Coverage   45.16%   45.16%
=======================================
  Files         111      111
  Lines       11962    11962
=======================================
  Hits         5403     5403
  Misses       5727     5727
  Partials      832      832

Flag
Coverage Δ

#linux
49.25% <ø> (ø)
:arrow_up:

#windows
40.49% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7b6fea...ff7e0d4. Read the comment docs.

/cc @AkihiroSuda

LGTM except a nit
v2 runtime needs this fix as well? (Can be a separate PR)

We have no issues with task creation because we are not using a snapshotter, so as mentioned above no mount calls are performed by the shim.

We should have integration test suites for both "root in userns" and "non-root outside userns" mode so as to clarify what are expected to work and what are not.

nit: Co-authored-by doesn't seem to be needed if this commit was solely authored by Julia

hi @georgethebeatle ,
The current implementation ignore the not mount point when the rootfs is provided. So I think we can think it in this way, check rootfs is provided or not. WDYT?
and it seems that we cannot mount without sudo and maybe we can add some user in /etc/fstab. just random thoughts.

@fuweid If you pass rootfs mounts using the WithRootFS task option it would still be mounted by the shim. However this will not work rootlessly and this is an intentional limitation of this PR.

@AkihiroSuda we have addressed your comments and v2 is now supported.

We should have integration test suites for both "root in userns" and "non-root outside userns" mode so as to clarify what are expected to work and what are not.

Do you think that adding those tests is in the scope of this PR or is this a suggestion for future work?

@georgethebeatle you can try oci.WithRootFSPath which shim will not mount it , because it is provided by client. But the shim still tries to unmount it after cleanup. That is the problem. If we can know that it is not mount point, we can skip it.

Test can be future work

@fuweid If you provide any mounts via oci.WithRootFSPath the shim WILL try to mount them, because the client will add them to the request here and later on the shim will range over them here. The shim will always call mount.UnmountAll on cleanup but note that we have modified that function to not do the unmount if  the mount argument is not a valid path. We have also made sure that in case no mounts were propagated to the shim by the client the rootfs will be set to "" and no rootfs directory will be created in the bundle dir. This way when UnmountAll is called it will return early and not attempt to do the unmount.
As far as I know we cannot check if the rootfs directory is a mount point as a non-root user, because it will involve making a mount syscall and checking if the error we got back was EINVAL, but as a non root user we are not allowed to call mount so we always get an EPERM. That's why we have taken the approach described above.

@AkihiroSuda @fuweid Do you think the PR is mergeable as it is or do we need to do some more work?

@georgethebeatle sorry for the late reply.

If you provide any mounts via oci.WithRootFSPath the shim WILL try to mount them, because the client will add them to the request here and later on the shim will range over them here.

If I understand correctly, the oci.WithRootFSPath is used to set the rootfs in oci runtime spec. shim will no mount it.

As far as I know we cannot check if the rootfs directory is a mount point as a non-root user, because it will involve making a mount syscall and checking if the error we got back was EINVAL, but as a non root user we are not allowed to call mount so we always get an EPERM.

I try to read the /proc/mounts info and check the path in there or not. I am not sure that it is good way. I make demo for this. Please check https://github.com/containerd/containerd/commit/9ed70657b7378a1d293b0c6855b04e486116ca4b
I use the following command to run it and it will not unmount the provided rootfs.
# busybox image and ignore the rootless container error :P
$ ctr -a /tmp/containerd.sock run -t --fifo-dir /tmp/1 --rootfs /home/vagrant/blobs/sha256 demo1 ls

ctr: OCI runtime create failed: rootless container requires user namespaces: unknown

cc @AkihiroSuda

@fuweid Your suggestion to check whether a path is a mountpoint prior unmounting is racy in the rootful case. In a concurrent scenario where two deletes are running in parallel it may happen that deleteA has checked whether something is a mountpoint and is about to unmount it. In the meanwhile, deleteB might have already umounted the path thus making deleteA unmount to fail (as the path has been already unmounted). In order to avoid the race we unmount straight away and ignore the EINVAL (i.e. not a mountpoint error).
In the rootless scenario we would not want to unmount at all (as we are running as an unprivileged user). In order to do that we use the empty string as a special marker value to indicate that there is no rootfs mount. I think that this is the simpler solution compared to checking the mount table and taking the risk of being racy.
Agree?

@fuweid We applied the change you suggested adding some debug logging:
diff --git a/mount/mount_linux.go b/mount/mount_linux.go
index b5a16148..86f4e87b 100644
--- a/mount/mount_linux.go
+++ b/mount/mount_linux.go
@@ -18,6 +18,7 @@ package mount

 import (
  "fmt"
+ "io/ioutil"
  "os"
  "path"
  "strings"
@@ -112,6 +113,16 @@ func unmount(target string, flags int) error {
 // are no mounts remaining (EINVAL is returned by mount), which is
 // useful for undoing a stack of mounts on the same mount point.
 func UnmountAll(mount string, flags int) error {
+ isMount, err := isMountpoint(mount)
+ if err != nil {
+   return err
+ }
+
+ if !isMount {
+   fmt.Printf("Skipping unmount of: %s\n", mount)
+   return nil
+ }
+
  for {
    if err := unmount(mount, flags); err != nil {
      // EINVAL is returned if the target is not a
@@ -127,6 +138,32 @@ func UnmountAll(mount string, flags int) error {
  }
 }

+func isMountpoint(path string) (bool, error) {
+ bs, err := ioutil.ReadFile("/proc/mounts")
+ if err != nil {
+   return false, err
+ }
+
+ lines := strings.Split(string(bs), "\n")
+ for _, line := range lines {
+   if line == "" {
+     continue
+   }
+
+   fields := strings.Fields(line)
+   if len(fields) != 6 {
+     continue
+   }
+
+   if fields[1] == path {
+     return true, nil
+   }
+ }
+
+ fmt.Printf("\n Mount table is \n%s\n\n", string(bs))
+ return false, nil
+}
+
 // parseMountOptions takes fstab style mount options and parses them for
 // use with a standard mount() syscall
 func parseMountOptions(options []string) (int, string) {

Then we ran containerd's integration tests for the v2 runtime with the following command:
for i in {0..100}; do printf "\n\n\n\n\n ################## ATTEMPT %d\n\n\n\n\n\n" $i; TEST_RUNTIME=io.containerd.runc.v2 make integration || break; done

It failed on the 80th run with the following error:
 Mount table is
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,nosuid,relatime,size=1012144k,nr_inodes=253036,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=204104k,mode=755 0 0
/dev/sda1 / ext4 rw,relatime,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=24,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12599 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
lxcfs /var/lib/lxcfs fuse.lxcfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
vagrant /vagrant vboxsf rw,nodev,relatime 0 0
vagrant_data /vagrant_data vboxsf rw,nodev,relatime 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=204100k,mode=700,uid=1000,gid=1000 0 0
/dev/sda1 /tmp/snapshot-suite-SnapshotterClient-791538560/work/prepare091312845 ext4 rw,relatime,data=ordered 0 0
/dev/sda1 /tmp/snapshot-suite-SnapshotterClient-039602127/work/check973865297 ext4 ro,relatime,data=ordered 0 0
overlay /tmp/snapshot-suite-SnapshotterClient-511760647/work/nextnextlayer overlay ro,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/20/fs:/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/15/fs 0 0
/dev/sda1 /tmp/snapshot-suite-SnapshotterClient-991477075/work/preparing ext4 rw,relatime,data=ordered 0 0
overlay /tmp/snapshot-suite-SnapshotterClient-991477075/work/nextlayer overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/9/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/17/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/17/work 0 0
overlay /tmp/snapshot-suite-SnapshotterClient-991477075/work/nextnextlayer overlay ro,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/17/fs:/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/9/fs 0 0

Skipping unmount of: /tmp/snapshot-suite-SnapshotterClient-979746360/work/prepare954751792
=== CONT  TestSnapshotterClient/PreareViewFailingtest
=== CONT  TestSnapshotterClient/StatActive
=== CONT  TestSnapshotterClient/StatComitted
=== CONT  TestSnapshotterClient/TransitivityTest
=== CONT  TestSnapshotterClient/RemoveIntermediateSnapshot
=== CONT  TestSnapshotterClient/MoveFileFromLowerLayer
=== CONT  TestSnapshotterClient/128LayersMount
=== CONT  TestSnapshotterClient/StatInWalk
=== CONT  TestSnapshotterClient/ViewReadonly
=== CONT  TestSnapshotterClient/Rename
--- FAIL: TestSnapshotterClient (0.02s)
    --- PASS: TestSnapshotterClient/RootPermission (0.07s)
        helpers_unix.go:32: unmount /tmp/snapshot-suite-SnapshotterClient-334983933/work/preparing
    --- PASS: TestSnapshotterClient/CloseTwice (0.08s)
    --- FAIL: TestSnapshotterClient/Chown (0.36s)
        issues.go:111: Check snapshots failed: directory diff between /tmp/snapshot-suite-SnapshotterClient-979746360/work/flat255284778 and /tmp/snapshot-suite-SnapshotterClient-979746360/work/check202078681
                - /opt
                - /opt/a
                - /opt/a/b
                - /opt/a/b/file.txt

                github.com/containerd/containerd/vendor/github.com/containerd/continuity/fs/fstest.CheckDirectoryEqual
                        /vagrant_data/src/github.com/containerd/containerd/vendor/github.com/containerd/continuity/fs/fstest/compare.go:53
                github.com/containerd/containerd/snapshots/testsuite.checkSnapshot
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/helpers.go:104
                github.com/containerd/containerd/snapshots/testsuite.checkSnapshots
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/helpers.go:132
                github.com/containerd/containerd/snapshots/testsuite.checkChown
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/issues.go:110
                github.com/containerd/containerd/snapshots/testsuite.makeTest.func1
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/testsuite.go:111
                testing.tRunner
                        /usr/local/go/src/testing/testing.go:777
                runtime.goexit
                        /usr/local/go/src/runtime/asm_amd64.s:2361
                check directory failed
                github.com/containerd/containerd/snapshots/testsuite.checkSnapshot
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/helpers.go:105
                github.com/containerd/containerd/snapshots/testsuite.checkSnapshots
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/helpers.go:132
                github.com/containerd/containerd/snapshots/testsuite.checkChown
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/issues.go:110
                github.com/containerd/containerd/snapshots/testsuite.makeTest.func1
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/testsuite.go:111
                testing.tRunner
                        /usr/local/go/src/testing/testing.go:777
                runtime.goexit
                        /usr/local/go/src/runtime/asm_amd64.s:2361
                snapshot check failed on snapshot 2
                github.com/containerd/containerd/snapshots/testsuite.checkSnapshots
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/helpers.go:133
                github.com/containerd/containerd/snapshots/testsuite.checkChown
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/issues.go:110
                github.com/containerd/containerd/snapshots/testsuite.makeTest.func1
                        /vagrant_data/src/github.com/containerd/containerd/snapshots/testsuite/testsuite.go:111
                testing.tRunner
                        /usr/local/go/src/testing/testing.go:777
                runtime.goexit
                        /usr/local/go/src/runtime/asm_amd64.s:2361
        helpers.go:67: drwx------       4096 /tmp/snapshot-suite-SnapshotterClient-979746360
        helpers.go:67: drwxr-xr-x       4096 /tmp/snapshot-suite-SnapshotterClient-979746360/root
        helpers.go:67: drwxr-xr-x       4096 /tmp/snapshot-suite-SnapshotterClient-979746360/work
        helpers.go:67: drwxr-xr-x       4096 /tmp/snapshot-suite-SnapshotterClient-979746360/work/prepare954751792

Looking at our debug logs it seems like UnmountAll skipped the unmounting of /tmp/snapshot-suite-SnapshotterClient-979746360/work/prepare954751792, because it was not in the mount table at the time of the check. Later on the test is making a diff of some directories and unexpectedly finds /tmp/snapshot-suite-SnapshotterClient-979746360/work/prepare954751792 which should have been removed, but wasn't because it has not been unmounted. The fact that it was not unmounted is confirmed by Skipping unmount of debug line, which appears only in the failing test logs. This line is missing in all the cases when the very same test passes.
We did the same experiment using the docker mount.GetMounts utility method in the isMountpoint check and got the same error reliably. Therefore we think checking the mount table is not a good option.
Any thoughts?
cc @AkihiroSuda

It's been a while since we last updated this PR. Do you have any thoughts or suggestions? How should we proceed?

@danail-branekov @georgethebeatle sorry for the late reply.

In a concurrent scenario where two deletes are running in parallel it may happen that deleteA has checked whether something is a mountpoint and is about to unmount it. In the meanwhile, deleteB might have already umounted the path thus making deleteA unmount to fail (as the path has been already unmounted). In order to avoid the race we unmount straight away and ignore the EINVAL

@danail-branekov mount.UnmountAll has already ignore the EINVAL. I think the case will not be issue, right?
// UnmountAll repeatedly unmounts the given mount point until there
// are no mounts remaining (EINVAL is returned by mount), which is
// useful for undoing a stack of mounts on the same mount point.
func UnmountAll(mount string, flags int) error {
        for {
                if err := unmount(mount, flags); err != nil {
                        // EINVAL is returned if the target is not a
                        // mount point, indicating that we are
                        // done. It can also indicate a few other
                        // things (such as invalid flags) which we
                        // unfortunately end up squelching here too.
                        if err == unix.EINVAL {
                                return nil
                        }
                        return err
                }
        }
}

@georgethebeatle  I was thinking we can check the mount table in the shim side. Anyway,
it seems that easy to fix this problem by the empty string.  :)

mount.UnmountAll has already ignore the EINVAL. I think the case will not be issue, right?
Not quite. The fact that the mount point check is racy can have 2 effects:

You check the mount table and rootfs is a mount point, then another thread unmounts rootfs, then you try to unmount rootfs which is no longer a mount point. This one is fine as you point out since we are ignoring EINVAL
You check the mount table and rootfs is not there, which means it is not mounted. In the meantime another thread mounts rootfs, but you are already past the check so you do not attempt to unmount which leads to a leaked mount point and an EBUSY error when you later on try to remove rootfs. This is the same problem that we managed to reproduce by running all tests in a loop until they fail using the "check the mount table" implementation. Therefore we think it is better to take the "invalid path" approach where we are telling UnmountAll whether or not to attempt unmount by passing an invalid (or non-existing) path.

Hi @georgethebeatle
Yeah. My point is that we can check mount table in the runtime(shim) side, not in the unmount function. The cleanupAfterDeadShim will check the rootfs is mount point or not. Since the task id is deleted after by the runtime.terminate action, we cannot create two same tasks in the same time. I think it can avoid some concurrent issues. But we cannot prevent the manually case. For example, the rootfs is provided and bundle/rootfs is empty folder. Before the cleanup, we can mount the random thing on the rootfs folder to make the cleanup failed.
Empty rootfs is easy to make it happen, I agree. We can do skip-unmount-checker in shim side, not in the unmount side, because we cannot let all the cover things in mount package. it is just my two cents. Thanks

@fuweid I feel like we are kind of on the same page but do not understand whether you want us to do some more work. Should we do something more to get this merged and what do you suggest we do?

@fuweid About your proposition that we do a mountpoint check in cleanupAfterDeadShim hoping that this would avoid concurrent issues I have several concerns:

As far as I see cleanupAfterDeadShim is called if the shim fails to start or after it exits as an exit handler. Maybe the exit handler is being called late enough to avoid concurrency, but I do not know the codebase well enough to be convinced, so this should be demonstrated by running the tests lots and lots of times without any failures (assuming that cleanupAfterDeadShim does the mountpoint check). As a whole I am not comfortable introducing a racy check and hoping that it will not be a problem.
The cleanupAftertDeadShim is only used in v1 so it is not a universal solution. In v2 containerd delegates to external shim binaries. To my best knowledge the v2 shim binary calls out to UnmountAll to ensure that the mount point is gone. If we have to apply your point to v2 this would mean that each binary would have to do the unmount itself. Not sure if that is desirable

What do you think?

Hi @georgethebeatle sorry for the late reply.
cleanupAfterDeadShim is only used in v1 shim. But in the v2 shim, there are two places to handle unmount:

If loadTask fails, the v2 task manager will try to load container and unmount rootfs.
v2 shim binary will unmount rootfs point during Delete API call.

I have checked your code.
diff --git a/runtime/v2/bundle.go b/runtime/v2/bundle.go
index 7139410..7897a1c 100644
--- a/runtime/v2/bundle.go
+++ b/runtime/v2/bundle.go
@@ -89,10 +89,6 @@ func NewBundle(ctx context.Context, root, state, id string, spec []byte) (b *Bun
                }
        }
        paths = append(paths, work)
-       // create rootfs dir
-       if err := os.Mkdir(filepath.Join(b.Path, "rootfs"), 0711); err != nil {
-               return nil, err
-       }
        // symlink workdir
        if err := os.Symlink(work, filepath.Join(b.Path, "work")); err != nil {
                return nil, err

And tried to run it with v2 shim binary in my local
root@ubuntu-xenial ~/g/s/g/c/containerd# ctr run --runtime io.containerd.runc.v2 --rm docker.io/library/alpine:latest pause date                                          me-cross-push!
ctr: failed to mount rootfs component &{overlay overlay [workdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/49/work upperdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/49/fs lowerdir=/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs]}: no such file or directory: unknown

I think we should not just skip the rootfs in v2. Just random thought: record the information about the provided rootfs in file(read later) or add empty checker in everywhere.
BTW, Do you consider to use v2 shim in your production? It seems that we can handle the unmount thing in custom shim binary.

Hi @fuweid
On a local environment with the PR changes we were able to successfully execute the following command:
ctr run --runtime io.containerd.runc.v2 --rm docker.io/library/alpine:latest test echo hi

We were able to get the error you are talking about by applying the diff you gave above to master (which is only a small part of the PR), but could you explain exactly what you did?
Apart from that we experimented with what you suggested: we moved the UnmountAll call to the exitHandler of the shim (in v1)
and made UnmountAll check whether the path is a mountpoint and unmount only when it is:

diff --git a/runtime/v1/linux/runtime.go b/runtime/v1/linux/runtime.go
index 545e2167..c2f889d3 100644
--- a/runtime/v1/linux/runtime.go
+++ b/runtime/v1/linux/runtime.go
@@ -190,12 +190,17 @@ func (r *Runtime) Create(ctx context.Context, id string, opts runtime.CreateOpts
      cgroup = v.(*runctypes.CreateOptions).ShimCgroup
    }
    exitHandler := func() {
+     log.G(ctx).WithField("id", id).Info("short cirquiting")
+     if err2 := mount.UnmountAll(filepath.Join(bundle.path, "rootfs"), 0); err2 != nil {
+       log.G(ctx).WithError(err2).Warn("Failed to cleanup rootfs mount")
+     }
      log.G(ctx).WithField("id", id).Info("shim reaped")
      t, err := r.tasks.Get(ctx, id)

Unfortunately the tests started failing with the same error as mentioned above: https://github.com/containerd/containerd/pull/3148#issuecomment-482546469:
 Mount table is
 sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
 udev /dev devtmpfs rw,nosuid,relatime,size=1007456k,nr_inodes=251864,mode=755 0 0
 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
 tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=204060k,mode=755 0 0
 /dev/sda1 / ext4 rw,relatime 0 0
 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0
 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0
 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0
 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0
 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0
 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0
 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
 debugfs /sys/kernel/debug debugfs rw,relatime 0 0
 mqueue /dev/mqueue mqueue rw,relatime 0 0
 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
 configfs /sys/kernel/config configfs rw,relatime 0 0
 /dev/loop0 /snap/core/6673 squashfs ro,nodev,relatime 0 0
 /dev/loop1 /snap/lxd/10601 squashfs ro,nodev,relatime 0 0
 tmpfs /run/snapd/ns tmpfs rw,nosuid,noexec,relatime,size=204060k,mode=755 0 0
 nsfs /run/snapd/ns/lxd.mnt nsfs rw 0 0
 vagrant /vagrant vboxsf rw,nodev,relatime,uid=1000,gid=1000,ttl=0,dmode=037777777777,fmode=037777777777,dmask=00,fmask=00 0 0
 vagrant_data /vagrant_data vboxsf rw,nodev,relatime,uid=1000,gid=1000,ttl=0,dmode=037777777777,fmode=037777777777,dmask=00,fmask=00 0 0
 /dev/loop2 /snap/core/6818 squashfs ro,nodev,relatime 0 0
 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=204056k,mode=700,uid=1000,gid=1000 0 0
 tracefs /sys/kernel/debug/tracing tracefs rw,relatime 0 0
 overlay /tmp/containerd-test-new-daemon-with-config143921677/state/io.containerd.runtime.v2.task/testing/.TestDaemonRuntimeRoot/rootfs overlay rw,relatime,lowerdir=/tmp/containerd-test-new-daemon-with-config143921677/root/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/tmp/containerd-test-new-daemon-with-config143921677/root/io.containerd.snapshotter.v1.overlayfs/snapshots/2/fs,workdir=/tmp/containerd-test-new-daemon-with-config143921677/root/io.containerd.snapshotter.v1.overlayfs/snapshots/2/work,xino=off 0 0
 overlay /tmp/containerd-test-new-daemon-with-config095602708/state/io.containerd.runtime.v2.task/testing/.TestDaemonRuntimeRoot/rootfs overlay rw,relatime,lowerdir=/tmp/containerd-test-new-daemon-with-config095602708/root/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/tmp/containerd-test-new-daemon-with-config095602708/root/io.containerd.snapshotter.v1.overlayfs/snapshots/2/fs,workdir=/tmp/containerd-test-new-daemon-with-config095602708/root/io.containerd.snapshotter.v1.overlayfs/snapshots/2/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/.TestDeleteContainerExecCreated/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/224/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/224/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/.TestContainerAttach/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/236/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/236/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/.TestContainerPTY/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/238/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/238/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/.TestContainerUser_UserIDAndGroupName/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/240/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/240/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/TestBindLowPortNonOpt/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/244/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/244/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/.TestTaskResize/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/248/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/248/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v2.task/testing/.TestContainerRuntimeOptionsv2/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/250/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/250/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/TestDeleteRunningContainer/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/253/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/253/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/TestContainerKill/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/254/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/254/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/.TestContainerUser_UserIDAndGroupID/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/256/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/256/work,xino=off 0 0
 overlay /run/containerd-test/io.containerd.runtime.v1.linux/testing/TestProcessForceDelete/rootfs overlay rw,relatime,lowerdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/1/fs,upperdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/255/fs,workdir=/var/lib/containerd-test/io.containerd.snapshotter.v1.overlayfs/snapshots/255/work,xino=off 0 0

 Skipping unmount of  /run/containerd-test/io.containerd.runtime.v1.linux/testing/TestContainerUser_UserIDAndGroupID/rootfs

This time the race would happen every time we run the tests, rather than once in 80 runs, so it seems to have gotten worse.
We are still reluctant to introduce a racey check. Do you have any further thoughts?
cc @AkihiroSuda what do you think?

Hi @georgethebeatle I just applied the change in master and run a container.
step 1: clean up
root@ubuntu-xenial ~/g/s/g/c/containerd# service containerd stop                                                                                me-testing!
root@ubuntu-xenial ~/g/s/g/c/containerd# rm -rf /var/lib/containerd                                                                                me-testing!
root@ubuntu-xenial ~/g/s/g/c/containerd# ls -al /var/lib/containerd                                                                                me-testing!
ls: cannot access '/var/lib/containerd': No such file or directory
root@ubuntu-xenial ~/g/s/g/c/containerd# rm -rf /var/run/containerd/runc                                                                                2 me-testing!
root@ubuntu-xenial ~/g/s/g/c/containerd# ls -al /var/run/containerd/runc                                                                                me-testing!
ls: cannot access '/var/run/containerd/runc': No such file or directory
root@ubuntu-xenial ~/g/s/g/c/containerd# git --no-pager diff                                                                                2 me-testing!
diff --git a/runtime/v2/bundle.go b/runtime/v2/bundle.go
index 7139410..51f0fa6 100644
--- a/runtime/v2/bundle.go
+++ b/runtime/v2/bundle.go
@@ -89,10 +89,13 @@ func NewBundle(ctx context.Context, root, state, id string, spec []byte) (b *Bun
                }
        }
        paths = append(paths, work)
-       // create rootfs dir
-       if err := os.Mkdir(filepath.Join(b.Path, "rootfs"), 0711); err != nil {
-               return nil, err
-       }
+       /*
+               // create rootfs dir
+               if err := os.Mkdir(filepath.Join(b.Path, "rootfs"), 0711); err != nil {
+                       return nil, err
+               }
+       */
+
        // symlink workdir
        if err := os.Symlink(work, filepath.Join(b.Path, "work")); err != nil {
                return nil, err

step2: make and restart
root@ubuntu-xenial ~/g/s/g/c/containerd# make && make install                                                                                me-testing!
+ bin/ctr
+ bin/containerd
+ bin/containerd-stress
+ bin/containerd-shim
+ bin/containerd-shim-runc-v1
+ bin/containerd-shim-runc-v2
+ binaries
+ install bin/ctr bin/containerd bin/containerd-stress bin/containerd-shim bin/containerd-shim-runc-v1 bin/containerd-shim-runc-v2
root@ubuntu-xenial ~/g/s/g/c/containerd# service containerd start

step3: pull image and run it with runtime v2 API
root@ubuntu-xenial ~/g/s/g/c/containerd# ctr image pull registry.hub.docker.com/library/busybox:1.25                                                                                me-testing!
registry.hub.docker.com/library/busybox:1.25:                                     resolved       |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:56bec22e355981d8ba0878c6c2f23b21f422f30ab0aba188b54f1ffeff59c190:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:e02e811dd08fd49e7f6032625495118e63f597eb150403d02e3238af1df240ba:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 10.8s                                                                    total:  640.6  (59.3 KiB/s)
unpacking linux/amd64 sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912...
done
root@ubuntu-xenial ~/g/s/g/c/containerd# ctr run --runtime io.containerd.runc.v2 --rm registry.hub.docker.com/library/busybox:1.25 test echo hi                                                         me-testing!
ctr: ttrpc: client shutting down: ttrpc: closed: unknown
root@ubuntu-xenial ~/g/s/g/c/containerd# journalctl -u containerd.service -f                                                                                1 me-testing!
-- Logs begin at Tue 2019-04-30 14:39:00 CST. --
May 07 15:56:40 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:40.828676376+08:00" level=debug msg="get snapshot mounts" key=test
May 07 15:56:40 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:40.837119389+08:00" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.containerd.runtime.v2.task/default/test
pid=13146
May 07 15:56:40 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:40.974091909+08:00" level=error msg="not found"
May 07 15:56:40 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:40.976926410+08:00" level=debug msg="remove snapshot" key=test
May 07 15:56:40 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:40.977740021+08:00" level=debug msg="event published" ns=default topic="/snapshot/remove" type=containerd.events.SnapshotRemove
May 07 15:56:40 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:40.978615668+08:00" level=debug msg="event published" ns=default topic="/containers/delete" type=containerd.events.ContainerDelete
May 07 15:56:41 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:41.015853110+08:00" level=debug msg="schedule snapshotter cleanup" snapshotter=overlayfs
May 07 15:56:41 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:41.016691814+08:00" level=debug msg="removed snapshot" key="default/3/test" snapshotter=overlayfs
May 07 15:56:41 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:41.017067204+08:00" level=debug msg="snapshot garbage collected" d=941.24µs snapshotter=overlayfs
May 07 15:56:41 ubuntu-xenial containerd[12728]: time="2019-05-07T15:56:41.017231622+08:00" level=debug msg="garbage collected" d=803.161µs

@georgethebeatle I think we can give up the mount check idea 😂 Focus on your original idea.

LGTM

Hey @crosbymichael @dmcgowan
I have question regarding this PR. It was merged to master in between 1.2.6 and 1.2.7 releases, but it is not available in 1.2.7. Why is that so? When will it be released and should we do something more so that it gets in the next release?

@georgethebeatle the master branch is used to work on containerd 1.3.0; there's release branches for the 1.0, 1.1.x and 1.2.x releases; to include something in one of those releases, fixes have to be back ported / cherry-picked (e.g. see https://github.com/containerd/containerd/pulls?q=is%3Apr+release+in%3Atitle+is%3Aclosed)


on :: (b -> b -> c) -> (a -> b) -> a -> c

While the general useWith and converge from Ramda aren't very well typed, I do find them to be useful functions.  One of my main use cases for useWith is to emulate Haskell's on function (this is also known as the psi combinator).
Example:
//Point :: {x, y}
const x = prop('x');

//Point -> Point -> Number
const leftmost = on(min, x);
const rightmost = on(max, x);

I struggle with the parameter order for this function. In Haskell it's typically used in infix form:
leftmost = min `on` x

As a result, I generally define it using the signature above, even though there could be an argument made that the opposite order makes just as much sense.
Is there enough interest for this to be part of Sanctuary?

Is there enough interest for this to be part of Sanctuary?

This function seems useful to me. Would you like to submit a pull request?
I think the type signature you provided is missing one a ->, by the way.

Thanks!  I was missing the a.  I will work on a PR.


support codegen for custom elements registered programatically

 Issue by sigmundch
Originally opened as dart-lang/sdk#17790

Codegen currently works if custom elements are annotated with @CustomTag and are visible from the imported HTML files.
We currently don't support generating code when classes are generated programatically.
Should we add support for this?
One idea to do this would be to include @UnregisterdCustomTag, which is only used for the static code generation, but leaves the registration to the code.

 Comment by sigmundch

John had a great idea that we can use the Polymer mixin as a way to identify all these classes. The only missing feature is what to do with the 'attributes' attribute on such classes.
When people have the following:
 - they use 'attributes' in the html
 - they manually call Polymer.register (no @CustomTag)
 - they do not include @published
then the codegen might be missing the fields published via the HTML. For now, it seems prudent to require that if users don't have @CustomTag, then they must use @published.
The alternative above, with @UnregisteredCustomTag could be useful for this too.

 Comment by sigmundch

Added this to the 1.5 milestone.

 Comment by jmesserly

Removed Library-PolymerBuild label.

 Comment by justinfagnani

I'd consider dropping support for the attributes attribute. @published seems better in just about every way, and we can afford to differ from polymer.js a little.
Custom binding delegates seems like a bigger problem, but we could add an attribute telling codegen to not parse expressions. If that attribute could be applied to any template, then it could be a hook for other binding delegates to run their codegen. This is of course a theoretical problem until we see another binding delegate.

 Comment by jmesserly

Removed Pkg-Polymer label.
Added Pkg-PolymerBuild label.

 Comment by jmesserly

Added Pkg-Polymer label.

 Comment by jmesserly

Removed Area-Polymer label.
Added Area-Pkg label.

 Comment by sigmundch

Removed Polymer-Milestone-Next label.

 Comment by sigmundch

Removed Polymer-P-1 label.
Added Polymer-Milestone-Next label.

 Comment by kasperl

Removed this from the 1.5 milestone.
Added this to the 1.6 milestone.

 Comment by sigmundch

Added PolymerMilestone-Next label.

 Comment by sigmundch

Removed this from the 1.6 milestone.
Added Polymer-P-1 label.


Re-add the host IPs in controller-0 .ssh/config, revert hostname

There's some magical things between .ssh/config and ansible that were
broken for HCI, when time comes to deploy ceph using the ceph.yml
playbook.
In parallel, this patch also reverts the hostname of the VMs to its
former "short" format (compute-0, etc) instead of a more FQDN-like
one (compute-1.hypervisor-1, etc).
That change was mostly needed for sushy since it consumes, by default,
the ansible_host.
Injecting that value in the /etc/hosts makes it easier to consume.
As a pull request owner and reviewers, I checked that:

[X] Appropriate testing is done and actually running

recheck  issue was with infra, not the PR.

/approve

/lgtm
HCI VA job ran without issues with the fix


Native Script Runner

Hello 👋
I am proposing that Coursier should ship with a script runner that can run scripts as native executables. Here is a POC (using Python but I'm happy to rewrite if proposal is accepted) https://gist.github.com/tabdulradi/da4977d006689f775ee9a9dc3d1a8bdd
Coursier contains all the needed functionalities like dependencies fetching and creating native executables from Jars. All we need is to also compile single files and cache the executables somewhere hidden from the user.
Must have Goals:

Ability to have external dependencies
Ability to run as native second time a script runs
Ability to set Scala version (not implemented in the POC)
Support multi line shebangs (to play nice with nix-shell interpreter)

Nice to have Goals:

Ability to switch between Graal, Scala-native or normal JVM runners (implemented in POC)
Not invent syntax for dependencies (implemented in POC as args in the shebang)

Non Goals:

Compile multi file scripts
Ability to share or distribute the executables

At first, I'd say I'm not sure it's something coursier itself should handle… There's nothing related to compilation in it. It'd be a better fit for Ammonite (see https://github.com/lihaoyi/Ammonite/issues/1119, plus it should be possible to have a native-image-based launcher for Ammonite to run such scripts, possibly by offloading their compilation in Ammonite to bloop), or tsk too.
That said, adding cargo or poetry -like features in cs (to compile or run things) has been brought up in the past, and having those would make sense for new comers I guess…
Let me write an issue about a possible way to extend cs externally, with new commands. Then maybe a project handling what you describe could be picked transparently as an "external sub-command" by cs.

Just so you know, the packaging capabilities of coursier are published in a specific module (io.get-coursier::coursier-launcher) that basically has no dependencies, so it would be easy to re-use that from another project.

Ok, jotted down a few ideas here.

Hi @tabdulradi, would you be okay to share your scripting use-case which involves multi line shebangs nix shell interpreter?
I'm asking because tsk doesn't use the classic shebang on purpose, to keep the source file valid Scala (and get great IDE support in exchange). I'm curious if that decision breaks any usecases (and which ones).

I am referring to NixOs and Nix package manager ability to download and install other interpreters  on the fly if the shebang looks some thing like this
#! /usr/bin/env nix-shell
#! nix-shell -i python -p python

print "Hello world!"

In an organisation where everyone has Nix installed, you don't need to ask users to install Python, Scala or whatever you need in your script.
However the target interpreter has to support multiple lines shebang. Something like NodeJs only supports one line shebangs then complain that second line isn't valid Javascript syntax.
Because Scalac doesn't understand shebangs I have to copy the file somewhere and comment out the header.

Thanks for elaborating Tamer. Sounds useful for an organization which is into Nix. Shame that Scala compiler doesn't like shebangs, it sort-of ruins the developer experience for people who would like to have IDE support for hacks like that (unless one would go for something like JBang edit command).
Since Scala is a flexible language, I imagined a way to support shebangs by introducing some extra values into scope, for example:
@ import scala.language.postfixOps
import scala.language.postfixOps

@ def / = ()
defined function /

@ def #!/ = new { def bin(slash: Any) = new { def env = "see? valid Scala" } }
defined function #!/

@ #!/bin/env
res17: String = "see? valid Scala"

but that becomes a rabbit hole quickly.

@tabdulradi I think scriptisto can be used to overcome this without changes to Scala? It still needs the first shebang line though.


Fix optional vnc container

What does this PR do?
This PR fixes a number of problems with the existing vnc testclient container:

rename the container to vnc since that is its purpose
add vnc.infra.ciab.test hostname entry into the static dns server configuration.
move the vnc container under the optional containers directory for better organization
updates the shared volume required to get access to shared CA x509 SSL certificate
fixes to allow firefox to start up in a  container without dbus or systemd
default vnc password is now random by default. Override via environment at runtime.
Port changed to 5909 to avoid port collision

New proposed directory structure for optional containers:
cdn-in-a-box/optional
cdn-in-a-box/optional/docker-compose.$NAME.yml
cdn-in-a-box/optional/$NAME/

Which TC components are affected by this PR?

[ ] Documentation
[ ] Grove
[ ] Traffic Analytics
[ ] Traffic Monitor
[ ] Traffic Ops
[ ] Traffic Ops ORT
[ ] Traffic Portal
[ ] Traffic Router
[ ] Traffic Stats
[ ] Traffic Vault
[x] Other cdn-in-a-box_______

What is the best way to verify this PR?

Add VNC_PASSWD with six alphanumeric password to variables.env
Start up the CIAB stack using the following alias:

alias vncdc='docker-compose -f docker-compose.yml -f optional/docker-compose.vnc.yml'
vncdc build
vncdc kill && vncdc rm -fv
docker volume prune -f
vncdc up

Connect with vnc client to localhost 5909 with $VNC_PASSWD
firefox should start as soon as the traffic portal is available.

Check all that apply

[ ] This PR includes tests
[ ] This PR includes documentation updates
[ ] This PR includes an update to CHANGELOG.md
[ ] This PR includes all required license headers
[ ] This PR includes a database migration (ensure that migration sequence is correct)
[ ] This PR fixes a serious security flaw. Read more: www.apache.org/security

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/2758/
Test PASSed.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/2816/
Test FAILed.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/2817/
Test FAILed.

Absolutely.

Documentation updated.  Depends on PR #3040

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/2903/
Test PASSed.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/2904/
Test FAILed.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/trafficcontrol-PR/2916/
Test PASSed.


Unsure about loading rules

I am using straight with both straight-use-package-by-default and use-package-always-defer set to t.
In particular I have one use-package statement to load auctex-latexmk like this
(use-package auctex-latexmk
  :after tex
  :commands my/Latexmk-compile
  :bind (:map TeX-mode-map
              ("C-c C-c" . 'my/Latexmk-compile))
  :init
  (setq auctex-latexmk-inherit-TeX-PDF-mode t)
  (defun my/Latexmk-compile ()
    ...)

  :config
  (message "Loading latexmk")
  (auctex-latexmk-setup))

My understanding (and experience loading other packages) is that the package auctex-latexmk will be loaded after tex, when I use the command my/Latexmk-compile or when C-c C-c in TeX-mode-map is pressed.
However, the package is not loaded and the config part is never executed in my case (even when calling the function my/Latexmk-compile or pressing C-c C-c or loading tex) and I am not sure why.
I also tried removing the :after and :bind parts in the previous code and calling my/Latexmk-compile, but auctex-latexmk was still not loaded and auctex-latexmk-setup was still not called.
This package is the only one that is giving problems and I am not sure why.
Another question about loading rules is that :init is supposed to run before a package is loaded. But I have seen where a mode is enabled in :init rather than in :config. This seems to work but I don't understand how the function to load the mode can be called before a package is loaded.

Apologies. This is related to use-package not straight.


Model update

Changed User.js password length to be 8
Changed Conversation.js

Made maxlength for header 50
Made references to author username

Updated typedefs accordingly

@kwubbenhorst These are the most up to date model fields. You can base your seeds on this!


Remove Cargo.lock from gitignore

Cargo.lock is required to make prost build in NixOS with standard build tools, I could make it myself and include it in nixos' repo but this would be a pain to maintain at each prost update. And it also seems like a good practice to include it in your rust projects, see : https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html

Hey @SCOTT-HAMILTON, the link you included says

I'm going to close this out since I don't think it's appropriate for a library-only crate, but please feel free to re-open if I'm overlooking something.


wsl hangs forever

Windows Version
Microsoft Windows [Version 10.0.22631.3880]
WSL Version
wsl.exe --version
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
No response
Distro Version
No response
Other Software
No response
Repro Steps
run wsl.exe
Expected Behavior
wsl should start normally
Actual Behavior
wsl hangs forever. This is also true for many, but not all, of the command line switches. For example, wsl.exe --version works just fine, but wsl.exe --list hangs forever as well.
Rebooting seems to solve the issue, but after the computer sleeps I will often (but not always) find wsl in this same error state. I first notice this about a week ago, perhaps after auto-installing KB5040442 and KB5039895.
I have read several of the other threads on this issue from a few years ago, but none of the solutions there have worked for me (my setup is pretty vanilla).
Diagnostic Logs
WslLogs-2024-07-17_09-57-43.zip

I was not able to include the kernel or distro version because those commands would not execute.

/dumps


Allow configuration through a block, simplify resetting config during tests

This creates a public interface for configuring Judoscale through a block, commonly used for setting up libraries with different configs, and will facilitate moving away from some of the ENV var configs in favor of suggesting this block config, where customers can then rely on ENV vars if they decide to do so.
It also exposes a way to reset the config so we can more easily do that between tests instead of relying on the  Singleton.__init__ approach to clear out the cached singleton instance.
I have updated the two places in the doc that used direct Config.instance access, but not touched the "Configuration" section as that is still referring to some of the ENV vars supported. I think the next step would be to remove support for those ENV vars and update the Readme there to show the new configuration block way.

@adamlogic looks like I forgot to assign this one for review prior to #26, but that one is based on the changes here.


Add Remove button for mutliple STL

This adds a remove button so that models can be removed from the platform after they are placed.  It is based on the new_multiple_stl branch so either merge this one and get both or merge first https://github.com/kennethjiang/OctoPrint-Slicer/pull/33 and then this one.  Doesn't matter to me.
Thanks!  Let me know if you have UI suggestions.

great. I guess I will just merge this PR and close #33
However you forgot to add remove.png to the repo. Can you add it so that I can merge?

Oops, forgot to git add.


Low test accuracy with resnet 110 cifar10

I train a few resnet v2 110 model with cifar10, and only get 92.x% accuracy on the test set. In the table, it shows test accuracy is about 93.x%. I wonder how to get that?

I tried the default learning rate, regularization in cifar10_resnet.py, and a few other hyper parameters, still get 92.x% on test set.

It should be replicable if you run GPU and epochs > 120

@roatienza do you mean use the default parameters, n=12, version=2, I can reproduce the 93.x% on the test set by running cifar10_resnet10.py? or do I need to change any hyper parameters? Could you share your hyper parameter settings to reproduce the result?

Yes. n=12, version=2. The best accuracy is about 93.15% as shown on the table.

@XinLiuNvidia  increasing batch_size to 128 got me to 93.x% accuracy on the test set.

@fuzzythecat thank you very much! After sync to the latest keras version, I'm able to reproduce the accuracy 93.x% on the test set.


feat: boot simulator if app fails to install due to simulator state 149

When building and running, sometimes the simulator may not be on a booted state, and thus the script will fail without trying to boot. This applies a retry functionality to boot the simulator prior to install.

Checking these comments today! 👀 Thanks for the feedback and the prompt review.


Np 47469 to strict query validation

Does this change introduce the possibility that we can be sql-injected?

Yes, but it is either allowing that or keep a strict validation on query param which might cause some queries to fail.
Also, since this is just a proxy to cristin, cristin should handle it themselves.
Suggestions for improvements?


Support perf metrics for web

Goal is to have the ITimerService and the Performance View (F1 > Startup Performance) working for the web. Not all values we currently have apply but things like loading the the main bundle or digesting contributions does apply.

with ea59ae20bee5e76b1d260e34e0b7b696d0b94a7d and 2e802ebcfea5bd69957a496cf6489ce2608869ac the timer service and the perf view is available in the web. What's missing is some critical perf mark to measure startup

@bpasero can you help me to get equivalent marks for willLoadWorkbenchMain and didLoadWorkbenchMain ( see here)

@jrieken I would think that you can add didLoadWorkbenchMain here:
https://github.com/microsoft/vscode/blob/c70ac014c39714d2b611407693288728bbc1b4b7/src/vs/code/browser/workbench/workbench.ts#L281
As for willLoadWorkbenchMain the only solution I can think of is adding it into the script tag of the HTML before we load other scripts (this would not make them appear for Codespaces though as they have their own HTML entry file):
https://github.com/microsoft/vscode/blob/c70ac014c39714d2b611407693288728bbc1b4b7/src/vs/code/browser/workbench/workbench.html#L30
and
https://github.com/microsoft/vscode/blob/c70ac014c39714d2b611407693288728bbc1b4b7/src/vs/code/browser/workbench/workbench-dev.html#L29
This needs tweaks to our CSP though.

vscode/src/vs/code/browser/workbench/workbench.html

How does that case work? There is no require call in that one

I actually do not remember anymore, quite sure I did this together with @alexdima. But how is that relevant, I think we just need to ensure to put the perf-start-marker as early as possible, no?

In the case of src/vs/code/browser/workbench/workbench.html, we use manual <script> tags to load the workbench, which is faster than creating script tags from the loader.js. The requests can be made in parallel by the browser as soon as the browser parses the .html and the evaluation still happens sequentially, but that is OK.
So if you want to know the time it takes to "require" workbench.main.js, I think you could try adding a <script> tag in the header and recording the Date.now() or perhaps use some special purpose HTML5 API -- https://www.html5rocks.com/en/tutorials/webperformance/basics/.

@alexdima where would I put the "after" mark? What's the signal that all of the workbench.main has been loaded?

What's the signal that all of the workbench.main has been loaded

nvm - figured that out

Adding renderer ready which is purely focused on the renderer and allows us to compare browser perf vs electron-renderer perf

@jrieken I like. I see that for web this is the very first entry in the HTML, but for desktop it is not. Shouldn't we put the renderer/started very high up into src/vs/code/electron-browser/workbench/workbench.html to compare the same thing?


Add option to control the rubber band stiffness

Feature request is related to

[x] embla-carousel (core package)
[ ] embla-carousel-react
[ ] embla-carousel-vue
[ ] embla-carousel-svelte
[ ] embla-carousel-autoplay
[ ] embla-carousel-auto-height
[ ] embla-carousel-class-names
[ ] embla-carousel-docs (documentation website)

Is your feature request related to an issue?
I want to control the rubber band effect when attempting to swipe outside of where the slides are.
Describe the solution you'd like
A number option labeled for example rubberBandStiffness or something similar
Describe alternatives you've considered
I haven't considered alternatives
Additional context
Video describing what I want to control:
https://user-images.githubusercontent.com/58148984/208880765-1110f762-76b1-44f7-a09f-f7aa152b1d39.mov

Hi @PumpedSardines, please respect my decision here #321.

Sorry, I must be blind because I didn't see that issue pinned. I understand your decision completely, sorry to bother you!


Fix typos

Thanks @slozier!  @omh1280 do you want to review this?

Looks good! Thanks @slozier!

Thanks @slozier @omh1280!


Update Sample Data for Final Demo and Ui.png

Pull Request Test Coverage Report for Build 318

12 of 19 (63.16%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 87.119%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/main/java/seedu/address/model/util/SampleEventUtil.java
0
7
0.0%

Totals

Change from base Build 317:
-0.1%

Covered Lines:
2746

Relevant Lines:
3152

💛  - Coveralls


Tests support django 1.10 && Shallow copy field

Updated runtests to support new django 1.10 config.

Coverage remained the same at 95.05% when pulling ea275814b020444f88af670f5bb8a084fa549b65 on shwoop:master into 970bf5b21b32acff92624deeb4b08e1029253209 on kmike:master.

Coverage increased (+0.2%) to 95.283% when pulling 2c8d1df99b2ee9bb9c25d2821f85a1208b8817df on shwoop:master into 970bf5b21b32acff92624deeb4b08e1029253209 on kmike:master.

Coverage increased (+0.2%) to 95.238% when pulling 33a3af5fcaf1771e1ff0c9ad8a47f19e85c98983 on shwoop:master into 970bf5b21b32acff92624deeb4b08e1029253209 on kmike:master.

Coverage increased (+0.1%) to 95.146% when pulling fb0a53a3bb0cb75c1e06fb96e926f957f445066a on shwoop:master into 970bf5b21b32acff92624deeb4b08e1029253209 on kmike:master.

Thanks, the changes look good!
Could you please confirm that new tests failed before the fix?

with local diff:
diff --git a/widget_tweaks/templatetags/widget_tweaks.py b/widget_tweaks/templatetags/widget_tweaks.py
index b3d48f8..4db2b22 100644
--- a/widget_tweaks/templatetags/widget_tweaks.py
+++ b/widget_tweaks/templatetags/widget_tweaks.py
@@ -20,7 +20,7 @@ def _process_field_attributes(field, attr, process):
     attribute = params[0]
     value = params[1] if len(params) == 2 else ''

-    field = copy(field)
+    # field = copy(field)

     # decorate field.as_widget method with updated attributes
     old_as_widget = field.as_widget

We get the new tests error, due to the edited attributes bleeding across re-renders:
[~/repos/django-widget-tweaks]$ ./runtests.py                                                                                *[master]
......x...............x..FFF.................
======================================================================
FAIL: test_field_double_rendering_id_class (widget_tweaks.tests.RenderFieldTagFieldReuseTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/alistair/repos/django-widget-tweaks/widget_tweaks/tests.py", line 347, in test_field_double_rendering_id_class
    self.assertEqual(res.count("id_1"), 1)
AssertionError: 2 != 1

======================================================================
FAIL: test_field_double_rendering_id_name (widget_tweaks.tests.RenderFieldTagFieldReuseTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/alistair/repos/django-widget-tweaks/widget_tweaks/tests.py", line 336, in test_field_double_rendering_id_name
    self.assertEqual(res.count("id_1"), 1)
AssertionError: 2 != 1

======================================================================
FAIL: test_field_double_rendering_name_class (widget_tweaks.tests.RenderFieldTagFieldReuseTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/alistair/repos/django-widget-tweaks/widget_tweaks/tests.py", line 358, in test_field_double_rendering_name_class
    self.assertEqual(res.count("n_1"), 1)
AssertionError: 2 != 1

----------------------------------------------------------------------
Ran 45 tests in 0.046s

FAILED (failures=3, expected failures=2)

Great, thanks!


0.3.3 Release

More generalized improvements to the server speed and server return, moved interpolation to ToolipsServables, and fixed stream closing without obtained IP by changing how IP is managed.
@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/109780
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.3.3 -m "<description of version>" 0e05d464cc551fff1928a8b87a583b9e4247cafa
git push origin v0.3.3

Oh right, I need to wait for ToolipsServables 0.1.6 -- oops

@JuliaRegistrator register

Registration pull request updated: JuliaRegistries/General/109780
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.3.3 -m "<description of version>" 0e05d464cc551fff1928a8b87a583b9e4247cafa
git push origin v0.3.3


Removed extra {{end}}

Description

Looks like with the recent PR, there was extra {{end}} introduced.
Checklist

[ ] Unit tests updated
[ ] End user documentation updated

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 69.938%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
13980

Relevant Lines:
19989

💛  - Coveralls


Missing dash in commit-lockfile-summary option

Describe the bug
commit-lockfile-summary -> commit-lock-file-summary + alias + check all docs and other occurrances

Refs https://github.com/NixOS/nix/pull/8220#issuecomment-1510425086

@roberth  I am wondering if this is correct in the nixpkgs repo? seems to me more like an nix issue?


Unable to build "sourcerer-app.jar" on macOS.

This is what happened.
$ gradle build

FAILURE: Build failed with an exception.

* Where:
Build file '/Users/khhan1993/Desktop/sourcerer-app/build.gradle' line: 22

* What went wrong:
A problem occurred evaluating root project 'sourcerer-app'.
> Could not initialize class org.jetbrains.kotlin.gradle.internal.KotlinSourceSetProviderImplKt

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 506ms

Java version
openjdk version "1.8.0_222"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.222-b10, mixed mode)

same issue , seems like issue with new gradle version 6.0.1 ,
was working well with gradle version 5.6.2

Updating the Kotlin version referenced in the Gradle build file to 1.3.61 and keeping the version of org.jetbrains.kotlin:kotlin-stdlib-jre8 at 1.2.31 solves this issue.

in short for Mac users:
brew update brew install kotlin


IntersectionObserver is not defined

I have got some error reports with the following message:
IntersectionObserver is not defined

Reports received from user-agent:
Facebook 115.0.0
Facebook 109.0.0
Facebook 114.0.0
Facebook 113.0.0

I think that polyfill.io cannot identify Facebook webview user-agent and cannot provide the correct polyfill.
So I have tried to add unknown=polyfill, but with no luck.
An error was caught:
Script error.

Also, setting unknown=polyfill will cause many errors with our scripts.
for example: getComputedStyle returns undefined in modern browser.
I have tried android webview and the browser has implemented getComputedStyle but due to the unknown user agent, polyfill.io will respond all polyfills.
therefore, unknown=polyfill cannot use at any time. I would like to suggest Polyfill.io not to polyfill if the function exists.

https://cdn.polyfill.io/v2/polyfill.min.js?features=IntersectionObserver,requestAnimationFrame&excludes=Element,getComputedStyle&unknown=polyfill

this request works fine now. But I don't know how about if browsers don't have Element and getComputedStyle. I think Element, getComputedStyle are not working properly.

Hi @shtse8.  To help us debug this, could you give us some more information:

The full User-Agent header of the browser in which you observed the problem
A reduced test case using a tool like jsbin.com that loads only polyfill.io and as much inline script as is needed to trigger the issue

Without this, I can make an educated guess that Facebook browser is being treated as an unknown UA, and that's why you need unknown=polyfill to make it work.  However, unknown=polyfill gives you all polyfills, which may include those that don't work in that browser.  Results may be unpredictable.  I would suggest adding flags=gated when using this mode.
We are already tracking an issue with identifying Facebook browser as #990, so if that's your issue you can follow it there, otherwise feel free to reopen with the info above.

Yes, I think it is the same issue. But I would not expect giving all polyfills will cause errors, once it is, it is very dangerous for those unknown user agent. why don't checking the existance of the methods instead of just overriding all?

@shtse8 flags=gated does exactly that, see https://polyfill.io/v2/docs/api.

@Gheoan Thanks. I miss this flag. how about the always flag? i can't understand the description.


Android sharing url incorrect when using sub-directory

My server URL is https://hauk.domain.com/hauk/ .
All works well to update the server. But the "Share tracking link" button in the Android app chops off the /hauk/ directory to
https://hauk.domain.com/?XXX123-YYY456
manually  changing the URL in the browser to
https://hauk.domain.com/hauk/?XXX123-YYY456
works fine .
I am using app version 1.0.1

apparently not. thanks! all is working now.

Good to hear! Enjoy Hauk :-)


Switch XLSX input

see https://github.com/GenomicsStandardsConsortium/mixs/issues/610

https://github.com/GenomicsStandardsConsortium/mixs/raw/issue-610-temp-mixs-xlsx-home/mixs/excel/mixs_v6.xlsx


4779_make_getmac_threadsafe

https://github.com/knowm/XChange/issues/4779

These test failures are unrelated, seems like an issue with develop.

it's seems that some error in(kraken futures)

no BTC in https://futures.kraken.com/derivatives/api/v3/instruments response (very strange)
lower case parsing "pf" prefix in adaptInstrumentsMetaData method
trying to deep in...
@makarid ?

I think I fixed it
post PR a little bit later

done
https://github.com/knowm/XChange/pull/4787

@douggie
is this fix just in case?
or is there code that does not work without this change?


City / District Name - Spelling Mistake

Hello there,
I am not sure that this is an issue or not but I just want to highlight to you.
I found two - Incorrect or Misspelled district/city name.

Ahmadabad - Should be Ahmedabad
Dohad - Should be Dahod

Ref: Districtwise Sheet In Google Sheets.
I apologize to take your time If you find this an Irrelevant Issue.

Data Issue. Not a bug to be raised here.


White Cars

When following all the directions for installation, all cars are white any fix for this. I have a RTX ready card.

The details for the temporary fix for the white car issue are in the RTX-Remix Discord GTASA channel.
RTX-Remix Discord Channel: RTX-Remix Discord Channel
GTASA Remix Discord Group: RTX-Remix GTASA Discord Group. (To join this group, you first need to join the RTX-Remix Discord channel.)

SARemix Vehicle Toolkit script completely fix this problem


[FEATURE] Support to expose historical data points

Is there an existing issue for this?

[X] I have searched the existing issues

Feature description
Context
In the context of jobs, Yace allows scraping CloudWatch metrics, and the frequency and amount of data points are controlled by:

period: <int>: statistic period in seconds
length: <int>: how far back to request data in seconds

All the examples in the examples folder, show how to scrape different types of metrics and the period and length happen to be always the same. I'm currently working with a client who adopted YACE to scrape metrics and we configured it to get data points with a granularity of 1 minute and 5 minutes worth of data.
Take the following yace-config.yaml as a concrete example:
apiVersion: v1alpha1
sts-region: eu-west-2
discovery:
  jobs:
  - type: alb
    regions:
    - eu-west-2
    period: 60
    length: 300
    addCloudwatchTimestamp: true
    dimensionNameRequirements:
    - LoadBalancer
    metrics:
    - name: TargetResponseTime
      statistics: [Average]

Expected Behaviour
We run YACE, via docker-compose, using the configuration above and we were expecting to see the following:

As YACE is by default configured to scrape every 5 minutes the metrics in Prometheus were delayed by 5 minutes AS EXPECTED
Prometheus was configured to scrape YACE every minute (again default configuration), so we were expecting to see 5 datapoints in Prometheus shortly after YACE made the datapoints available in the /metrics endpoint. However, only the last data point was presented NOT EXPECTED

I spent a fair amount of time trying to understand if we misconfigured YACE but after enabling the debug logs, spending a fair amount of time looking at the source code, and running YACE locally I realised that's how is currently implemented. See https://github.com/nerdswords/yet-another-cloudwatch-exporter/blob/c7807a770bb427f8ddb2c7becac51185fb3e8230/pkg/clients/cloudwatch/v1/client.go#L120-L128
I believe this is NOT a BUG but a design decision to only include the latest data points and not any historic data as Prometheus will reject any samples that are too old.
Having said that, I believe that it might be really useful to be able to include historic data points if the period is small and the length does not go too far (making sure Prometheus is still happily ingesting all the samples)
New Feature
Implementing the approach above could be a bit controversial, and it will introduce breaking changes (or extra behaviour when the length is bigger than the period setting). Therefore, I'm proposing to introduce a new feature, e.g. -enable-feature=allow-multiple-datapoints or -enable-feature=allow-historical-datapoints whereby all the data points are included. So, in the example above, rather than getting the latest data point, the metrics page will include the 5 fetched from the CloudWatch API
What might the configuration look like?
No extra configuration would be required as I'm suggesting controlling this new behaviour globally via a new feature flag but I'm open to suggestions as it could be useful to have a fine-grain control by including an extra config setting at the job level
Anything else?
I'll include more details in terms of images and logs later on but I'm happy to give it a go, although it's been a while since I've done any serious development and I don't have huge experience with golang 😅

I've forked this repo and I've been playing with the code to see if I can include historical metrics. Hopefully, I'll have something to push in a branch and maybe a draft PR.
I think it will make more sense to have the addHistoricalMetrics flag at the job level and not as an extra feature flag as you might only want to enable this for a specific job. I've been looking at how the addCloudWatchTimestamp has been implemented.

@cristiangreco we initially did that to get the same level of metrics as CloudWatch but we want to have a more flexible control at the expense of metrics being "late". So, ideally, I want to keep the scraping interval high but being able to go far in the past so the metrics scrape by our prometheus also fetches the "historical" metrics. By always having the "latest" endpoint we won't be able to achieve that.
I totally understand that this is specific edge case when you're trying to "mirror" CloudWatch metrics with the original timestamp but I think it will also give you flexibility if you want to push "high resolution" metrics from CloudWatch metrics (i.e. anything under 1 minute. With the current approach if I have metrics produce at for example, every 15 seconds. I would have to configure the scraping interval to at least 15 seconds.
Also, from our testing, we have seen situations where if the metric is produced every minute, setting the default interval to 1 minute too might not be enough (usually the interval should be set to half of the rate of the original source, in this case 30 seconds). Also we have seen situations whereby the CloudWatch metric is not ready at that specific time so when the next scrape interval happens it will be taken the latest one but you will be missing a data point.
I know the current PR I currently have requires more work which I'm happy to complete. Just wanted to double check with you guys if you see it as a good feature to be included in the main release

If you guys think this feature has "legs" I'll be more than happy to progress it further this month. I have a list of things to improve (implementation-wise)
I've tested the fork/feature branch for our use case I can confirm that the visualizations in Grafana look exactly the same as the ones you get in AWS CloudWatch metrics

This fits our desired usage pattern as well. We would rather save the CloudWatch expense and scrape every 5 minutes, accepting the lag. That said, it would be ideal to have 60 second granularity from that 5m scrape. Realistically requiring accepting out of order metrics on scrape as well.


[Boxplot] Allow configuration of y-axis range

Is your feature request related to a problem? Please describe.
The y-axis range can't be specified by the user in boxplot charts.
This results in a complex (sometime impossible) reading of the chart, due to outliers
Describe the solution you'd like
Users should be able to specify the range for the y-axis in Chart Options, like any other chart with a numeric axis.
Describe alternatives you've considered
/
Additional context
/

If Superset wants to be used for statistical analyses, this is a mandatory point (and it would be more coherent with other charts)

Moving this to an Ideas thread... it seems like a perfectly reasonable feature request, and doesn't seem like it'd be a terribly difficult PR if you (or anyone finding the thread) is able to volunteer the contribution.


Copy html block to helper? (help needed)

In my file is a documentation and a markup part. For syntax highlighting I want the same code (pink rectangle) in my markdown block. Is there an easy way to simply "copy" the markup?

I can't access the content of the parent file context with options.fn(this) in my markdown helper. I searched for about 3 hours for now (github issues, stackoverflow, handelbars documentation) and didn't find any solution :-(. I dont want to copy&paste the html markup each time in my markdown block... 😕

You could use an inline partial
Example: https://jsfiddle.net/9D88g/855/

Works like a charm! Thanks! :-)


[DATASET GROUPS] Different color for RESET button?

It took me a few moments to find it when I needed it.
I leave the solution up to you

@vaunapuu
This is the new button:


init.py : NameError: name 'os' is not defined

Hi Team,
During my initial clone, I have observed below error :
It seems we need to import os in init.py
(venv) tusharniras@tusharniras-macbookpro:~/Desktop/sanbox/flask-admin-boilerplate$ python app.py
Traceback (most recent call last):
  File "/Users/tusharniras/Desktop/sanbox/flask-admin-boilerplate/app.py", line 4, in <module>
    from views import *
  File "/Users/tusharniras/Desktop/sanbox/flask-admin-boilerplate/views/__init__.py", line 3, in <module>
    from model import *
  File "/Users/tusharniras/Desktop/sanbox/flask-admin-boilerplate/model/__init__.py", line 3, in <module>
    from helpers.database import *
  File "/Users/tusharniras/Desktop/sanbox/flask-admin-boilerplate/helpers/database.py", line 2, in <module>
    import configuration
  File "/Users/tusharniras/Desktop/sanbox/flask-admin-boilerplate/configuration/__init__.py", line 11, in <module>
    app.config['MAIL_PASSWORD'] = os.environ['MAIL_PASSWORD'] #password
NameError: name 'os' is not defined
(venv) tusharniras@tusharniras-macbookpro:~/Desktop/sanbox/flask-admin-boilerplate$

requested the pull request


Forward / backward for mp4 does not work on iPad

Hello,
using latest piwigo 2.9.3 with latest Darkroom 2.3.2 theme.
Unfortunately I have some problems to play my *.mp4 on my iPad 2017.
Videos (1920x1080) are playing, but I am not able to forward or backward the video. I can do forward and backward, when switching into full-screen mode.
On my iPhone 8 I do not have these issues.
Any hint is welcome.
Regards
kuzco

Hello,
any news on that ?
Is there a way to downgrade the theme, so that I can check which version does not have this issue ?
Regards
kuzco

Hmm, I can‘t reproduce this on my iPad Air...

Old version can be found on PEM or here in the releases tab.
I don't know what you mean with 2. Link thumbnails to PhotoSwipe -> always?

Thnaks for tzhe info.
I have foind the old versions, but how can I install it. Just disable and delete the actual theme, and after that copy the "old" version in the theme-folder ?
Please ignore point 2 - have installed a backup and full-screen works now.


Configuring RPC interface

Getting a error , where the RPC server cannot be found
PHP Fatal error:  Uncaught RuntimeException: Error creating resource: [message] fopen(http://<IP_ADDRESS>:9001/RPC2): failed to open stream: Connection refused
[file] /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php
[line] 324 in /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php:253
Stack trace:
#0 /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php(335): GuzzleHttp\Handler\StreamHandler->createResource()
#1 /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php(52): GuzzleHttp\Handler\StreamHandler->createStream()
#2 /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(66): GuzzleHttp\Handler\StreamHandler->__invoke()
#3 /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/Middleware.php(29): GuzzleHttp\PrepareBodyMiddleware->__invoke()
#4 /mnt/e/work/code/web/twitter/src/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php(70): GuzzleHttp\Mid in /mnt/e/work/code/web/twitter/src/vendor/lstrojny/fxmlrpc/src/fXmlRpc/Exception/TransportException.php on line 35

Does supervisord need to be configured to be able to use this library ? This page suggests so: http://supervisord.org/xmlrpc.html#rpcinterface-factories . But I can't find anything on how to configure this library.
What else could be the issue ?

We're running our supervisor instances at our company in Docker containers which run quite plainly like this (lots of comments included...):
[unix_http_server]
file=/var/run/supervisor.sock   ; (the path to the socket file)
chmod=0700                       ; sockef file mode (default 0700)

[inet_http_server]
port = *:9001

[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor            ; ('AUTO' child log dir, default $TEMP)
nodaemon=true

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = /etc/supervisor/conf.d/*.conf

The author of this package provides us a docker-compose.yml with an example supervisor container (from Dockerhub supervisorphp/supervisor:latest). Didn't test it with this image, we're installing our supervisord via apt-get install -y supervisor in the Dockerfile.
Are you running supervisord locally or in a Docker container? Any firewalls blocking access?
By the way, it looks a little bit like you're using WSL (/mnt/e/ looks pretty much like these Windows-HDD-Linux-Mountpoints). I'm not sure what you're trying to achieve.

Solved -
conf file was missing this:
[inet_http_server]
port = <IP_ADDRESS>:9001


DPDK: skip TC VERIFY-OVS-DPDK on non debian platforms

VERIFY-OVS-DPDK supports only DEBIAN based platforms for the moment, so it makes sense to skip the test on the other platforms.

Is any way to skip then against upstream pipeline?

Is any way to skip them against upstream kernel?

This should be handled by excluding the dpdk tests in the upstream pipeline in my opinion.

Thanks Chris for your reply! Yes, that's what I did currently.