[FEATURE] Reclaim/Shrink space of volume

Hello, I khow volumes can grow only, but what if I need to reclaim space of deleted files?
Is there other choice but remove/create volume?

@IAkumaI
I don't think there is another way to reclaim the space of deleted files. Regarding the total size of the volume, you might reclaim some by removing old snapshots.

Maybe we can try to implement trim on iscsi target.

@yasker Thanks for the explanation.
I experimented with running fstrim directly on the underlying file, and it did free up space. I'm afraid that could cause inconsistencies though. Do you think there is a feasible way to avoid losing the consistency, given that I am willing to avoid using snapshots if that's the cost?

@SeMeKh Losing the snapshot is not really an option. If you take a look at Replica Operations section in https://rancher.com/blog/2017/announcing-longhorn-microservices-block-storage/ , snapshots are the foundations of Longhorn. We rely on holes in the snapshots to know where is the data stored. TRIM can potentially change that. I think TRIM may change previous data into holes, so a read before TRIM supposes to read at snapshot C, but since snapshot C was trimmed and some new holes are created, Longhorn would see the hole (instead of empty data), continue reading on snapshot B and A, which result in data reading error.
It's just an assumption at this point. We can do more experiments and understand more about the potential impacts.

This has been a problem day one since hypervisors came along. You create for an example a 100GB disc and initially the VM installation takes up 30GB. But since the filesystems writes new data pretty much randomly over the the disc after a while the hypervisor will report 100GB usage even though you might only use 60GB inside the VM. Same thing seems to apply here with the containers and longhorn. For OS and hypervisors it has been solved with TRIM,UNMAP,discard,deletenotification etc. Anything is possible and nothing is free so it would be good if this can be prioritized otherwise it feels like a step back in the storage evolution.

I think the tricky part is the way snapshots are layered on a replica. Longhorn relies on hole detection to realize that data is on a snapshot layer below it. Say that snapshot has data when a hole is punched in the active image, then what should be all 0's turns into old-data pulled from the snapshot. From a security perspective (especially with iSCSI) if you zero a block, that data better not come back.
A couple ideas that could implement TRIM without adding metadata.

Check all the snapshot layers for holes before punching a hole. Only punch a hole if all of the underlying layers have holes as well
periodically scan for areas where a hole could be punched.

Those ideas might be more difficult than it's worth and there is a price to them. If you have a chain of 20 snapshots, you have to check for a hole in 20 files. Not exactly ideal.
I'm still interested in this feature, but after hearing how tricky this is, I'm willing to wait. It's not a problem for us. SSD TRIM is helps extend the life and performance of the drives, so passing it down to the underlying hardware is still something to keep an eye on, in my opinion.

I have a related but may be a bit different question. How to reclaim space by a removed first snapshot? My 5G volume became fully utilized by the stored ~300MB data and some experimentation with dd to see how a volume grows, then I deleted a replica (as part of learning Longhorn to also see what happens), it created a snapshot automatically to recover the lost replica, then I removed that snapshot but it never really disappeared.
# in the attached pod
$ df -h
Filesystem                                                            Size  Used Avail Use% Mounted on
/dev/longhorn/pvc-c0747652-32ce-4f68-b769-f7ff104ef0fd                4.9G  312M  4.6G   7% /config

# on a node
$ ls -la /var/lib/longhorn/replicas/pvc-c0747652-32ce-4f68-b769-f7ff104ef0fd-435f8290/
total 5119048
drwx------. 2 root root        233 Mar 22 22:11 .
drwxr-xr-x. 4 root root        120 Mar 22 20:44 ..
-rw-------. 1 root root       4096 Mar 23 09:41 revision.counter
-rw-r--r--. 1 root root<PHONE_NUMBER> Mar 23 09:41 volume-head-004.img
-rw-r--r--. 1 root root        178 Mar 22 22:06 volume-head-004.img.meta
-rw-r--r--. 1 root root        194 Mar 22 22:06 volume.meta
-rw-r--r--. 1 root root<PHONE_NUMBER> Mar 22 22:11 volume-snap-275d6f37-451e-45a2-91e1-d26ec2dfd8b5.img
-rw-r--r--. 1 root root        158 Mar 22 22:11 volume-snap-275d6f37-451e-45a2-91e1-d26ec2dfd8b5.img.meta

Still on the node by ncdu:

On the UI:

I would expect deleting a snapshot removes that snapshot from the disk, and only the head remains if it was the first snapshot. If it's not the case for the 1st snapshot but only for the 2nd and later, I can accept that as something would still be 5G as far as I understand, if not the first snapshot, then the volume head as Longhorn operates on the block level.
And this is where I can join the original conversation, it would be great to be able to reclaim that 4.7G space lost by experimentation (than later by some IO operations). Maybe I could run fstrim from within the attached pod? It's effect could sync to all replicas after that, couldn't it?

After reading this masterpiece (https://github.com/longhorn/longhorn/issues/1555#issuecomment-671097588), I feel I've run into the same issue as others :grinning:
However, these two questions might still be valid:

Why is the removed snapshot still present on the disk? Maybe it was the first and this is the way Longhorn works with the volume head never replacing back the first snapshot?
Could we fstrim from within the attached pod to propagate the change to the replicas as well? So not from the node which can be of course inconsistent but from the application using the volume? Or not supporting TRIM is also true from this viewpoint?

I could reclaim the 4.7G space on all replicas with a simple script to recreate the volume with a backup to a hostPath PV. As Jellyfin temporarily stores movie transcodes on the config volume which can take up hundreds of megabytes, I can semi-automate the PV recreation this way when the Actual Size will take up too much space over the ~300MB of base files.
migrate-longhorn.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: migrate
  namespace: media
  labels:
    app: migrate
spec:
  selector:
    matchLabels:
      app: migrate
  template:
    metadata:
      labels:
        app: migrate
    spec:
      containers:
        - image: alpine
          name: alpine
          command:
          - /bin/sh
          args:
          - -c
          - sleep 50000
          volumeMounts:
            - name: config
              mountPath: "/config"
            - name: config-bak
              mountPath: "/config-bak"
      volumes:
        - name: config
          persistentVolumeClaim:
            claimName: jellyfin-config
        - name: config-bak
          hostPath:
            path: /srv/hostpath/jellyfin
            type: DirectoryOrCreate
      nodeName: node2

recreate-longhorn.sh
#!/bin/bash

set -xe

kubectl -n media scale deployment jellyfin --replicas=0
kubectl -n media rollout status deployment jellyfin

kubectl -n media apply -f migrate-longhorn.yml
kubectl -n media rollout status deployment migrate

kubectl -n media exec -it deployment/migrate -- sh -c 'ls -la /config /config-bak'
kubectl -n media exec -it deployment/migrate -- sh -c 'rm -rf /config-bak/*'
kubectl -n media exec -it deployment/migrate -- sh -c 'cp -a /config/* /config-bak'
kubectl -n media exec -it deployment/migrate -- sh -c 'rm -rf /config-bak/lost+found'
kubectl -n media exec -it deployment/migrate -- sh -c 'ls -la /config /config-bak'

kubectl -n media delete -f migrate-longhorn.yml

yq r jellyfin.yml -d 0 | kubectl delete -f -
yq r jellyfin.yml -d 0 | kubectl apply -f -

kubectl -n media apply -f migrate-longhorn.yml
kubectl -n media rollout status deployment migrate

kubectl -n media exec -it deployment/migrate -- sh -c 'ls -la /config /config-bak'
kubectl -n media exec -it deployment/migrate -- sh -c 'cp -a /config-bak/* /config'
kubectl -n media exec -it deployment/migrate -- sh -c 'ls -la /config /config-bak'

kubectl -n media delete -f migrate-longhorn.yml

kubectl -n media scale deployment jellyfin --replicas=1
kubectl -n media rollout status deployment jellyfin

The yq r jellyfin.yml -d 0 output is the config volume claim:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: "jellyfin-config"
  namespace: "media"
spec:
  accessModes:
    - "ReadWriteOnce"
  resources:
    requests:
      storage: "5Gi"
  storageClassName: "longhorn"

It could be done more efficiently with emptyDir volume and without the execs using container command/args but this was easier to debug and I felt safer not to have accidental data loss.

Hi, we have this problem too. We don't really need snapshots since we are using Prometheus, and the longhorn volume is monotonically increasing despite us having set a retention size. Is there a work-around or anyway forward with using longhorn volume for write-delete heavy applications?

@kirinnee Currently we don't support trim, so the only advice I can give don't over provision your PVC, allocate the maximum expected file system size as your PVC volume size. Then if you don't use snapshot, your volume will not grow bigger then the maximum capacity since each snapshot is bound by that capacity. (live head is also a snapshot)

@joshimoo  Hi, it seems to still grow bigger than the maximum capacity. I'm not sure if I'm doing this right - correct me if I'm wrong.

Just in case, there's no snapshot with regards to this volume:

@kirinnee can you flip the switch to show System Hidden I am guessing you had some replica rebuild events, that left system snapshots behind?

Hi, that seems like the case. Can I trouble you to give me some readings on how replica rebuild events are triggered?

I have the same problem as @kirinnee . My PVC size is 20Go, but it's seem possible to grow up as 37Go ...

At this miment, I have a volume that grow without activity (PVC 20Go with now 26,1Go size showed by longhorn). And in the pod, with a df -h, I only have 2,8Go, so, it's weird :).

I have the same problem with some Volumes. Volumes don't have any snapshots and there is nothing when I click the Show Hidden button as well. This is going to be a huge problem for us in the near future because we can't reclaim disk space when we delete data.

same problems here, any progress?

Same thing. For example I have a volume for a Clickhouse database with nominal size of 30Gi. The actual data takes only 3-400MBs but the Longhorn volume's actual size is already at 6.5GB after a few days and I don't know why. There have been no rebuilds, there are no snapshots either created by me/recurring job, nor system snapshots.

Having a manual or automatic or some kind of scheduled space reclamation option would be a perfect solution for this issue. Everyone always deletes some data from the filesystem all the time. Backups are created using system snapshots as well and that is also increasing the actual size of the volume.

@yasker is there any news on when this problem will be fixed?

Not at the moment, best advice is to not over provision your volume during creation.
If you are only expecting 10GB usage then no need to provision 100GB, since you can grow the pvc -> volume in the future when needed.

Would be nice to reclaim the space. Makes it quite hard to see how much space is left in the volume.

I'm assuming that trimming the filesystem will effectively mark blocks as "don't care" blocks, i.e. the file system isn't using them and if it uses them in the future it will overwrite them.  ext4 and xfs both have discard mount options which look like they might automatically trim the block device as it is being used.
TGT can receive an UNMAP SCSI command, but the longhorn block device in tgt doesn't handle it.  I believe the UNMAP SCSI command corresponds to the trimming.   We could test added support for the UNMAP command and punch a hole in the volume's sparse file for the offset and length specified in the command.  If there are no snapshots, this will zero the command.   If there are snapshots, reading data at the unmapped offset could return data from one of the snapshots.  If the potential for returning inconsistent data may not be important for filesystem if it unmapped the data.   We should confirm that the filesystem is consistent after unmapping.
To try this out, we will need to:

Make tgt handle the UNMAP SCSI command and verify that the discard mount option or manually running fstrim cause the UNMAP SCSI command to be sent to the longhorn block device.
Add unmap command with offset and length to liblonghorn.
Add unmap to pkg/dataconn in longhorn-engine and forward all unmap commands from the longhorn-engine controller to all replicas.
Add support to the longhorn-engine replica to punch holes when it receives an unmap command.
Verify that an ext4 or xfs filesystem mounted with the discard option works with work flows that involve creating and deleting many files and the disk usages for that volume is smaller than without the change.

will xfs file system make any different ?

https://docs.portworx.com/reference/cli/volume-trim/

@keithalucas we need to be careful about the data we returned if a block is claimed as unmapped.
I agree that if a block is unmapped/trimmed, the user shouldn't care about it. However, there are two concerns if there are snapshots exist in the system:

As a block storage provider, what's the defined behavior for reading if a block is trimmed? Would the user expect 0 or anything from reading? We need to find more info about it or define it ourselves.
Would it be a security concern if we just read through the previous snapshots? since that data can still be directly read from the disk if you bypass the filesystem.

Agreed on the security concern of getting back data that was previously trimmed.
I have a couple other points I figure I'd add in:
There's also this implicit agreement that if a block is TRIMed, it will be zeros if/when read in the future. This may or may not be written into a spec anywhere, but the behavior is ubiquitous if the device supports trim.
I recognize your proposed work around though. It is a clever solution.
I'd like to see this solved eventually, but I expect it'll just take managing a block map or mask of some sort.

Here is how we shrink'ed a disk manually: https://github.com/longhorn/longhorn/discussions/4178.

Scale all deployments using the volumes to 0
In production ??
Let's continue trying to get a production solution
On Sat, Jun 25, 2022, 20:32 Iosif Nicolae @.***> wrote:

Here is how we shrink'ed a disk manually: #4178
https://github.com/longhorn/longhorn/discussions/4178.
—
Reply to this email directly, view it on GitHub
https://github.com/longhorn/longhorn/issues/836#issuecomment-1166330707,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADHJICQNBCNP36SYGO3NDFDVQ47DFANCNFSM4JFXS6YA
.
You are receiving this because you commented.Message ID:
@.***>

Hi, maybe we can escalate the priority of this issue, it will no longer be an enhancement feature, but a level 1 bug, as it has caused serious production incidents in our production environment.
The explanation is as follows:
When the actual size of the volume exceeds the nominal size, longhorn will consider the disk space of this node to be unschedulable, which will cause all pvc mounted on this node to hang (if the pod is restarted), resulting in the node hang up, causing the service deployed in the cluster to hang up (ie: serious production accident)
What's even scarier is that the data recovery of the cluster is catastrophic. If the pvc volume is not backed up, the data will be completely lost, because the actual size of the volume is constantly expanding, and the node to which the volume is mounted has been marked as unschedulable. At this time, if the program continues to write a large amount of data to this volume, the volume will be completely damaged and cannot be repaired (the node hard disk actually still has a lot of space df -h)
If the volume of data is too large, the recovery time of the volume will also be very, very long, starting at least half a day, and your production environment is always in a waiting state (suspended) during this process.
We have tested in the production environment, if the amount of data is only 300GB, it will only take about an hour for us to perform remote recovery through data replication, and it will take at least several hours or even longer to achieve remote recovery through longhron.
Summary: Longhorn is currently not suitable for scenarios with large amounts of data (such as logs, monitoring), nor for scenarios with large amounts of data deletion.

Thank you for @keithalucas's investigation and reporting. In brief, users should take their own risk of the data from the trimmed part being read if someone has access to the raw block device of the filesystem, right?

When the actual size of the volume exceeds the nominal size, longhorn will consider the disk space of this node to be unschedulable, which will cause all pvc mounted on this node to hang (if the pod is restarted), resulting in the node hang up, causing the service deployed in the cluster to hang up (ie: serious production accident)

I am suspecting the exceeded data on the cluster nodes triggers the Kubernetes node disk pressure issue. Then the Kubernetes node becomes unscheduable.
Right now there is no way for Longhorn to prevent the volume from continuing consuming space even if the rest free disk space is limited. By default, Longhorn enables volume over-provisioning. It's possible to trigger the disk pressure issue/Kubernetes node unscheduable issue as well if you create a large volume (of which spec size is greater than the disk free space) and try to fill up it (no snapshot in this case hence the actual size won't exceed the spec value). To avoid the issue(it's hard to eliminate it now), it's better to set an appropriate over-provisioning value based on the use case.

This is really level 1 bug for us, as @yl-yue says! 😰
Is it somehow possible to fix this bug before release v1.4.0 (v1.4.0 is planned for january 2023😯)?
Because of this bug we have to use NFS instead of Longhorn for multiple production workloads. 😢

Typically this kind of important feature won't be put into patch releases (v1.3.x) since it may have unpredictable impacts on the stability/robustness.

I want to advise my way how to reduce Actual size without data loss.
It is necessary to reduce the number of replicas to 1, and then increase.
For example You have pvc-123 on node 1. The original size is 20GB, the actual size is 29GB(In long UI), Let's increase the replica to 2. Now pvc-123 node1 and pvc-456 node2. After the rebuild, we remove pvc-123 and the actual size will decrease.

issue

#3888 #2510 #3423 #3120

discussions

https://github.com/longhorn/longhorn/discussions/4178

Please make it happen, I have some volumes that grow up to 100+ GB because that space was used at some point. one of the volumes now has about 8 GB in files and it's just ridiculous to deal with replicas and backups of 100GB...
Limit the max size is not really a solution, we allocate large volumes so we don't need to manually adjust them once we require more storage...

If the actual size of the volume doesn't get shrunk, then how can we monitor the volume.
In Our case we created a Grafana Dashboard for monitoring Longhorn Volumes and was planning to create an Alert once the usage is above 70% , but when usage level goes below the threshold the alerts still remain active as Longhorn Volume doesn't get shrunk.

@shuo-wu It seems straightforward but still needs an LEP for the feature review/understanding of the overall features and user behavior, especially about snapshot management. Added require/LEP.

+1 really great feature

Verified on master-head 20221213

longhorn master-head (aa3998e)
longhorn-manager master-head (4a758e7)
longhorn-instance-manager master-head (7026fbc)

The test steps
test-filesystem-trim
Result Passed
Case 1: Test filesystem trim during writing

The script execution succeed, we didn't get the error message on the console.

Case 2: Test flag unmapMarkSnapChainRemoved unchanged after volume engine upgrade

After upgraded volume with another engine image,
we saw the "Get backend <replica address> UnmapMarkSnapChainRemoved true"
we could not see the "Set backend <replica address> UnmapMarkSnapChainRemoved true"

cc @shuo-wu

Amazing fix
בתאריך יום ג׳, 13 בדצמ׳ 2022, 9:24, מאת oscni @.***>:

Great working this out! :)
—
Reply to this email directly, view it on GitHub
https://github.com/longhorn/longhorn/issues/836#issuecomment-1347852945,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADHJICUEVSAUHJXYFLFRK7DWNAQCPANCNFSM4JFXS6YA
.
You are receiving this because you commented.Message ID:
@.***>

Verified on master-head 20221213

longhorn master-head (aa3998e)
longhorn-manager master-head (4a758e7)
longhorn-instance-manager master-head (7026fbc)

The test steps

test-filesystem-trim

Result Passed
Case 1: Test filesystem trim during writing

The script execution succeed, we didn't get the error message on the console.

Case 2: Test flag unmapMarkSnapChainRemoved unchanged after volume engine upgrade

After upgraded volume with another engine image,
we saw the "Get backend <replica address> UnmapMarkSnapChainRemoved true"
we could not see the "Set backend <replica address> UnmapMarkSnapChainRemoved true"

cc @shuo-wu
@roger-ryao : great to hear that but  the longhorn volume still shows more space than the actual size after deleting the content, right? Am I misunderstanding something here? Some info bellow

Longhorn: v1.4.0(install from master branch)
Create Longhorn Volume: 10GB
Allow snapshots removal during trim:enabled(true)
Create some file tests then delete them.
Show file system info:

Show volume from Longhorn UI:

@tommy04062019 did you trigger filesystem trim?

@tommy04062019 did you trigger filesystem trim?

Do you mean I have to go to Longhorn UI, choose the Volume and click on Trim Filesystem? I did, but it has failures

@shuo-wu @longhorn/dev
Do we have any prerequisite before trigger Trim Filesystem?
Just create workloads to use Longhorn volume, like:
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/deployment.yaml
or
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/statefulset.yaml
Then triggered Trim Filesystem from Longhorn UI for each volume, I can observe different error messages shown up like:
unable to trim filesystem for volume test-1: zero or multiple mount points for volume test-2: [/var/lib/kubelet/plugins/kubernetes.io/csi/driver.longhorn.io/e063cdf36f817a24e97839b0799c023644dd1c31c668bda6481869027035a655/globalmount /var/lib/kubelet/pods/c63d1f31-0a5c-40c1-b4d6-2a70f7f14200/volumes/kubernetes.io~csi/test-1/mount]
or:
unable to trim filesystem for volume pvc-3155fa88-7f2d-423b-9e95-7c840c5d916f: cannot find volume pvc-3155fa88-7f2d-423b-9e95-7c840c5d916f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

cc @derekbit @shuo-wu

unable to trim filesystem for volume test-1: zero or multiple mount points for volume test-2

There are two mountpoints
/dev/longhorn/<pvc name> on /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc name>/globalmount type ext4 (rw,relatime)
/dev/longhorn/<pvc name> on /var/lib/kubelet/pods/xxxx/volumes/kubernetes.io~csi/<pvc name>/mount type ext4 (rw,relatime)

The first on is created by NodeStageVolume and the second one is a bind mount created by NodePublishVolume.
The request is rejected by https://github.com/longhorn/longhorn-manager/blob/master/util/util.go#L922.
From the fstrim spec, trimming a volume with multiple mountpoints is not forbidden. Not sure if there is any concern.

The second mount is just a bind mount of the global mount, so we should treat it the same. We should do fstrim against the global mount.
@roger-ryao this function did not work, so we need to revisit how we test it. cc @longhorn/qa
@tommy04062019 thanks for the feedback :+1:

@yangchiu Please re-validate this. Thanks.

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume:
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message:
unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

Same error with RWO:
unable to trim filesystem for volume pvc-858062c5-baef-472b-99fa-feb98a785574: cannot find volume pvc-858062c5-baef-472b-99fa-feb98a785574 mount info on host: failed to execute: nsenter [--mount=/host/proc/985/ns/mnt --net=/host/proc/985/ns/net fstrim /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-858062c5-baef-472b-99fa-feb98a785574/globalmount], output , stderr fstrim: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-858062c5-baef-472b-99fa-feb98a785574/globalmount: FITRIM ioctl failed: Remote I/O error : exit status 1

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

Same error with RWO: unable to trim filesystem for volume pvc-858062c5-baef-472b-99fa-feb98a785574: cannot find volume pvc-858062c5-baef-472b-99fa-feb98a785574 mount info on host: failed to execute: nsenter [--mount=/host/proc/985/ns/mnt --net=/host/proc/985/ns/net fstrim /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-858062c5-baef-472b-99fa-feb98a785574/globalmount], output , stderr fstrim: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-858062c5-baef-472b-99fa-feb98a785574/globalmount: FITRIM ioctl failed: Remote I/O error : exit status 1

@yangchiu
I never tried RWX. Will check it again. But the error message means the mount point is not existing.
@tommy04062019
RWX as well? The error message shows up error. Can you provide the steps to reproduce?

@tommy04062019
The error message shows io error. Can you provide the steps to reproduce?

@derekbit Steps I made to test as bellow:

Install longhorn with v1.4.x-head version

Run deployment with  volume of 10GB : https://github.com/longhorn/longhorn/blob/master/examples/deployment.yaml

Create some test file and then deleting them.

Go to Longhorn UI

Execute Trim Filesystem and it shown an error as I comment above

fstrim: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-858062c5-baef-472b-99fa-feb98a785574/globalmount: FITRIM ioctl failed: Remote I/O error : exit status 1 --> I think fstrim command should be: fstrim fstrim: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-858062c5-baef-472b-99fa-feb98a785574 without globalmount

@tommy04062019 Sorry but since I'm failed to reproduce the error you encountered, could you help to provide more information about your test steps? Like how/what you created/deleted the test files? did you take snapshots or do any other operations? And if other environment information like OS, k8s distro version can be provided would be very appreciated. Thank you!

@tommy04062019 Sorry but since I'm failed to reproduce the error you encountered, could you help to provide more information about your test steps? Like how/what you created/deleted the test files? did you take snapshots or do any other operations? And if other environment information like OS, k8s distro version can be provided would be very appreciated. Thank you!

This is my environment:

Rancher : v2.5.15 (or v2.7.0)
Kubernetes Version: v1.21.12
OS worker: Ubuntu 20.04

Here is how I tested created/deleted files.

After pod is running I access to shell
cd to the mountpoint of Longhorn volume in the pod.

dd if=/dev/zero of=2gbfile-1 bs=2G count=1
dd if=/dev/zero of=2gbfile-2 bs=2G count=1
sync
rm 2gbfile-1

On Longhorn UI, I did not take any snapshots, I just go to the volume and execute Trim Filesystem function.

Tell me if you need more info. Thank

@tommy04062019 Sorry but since I'm failed to reproduce the error you encountered, could you help to provide more information about your test steps? Like how/what you created/deleted the test files? did you take snapshots or do any other operations? And if other environment information like OS, k8s distro version can be provided would be very appreciated. Thank you!

This is my environment:
* Rancher : v2.5.15 (or v2.7.0)

* Kubernetes Version: v1.21.12

* OS worker: Ubuntu 20.04

Here is how I tested created/deleted files.
* After pod is running I access to shell

* cd to the mountpoint of Longhorn volume in the pod.

dd if=/dev/zero of=2gbfile-1 bs=2G count=1
dd if=/dev/zero of=2gbfile-2 bs=2G count=1
sync
rm 2gbfile-1

* On Longhorn UI, I did not take any snapshots, I just go to the volume and execute `Trim Filesystem` function.

Tell me if you need more info. Thank

With ubuntu 20.04 + k3s v1.21.12+k3s1 + longhorn v1.4.x-head and following the above operations, I'm still unable to reproduce this. Currently I don't have any clue. @derekbit Do you have any insight into what will possibly cause this error?

@tommy04062019 Sorry but since I'm failed to reproduce the error you encountered, could you help to provide more information about your test steps? Like how/what you created/deleted the test files? did you take snapshots or do any other operations? And if other environment information like OS, k8s distro version can be provided would be very appreciated. Thank you!

This is my environment:
* Rancher : v2.5.15 (or v2.7.0)

* Kubernetes Version: v1.21.12

* OS worker: Ubuntu 20.04

Here is how I tested created/deleted files.
* After pod is running I access to shell

* cd to the mountpoint of Longhorn volume in the pod.

dd if=/dev/zero of=2gbfile-1 bs=2G count=1
dd if=/dev/zero of=2gbfile-2 bs=2G count=1
sync
rm 2gbfile-1

* On Longhorn UI, I did not take any snapshots, I just go to the volume and execute `Trim Filesystem` function.

Tell me if you need more info. Thank

With ubuntu 20.04 + k3s v1.21.12+k3s1 + longhorn v1.4.x-head and following the above operations, I'm still unable to reproduce this. Currently I don't have any clue. @derekbit Do you have any insight into what will possibly cause this error?

@yangchiu : I have just tested agained, what a surprise, somehow it works. I'm sure I haven't changed anything since reporting this bug. This error not only shows up once but many times. May be I will tested on new cluster one more time. Let's wait and see

@yangchiu : I tested with new install on another cluster. It seems ok now. Need more time to obverse it, I will report then if it happen again. Also, I have a question:

If I have hundred to thousands volumes, I have to go to Longhorn UI and trigger Trim Filesystem manually, right? Do we have any way to do this automatically?

Do we have any way to do this automatically?

@tommy04062019
No for now. Can you create a ticket for the feature request?

Do we have any way to do this automatically?

@tommy04062019 No for now. Can you create a ticket for the feature request?

Yeah, sure, I will create one

BTW, I cannot reproduce the IO error as well. You can provide the support bundle for futher investigation if encounter it again. Thank you.

May be there are something wrong with Longhorn on my old cluster. It works on a new Longhorn cluster, I will check this more closely, will inform u with the bundle when having failed.

@tommy04062019 https://github.com/longhorn/longhorn/issues/4932 is planned.

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

@yangchiu
For a RWX volume, the /dev/longhorn/xxxx is mounted to /export/xxxx in a share-manager pod, so you cannot see the mountpoint on the host and hence got the error message No such file or directory.
A NFS mountpoint doesn't support discard/trim operation as well. For the RWX volumes, need to figure out a way to do trim operation. Let me create a ticket for the RWX trim feature.

We should add this limitation together with #5128 , but need to think of a validation way to avoid this unsupported operation.

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

@yangchiu For a RWX volume, the /dev/longhorn/xxxx is mounted to /export/xxxx in a share-manager pod, so you cannot see the mountpoint on the host and hence got the error message No such file or directory. A NFS mountpoint doesn't support discard/trim operation as well. For the RWX volumes, need to figure out a way to do trim operation instead of nsenter .... fstrim <mountpoint>. Let me create a ticket for the RWX trim feature.

@derekbit : I got 1 error while using RWX Volume (On v1.3.0, v1.3.2, v1.4.x)

Maybe I should create another ticket for it.

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

@yangchiu For a RWX volume, the /dev/longhorn/xxxx is mounted to /export/xxxx in a share-manager pod, so you cannot see the mountpoint on the host and hence got the error message No such file or directory. A NFS mountpoint doesn't support discard/trim operation as well. For the RWX volumes, need to figure out a way to do trim operation instead of nsenter .... fstrim <mountpoint>. Let me create a ticket for the RWX trim feature.

@derekbit : I got 1 error while using RWX Volume (On v1.3.0, v1.3.2, v1.4.x)
Maybe I should create another ticket for it.

Yeah, please provide us the steps and support bundle. Many thanks. :)

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

@yangchiu For a RWX volume, the /dev/longhorn/xxxx is mounted to /export/xxxx in a share-manager pod, so you cannot see the mountpoint on the host and hence got the error message No such file or directory. A NFS mountpoint doesn't support discard/trim operation as well. For the RWX volumes, need to figure out a way to do trim operation instead of nsenter .... fstrim <mountpoint>. Let me create a ticket for the RWX trim feature.

@derekbit : I got 1 error while using RWX Volume (On v1.3.0, v1.3.2, v1.4.x)
Maybe I should create another ticket for it.

Yeah, please provide us the steps and support bundle. Many thanks. :)

Sorry, it's my fault,I forgot to install nfs client on worker nodes:)

@derekbit Are we able to trim filesystem for a RWX volume?
I create a workload using Longhorn RWX volume: kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/examples/rwx/rwx-nginx-deployment.yaml
And try to trim filesystem from Longhorn UI, it gives the following error message: unable to trim filesystem for volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f: cannot find volume pvc-dd108b65-2e23-46dd-aaf6-bf3f38ac383f mount info on host: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net fstrim ], output , stderr fstrim: cannot get realpath: : No such file or directory : exit status 1

@yangchiu For a RWX volume, the /dev/longhorn/xxxx is mounted to /export/xxxx in a share-manager pod, so you cannot see the mountpoint on the host and hence got the error message No such file or directory. A NFS mountpoint doesn't support discard/trim operation as well. For the RWX volumes, need to figure out a way to do trim operation instead of nsenter .... fstrim <mountpoint>. Let me create a ticket for the RWX trim feature.

@derekbit : I just tested case with RWX. I think we can somehow access to shared-manager pod from instance-manager-e-... pod  when trigger Trim Filesystem on RWX volume, so  fstrim job will be executed in shared-manager-<pvc-name> pod.
Steps to test:

Create a workload using RWX with 1GB of data.

Create one test file with 200MB: dd if=/dev/zero of=test bs=200M count=1

Delete test file --> Trigger Trim Filesystem --> Error as above

Try to access to shared-manager-<pvc-name> pod and run fstrim with the volume

And then Longhorn UI shown as expected. Volume was trimmed

Thanks, @tommy04062019 , this is good feedback as you have done this to the mount point in the share manager pod. This will be the way when we implement trim support for RWX volume.

@yangchiu I will hang over the remaining testing to you and let's wrap this up. Thanks.

@derekbit Will we show any message on UI when trying to trim a volume with backing image?

@derekbit Do we support trim an encrypted volume?

@derekbit Will we show any message on UI when trying to trim a volume with backing image?

Timing a volume backed by a backing image is supported, even the layer/disk of the backing image will not be altered. Therefore, there is no specific message for it IIRC. Do you think we should notice users?

@derekbit Do we support trim an encrypted volume?

No, due to https://github.com/longhorn/longhorn-manager/pull/1613#issuecomment-1352559588. We need to mention this in the doc, https://longhorn.io/docs/1.4.0/volumes-and-nodes/trim-filesystem/. @shuo-wu

@derekbit Will we show any message on UI when trying to trim a volume with backing image?

Timing a volume backed by a backing image is supported, even the layer/disk of the backing image will not be altered. Therefore, there is no specific message for it IIRC. Do you think we should notice users?

Not sure if we need document for this or not?

Verified passed on v1.4.x-head (longhorn-manager 413f2b3).

Create Longhorn volume from UI or dynamic provisioning, the volume can be trimmed without error.
For RWX volume or encrypted volume, trigger trim filesystem will show no valid mount point error message on UI.
For volume with backing image, trigger trim filesystem will do nothing, and won't show any message on UI.

For volume with backing image, trigger trim filesystem will do nothing, and won't show any message on UI.

@yangchiu Could you test this again after adding/removing some data to the volume backed by a backing image? See if the removed data actually trimmed.
cc @derekbit

@derekbit Will we show any message on UI when trying to trim a volume with backing image?

Timing a volume backed by a backing image is supported, even the layer/disk of the backing image will not be altered. Therefore, there is no specific message for it IIRC. Do you think we should notice users?

Not sure if we need document for this or not?

YES, we need to update the doc https://longhorn.io/docs/1.4.0/volumes-and-nodes/trim-filesystem/ before closing this issue. @shuo-wu

For volume with backing image, trigger trim filesystem will do nothing, and won't show any message on UI.

@yangchiu Could you test this again after adding/removing some data to the volume backed by a backing image? See if the removed data actually trimmed.
cc @derekbit

Tested on v1.4.x-head:
(1) Create a volume with backing image of size 1.1 Mi, the actual size shown on Longhorn UI is 1.1 Mi
(2) Write extra 16 Mi data to the volume, and then delete it:
# dd if=/dev/urandom of=/data/guests/test-1 bs=1M count=16
# rm /data/guests/test-1

(3) The actual size shown on Longhorn UI becomes 17.2 Mi
(4) Trigger trim filesystem, the actual size is back to 1.1 Mi

The doc has been updated. Closing the issue.


update Roslyn references

This includes updating the document highlights API exposed by Roslyn.
NOTE: this PR is initially expected to fail as I have had to include a dummy package version number.  I'll update this PR with the real package number once it's available.
@KevinRansom @Pilchie @cyrusnajmabadi

What are you meaning by "official" packages?  I would expect signed packages to be published to myget as part of the signed build that Roslyn is going to insert.

@Pilchie that's what I meant; a signed package on MyGet that I can reference.

@brettfo I think https://github.com/dotnet/roslyn/pull/19525 being done in a non-breaking way means this isn't urgent to get from MyGet, but we do still need to address the completion and Tokenizer issues right?

Correct.  As soon as packages exist I'll push this through anyways, just to have it done.  It'll also fix an issue with our glyphs in the completion list since the backing enum has changed.

@dotnet-bot test vsi please // this should pick up recent VSI test fixes

@dotnet-bot test this please


Existing ids are not taken into account when generating new ids.

I use Microsoft.EntityFrameworkCore.InMemory to setup a db context for test.
When I try to add a record SaveChanges fails with the error "An item with the same key has already been added. Key: 1"
InMemoryDbTest.zip

A solution that works for me is using negative numbers for ids of records that are added during test setup.
Negative numbers will never conflict with auto generated values.


Feature: integrate NZ paleo timing rate data to opensha

SRM want to run inversions using Paleo timing constraints.
Done when

[ ] https://github.com/GNS-Science/nshm-paleo-db/blob/main/NZPaleoseismicSiteDB_v1.0.xlsx integrate with enum. reader etc so data is available for constraints

@voj data from @earjcr1  is now available here https://nshmrevisionproject.slack.com/archives/C02E5PXMB1D/p1637181658088700

merged


setting up etl

Describe your changes
Setting up script for data extraction
Issue number
Closes #X
Checklist before requesting a review

[x] Performed a self-review of my code
[x] Formatted my code with pkgmt format

:books: Documentation preview :books:: https://ploomber-sql--98.org.readthedocs.build/en/98/

Hi @jpjon
Can you please give this a read and let me know if the flow is clear, the challenges are reasonable, and if I did a good job of conveying the workflow?

The module is clear and concise.
The only feedback I have is that when I scroll all the way down from the Poetry section, the "Next" hyperlink does not go to the Designing a Movie Recommender System Pipeline section.
Also, when I'm on the Poetry section, the table of contents bar to the left does not contain the Designing a Movie Recommender System Pipeline section

I had to rebuild the docs a few times before the menu updates

The only feedback I have is that when I scroll all the way down from the Poetry section, the "Next" hyperlink does not go to the Designing a Movie Recommender System Pipeline section.

What does it direct you to?

It directs me to the Cloud-based options for deployment of applications section

are you able to open index.html and visit the "Designing a Movie Recommender System Pipeline" page?

Yup, I'm able to visit that page.


More options

[ ] Autohide (if no activity)
[ ] Download and upload
[ ] Hide KB/s label

I second the upload AND download option.
There's always activity or the connection would fail; netmonitors normally include a speed label.
Apk size is the real issue --- but a much desired app. Thanks!


Cant login

Hello ! I have a question, when I click on login with google or PTC, a message appears "invalid pokemon ids. only numbers and commas allowed",
I used the app yesterday, and it was working fine :(
what can I do ??

first of all there is an update.
if you didnt install it go a head and do it.
test it and replay.

Try the latest version.


Update stable version in repo from 1.1.6 to 2.1.0

Think about update stable version to 2.1.0
Version: stable=1.1.6 (364 days old) => latest=2.1.0 (15 days old)
Installs: stable=176 (83.02%), latest=22 (10.38%), total=212
Click to edit

This issue seems to be outdated.
This issue suggests to update the stable version of this adapter to 2.1.0 but in the meantime an update to version 2.1.2 is suggested.
So this issue will be closed and replaced by an updated one.


Failed to parse URI

When trying to download and install the modpack I have created, an issue occurs for one of the mods on the list that packwiz cannot find. The mod in question is InventoryHud+ for fabric 1.17 (latest version).
I am using MultiMC with the packwiz installer bootstrap + custom command.
Error log found in the popup gui: https://pastebin.com/6YzdqiMn

Duplicate of #5
My bad, checked the wrong github repo for duplicates first.


Can't run Synpress with Playwright with docker

I follow this issue to setup playwright+ synpress to run with docker in server Ubuntu 20, but still can't run metamask.acceptAccess
synpress  |
synpress  |   1) [chromium] › connect-wallet.spec.ts:12:5 › connect wallet using default metamask account ──────
synpress  |
synpress  |     Error: [switchToMetamaskNotification] Max amount of retries to switch to metamask notification window has been reached. It was never found.
synpress  |
synpress  |       20 |     .click();
synpress  |       21 |   await page.getByRole('button', { name: 'Confirm' }).click();
synpress  |     > 22 |   await metamask.acceptAccess({ allAccounts: false, signInSignature: true });
synpress  |          |   ^

My project : https://github.com/giapt/metamask-test
What can I do

I clone your repo - and it seem work - can you try reinstall dependencies and chrome driver again ?

Thanks, @cuongpo , it works when I run the test direct on my Macbook too, but I need to run with docker, then push to run on Jenkin ci/cd server
Error display when run with docker

Hey @giapt,
You either have to run your browser in headless mode or follow these docs:
https://playwright.dev/docs/next/ci#running-headed
Use xvfb-run to emulate desktop (because you're running browser in headful mode without desktop emulation).
I'm closing this issue as it's related to playwright itself, not synpress.
Thanks,
Jakub.


[Bug][MasterServer]  workflow is always running: state in t_ds_process_instance is always  1   but state in t_ds_task_instance is always 0 or 8 or something else

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
workflow is always running: state in t_ds_process_instance is always  1   but state in t_ds_task_instance is always 0 or 8 or something else
What you expected to happen
workflow finished es expected
How to reproduce
run a large batch of workflow
Anything else
No response
Version
dev
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

branch: 2.0.2-prepare

Hi, can you describe how to reproduce this problem?


:zap: perf(tabs): improve render speed for LCP

Description of this issue

check amount of initial renderers
check if there are any external sources to load after LCP
check if there are any animation to do after LCP or with the first user action

Code Reproduction URL
http://localhost:4000/test/seo/07-tabs.html
Performance
91
Largest Contentful Paint
3.2
Speed Index
2.2
Additional Information
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

do #1501 first

done with #1501


Not Recognising the Password (I gave my own password and tried on myself)

Tool not recognising the password, I tried giving my own password but not shown any success or this password is right like that if you able to fix this issue it really helpful

yes, it's depreciated, but I don't think I'm going to try to correct it because with my studies and personal projects I don't have the time for that.


Add ShardID to valid attributes

What changed?
Adding ShardID as valid attribute

Why?
Related to https://github.com/uber/cadence/pull/5099
Indexing fails if you will try to use "config/dynamicconfig/development_es.yaml"

How did you test it?

Potential risks

Release notes

Documentation Changes

Pull Request Test Coverage Report for Build 0186e4af-bce5-45e7-a64d-a74a3973f0c2

0 of 0   changed or added relevant lines in 0 files are covered.
109 unchanged lines in 18 files lost coverage.
Overall coverage decreased (-0.03%) to 57.049%

Files with Coverage Reduction
New Missed Lines
%

client/history/client.go
2
38.1%

client/history/metricClient.go
2
45.3%

common/task/parallelTaskProcessor.go
2
92.75%

common/task/weightedRoundRobinTaskScheduler.go
2
88.6%

service/history/handler.go
2
47.15%

service/history/task/transfer_active_task_executor.go
2
72.15%

service/history/task/transfer_standby_task_executor.go
2
86.82%

common/persistence/nosql/nosqlplugin/cassandra/workflow.go
3
59.1%

service/matching/taskListManager.go
3
81.15%

common/persistence/serialization/parser.go
4
63.91%

Totals

Change from base Build 0186e47c-902b-4c78-9a2f-38a5b8fc9749:
-0.03%

Covered Lines:
85233

Relevant Lines:
149404

💛  - Coveralls


Fix WorkloadLabels parsing from metadata

A recent PR, #16124, added parsing WorkloadLabels from the metadata.
This did not have the intended effect, as we actually ignored these
labels always because they were not strings. This change fixes this and
adds unit tests.
Please provide a description for what this PR is for.
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

/cherrypick release-1.3

@rshriram: new pull request created: #16694

In response to this:

/cherrypick release-1.3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Regex types

Given a type like:
type ColorHex = /^#([0-9a-f]{3}|[0-9a-f]{6})$/i

the following declarations would be valid:
const c1: ColorHex = '#000';
const c2: ColorHex = '#FF9900';

This is related to, but different from #86.

What if we treat match and matchAll as special functions?  We'll likely want to do this in other situations as well.  We can make the types opaque for now.  Let's introduce Regex and RegexMatch as two new types and provide types
for match and matchAll.
fn match(regex: Regex) -> RegexMatch | null;
fn matchAll(regex: Regex) -> RegexMatch[];

When we infer the calls to these functions, we replace the RegexMatch type with an object literal type containing the structure that's encoded into Regex type.
TODO:

[ ] add RegexMatch type
[ ] add overloads for match and matchAll to the String (to start we can replace the existing signatures)
[ ] add logic to covert RegexMatch type when infer function calls


Doc Feedback: invalid "WSO2 ESB" link

Location : https://apim.docs.wso2.com/en/latest/getting-started/overview/#api-gateway
WSO2 ESB link is not properly created - missing a '(' character after a ']' character:

The API Gateway is a runtime, backend component (an API proxy ) developed using [WSO2 ESB]https://apim.docs.wso2.com/en/latest/getting-started/about-this-release/)

closing issue since the PR is merged.


I forgot to run update.py

I forgot to run update.py before I made that PR, can you fix that?

right I shouldn't have merged it...


Spelling and grammar mistakes

There are spelling and grammar mistakes in vector.md (https://github.com/Bhupesh-V/30-seconds-of-cpp/blob/master/vector/vector.md). I will work on them!

I submitted a PR but it says that build has failed? Could you please look it over? I did not add any code. Thank you for your time.


🍒 Cherry pick PR #12013 to staging 🍒

🍒 Cherry pick https://github.com/Expensify/App/pull/12013 to staging 🍒

🚀 Cherry-picked to staging by @OSBotify in version: 1.2.18-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by @chiragsalian in version: 1.2.18-10 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/AndrewGable in version: 1.3.28-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[PE-D][Tester D] fullscreen help button bug

when im on full screen mode of the app and click on the help button, it would bring me to another full screened page that is displayed here, rather than a pop up

remedy: you might want to juz restrict the resizing of the windows as we had a lot of bugs from here also

Labels: type.FunctionalityBug severity.Medium
original: dominic2412/ped#13

Behaviour of pop-up is due to macos.

can put in UG as a known issue as well


"Bump 0.0.3"

Reverts aliyun/alibabacloud-api-jetbrains-toolkit#21

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (e22f1c1) 2.05% compared to head (498ece2) 2.05%.

Additional details and impacted files
@@          Coverage Diff           @@
##           master     #22   +/-   ##
======================================
  Coverage    2.05%   2.05%
======================================
  Files          11      11
  Lines         390     390
  Branches       51      51
======================================
  Hits            8       8
  Misses        381     381
  Partials        1       1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Question on high CPU usage

I had a cron job running inside a docker container, but the cpu usage is really high.

My code looks like this
func main() {
	c := cron.New()
	if err := c.AddFunc("@midnight", func() {
		log.Printf("Start crawling latest data on : %s", time.Now())
		start := time.Now()
		dailyUpdate(10000)
		elapsed := time.Since(start)
		log.Printf("%s took %s\n", "DailyUpdate", elapsed)
	}); err != nil {
		log.Fatal(err.Error())
	}
	log.Println("Start the crawler")
	for {
		c.Start()
	}
}

I want to ask if my way of using it is wrong. Thank you

Yes, replace
for {
   c.Start()
}

with
c.Run()


Updated the README.md with @hello-pangea/dnd

Updated the README.md with references to install @hello-pangea/dnd
Description
Even with the list of dependencies containing @hello-pangea/dnd, I think it would be interesting to contain this information in the project description;
How Has This Been Tested?
This is just an update on the README.md

Something's off with your fit history here. There's 44 files being "changed".
I'm not opposed to putting a blurb on the readme. If you get the history sorted, the readme addition looks good to me.

@christopher-caldwell sorry, created the PR via Github web interface, I think this must be a bug. I'll create a PR forking the project.

What is the status of this README update? I still see no mention of installing @hello-pangea/dnd as a dependency, even though I see it listed as a peer in your package.json.

@DrZanuff said he was going to fix this git issue, but it's been awhile. I'll close this PR and just update the readme.

This has been fixed by this PR.


/etc/container_environment/PATH ignored

I'm trying to set the PATH variable for my container, but it is not being picked up:
root@b83a0bcb0ea5:~# cat /etc/container_environment/PATH
/home/app/rails/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
root@b83a0bcb0ea5:~# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Am I missing something, or is this a bug?
I saw the merged PR #86, but it does not concern PATH?

I do not currently have the time to investigate this, perhaps @kingdonb or @hyperknot ?


Track NumJs memory performance problem: flatten() memory out

https://github.com/nicolaspanel/numjs/issues/21

Update:
https://github.com/nicolaspanel/numjs/issues/21#issuecomment-319301957


fix: proto query fixed

Query params don't accept sdk.Coin: https://github.com/grpc-ecosystem/grpc-gateway/blob/094a6fe78b3ca888297d090185cdf30f0e42e157/runtime/query.go#L395-L404

I ran make proto-all which updated the swagger docs

Thanks, make proto-all never worked for me so I'm using a custom cmd to generate the files.


Launch language server with environment from .env file (fixes #60)

This mostly fixes https://github.com/prisma/vscode-graphql/issues/60 by launching the GraphQL Language Server with variables from .env added to its environment.
The bigger issue however is that there is no visible error when the language server fails to parse the config file (e.g. when an environment variable doesn't exist at all, or for any other reason).

Thanks for the PR @peterholak. Merging this, I will notify on the issue when this is released.

This is available in 0.1.8 🙌


Resolve dependencies in the reverse BFS order

Resolve dependencies in the reverse BFS order, it will provide right objects lifetime when JetBrains Lifetime library is used to control lifetime of objects.

Need to think about better syntax. "Aware" stage won't work at all.

CreateByReflection and other instantiating build actions now build unit (ParameterInfo[], ServiceTag.Argument) rather than building each argument one by one. It allows having build dedicated build action which will build arguments for parameters in any needed order.


NO-ISSUE: Fix simulator

Fixing simutalor configuration and pathes
Fixing small code issues
Adding auto approve for devices that were created by simulator

Nice, thank you!,
Do you think it could somehow be added to https://github.com/flightctl/flightctl/blob/main/.github/workflows/pr-smoke-testing.yaml so we make sure it doesn't break over time?, it may need to exit 1 / 0 under some conditions (may be a flag to exit after enrolling devices or something..)
But just an idea for a follow up.

@tsorya one of the changes is influencing the enrollment request ID discovery from e2e tests on the console: https://github.com/flightctl/flightctl/actions/runs/10442439645/job/28917390092?pr=466#step:10:87

Nice, thank you!,
Do you think it could somehow be added to https://github.com/flightctl/flightctl/blob/main/.github/workflows/pr-smoke-testing.yaml so we make sure it doesn't break over time?, it may need to exit 1 / 0 under some conditions (may be a flag to exit after enrolling devices or something..)
But just an idea for a follow up.

Good idea, added an option to run simulator for some amount of time and in case we are able to run it should stop it after given duration and exit 0 and in case there is an issue with agent.Run we should exit 1

e2e failed on
Waiting for root prompt on device 293gupbavh35npjankertq0tsetqd0ln8usitf7kdjk1d4kvpcng console
don't think it is related to the change

/retest

/retest

/retest

Was it really a flake?
I thought it was due to the changes in "get id from console" changes.
@hexfusion let's document if we see flakes, we should fix them where in our hand.

Looks great, I was trying to merge, but the commits need the signature
Merging is blocked
The base branch requires all commits to be signed. [Learn more about signing commits.](https://docs.github.com/authentication/managing-commit-signature-verification/about-commit-signature-verification)

can you sign the commits?
Thank you :-)

done


Profile Page is Not working after copy share link from social card

When We copy share link in social card 'https://git-o-get.mridul.tech/profile/Diwanshumidha' the link is missing and showing 404 page
I Would like to add this feature

The actual URL is https://git-o-get.mridul.tech/user/Diwanshumidha
The slug is /user/[slug] not /profile/[slug]

Thanks for raising the issue. Please raise a PR after fixing the issue

@Mridul2820
can you please explain me what should be there in Profile/[id] page like all the requirements

It will be only /user/[slug]

So Sharecard will copy https://git-o-get.mridul.tech/user/adrianhajdin instead of https://git-o-get.mridul.tech/profile/adrianhajdin


Real-Time Query

I'm a little confused about the real-time implications of MTTS-CAN. From what I can gather, based on the model itself, it will only operate on a complete video and does so in isolation (i.e, the temporal information is only leveraged 'per-prediction' to the model and nothing is stored or leveraged for subsequent calls to the model), so while it can retrospectively give a HR and BR estimation for each frame provided to the model it can only do so after being provided all the frames at once for a prediction. Is my understanding of this correct, and that is it not possible to process a live video feed; providing a live HR and BR estimation actually in real-time?
If that is correct then my followup question would be, in your experiementation what is the minimum number of frames that can still provide an accurate estimation? As the only alternative to get would be to process n-previous frames each time a new live frame is received and only use the end values output by the model each time

When you want to do l ive-prediction, you need to input at least 10 frames or a multiplier of 10 frames (e.g., 20, 100, 200).


All console messages should appear in Notebook output

Some console log messages appear on the CLI but aren't making it through to the notebook output.

"All model batches have been executed successfully"
"The target environment has been updated successfully"

Likely fix is flushing the buffer, but some of these messages are interleaved between other outputs that are correctly appearing.

@treysp is this still an issue, we upgraded rich to use the notebook extension

Yes, some info is still omitted:

"All model versions have been created successfully"
List of individual model times
"Evaluating models"
"All model batches have been executed successfully"
"The target environment has been updated successfully"
"Creating new model versions" appears twice

CLI

Notebook

I tried reproducing this today but it looks like I see the messages you mentioned @treysp:


DOCS: added Readme file to the FilterSolutions example directory.

DOCS: added Readme file to the FilterSolutions example directory.

Hi and thanks for the contribution.
Unfortunately, we have moved the examples to a new repository and removed the example from pyaedt (see #5242).
The repository is https://github.com/ansys/pyaedt-examples


Example:
library(datasets)

s <- split(airquality, airquality$Month)
str(s)

results in (first 3 lines):
List

 of

5

$

5

:
'data.frame':

31

 obs. of

6

 variable

s

:

  ..

$

Ozone

:
 Factor w/ 68 levels "1","10","108",..:

35 30 8 15 68 23 20 16 58 68

 ...

Instead of the expected
List of 5
 $ 5:'data.frame':	31 obs. of  6 variables:
  ..$ Ozone  : int [1:31] 41 36 12 18 NA 28 23 19 8 NA ...

Fixed.


functionality for using in jupyter (not always writing / reading from…

… files)
It's likely this might need tests or documentation to ensure it doesn't break any new implementations. Let me know how best to go about this! Thanks again for reaching out :+1:

Looks great, many thanks @kjam !


expand on the LEARN, APPLY and CONNECT points, and also added in some…

… emoji to make the reading more fun

Merge PR


Use logging

Currently, it's pretty hard to trace errors when they happen. It would be helpful to use a wide-spread logging library and insert calls around the code.

Suggestions:

[ ] slog-rs (rich but heavy)
[ ] log (logging facade) (very minimal)
[ ] dedicated lightweight implementation similar or on top off log

slog-rs might be a bit overkill, and although it could come in helpful (ideally we will have several tasks running in parallel handling network), I would prefer to keep the core libraries free of bloat.
I like the idea of using log, it seems simple and small enough, and end-users are the ones deciding on the final logging implementation.
Not sure if rolling our own logging is a good idea: log already exists, and has been tested extensively, with hundreds of projects using it.

slog-rs might be a bit overkill, and although it could come in helpful (ideally we will have several tasks running in parallel handling network), I would prefer to keep the core libraries free of bloat.
I like the idea of using log, it seems simple and small enough, and end-users are the ones deciding on the final logging implementation.
Not sure if rolling our own logging is a good idea: log already exists, and has been tested extensively, with hundreds of projects using it.

I absolutely agree, if we just want grammers to be a reusable library it should be quite sufficient.

I too vote for log. There's no real reason to keep anything more complicated inside of the library.

I think logging is in a good state (both the sender and update handling make a decent use of logging). We can still add more logging calls as we see fit, but I think this issue has served its purpose, so closing.


add extra monitoring helpers from fork

I found these to be useful to monitor the status of the running servers:
https://github.com/Aeneas1/PowerShellGSM/commit/01e8c807ec4ae8046e07e19a068485ee818304b2
https://github.com/Aeneas1/PowerShellGSM/commit/cba08bb95ece29432e7f5dd3e7597235f38b6c15
https://github.com/Aeneas1/PowerShellGSM/commit/77c4d8ce25d9cdde0abbf8e36ca71c4251734df2
https://github.com/Aeneas1/PowerShellGSM/commit/aedbd5013ee87289ab65343d9c8d4b3ce558cf9b

oh that is still WIP, but feel free to use what ever you like.
my plan is to replace the windows tasks cause a server with 5+ gameserver is not useble, with tasks running every 5min.

nice @Aeneas1
Had a similar thought about replacing tasks und  using nssm for adding services (used it in another project as a cheap solution)
What were you thinking about being a solution @Aeneas1

i will do it with the monitor thread.
the monitor will check if the a server is alive, need a update or what ever.
so only the monitor is running. then is only 1 windows task needed to check if the monitor is running.
i can try to improve the monitor in the next days.

At this point I was thinking about rewritting the whole thing in C# as an application with a single task-scheduler task to check if it's still alive. That application would take care of monitoring everything.
I clearly don't have the time for that right now though.

If you do, let a C# service handle the chron jobs, don't use the scheduler.


Feature/draw line cap style

Simple PR; just wanted to be able to set the line cap style when drawing a line

Looks good. Thanks @taxilian!

Version 1.6.0 is now published with this feature. The full release notes are available here.
You can install this new version with npm:
npm install<EMAIL_ADDRESS>
It's also available on unpkg:

<EMAIL_ADDRESS><EMAIL_ADDRESS>
As well as jsDelivr:

<EMAIL_ADDRESS><EMAIL_ADDRESS>

Sorry I'm late to the party, but was the breaking change necessary? (required prop)
Couldn't we have lineCap?: LineCapStyle; instead of lineCap: LineCapStyle | undefined;?

Hello @antoinerousseau!
You’re absolutely correct. This addition should not have introduced a breaking change. The new prop should be optional. My apologies, I should’ve caught this before releasing. I’ll publish a patch for this as 1.6.1 as soon as I get back to my desk today. Thanks for calling this out!

Ok cool, and no worries!

My apologies; I was following what appeared to be the conventions of the file =] I'll be more careful next time.

@taxilian No worries! You did follow the correct conventions for the files you changed. Unfortunately, in order to avoid a breaking API change it's necessary to violate those conventions. Try as we might, it's quite difficult to prevent these little things from slipping though now and again.

@antoinerousseau Version 1.6.1 is now published. It contains the fix for this issue. The full release notes are available here.
You can install this new version with npm:
npm install<EMAIL_ADDRESS>
It's also available on unpkg:

<EMAIL_ADDRESS><EMAIL_ADDRESS>
As well as jsDelivr:

<EMAIL_ADDRESS><EMAIL_ADDRESS>

wow that was fast, thanks!


Error using old Git versions

Versioneer appears to make some assumptions about the Git version being used. In particular, the version 1.7.1 included in CentOS 6 causes errors like the following. Ideally, this would be fixed so that I don't have to give my users special instructions on how to update their Git. However, at a minimum, versioneer should detect the error so that I get a 0+unknown version instead of a hard error. Right now I can't even install the software on such machines.
This is using versioneer 0.18 from conda-forge.
Repro steps at bottom of post.
Begin: setup.py build_ext --xtcdata=/lcls2/install -f --inplace
usage: git rev-list [OPTION] <commit-id>... [ -- paths... ]
  limiting output:
    --max-count=nr
    --max-age=epoch
    --min-age=epoch
    --sparse
    --no-merges
    --remove-empty
    --all
    --branches
    --tags
    --remotes
    --stdin
    --quiet
  ordering output:
    --topo-order
    --date-order
    --reverse
  formatting output:
    --parents
    --children
    --objects | --objects-edge
    --unpacked
    --header | --pretty
    --abbrev=nr | --no-abbrev
    --abbrev-commit
    --left-right
  special purpose:
    --bisect
    --bisect-vars
    --bisect-all
Traceback (most recent call last):
  File "setup.py", line 45, in <module>
    version=versioneer.get_version(),
  File "/lcls2/psana/versioneer.py", line 1480, in get_version
    return get_versions()["version"]
  File "/lcls2/psana/versioneer.py", line 1453, in get_versions
    pieces = from_vcs_f(cfg.tag_prefix, root, verbose)
  File "/lcls2/psana/versioneer.py", line 1110, in git_pieces_from_vcs
    pieces["distance"] = int(count_out)  # total number of commits
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Reproduction steps:
docker run -ti slaclcls/travis:centos6-py3.6-versioneer-bug187

From inside this container, run:
source activate myrel
git clone https://github.com/slac-lcls/lcls2.git
cd lcls2
./build_travis.sh -p install

I made the following patch:

-        pieces["distance"] = int(count_out)  # total number of commits
+        pieces["distance"] = 0
+        if count_out != None:
+          pieces["distance"] = int(count_out)  # total number of commits

Then, all is working (I'm not really interesting about distance between commits, but it works)


Detail page

Make a detail page or may be popup that display the complete info related to this item @Git21221

@zainbinshakoor sure go ahead, make a pr with proper rules #68

@Git21221 sure

#71


RegExpExecArray is missing a type definition for the groups property

TypeScript Version: I'm using the ts checker with javascript in VS Code so I'm not 100% sure on the version I'm using.

Search Terms:
RegExpExecArray
RegExpExecArray groups
exec groups
regex groups
Code
// GF116 [GeForce GTX 550 Ti] --> GeForce GTX 550 Ti
const regex = new RegExp(/.+\[(?<gpuname>.+)\]/);
regex.exec('GF116 [GeForce GTX 550 Ti]').groups.gpuname;

Expected behavior:
No error.
Actual behavior:
Property 'groups' does not exist on type 'RegExpExecArray'.ts(2339)
Related Issues:
https://github.com/Microsoft/TypeScript/issues/17094

Duplicate of #29465.

@MartinJohns looks like RegExpExecArray also needs the fix.
Trying the slackbliz from that issue with this code still shows this.
Property 'groups' does not exist on type 'RegExpExecArray'.
any

// GF116 [GeForce GTX 550 Ti] --> GeForce GTX 550 Ti
const regex = new RegExp(/.+\[(?<gpuname>.+)\]/);
regex.exec('GF116 [GeForce GTX 550 Ti]').groups.gpuname;

Trying the stackblitz from that issue with this code still shows this.

The issue explicitly says that the Stackblitz website is using outdated versions...
When you set your target to es2018, then your code works. https://github.com/Microsoft/TypeScript/issues/29465#issuecomment-455489648

Trying this on the playground again no longer works. I tried es2018, es2019, es2020 and esnext.
Version: v3.9.0-dev.20200407
Config:

const regex = new RegExp(/^.+: ID (?<id>\S+)(?<name>.*)$/);
const line = 'test string';
const groups = regex.exec(line)?.groups;

Hovering over ?.groups shows "Property 'groups' does not exist on type 'RegExpExecArray'.(2339)".

Same on 3.9.3 for me.

This is still an issue. Can it please be reopened?

Still an issue, target & lib => 2020

@RyanCavanaugh please reopen this.


failed to connect to server

2024-03-08 19:28:11 server-1  | 2024-03-08T11:28:11.395Z - error: [gme:server:auth:gmeauth] MongoError: failed to connect to server [mongo:27017] on first connect [Error: getaddrinfo ENOTFOUND mongo

It looks like this might be related to https://github.com/symbench/electric-circuits/issues/172#issuecomment-1985808963. Let me know if the problem persists after using a different MongoDB version.

Thank you very much! I have resolved this problem with re-pulling the image


[alpha9] It's giant lags after pressing "k"(open library)

Latest Log (if necessary):
debug.log
latest.log

That's interesting, going over the logs now. There shouldn't be a huge amount of data that's transferred, but there might be a bug somewhere.

The main error had to do with the rampant life infusion FX not being properly registered, which I've adjusted. We'll revisit this after the next alpha to see if that fixes it.

It seems that it doesn't cause lags anymore


Cancel drawing interaction with escape key.

The Esc/Escape key will cancel the drawing interaction.
Right click to remove the last vertex will executed conditions [array] methods. This allows to undo the last section from routes in the measure_distance plugin.

After finishing the interaction how do we get rid of the drawn geometry?


Move into root

OK, I applied a change that fixes the lambda as it gets deployed, but breaks the test run!. Will sort it...

OK, I applied a change that fixes the lambda as it gets deployed, but breaks the test run!. Will sort it...

It's a problem that isn't really because of your changes.
When using the tests you have to use ingestion.module, but if you zip up ingestion with terraform Lambda won't recognize it because the folder gets ignored during the zip (you can't force it to be included... I checked).
I think your approach of using a local src & test is decent, and means that you can zip up the src folder so the imports are the same for Lambda and for local.
Might do some testing later to try and see the best way to approach.


message edit: Can horizontally resize past the edge of the page

If you try to use the resize icon in the bottom right corner of the textarea, weird things can happen:

Since that resize icon is basically nonfunctional besides that, I think it would be be best just to get rid of resize. Alternatively, we could enable vertical resize to make it kind of useful, and then lock the horizontal resizing.

Thanks @cory2067! Let's enable vertical resize and lock horizontal resize, rather than getting rid of it.

@zulipbot claim


Polyway to Test Components - Using polymer 0.16.X and test 0.12.x?

Hia
Have following along with the changes and [discussion] (https://github.com/dart-lang/test/issues/91)  in test packages to support polymer.
I get the <link rel="x-dart-test" href="unit_name_test.dart"> and understand that changes in polymer.init.dart.
But I really need some help.
My previous approach was to include the unit test in for polymer transformation. In the test itself I would init polymer and onReady', create the with new Elementwith a nullSanitizer and thenappend(_element)` this all worked well with async/await..  Then I could query and find the element, check it's shadow dom etc.
With the changes to test, I am really very unclear what is the best way to structure these unit test.
So could you help with maybe an example of our new poly'way'. I can see that you have got it working from 0.16.2 . But it just not coming together for me...fog is not lifting. Any help would be appreciated...
In my ideal world I could:

Not have to register the polymer test in polymer section of the pubspec.yaml.
I could create a file in my test directory call foo_bar_test.dart, hopefully avoid having a custom HTML unless there was JS script dependences I needed to access.
In the unit test foo_bar_test.dart I would just do something like:

import 'dart:html';
import 'package:test/test.dart';
export 'package:polymer/init.dart'

main(){

Element _testEl = createElement(<my-foo id="good_foo"></my-foo>);

test('foo should do a little dance, well at least appear on the page.',() async{
    document.body.append(_testEl);
    Element testSubject = await document.querySelector('#good_foo');
    expect( testSubject.shadowRoot, isNotNull);
});

}

That again is ideal world... and the above would just run along side all other test when I pub run test -p content-shell. Stepping back to reality an annotation would be fine... @polyTest have using @initMethod annotation  above main(){ would not kill anyone..
I am just not sure of the way to proceed... I know I don't want to have anything in my setup that needs to await the zone to showup...
So a gist with a example that  we will call the poly way would really help me... Maybe that will lift my fog of war on this problem..
Thanks in advance.

I  guess you need a custom html file because otherwise there is no way to register an entry point in pubspec.yaml which is necessary. Instead of maiin just use @whenPolymerReady annotation and probably a different name than main (like init)

Yes, @zoechi is correct here. You need an html file because the polymer transformer starts from html not dart code, and you should use an @whenPolymerReady annotated method to define your tests.
If you need something more custom you can write a custom main like before, but you need to return a future which does not complete until all your test methods have been called.

@zoechi & @jakemac53 .. Thanks for the advice... it is probably just must be me not getting it...
I put it all down on a Gist. I hope you have a few moments to look at it and let me know where I am off track it would be great... I just can get it working.
I have tried to init poly from the HTML side using <script type="application/dart">export 'package:polymer/init.dart';</script>  but the two script tag thing throws the Script Compacting error.
Clearly if I get rid of
<link rel="x-dart-test" href="poly_foo_test.dart">
<script src="packages/test/dart.js"></script>

All is good with the polymer side but not the test side, test fails to load the test file. I have also combed all the doco out their on the Polymer Site and and everything about test. Without luck, so I am sure it is something really simple and not worth writing about...
Thanks again guys...

Looks fine except the test transformer is missing at the end of your pubspec.yaml (see test readme, I am just on my phone)

https://gist.github.com/jakemac53/b42e8e46d83c9e6474fc is a working example

Thanks guys.. it was that one transformer line...


Fix issues  for HelpText.AutoBuild configuration (issues #224 , # 259)

HelpText.AutoBuild can't apply setting on HelpText to add AdditionalNewLineAfterOption and other setting.
These setting are applied only when parser raise parsing errors.
This PR fix this bug and fix issue #224 , #259 .
You have a complete control on HelpText configuration in custom help.
Also, new extension methods IsHelp() /IsVersion() is added to simplify checking errors if it have help/version option.
        var parser = new Parser(x =>
        {
            x.HelpWriter = null;
        });
        var result = parser.ParseArguments<Simple_Options>(new[]{"--help"});
        //generate custom help
        result .WithNotParsed(errs =>
        {
            var helpText = HelpText.AutoBuild(result,
                h =>
                {
				    //configure help
                    h.AdditionalNewLineAfterOption = false;
                    return h;
                }
                , e => e);
             //print help screen
            Console.WriteLine(helpText);

        });

This PR also fix #414 and #455


Cannot delete token

I started adding service, but didn’t finish it. However, the token was still aded in the app. Because I didn’t finish it the first time, I had to add it again, and now I have two entries for the same service and it’s annoying trying to remember which is actually in use. Especially since I can’t seem to change icon for them either.

@andreaslif Can you swipe the token right to remove the one you do not want?

Ah, yes. That worked. I tried swiping from right to left before, since that's the typical iOS behaviour.
Maybe it would be a good idea to change that? I think it would also be good if swiping just revealed the delete button underneath and then you have to tap that button to confirm the deletion. Like iOS Mail/notes/reminders etc. work. What do you think?

Yes agreed, I'd like to improve the 'removal' functionality but haven't had time to do so. I am considering using https://github.com/SwipeCellKit/SwipeCellKit for it

I see. Seems like this would work also?
https://developer.apple.com/documentation/uikit/uiswipeactionsconfiguration

I see. Seems like this would work also?
https://developer.apple.com/documentation/uikit/uiswipeactionsconfiguration

This is a property of a table view, but our tokens view is using a collection view.


Shibboleth sessions are restored on next login after logging out

Describe the bug
Impossible to log in via username/password after logging out of a Shibboleth session

The _shibsession_* cookie remains even after logging in
Upon the next password login this cookie is used instead of the username/password, causing the previous Shibboleth user to get logged in again instead of the new password user.

To Reproduce
Steps to reproduce the behavior:

Log in via Shibboleth

You should see a _shibsession_* cookie in your browser's dev tools

Log out

The _shibsession_* cookie remains

Log in via password (as a different EPerson)

You will be logged in as the previous "Shibboleth EPerson" instead
It's impossible to log in as a "password user" until the session cookie is manually cleared

Expected behavior
Shibboleth sessions should not bleed over into the next "password session"
Preliminary investigation
Confirmed that this issue cannot be addressed from the frontend, as the _shibsession_* cookie is HttpOnly. We may be able to solve this by explicitly ignoring the cookie in REST if a login attempt is being made with username/password.
Related work
Discovered while working on https://github.com/DSpace/dspace-angular/pull/1805

@ybnd : I think another way to approach this would be to address #7799
I see we actually have an old PR to address this from @paulo-graca : #2758  But it looks to have been abandoned.  Maybe it's time to look at whether Single Logout for Shibboleth makes sense?

We ran into the same issue. If I understand correctly, this appears to be caused by the AuthenticationService attempting each AuthenticationMethod retrieved through getAuthenticationMethodStack. This AM Stack appears to be built through getPluginSequence, its docstring states that:
"The order of the plugins in the array is the same as their class"
I tried changing the order of the AuthenticationMethods in local.cfg (commenting out other occurances in authentication.cfg), hoping that PasswordAuthentication will be attempted before ShibAuthentication, so that it now looks like this:
plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.PasswordAuthentication
plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.ShibAuthentication
This appears to have solved the issue for us.


[WIP] Update to python 3

solves Statoil/everest#363

I am not familiar with the details of this, but to me it seems weird to focus on Python 3.4 instead of 3.6.

I agree that we will probably use Python 3.6, I think that the lower the better, and would actually propose compiling it against Python 3.3!  The reason is that since Python 3.4, 3.5, 3.6, and soon 3.7 are all backwards compatible, the lowest version we can build with, the more versions we can support.
In addition, Python 3.3 (or is it 3.5?) is the one that ships with RHEL SCL which we might have to support, but not in the near future.
However, I believe this is just an exercise from Elena's perspective.  If we manage to get it to compile with Python 3.4, I ass/u/me we will be able to compile it with 3.x for any x in [2,7].

My motivation here is libboost-python in my machine (ubuntu trusty) only reaches till version 1.54 and compiles to python 2.x and 3.4. In the latest version of ubuntu, libboost is supported til 1.6x
So far I haven't found the way to use by default the 3.4 version when linking. In travis is using /usr/lib/x86_64-linux-gnu/libboost_python-py27.so.1.54.0 when /usr/lib/x86_64-linux-gnu/libboost_python-py34.so.1.54.0 also exists in the same directory (at least on mine it does)
I've been browsing around and haven't found the solution yet.

In addition, Python 3.3 (or is it 3.5?) is the one that ships with RHEL SCL which we might have to support, but not in the near future.

3.5

superseded by #20


StreamDeck XL key press sequence lost in UI

Steps to repro:

Create a new Key Press button action
In the Key Sequences window create one or more sequences
reopen the Key Sequences window and observe no entries

The key press binding sill shows in the main window and pressing the button on the Stream Deck still fires the key press as expected
[dcsfp_profile.zip](https://github.com/DCSFlightpanels
https://user-images.githubusercontent.com/161126/153693206-b80b7850-2986-4ac1-bc0a-4bbaf4276f35.mp4
/DCSFlightpanels/files/8052553/dcsfp_profile.zip)

That single key press is not a sequence. If you add a minimum of 2 key presses you will see that they are shown. It is the same for the Saitek panels, if you add only one key press in the key sequence window it will be treated as a single key press, not a sequence.

Well if that is intended then fine but it is non-inuitive.  I expected to see it in there and be able to see/modify the settings on that one key.


Add an option to enable/disable lifts when loading osu songs

The osu song loader adds a lift to the end of each hold. I think by default this should be OFF and the user has the option to turn it ON. The purpose of converting should be to play the song as you would in Etterna.

done 315cca5b071d58370354a982da6cf5b5a25360d9 4bfada032c155f514e8fd5e14c82dcd20b3221e6
added as an option to debug menu F8 as well 339b91b3b30403e1ca35d9a192e23e636d90f6fc


[io] is_specfile PY3 bug when reading binary data

This solves the immediate issue (#1416).
But the current test to find whether a file is a specfile is not very well suited for investigating large binary files.
The issue is that we assume the data is text, and we loop for line in f: to test for a #f or #s in the first 10 lines. This means that we might end up reading an entire data file to find out that it is not a Specfile, if there are not at least 10 \n characters in the file.

There is no perfect solution. I don't think that we can guarantee that the first  character (after stripping whitespaces) will always be a #. I also don't think we can assume that a specfile must only contain ascii characters, if we want to support files generated by third party scripts.

There should not be any stripping of white spaces at the beginning of the lines. We support specfile and not something resembling it.
Concerning the problem. What about an fread of a maximum of 2500 bytes maximum and apply the 10 lines rule to it? For instance, one should only have to look for "\n#F" ot "\n#S" in that maximum of 2500 bytes.

It sounds reasonable

Done.


chore(fp-ts): refactor fetchImgixDominantColor return type

This is part of the Gatsby fp-ts refactor.

Current dependencies on/for this PR:

next

PR #190

This comment was auto-generated by Graphite.


UEFI support for KVM/vCenter

Hi,
Do opennebula have UEFI support?
We tried to change bootloader and custom templates but no luck.
I have tried to boot iso with uefi using virt-install and its working perfectly.
Please suggest the workaround , because we didn't find any documentation regarding uefi on your docs.

Given the fact that some VM now require UEFI to boot, I would say it can be an important addition. OVMF is widely available on nearly all linux distributions, and on the ONE side it would require just the addition of a "BIOS" section.

A simple thing is to do:
    <os><loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader></os>

But this can have issues due to the default nvram location being /var/lib/libvirt/qemu/nvram which is not accessible by opennebula user oneadmin by default. It would be better to store the nvram with the disk images in the datastore directory as with the context disk. This can be done with the <nvram> option.

Any progress on adding UEFI support? Or otherwise a work-around?

I've just open an issue on libvirt project: https://gitlab.com/libvirt/libvirt/-/issues/159
With my modification, i just need to add:
  DATA = "
  <os firmware='efi'>
    <type>hvm</type>
    <loader secure='no'/>
  </os>
",
  TYPE = "KVM" ]```

to template configuration and VM starts with EFI support.

There's a workaround for this that doesn't require waiting on upstream changes to libvirtd.  Mount /var/lib/libvirt/qemu/nvram with a filesystem that supports setting uid=oneadmin.  libvirtd doesn't know the difference & UEFI VMs, including migrations, work fine.
There are multiple filesystems that support uid= (see man mount).  Below is how I do it, using a loopback device, autofs, & vfat.   If you don't want to use autofs, /etc/fstab works.   Loopback devices aren't required, either.  Any device that can be formatted & mounted with uid=oneadmin will work.
cd /var/lib/libvirt/qemu
dd if=/dev/zero of=/var/lib/libvirt/qemu/nvram.img bs=100M count=2 # 200M
chown root. /var/lib/libvirt/qemu/nvram.img && chmod 0600 /var/lib/libvirt/qemu/nvram.img # only root needs to rw the image
losetup -fP /var/lib/libvirt/qemu/nvram.img && losetup -a # add & print loopback devices
mkfs.vfat /dev/loop0 # format image with vfat
losetup -d /dev/loop0 # remove loopback device

# install autofs; setup direct automount map
chown oneadmin. /var/lib/libvirt/qemu/nvram # autofs will change the mode via umask
echo "/- /etc/auto.qemu-nvram" > /etc/auto.master.d/nvram.autofs
echo "/var/lib/libvirt/qemu/nvram -fstype=vfat,uid=oneadmin,gid=oneadmin,umask=0077 :/var/lib/libvirt/qemu/nvram.img" > /etc/auto.qemu-nvram
systemctl restart autofs

touch /var/lib/libvirt/qemu/nvram/test && ls -l /var/lib/libvirt/qemu/nvram && rm /var/lib/libvirt/qemu/nvram/test

@tinova PRs to be approved:

Code
Docs

Having some experience with UEFI boot with OIpenNebula and after looking at the code I would like to understand some of the insights backing the current implementation?

There is no option/definition of the secure attribute set for the loader element at https://github.com/OpenNebula/one/commit/121b4b096ee36d3e1756b868a79e7d03dfb8b13d#diff-4f32f4a94d596188452be83043bf54091d4ded4e0d9e4b8fde332e124a3fa9fbR807 . It is an assumption that all distributions had qemu-kvm builds with secure=on enabled by default?

There is no option to define the SMM feature (SMM is needed for the Secure Boot). So it is assumed that it is active and enabled by default?

Also, the NVRAM is a container for variables stored by the VM OS. For example, it is used as a secure key-value store by Windows (enterprise) software like boot order, programs activation, licenses, etc. That said (IMHO) the backing entity for the NVRAM container should be handled by the storage drivers because it could be a plain file or a block device and considered as (sort of) an additional "disk" that must be included in the entire VM lifecycle, including backups, snapshots, etc.

What do you think?

* Also, the NVRAM is a container for variables stored by the VM OS. For example, it is used as a secure key-value store by Windows (enterprise) software like boot order, programs activation, licenses, etc. That said (IMHO) the backing entity for the NVRAM container should be handled by the storage drivers because it could be a plain file or a block device and considered as (sort of) an additional "disk" that must be included in the entire VM lifecycle, including backups, snapshots, etc.

I  agree, I too would like to see this handled by the storage drivers.

About storage, it is good idea to be handled by storage probably we can reuse the volatile disk or files & kernels datastore.
This will be implemented as a separated issue because of the implications and side effects on storage drivers.
In the meantime, note that this file is copied into the VM directory so it is consider during the VM life-cycle (i.e. migrations, undeploy/resume operations...).

In the meantime, note that this file is copied into the VM directory so it is consider during the VM life-cycle (i.e. migrations, undeploy/resume operations...).

Yes, this is covering the case when there is a file but it working with block devices too and in that case a there is no file copy (there is dd to the block device instead)... If the System TM_MAD is used, then the storage driver will take care to prepare the NVRAM file - File or symlink to a block device. Also, with the current implementation, it is not possible to backup the VM because there is no handling of the NVRAM files the backup extension implemented in the ruby bindings :)
It is not a strictly critical issue because our current implementation could continue to be used, but I prefer such tweaks to be as streamlined as possible. In our implementation, the NVRAM file is prepared in the contextualization script and the domain XML is tweaked with a deploy-tweak. Migrate, undeploy, etc cases are handled in the relevant TM_MAD scripts too. The setup works with all recent opennebula versions currently in production, btw :)

Yes but note that for 6.2 it is always a file as it is copied from a file. Also note it is same case as with the checkpoint file.
I understand the idea, just wanted to point out, that any change on the NVRAM will be preserved if you migrate or undeploy the VM, for the current implementation.

Yes, noted. It is indeed the same case with a ready resolution but not accepted for a long time...  :-/

Having some experience with UEFI boot with OIpenNebula and after looking at the code I would like to understand some of the insights backing the current implementation?
There is no option/definition of the secure attribute set for the loader element at 121b4b0#diff-4f32f4a94d596188452be83043bf54091d4ded4e0d9e4b8fde332e124a3fa9fbR807 . It is an assumption that all distributions had qemu-kvm builds with secure=on enabled by default?
There is no option to define the SMM feature (SMM is needed for the Secure Boot). So it is assumed that it is active and enabled by default?

Thanks for the heads up. We forgot to include the secure tag and enable SMM (both required for secure boot to work). So it was only working for UEFI no secure. This commit solves the problem.


Add footer in info component

Add footer in info component
Change Type (required)
Indicate the type of change your pull request is:

[ ] patch
[ ] minor
[ ] major

Does your PR have any documentation updates?

[ ] Updated docs
[ ] No Update needed
[ ] Unable to update docs

Mock tests are not properly configured for this branch hence had to rebase to bazel-6 jvm , so closing this


Master

Added support for Objective-C.
Updated documentation for Objective-C.

@bontoJR  Thanks for adding Objective-C support. I just have one request - could you rebase your commits into one and also improve commit message (use the imperative mood)?