v0.34.1 reproducible StorageClass bug: false positive error, misleading error message, and a suggestion

Description
(Migrated from https://github.com/aws/karpenter-provider-aws/issues/5757), per request of Karpenter Maintainer @jmdeal
Versions:

Chart: https://gallery.ecr.aws/karpenter/karpenter
Chart Version: v0.34.1 (Currently latest)
Kubernetes Version: 1.29.0 (probably not relevant)

Summary of Bug:

It's one of many storageclass edge case bugs
Since it's an edge case, I've put together detailed notes & IaC that should allow easy E2E reproducibility :), within < 2 hours, that's 1 hour to read this, ~20 min of hands on, and ~40 min of waiting for infra to spawn.
3 main issues:

False Positive Error message
Misleading Error message makes it sound like Karpenter is trying to edit a PVC or PV
Suggestion: While researching this, I've found multiple similar reports. I believe it's something of a whack a mole issue that keeps reappearing. (a fix gets implemented to fix 1 edge case, but that just results in the creation of a new edge case.) I theorize the reason the issue is so hard to permanently fix, is the approach used to deal with the problem. So further down I'll explain a suggestion for another approach that could be used either in addition to or instead of the current edge case riddled method of dealing with it.

Observed Behavior:

If you follow the reproducibility notes to have Terraform provision EKS with Karpenter, and deploy a manually provisioned a PVC & PV (edge case scenario of not using a StorageClass), then scale workload to trigger karpenter to scale up a node, something like the following will show in logs:

"failed updating volume limits for volume with unbound PVC, no storage class specified"
Full Log for Context:
{"level":"ERROR","time":"2024-02-28T03:39:03.801Z","logger":"controller.state.pod","message":"failed updating volume limits for volume with unbound PVC, no storage class specified","commit":"17d6c05","pod":"default/nfs-client-59d9f46ff6-dhhdx","pod":"nfs-client-59d9f46ff6-dhhdx","volume":"pvc","pvc":"nfs-client-pvc"}

It seems to be a False Positive Error message, because the scale up of a new node will happen correctly, completely ignoring the error like it was never a problem.
Misleading Error Message:

The above error message is based on the reproducible example code and at first glance this bug appears to mostly be a cosmetic issue, that's not that important to fix.
I'd like to share the short version of a real world story, to achieve 2 goals:

give some context of why fixing it matters
give context on why the error message should be changed.

A team of engineers was adopting/onboarding Karpenter for the first time, and between editing Karpenter and Terraform code an EKS security group got misconfigured, (you know that thing that can break EKS's networking in a way that's hard to figure out and debug, because it can break networking in a way where it's only half broken.) When that bug was encountered Karpenter wasn't working, because Kubernetes networking was broken, because an EKS security group was misconfigured, but we didn't know that at the time. What we saw was that Karpenter wasn't working, and we observed that Karpenter was giving an error message like the above. So the error message showing up when it shouldn't have lead us to misattribute the error, and waste 2 days messing with PVCs and reading up on Karpenter.
If we never saw the false positive error, we wouldn't have gone on a wild goose chase, and would have realized the real error sooner.
Myself and other engineers misinterpreted the way that "failed updating volume limits for volume with unbound PVC, no storage class specified" is worded as meaning:
Karpenter had experienced an error when it failed to update/edit a PVC or PV. (we thought it was trying to add volume limits to a PVC.)
That was very concerning, as I was like Karpenter has no business editing a PVC, read I can understand, but edit, no. That prompted me and another engineer to do a deep dive learning session to research / fully understand how karpenter works. I now realize after reviewing the source code that it was only reading the PVC/PV and not attempting to make any edits. I now understand that Karpenter was updating it's understanding of PVC scheduling constraints, but I only came to that conclusion after alot of research of docs, issue tickets, and source code.

Expected Behavior:

The fact that karpenter scales up a node, tells me things worked / there's no actual issue.
If there's no actual issue, then no error message should appear in logs, a warning message would be fine.
Ambiguous Error Messages that can be misinterpreted; specifically, "failed updating volume limits for volume with unbound PVC, no storage class specified" should be avoided in favor of clear easy to understand messages.
A better alternative would be something like: 'Error: karpenter has failed to detect CSI volume limit, associated with pvc "$NAME", this is used as a scheduling constraint. It's normally found by reading pvc, pv, sc, and csinode objects.'

Useful Background Context

It seems like https://github.com/aws/karpenter-provider-aws/issues/4574 might be related, the error message is a little different, and this has more details than that. I think fixing this will fix that.
After doing 3 days of deep dive research, I know understand that Karpenter works like this:

Karpenter is designed around the concept of layered constraints.
All pods follow the pattern of being constrained by configuration and constraints that are embedded in the EC2NodeClass & NodePool custom resources. For example: A NodePool will reference a EC2NodeClass, which might use a subnet tag selector, and if you describe the EC2NodeClass for it's status you'll might see it select 3 subnets/AZs that a Karpenter NodeClaim could be provisioned into.
Something I learned from doing a deep dive is that Stateful Pods mounting PVCs are given more than 1 addition layer of constraints.

One layer was obvious, imposed by AWS, and is specific to pods using PVCs backed by EBS CSI driver. EC2 instances can't do cross AZ mounts of EBS volumes, so Karpenter needs to know what AZ to provision a new node in. Karpenter dynamically determines the AZ constraint of such pods at runtime:

If the PV already exists, karpenter will check the PVs metadata to see what AZ to spin up the node in.
If the PV doesn't exist, karpenter will check the storage class, in the case of ebs.csi.aws.com provisioner, that often has a subset of allowed AZs. Karpenter's Docs give an example where the storage class further constrains stateful nodes to 2 AZs.

^-- This part was all review to me and made sense. I kept asking myself after reading like 20 issue tickets related to storage class edge cases, Why does karpenter care so much about ALL storage classes? (There are plenty of examples like EFS, NFS, Rook-Ceph, Linstor's-DRBD, etc. that can mount across AZs, so why care so much about detecting all storageclasses?)
That's when I learned PVCs and storage classes have a 2nd layer of constraint imposed by the CSI spec. "Some (most), storage providers may have a restriction on the number of volumes that can be used in a Node. ... To support volume limits in a CSI driver, the plugin must fill in max_volumes_per_node" So most storage classes have a CSI driver that imposes an additional scheduling constraint, and that's why karpenter cares about accurately detecting all storageclasses.

Proposal to Consider when implementing solution

Pointing out a problem (for context):

There's like 20+ issues related to storage class edge case bugs, they've long existed, and keep popping up.
From what I can tell there are more than a few cases like issue #3647 where someone would open an issue to report an edge case bug, that was then solved by improving kaprenter's ability to auto-detect storage classes. Thing is there's been a few times where it seems like fixing 1 edge case results in a new edge case that created a new bug (sig issue 664), that then resulted in new logic for non-existant StorageClasses, but then that fix also fixed one edge case while creating a new edge case. Then logic had to be added for all kinds of edge cases "", (storageclass like "manual", which could be listed in pvc, but a non-existant actual storage class), this was all iteratively figured out over time, and got complex enough to add some volume related unit tests.
The logic for handling all these edge cases is complicated enough that it's no surprize that there's bugs.

Suggestion of another way to handle the problem

Karpenter currently dynamically figures out the scheduling constraints of stateful workloads that mount PVCs, by reading pvc, pv, sc, and csinode objects.
Karpenter doesn't seem to have a solid strategy for dealing with edge cases like local storage class, other instances of using no-provisioner, and just manual PVC assignment without a storage class, like shown in the below bug reproduction steps.apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner

My suggestion is that rather than continueing to complicate the storage class auto detection logic even further. It might be worth considering a new approach that could be used to compliment the current auto detection methodology, (and it could be used to deal with the remaining edge cases vs try to perfect the auto detection further with more complexity.)
Instead of trying to dynamicaly auto detect every possible edge case, what if you just take an approach of surfacing a warning message in karpenter logs like "Error: karpenter detected an edge case when trying to schedule pvc xyz, please see linked doc, on how to configure edge case resolution"
Then that linked doc could give tips on how karpenter auto detection works, how to improve karpenter's ability to do auto detection, and offer an optional configmap that can be used to pass explicit instructions on how you'd like karpenter to handle this edge case. (like if it detects storage class local-storage, but it get's confused about unspecified volume limit of no-provisioner, the configmap can let the user specify a volume limit to use when dynamic detection of constraints fails / when that edge case is detected)
The reason I suggest an optional configmap, is I think it's not uncommon for users to use something like awscdk, or terraform and just say karpenter enabled give me explicitly pinned version of current latest, and not want to need to learn how to override the default helm values. An optional configmap would be easier to patch in some edge case config without having to customize default helm chart install, and thus result in a better UX.

Easy E2E Reproduction Steps (TF, YAML, notes, etc.)
1.) Download TF's EKS Karpenter Example (The git commit 19b79df, has little significance, it was latest as of Feb 29th, just listing it for reproducibility sake)
cd ~
git clone https://github.com/terraform-aws-modules/terraform-aws-eks.git
cd terraform-aws-eks
git checkout 19b79df
cd examples/karpenter

2.) edit line 172 of main.tf, which refers to the version of the karpenter helm chart
version             = "v0.34.1"
3.) deploy the EKS cluster, takes ~15min
terraform init
terraform apply

4.) Make sure karpenter's running.
(Note there's a bug in the Terraform example where the karpenter pods won't run, because karpenter namespace is configured to be backed by fargate notes, but karpenter pods don't tolerate that, we'll fix it with a quick patch.)
aws eks --region eu-west-1 update-kubeconfig --name ex-karpenter

tee toleration.patch << EOF
spec:
  template:
    spec:
      tolerations:
      - effect: NoSchedule
        key: eks.amazonaws.com/compute-type
        value: fargate
EOF

kubectl patch deploy karpenter -n=karpenter --patch-file toleration.patch
kubectl scale deploy karpenter -n=karpenter --replicas=1
kubectl get pods -A
kubectl get deploy karpenter -n=karpenter -o yaml | grep -i image
# image: public.ecr.aws/karpenter/controller:v0.34.1@sha256:94...

5.) Deploy nfs-server that has zero dependencies
tee nfs-server.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  name: nfs-server
spec:
  ports:
  - name: nfs
    port: 2049
  selector:
    app: nfs-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-server
spec:
  replicas: 1
  selector:
    matchLabels:
      role: nfs-server
  template:
    metadata:
      labels:
        role: nfs-server
    spec:
      nodeSelector:
        kubernetes.io/arch: amd64 #The image only works on x86_64, not arm64
      containers:
      - name: nfs-server
        image: k8s.gcr.io/volume-nfs:0.8
        resources:
          requests:
            memory: "1Gi"
            cpu: "1"
          limits:
            memory: "1Gi"
        ports:
        - name: nfs
          containerPort: 2049
        securityContext:
          privileged: true
        volumeMounts:
        - name: node-local-storage
          mountPath: /exports
      volumes:
      - name: node-local-storage
        hostPath:
          path: /data/nfs
          type: DirectoryOrCreate
---
EOF

kubectl apply -f nfs-server.yaml

6.) get nfs-server's pod IP, and store it into a variable
export NFS_POD_IP=$(kubectl get pod -l=role=nfs-server -o=jsonpath="{range .items[*]}{.status.podIP}")
echo $NFS_POD_IP

7.) After making sure NFS_POD_IP's value seems reasonable, deploy nfs-client
tee nfs-client.yaml << EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-client-pvc
  namespace: default
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
  storageClassName: ""
  volumeMode: Filesystem
  volumeName: nfs-client-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-client-pv
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 5Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: nfs-client-pvc
    namespace: default
  mountOptions:
  - nfsvers=4.2
  nfs:
    path: / #Note: kubelet (worker node agent) is what tries to resolve the server.
    server: $NFS_POD_IP #pod ip of nfs-server, (b/c kubelet can't resolve service of type cluster ip)
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client
spec:
  replicas: 1
  selector:
    matchLabels:
      name: nfs-client
  template:
    metadata:
      labels:
        name: nfs-client
    spec:
      containers:
      - image: busybox:latest
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "512Mi"
        command: #based on https://github.com/mr-pascal/medium-k8s-nfs/blob/master/nfs-busybox-deployment.yaml
        - sh
        - -c
        - 'while true; do date > /mnt/index.html; hostname >> /mnt/index.html; sleep $(($RANDOM % 5 + 5)); done'
        imagePullPolicy: IfNotPresent
        name: busybox
        volumeMounts:
        - name: pvc
          mountPath: "/mnt"
      volumes:
      - name: pvc
        persistentVolumeClaim:
          claimName: nfs-client-pvc
---
EOF

kubectl apply -f nfs-client.yaml

8.) scale the nfs client to trigger a scale up
kubectl scale deploy/nfs-client -n=default --replicas=10
#^-- some replicas will be pending but that's enough to trigger a scale up.

kubectl logs deploy/karpenter -n=karpenter
#^-- will show the error
#v-- Then these will show it's a false positive / ignoreable error because they come up
kubectl get pods
kubectl get nodes

kubectl exec -it deploy/nfs-client -n=default -- cat /mnt/index.html
# ^-- side note if you periodically rerun this, you'll see they're all accessing the same volume

Standard Boiler Plate:

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

/assign jmdeal

Thanks again for the detailed write-up and reproduction steps, this made reproducing and diagnosing the problem a breeze. In this case, this error being reported at all is certainly a bug given that the PVC is already bound to a PV. This was a miss on my part in my fix for non-empty storage classes. If the PVC is bound to a volume but we can't resolve the driver we should exit early instead of trying to resolve the driver via the storage class.
On your point about the ambiguous error message, I think you're right that it should be made more clear that what is failing to be updated is Karpenter's internally tracked volume limits, not the PV / PVC / CSINode. I'm thinking rewording as failed tracking volume limits for volume with unbound PVC, no storage class specified would be sufficient there.
I think your immediate problem is a relatively straightforward fix but you raise an interesting point with configuration via configmap. I think full discovery of CSI driver driven limits is reasonable without; with the exception of this spurious logging I believe we've hit the main edge case. It is something interesting to consider though if we expand our storage based discovery further.

Responding to: "I'm thinking rewording as failed tracking volume limits for volume with unbound PVC, no storage class specified would be sufficient there."
It'd be nice if it was more like: failed tracking volume limits for volume with unbound PVC, no storage class specified. (csi spec defined volume limits is a scheduling constraint).
I think most users aren't even aware that csi spec introduces a volume limit, because if a user reviews storageclass.yaml, pvc.yaml, pv.yaml, it's never or rarely listed. Without the extra explanation the error only makes sense to an expert like yourself, but won't make sense to those who lack a deep understanding, I think it'd be worth adding the extra explanation to help average user understand better.
Your logic sounds good, thanks for looking into it, and glad the reproducibility went smooth.

Forgot that the PR gets auto-merged when approved now that this project is in kubernetes-sigs, I think the log message surfaced in it is a good compromise. It specifically calls out that they are CSI Volume Limits and the first thing I find when googling that is the kubernetes docs you linked in your write-up. It feels like a good compromise by including enough information to make the message diagnosable without making it too verbose.


drivertest: add arm-m signal through isr case

Summary
cmocka case for call signal in exception direct handler.
Impact
case only,  fix cmake support for cmocka_driver_mps2_zerointerrupt
Testing
CI-test, local qemu mps2

[Experimental Bot, please feedback here]
Yes, this PR appears to meet the basic NuttX requirements, although it's missing some details.  While concise, it could be improved by providing more specifics in the Impact and Testing sections.  For example:

Impact: While it says "case only," it doesn't explicitly answer all the impact questions (e.g., Impact on user, build, hardware, documentation, security, compatibility).  Even if the answer is "NO" for most, stating that explicitly is helpful for reviewers. The "fix cmake support" comment should be elaborated on – what exactly was fixed and why?
Testing: "CI-test, local qemu mps2" is a good start, but more detail would be beneficial.  What specific CI tests were run? What configuration was used for qemu-mps2?  While the template asks for "Testing logs before change" and "Testing logs after change," providing the full logs is usually not necessary.  Instead, summarize the key differences or observed improvements. For example, "Before:  Assertion failed. After: Test passed."

While the summary is brief, it does address the core questions.  The provided information suggests that the changes are limited in scope (a new test case and a CMake fix), which simplifies the review process. However, providing explicit answers to all the questions, even if they are negative ("NO"), strengthens the PR and makes it easier for reviewers to quickly assess the changes.

@jasonbu please include documentation about drivertest, the entry is empty: https://nuttx.apache.org/docs/latest/applications/testing/drivertest/index.html
Also please remove the Vela specific reference, since it is integrated into mainline is doesn't depend on Vela.


Generic composition of two generic functions produces a non-generic composed function

This is a bug, please see it reproduced on flow.org/try.
A bit mind twisting, but it's quite simple. A naive parametric compose2 function type
type Compose2 = <A, B, C>(B => C, A => B) => (A => C);

produces a non-parametric function type. Once it gets called it breaks all the following calls, as they will conflict with all previous calls. Same would happen with an even more naive non-parametric compose function like this:
const compose = f => g => x => f(g(x));

Possible workarounds

Compose manually as a true Jedi 😌
Compose semi-manually using some ugly syntax like this:

const f_g = <X>(x: X): $Call<typeof f, X> => f(g(x))

Have a handful of differently named compose functions 😂

What did not help

Making more generic types
Having 100% flow coverage
Typing compose() with $Compose

Smaller example


Revert postgres version to 13

Bump from version 13 to 14 causes data incompatibility and needs to be very carefully considered.
Ref: https://github.com/inventree/InvenTree/pull/7699

As mentioned in https://github.com/inventree/InvenTree/pull/7699#issuecomment-2244855413 I think the quickest / cleanst solution to get a "safe" image out to latest is to just revert #7699 completly


⚠ Bofrost.es has degraded performance

In c5b85b6, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 2081 ms

Resolved: Bofrost.es performance has improved in c1b9df8 after 7 minutes.


Create to-do list

My aim is to start with very small steps so that everyone can contribute.
Features I would like to have in the structure Nuxt 3

[ ] Auth
[ ] Mobile App
[ ] Admin Panel
[ ] Web
[ ] Api
[ ] GraphQL
[ ] Image Upload
[ ] Stripe - Payment - Even more than that
[ ] Drizzle + PostgresQL
[x] Docs Website https://github.com/oku-ui/docs/issues/84
[ ] ... more

If you have anything you want, please let me know in the comments, maybe it will help me to move forward. I am open to all feedback.I am sure the contributions will increase as the structure slowly emerges. Especially follow pr.

Seeing Redis and Kafka would be nice.


Purge Marathon state before and after tests

High Level Description
Tests often flake-out due to a marathon deployment timing out or some other odd deployment condition. To remedy this, purge all marathon workloads before and running EVERY test.
Related Issues
https://jira.mesosphere.com/browse/DCOS-14108
dcos-test-utils PR: https://github.com/mesosphere/dcos-test-utils/pull/40
Checklist for all PR's

[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[ ] Read the DC/OS contributing guidelines
[ ] Followed relevant code rules Rules for Packages and Systemd

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

This needs to point back to master before going on a train, moved to holding for now to not hold up the train.

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@lingmann This is ready to go again. Not a silver bullet but definitely an improvement

Hi @jgehrcke for future PR's please make sure we have (2) +1 reviews before adding the shipit-label. I know that is implicit here based on your comments and interactions on this PR, but you'll notice that our new check mergebot/review/approved/min_2 did not go green due to this. Just a quick FYI. THANKS!


Clean lint bill of health, format source

As per #309, this runs make format with a little bit of manual help on some of the gnarliest code, and takes minimal steps to get make lint to exit 0.
I've also tried adding to the CI setup, we can discuss that further. Not certain if my initial configuration is going to work anyway, I haven't tried local Drone testing yet—to my understanding the Drone CLI has changes in the latest version, I'm not sure what version our CI infrastructure runs but I believe it isn't the latest yet so I don't know what CLI to install… Edit: Seems to work 😄
See also the "soft limit" line length idea on #309 and let me know what you think of that, haven't done it yet.

GitHub tricks reminder: diff omitting whitespace-only changes.

Rebased to resolve conflicts. Any further thoughts about this? What do we need to move forward?
cc @ChrisCoffey

The drone checks were the only thing that would have prevented me from saying this is good to go. I'm comfortable landing this & seeing how it is received over the next couple weeks.

The drone checks were the only thing that would have prevented me from saying this is good to go. I'm comfortable landing this & seeing how it is received over the next couple weeks.

Just curious, how do you think we could improve it?
I'll go ahead with this as-is then, will be good to have out of the way for other stuff in the works. We can of course tweak over time if anything about the build setup or specific linter settings proves onerous.


Wrong default value for pcap filename in samples

Sample applications have a composite parameter to specify the pcap file.
for instance:
expression $pcapFilename: getSubmissionTimeValue("pcapFilename", "../../SampleNetworkToolkitData/sample_dns+dhcp.pcap" );
The default values needs to be changed to the right path:
"../../SampleNetworkToolkitData/data/sample_dns+dhcp.pcap"
The files are located in the data of the SampleNetworkToolkitData project.
This would help using the samples in StreamsStudio.

I agree. I'll clean this up.

fixed in 3.0.0


[ffmpeg] Fix detection of x264.

Should prevent ffmpeg from silently failing on newer x264 versions.

Thanks!


backport to 1.13: Renewed certificates used in unit and integration tests.

Commit Message:
Some certificates used in unit and integration tests expired in middle of December 2020.
This PR contains renewed certs.
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]

/retest

We can work out a plan going forward somewhere else, approving this since we should unblock release branch tests.

We can work out a plan going forward somewhere else, approving this since we should unblock release branch tests.


it returl error like  "psql: FATAL:  Peer authentication failed for user "postgres""

hi,
i want to create django-cms-demo installation in my system
whem i give command like
make install
so it return error
make[1]: Leaving directory /home/keyur/django_cms_3.2/django-cms-demo-master' make database make[1]: Entering directory /home/keyur/django_cms_3.2/django-cms-demo-master'
psql -U postgres -c 'DROP DATABASE djangocms_demo_local;'
psql: FATAL:  Peer authentication failed for user "postgres"
make[1]: [database] Error 2 (ignored)
psql -U postgres -c 'CREATE DATABASE djangocms_demo_local;'
psql: FATAL:  Peer authentication failed for user "postgres"
make[1]: *** [database] Error 2
make[1]: Leaving directory `/home/keyur/django_cms_3.2/django-cms-demo-master'
make: *** [install] Error 2
after i try to create db manually in my local system but still it return error
pls help how can i solve this error

Hello @kartikdanidhariya the demo project runs currently on the unreleased 3.2.1 branch for testing. Your mentioned problem has been fixed on Friday, so doing another make install after pulling should do it.
If you wanna go save just change the following line: https://github.com/divio/django-cms-demo/blob/master/requirements.txt#L7
to
https://github.com/divio/django-cms/archive/master.zip

hi,
Thanks for reply, download latest zip from your repo and after when i give make install command so it return same error
so how can i solve

@kartikdanidhariya what postgres user do you have? The install script assumes you have the default "postgres" user enabled:
https://github.com/divio/django-cms-demo/blob/master/Makefile#L8
You can override using make install DBUSER=myCustomUser


Add a max metadata size to not error on some package pulls

Description
This hotfixes a potential issue where metadata exceeds the default maxmetadata size in oras.
Related Issue
Fixes #N/A
Type of change

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Other (security config, docs update, etc)

Checklist before merging

[ ] Test, docs, adr added or updated as needed
[X] Contributor Guide Steps followed

Closing as this was not the root cause of the issue.


updates to deps, testing, ci, msvs handling

Updates dependencies
Updates test handling

Replaced deprecated mocha-istanbul with nyc
Updates test 245 crypto_stream verify parameters bad param 1 number for newer Node.js versions
Still uses old behavior for older Node versions since some are still under maintenance LTS

Replaces old Travis CI config and badge with GitHub workflow and badge
Newer versions of Visual Studio use latest MSVC toolset build of libsodium binaries
(see C++ binary compatibility between Visual Studio versions)

The replacement of Travis had more to do with the original config failing on the new Travis site and GitHub workflows being free than any other reasons. The only thing I find the workflow CI being unable to test is older Visual Studio versions.
Also if you want to add libsodium 1.0.17 or 1.0.18 (still receiving updates) to paixaop/libsodium-bin but don't have the time to update the Windows install script, let me know and I can open another PR in this repo when I see libsodium-bin update

bumperino


requests.exceptions.MissingSchema: Invalid URL 'None': No scheme supplied. Perhaps you meant https://None?

This share link will expire in 24 hours. If you need a permanent link, visit: https://gradio.app/introducing-hosted (NEW!)
Interface loading below...
ERROR - ViLT - Failed after 0:01:00!
Traceback (most recent calls WITHOUT Sacred internals):
File "C:\Users\qr\Downloads\ViLT-master\demo.py", line 233, in main
interface.launch(debug=True)
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\gradio\interface.py", line 463, in launch
while not networking.url_ok(share_url):
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\gradio\networking.py", line 462, in url_ok
r = requests.head(url, timeout=3)
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\requests\api.py", line 100, in head
return request("head", url, **kwargs)
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\requests\sessions.py", line 575, in request
prep = self.prepare_request(req)
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\requests\sessions.py", line 486, in prepare_request
p.prepare(
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\requests\models.py", line 368, in prepare
self.prepare_url(url, params)
File "C:\Users\qr\miniconda3\envs\label\lib\site-packages\requests\models.py", line 439, in prepare_url
raise MissingSchema(
requests.exceptions.MissingSchema: Invalid URL 'None': No scheme supplied. Perhaps you meant https://None?

I have the same problem, please have you solved it

I have the same problem, please have you solved it
No, I 've change my research topic


Change "npm install" to "npm ci"

Hopefully this will lessen the flakiness in CI.

Would be good to do this sooner rather than later because this seems to cause a fair bit of CI pain.

#1727

Believe this is fixed by the merged PR #1727


refactor: update types for manual integrations

What is the purpose of this pull request?

What problem is this solving?

Types of changes

[ ] Feat: (new functionality)
[ ] Bug fix: ([#ref number] non-breaking change which fixes an issue)
[ ] Chore: (improvements that will not reflect in production behaviour)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Requires change to documentation, which has been updated accordingly.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#175
#174  👈
#173
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @keeganpotgieter and the rest of your teammates on  Graphite

Merge activity

Oct 16, 6:16 PM EDT: A user started a stack merge that includes this pull request via Graphite.


Create a new music track

For the presentation at code.talks in October, I want a new track:

Working title “Autumn in Hamburg”
Tempo 80 bpm
Pachelbel chord progression
At least 16 channels

“Autumn in Hamburg” is the track used in the second Codetalks 2019 demo


Refactor #[derive(GraphQLEnum)] macro

Solution
Refactor #[derive(GraphQLEnum)] macro to be more similar to the other proc macros.
Checklist

Created PR:

[x] In draft mode
[x] Name contains Draft:  prefix
[x] Name contains issue reference
[x] Has assignee

[x] Documentation is updated (if required)
[x] Tests are updated (if required)
[x] Changes conform code style
[x] CHANGELOG entry is added (if required)
[ ] FCM (final commit message) is posted

[ ] and approved

[ ] Review is completed and changes are approved
Before merge:

[ ] Milestone is set
[ ] PR's name and description are correct and up-to-date
[ ] Draft:  prefix is removed
[ ] All temporary labels are removed

FCM
Refactor #[derive(GraphQLEnum)] macro (#1047)


React 16 : componentDidUpdate Warning: Scheduled a cascading update

I profiled the performance of my application using react redux by following this article by Ben Schwarz.
In the user timing section, i get these warnings (with a no entry sign):

There is two messages:

(Committing Changes) Warning: Lifecycle hook scheduled a cascading update
Connect(MyComponent).componentDidUpdate Warning: Scheduled a cascading update

I made some search but i found nothing special. It seems related to the componentDidUpdate function of the connect HOC of react-redux.
What does these messages means ?

It's likely a side effect of how we do subscriptions. But it's nothing to worry about.

@gontard Did you get any useful info on these warnings ? I got the same

@Iliyass no. I open this issue, ask a question on stackoverflow and even chat about this in the #react-internals chan of reactiflux on discord. But i did not get more details.
Since @timdorr said to not worry about this so I did not look further
I think it might a new warning of React 16 and in this case a false positive.

Eh. Really unfortunate that this approach causes people to learn to ignore an otherwise valid and useful warning. Hopefully we can figure out a solution with the new context API soon. :-)

@gaearon , @acdlite : can either of you give some additional info on what the warning actually means, both in terms of the cause and resulting behavior?

That is picture from chrome dev tools, react profiling. Seems to this issue can significantly increase render time, just look at the gap between two render cycles. Can we fix it somehow?

Okay, after some researching, I fix it in fork, https://github.com/Amareis/react-redux/commit/1303cb5f18d52b5b9509af1420fefa45f6538b94 there is commit. May be i broke something, i dunno, but at least it works as expected.

A PR would be awesome.

On the next week I'll improve my solution and will make some PR.

can either of you give some additional info on what the warning actually means, both in terms of the cause and resulting behavior?

It means what it says: you scheduled a cascading update. "Cascading" means an update inside an update. React first reconciles the tree by calling render, then commits DOM changes, then calls lifecycle hooks.  This is wasteful.

@Amareis : it would also be helpful if you could put together an example project that demonstrates the behavior, and some benchmarks that compare the behavior before and after, and in React 15 vs React 16.

@Amareis Keep in mind that the cascading updates was a solution to the problem with connected child components getting newest state before they got new props from their connected parent components. I believe the test for that is should pass state consistently to mapState.

Ok, I look into this.

I start my investigation. Seems to this issue don't appears on simple samples, i.e. https://github.com/notrab/create-react-app-redux, need to something more complicated.

Okay, in my case it behavior was caused by little quick'n'dirty hack - pseudo-selector, which returned data from local storage without any memoization, so objects weren't equals by ===. Without that hack all works as expected and no cascading updates are called.
But I have a simple question. How we can prevent this errors? I bet that @gontard have similar issue it his code, but how he can track this issue without reducing code in search of bad string? For example, why-did-you-update don't complains about this issue (because of new render scheme in react 16, may be?..)

@Amareis Thanks for your investigation. I thought that it was an issue in react-redux but if it doesn't happen in simple case it might be a problem on my side.
 This is wasteful.

Does that means that there is an issue in the connect HOC of react-redux ?  (Connect(MyComponent).componentDidUpdate Warning: Scheduled a cascading update)

@gontard it's pretty easy. Something in your connect(SelectionRectangleContext) props is refreshed too often. Check mapPropsToState of this component, some selector must be memoized to avoid this warning. I don't show my code because it's don't very relate to issue.

@Amareis I did not found a similar problem with my selectors.
But it seems to be an issue with nested connected component. When i have only a parent connected component, i did not get the warning. But when i add a child connected  component, the warning appears.
It confirms what @jimbolla said:

Keep in mind that the cascading updates was a solution to the problem with connected child components getting newest state before they got new props from their connected parent components

@jimbolla @gaearon So if I understand this thread correctly, and this warnings pops up when you have nested connected components (that's the case in my code) - is there any way to "fix" these warnings and the possible performance hit of the cascading udpates?

I had the same issue and I discovered that one of my selectors was always returning a new instance. When I memoized my selector this warning disappeared from the profiler.

I'm receiving a similar warning but with componentDidMount instead of componentDidUpdate. Could this be related? Also I'm using redux-form.

If I get a lifecycle hook scheduled a cascading update in the performance timeline in chrome:
1.) Is there any way to find out more details about which lifecycle hook in which component actually scheduled this update?
2.) Is there an easy way to find out if I did something wrong or if this is an expected case from react-redux?

@mreishus : per the rest of the discussion in this thread, the warning appears to be due to how React-Redux v5 uses componentDidUpdate to trigger nested subscriptions.
That behavior will be changed in v6.

I found a case in my application where this error was my fault, and not react-redux's.  I had a componentDidUpdate that dispatched actions sometimes.  I assume that would still trigger this warning even after v6 of react-redux.  But it's difficult for me to look in the timeline and determine exactly what is happening when I see the cascading update error message.  I suppose this is outside of the scope of react-redux and I'll ask for help in react.  I

Hi guys, sorry for bringing this issue up.
I'd like to ask @gontard if you were able to come up with a minimal app to reproduce this issue because I'm pretty much trying to achieve the same.
Also, I'd like to ask @gaearon if it was fixed on react-redux side?
Because as I understood from this comment of yours it could be related to react-redux, and I want to clarify if this is still "could" be a react-redux issue or this is my fault (which I think actually is)

@eko24ive : Dan doesn't actively maintain Redux or React-Redux any more.
We just released React-Redux 6.0, which has a different internal implementation.  Please upgrade to that, as I would expect that it resolves this issue.

Hi @markerikson and thank you for the quick update.
I'll try to upgrade to 6.0 and provide an update of my results to this issue.


ENH: Use DCT and DST from pocketfft

Reference issue
Addresses part of #10488
What does this implement/fix?
This adds all variants of discrete (co)sine transform to _pocketfft:
dct, idct, dst, idst, dctn, idctn, dstn, idstn
Additional information
The tests are currently failing and the issues seem genuine: the failing tests are against textbook definitions and FFTW generated data.
cc: @mreineck

The compilation failure on Windows seems due to me not explicitly capturing cdim for a few lambdas. I removed these captures because clang told me they were not needed, and g++ didn't complain. Is this a known MSVC quirk?

I do have a vague memory of running into similar issues with lambdas capturing constexpr values. This also suggests that there's at least a similar bug in MSVC:
https://developercommunity.visualstudio.com/content/problem/367326/problems-with-capturing-constexpr-in-lambda.html

Guess I'll put them back in then ...

Hopefully fixed upstream!

Do you see actual failures in the FFT routines? Is there a way I can selectively run the FFT tests on a local checkout of your branch?

Is there a way I can selectively run the FFT tests on a local checkout of your branch?

From the scipy root directory (not scipy/scipy) run
python3 runtests.py -j4 -sfft

This will just run the scipy.fft test suite. To narrow it down further you can just run the real tests using -sfft._pocketfft.tests.test_real_transforms

Looking at the first test that fails for me:
self = <scipy.fft._pocketfft.tests.test_real_transforms.TestDCTIDouble object at 0x7f63575df320>

    def test_definition_ortho(self):
        # Test orthornomal mode.
        for i in range(len(X)):
            x = np.array(X[i], dtype=self.rdt)
            dt = np.result_type(np.float32, self.rdt)
            y = dct(x, norm='ortho', type=1)
            y2 = naive_dct1(x, norm='ortho')
            assert_equal(y.dtype, dt)
>           assert_array_almost_equal(y / np.max(y), y2 / np.max(y), decimal=self.dec)
E           AssertionError:
E           Arrays are not almost equal to 10 decimals
E
E           Mismatch: 100%
E           Max absolute difference: 0.2636039
E           Max relative difference: 1.
E            x: array([ 1.0000000000e+00, -4.0863458189e-01, -1.1102230246e-18,
E                  -4.8518399963e-02, -3.5527136788e-17, -2.0000000000e-02,
E                   0.0000000000e+00, -1.2596161837e-02,  1.1102230246e-18,
E                  -1.0250856309e-02,  0.0000000000e+00])
E            y: array([ 0.7363961031, -0.4500559381,  0.0414213562, -0.0899397562,
E                   0.0414213562, -0.0614213562,  0.0414213562, -0.0540175181,
E                   0.0414213562, -0.0516722125,  0.0292893219])

dt         = dtype('float64')
i          = 0
self       = <scipy.fft._pocketfft.tests.test_real_transforms.TestDCTIDouble object at 0x7f63575df320>
x          = array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10.])
y          = array([ 2.23606798e+01, -9.13734703e+00, -2.48253415e-17, -1.08490440e+00,
       -7.94410929e-16, -4.47213595e-01,  0.00000000e+00, -2.81658741e-01,
        2.48253415e-17, -2.29216115e-01,  0.00000000e+00])
y2         = array([ 16.46631745, -10.06355671,   0.92620968,  -2.01111409,
         0.92620968,  -1.37342328,   0.92620968,  -1.20786842,
         0.92620968,  -1.1554258 ,   0.65492915])

If I run the same transforms interactively with pypocketfft and scipy fftpack (scipy 1.3), I get identical results:
martin@debian:~/codes$ ipython3
Python 3.7.3 (default, Apr  3 2019, 05:39:12)
Type "copyright", "credits" or "license" for more information.

IPython 5.8.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import pypocketfft as ppf; import numpy as np; import scipy; import scipy.fftpack

In [2]: x= np.array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10.])

In [3]: ppf.dct(x, inorm=1, type=1)
Out[3]:
array([ 2.23606798e+01, -9.13734703e+00, -2.29659899e-17, -1.08490440e+00,
       -7.35466158e-16, -4.47213595e-01, -5.89447706e-17, -2.81658741e-01,
        2.29659899e-17, -2.29216115e-01,  0.00000000e+00])

In [4]: scipy.fftpack.dct(x, type=1)/np.sqrt(2*10)
Out[4]:
array([ 2.23606798e+01, -9.13734703e+00,  0.00000000e+00, -1.08490440e+00,
       -7.44760246e-17, -4.47213595e-01,  7.44760246e-17, -2.81658741e-01,
        0.00000000e+00, -2.29216115e-01,  0.00000000e+00])

I don't really understand the error log however: there are two different values for the array x, for example.

Nevermind, I think I understand the x confusion now...
Still, pypocketfft returns the same array as the classic scipy.fftpack, so why does the test expect sometning different?

Maybe the normalisation factors are different? What do you get if you normalize fftpack using norm="ortho"?

You are right. With 'ortho' I get the array starting with 16....
Is there some convention I have overlooked? The overall normalization factor for this transform should be 2*(11-1), and ortho should apply its square root, right?

Looking at the fftpack wrapper, it normalizes some element differently (first and sometimes last):
https://github.com/scipy/scipy/blob/2f34eaf5d62a8fa40a84c287d2ea4cbc5b3d7e3b/scipy/fftpack/src/dct.c.src#L87-L100

This completely kills the symmetry that ortho is supposed to provide. Do we really want this?

From wikipedia:

Some authors further multiply the x[0] and x[N-1] terms by √2, and correspondingly multiply the X[0] and X[N-1] terms by 1/√2. This makes the DCT-I matrix orthogonal, if one further multiplies by an overall scale factor of √(2/(N − 1))

So, it sounds like this is a necessary condition for orthogonality.

Unfortunately, it also sounds like this would need to be handled by the 1d cosine transform within the pocketfft_hdronly.h code, rather than only touching the pypocketfft interface.

That's really unfortunate, because it means we can no longer use the fct parameter for normalization purposes.
Should we just add a bool ortho flag to the routines, or do you have another idea?

I can't see any way other than adding a bool ortho parameter to general_dcst. Having an overall fct as well should still be okay though, I guess it depends on if you expect any other normalization factors to be useful.

Yes, I think it's good to keep fct. I'll try to implement the changes at some point tomorrow.

I just added experimental support. Went quicker than I expected :)

The benchmark results look good.
The two implementations have very similar performance scaling with fftpack being faster for some values and pocketfft faster for others. As expected, pocketfft is much faster for multi-dimesional transforms.

"smooth-sized" (i.e. length is a product of small primes) type I transforms are slower by a factor of roughly 2.

The DCT-I seems to be the only transform that is consistently slower in pocketfft but if it's that or poor accuracy then I think it's a worthwhile trade off.

Full benchmark results
· Discovering benchmarks
· Running 4 total benchmarks (1 commits * 1 environments * 4 benchmarks)
[  0.00%] ·· Benchmarking existing-py_usr_bin_py
[ 12.50%] ··· Running (fft_basic.RealTransforms1D.time_dct--)..
[ 37.50%] ··· Running (fft_basic.RealTransformsND.time_dctn--)..
[ 62.50%] ··· fft_basic.RealTransforms1D.time_dct                             ok
[ 62.50%] ··· ====== ====== =============== ============
              --                       module
              ------------- ----------------------------
               size   type   scipy.fftpack   scipy.fft
              ====== ====== =============== ============
                75     I       8.32±0.5μs    6.40±0.2μs
                75     II       8.87±1μs     5.51±0.6μs
                75    III       7.50±1μs     5.21±0.3μs
                75     IV      6.96±0.2μs    5.45±0.2μs
               100     I       7.56±0.2μs    6.40±0.4μs
               100     II      6.64±0.4μs    5.73±0.4μs
               100    III      7.06±0.8μs    5.12±0.1μs
               100     IV      7.21±0.3μs    5.07±0.2μs
               135     I       12.1±0.7μs    10.5±0.7μs
               135     II      7.30±0.3μs    6.03±0.3μs
               135    III      7.64±0.4μs    5.93±0.5μs
               135     IV      7.66±0.8μs    7.36±0.2μs
               256     I       10.7±0.9μs    11.2±0.7μs
               256     II       9.02±2μs     6.66±0.4μs
               256    III      7.93±0.6μs    5.99±0.3μs
               256     IV      8.03±0.4μs    6.85±0.6μs
               313     I        9.41±1μs     9.62±0.4μs
               313     II       82.7±4μs     19.0±0.5μs
               313    III       81.3±3μs      19.5±1μs
               313     IV       78.0±3μs      21.3±4μs
               512     I        19.7±1μs      24.6±1μs
               512     II      12.2±0.8μs    8.31±0.7μs
               512    III       9.65±1μs     10.3±0.8μs
               512     IV       12.0±1μs      9.46±2μs
               675     I        117±8μs       105±6μs
               675     II      10.5±0.7μs    10.1±0.4μs
               675    III      11.4±0.8μs    10.2±0.5μs
               675     IV      12.2±0.7μs    11.4±0.7μs
               1024    I        18.3±1μs      28.8±1μs
               1024    II      13.0±0.7μs    11.3±0.5μs
               1024   III      12.3±0.7μs    10.9±0.8μs
               1024    IV       13.8±1μs     11.3±0.2μs
               2025    I        29.5±4μs      48.0±2μs
               2025    II       23.9±1μs     20.7±0.7μs
               2025   III       23.5±1μs      20.4±1μs
               2025    IV       25.5±1μs      22.3±1μs
               2048    I        55.4±6μs      96.4±3μs
               2048    II       20.7±1μs      19.6±1μs
               2048   III      19.5±0.4μs     20.3±1μs
               2048    IV      24.4±0.8μs     20.4±1μs
              ====== ====== =============== ============

[ 75.00%] ··· fft_basic.RealTransforms1D.time_dst                             ok
[ 75.00%] ··· ====== ====== =============== ============
              --                       module
              ------------- ----------------------------
               size   type   scipy.fftpack   scipy.fft
              ====== ====== =============== ============
                75     I       7.40±0.1μs    6.19±0.4μs
                75     II      6.71±0.2μs    5.10±0.2μs
                75    III      7.61±0.8μs    4.97±0.1μs
                75     IV       7.24±1μs     6.34±0.5μs
               100     I        17.1±2μs     11.4±0.3μs
               100     II       8.37±1μs     6.42±0.6μs
               100    III      8.00±0.5μs    6.10±0.8μs
               100     IV       8.13±1μs     5.58±0.7μs
               135     I       10.4±0.5μs    7.24±0.9μs
               135     II      8.28±0.6μs    6.82±0.4μs
               135    III      7.93±0.6μs    6.04±0.6μs
               135     IV      9.30±0.6μs    6.82±0.5μs
               256     I        61.4±8μs      46.6±4μs
               256     II       8.98±1μs      7.66±1μs
               256    III       9.01±1μs     6.63±0.4μs
               256     IV      8.33±0.4μs    9.34±0.9μs
               313     I        32.1±1μs      28.9±1μs
               313     II       79.8±3μs     19.3±0.6μs
               313    III       82.4±3μs     20.2±0.7μs
               313     IV       87.1±3μs      21.5±2μs
               512     I       13.9±0.7μs    15.0±0.6μs
               512     II       11.6±2μs     9.37±0.9μs
               512    III       10.0±1μs     8.25±0.5μs
               512     IV      10.2±0.3μs     9.47±1μs
               675     I        12.6±1μs     16.6±0.9μs
               675     II      11.3±0.3μs    10.1±0.5μs
               675    III      11.4±0.4μs     10.6±1μs
               675     IV      14.3±0.7μs    12.2±0.7μs
               1024    I        20.8±1μs      30.9±1μs
               1024    II       14.7±1μs      14.3±2μs
               1024   III       14.0±1μs      13.0±2μs
               1024    IV       15.1±1μs     12.3±0.7μs
               2025    I        937±20μs      261±8μs
               2025    II      23.6±0.8μs     23.0±1μs
               2025   III       21.9±1μs      21.9±2μs
               2025    IV       27.2±3μs      23.0±2μs
               2048    I        565±40μs      263±9μs
               2048    II       21.7±1μs      21.4±2μs
               2048   III      21.9±0.9μs    21.6±0.9μs
               2048    IV       27.0±3μs     20.7±0.8μs
              ====== ====== =============== ============

[ 87.50%] ··· fft_basic.RealTransformsND.time_dctn                            ok
[ 87.50%] ··· ========== ====== =============== =============
              --                            module
              ----------------- -----------------------------
                 size     type   scipy.fftpack    scipy.fft
              ========== ====== =============== =============
                75x75      I        388±40μs       236±20μs
                75x75      II       154±5μs        130±8μs
                75x75     III       159±8μs        121±6μs
                75x75      IV       181±3μs        127±3μs
               100x100     I        269±10μs       243±20μs
               100x100     II       212±6μs        184±20μs
               100x100    III       204±10μs       159±3μs
               100x100     IV       221±8μs        164±7μs
               135x135     I      1.51±0.04ms      804±30μs
               135x135     II       311±10μs       232±9μs
               135x135    III       289±10μs       229±5μs
               135x135     IV       329±10μs       246±6μs
               313x363     I       11.1±0.1ms     6.29±0.3ms
               313x363     II      28.2±0.7ms     3.80±0.5ms
               313x363    III       28.9±1ms      3.68±0.3ms
               313x363     IV      28.4±0.7ms     3.89±0.2ms
               1000x100    I       2.12±0.2ms     2.52±0.1ms
               1000x100    II     1.41±0.09ms     1.03±0.1ms
               1000x100   III     1.29±0.07ms     1.01±0.1ms
               1000x100    IV     1.58±0.03ms    1.01±0.06ms
               256x256     I      1.30±0.06ms     1.38±0.2ms
               256x256     II       860±40μs       615±70μs
               256x256    III       883±40μs       628±30μs
               256x256     IV     1.01±0.02ms      629±20μs
              ========== ====== =============== =============

[100.00%] ··· fft_basic.RealTransformsND.time_dstn                            ok
[100.00%] ··· ========== ====== =============== =============
              --                            module
              ----------------- -----------------------------
                 size     type   scipy.fftpack    scipy.fft
              ========== ====== =============== =============
                75x75      I        236±20μs       184±9μs
                75x75      II       168±8μs        123±4μs
                75x75     III       157±5μs        122±4μs
                75x75      IV       177±7μs        142±10μs
               100x100     I      1.77±0.09ms      771±10μs
               100x100     II       212±10μs       159±10μs
               100x100    III       200±7μs        164±6μs
               100x100     IV       242±10μs       173±5μs
               135x135     I        424±10μs       389±8μs
               135x135     II       319±10μs       257±20μs
               135x135    III       295±10μs       243±4μs
               135x135     IV       349±20μs       282±20μs
               313x363     I       10.2±0.2ms     5.97±0.1ms
               313x363     II      27.9±0.4ms    3.67±0.09ms
               313x363    III      28.4±0.7ms     3.50±0.1ms
               313x363     IV      28.0±0.2ms     3.76±0.2ms
               1000x100    I       9.55±0.2ms     5.18±0.2ms
               1000x100    II     1.48±0.09ms    1.10±0.03ms
               1000x100   III     1.32±0.04ms    1.03±0.02ms
               1000x100    IV      1.81±0.3ms    1.16±0.06ms
               256x256     I       26.7±0.7ms     10.7±0.3ms
               256x256     II       959±9μs        661±30μs
               256x256    III       881±50μs       633±20μs
               256x256     IV     1.08±0.03ms     689±200μs
              ========== ====== =============== =============

Running benchmarks for Scipy version 1.4.0.dev0+add1cae at /mnt/d/git/scipy/build/testenv/lib/python3.7/site-packages/scipy/__init__.py

"smooth-sized" (i.e. length is a product of small primes) type I transforms are slower by a factor of roughly 2.

I just realized that this statement is sloppy and incorrect, sorry. A type I DCT of length N is computed via a length (N-1) FFT by fftpack and a length 2*(N-1) FFT in pocketfft. So it doesn't actually mean anything if N is "smooth-sized"; (N-1) needs to be smooth-sized instead for the statement to be true.
For type I DSTs, (N+1) needs to be smooth-sized.
In such cases fftpack should be faster unconditionally.

Thanks for pointing that out @mreineck, I've updated the benchmarks to make the FFT sizes smooth (could you check that for me?). It's still not entirely one sided but it does definitely tip the numbers in fftpack's favour.

Updated benchmark results
· Discovering benchmarks
· Running 4 total benchmarks (1 commits * 1 environments * 4 benchmarks)
[  0.00%] ·· Benchmarking existing-py_usr_bin_py
[ 12.50%] ··· Running (fft_basic.RealTransforms1D.time_dct--)..
[ 37.50%] ··· Running (fft_basic.RealTransformsND.time_dctn--)..
[ 62.50%] ··· fft_basic.RealTransforms1D.time_dct                             ok
[ 62.50%] ··· ====== ====== =============== =============
              --                        module
              ------------- -----------------------------
               size   type   scipy.fftpack    scipy.fft
              ====== ====== =============== =============
                75     I       6.16±0.6μs     5.00±0.3μs
                75     II       7.47±1μs      4.39±0.2μs
                75    III      6.43±0.3μs     4.57±0.2μs
                75     IV      6.87±0.9μs     4.45±0.2μs
               100     I        9.10±2μs      5.38±0.1μs
               100     II      6.53±0.2μs      6.03±1μs
               100    III      6.52±0.2μs    4.56±0.06μs
               100     IV      6.81±0.8μs     4.52±0.4μs
               135     I       6.57±0.2μs     5.71±0.2μs
               135     II      7.09±0.6μs     5.69±0.6μs
               135    III       6.94±1μs      5.29±0.1μs
               135     IV       8.34±3μs       6.07±2μs
               256     I       8.17±0.5μs     7.11±0.5μs
               256     II       12.4±3μs      5.83±0.3μs
               256    III       8.46±1μs      5.92±0.4μs
               256     IV       7.73±1μs      6.27±0.4μs
               313     I        77.3±2μs      34.6±0.9μs
               313     II       77.2±3μs       18.8±3μs
               313    III      78.8±10μs       18.5±2μs
               313     IV       78.5±2μs      18.1±0.4μs
               512     I       8.99±0.5μs      10.9±1μs
               512     II      9.21±0.3μs      8.91±1μs
               512    III       9.02±1μs      7.44±0.3μs
               512     IV       9.94±1μs       8.63±1μs
               675     I        11.4±1μs       13.4±4μs
               675     II      10.8±0.6μs     9.41±0.9μs
               675    III      10.8±0.4μs     10.2±0.7μs
               675     IV       12.0±1μs      10.9±0.9μs
               1024    I       13.1±0.9μs     16.0±0.4μs
               1024    II      12.9±0.9μs     11.9±0.5μs
               1024   III      12.0±0.4μs     10.5±0.5μs
               1024    IV      13.0±0.3μs     11.1±0.2μs
               2025    I       19.5±0.5μs      37.8±2μs
               2025    II      22.3±0.6μs     20.4±0.5μs
               2025   III       27.6±6μs       19.0±1μs
               2025    IV       24.3±3μs       21.8±2μs
               2048    I        21.4±2μs      34.3±0.7μs
               2048    II       21.7±2μs       21.9±3μs
               2048   III      20.2±0.9μs     16.8±0.5μs
               2048    IV      22.4±0.7μs      20.7±1μs
              ====== ====== =============== =============

[ 75.00%] ··· fft_basic.RealTransforms1D.time_dst                             ok
[ 75.00%] ··· ====== ====== =============== =============
              --                        module
              ------------- -----------------------------
               size   type   scipy.fftpack    scipy.fft
              ====== ====== =============== =============
                75     I       6.52±0.4μs     5.44±0.2μs
                75     II      6.73±0.2μs     4.62±0.3μs
                75    III      7.18±0.7μs     4.29±0.1μs
                75     IV      6.63±0.2μs     4.98±0.4μs
               100     I       6.51±0.1μs     5.34±0.2μs
               100     II      7.26±0.5μs     5.61±0.2μs
               100    III      6.86±0.5μs    4.51±0.06μs
               100     IV       7.39±1μs      4.67±0.1μs
               135     I       6.68±0.2μs     7.64±0.7μs
               135     II      6.81±0.3μs     5.41±0.2μs
               135    III      7.30±0.8μs     6.03±0.8μs
               135     IV       8.05±1μs       7.09±1μs
               256     I       7.34±0.2μs     8.92±0.5μs
               256     II       8.31±1μs      6.10±0.3μs
               256    III      7.64±0.5μs     5.95±0.1μs
               256     IV      7.63±0.3μs     6.57±0.4μs
               313     I        78.8±5μs       34.4±2μs
               313     II       77.9±2μs       18.9±1μs
               313    III      83.5±10μs      19.3±0.6μs
               313     IV      77.5±0.8μs     18.6±0.4μs
               512     I       9.00±0.4μs     11.1±0.9μs
               512     II      9.17±0.4μs     8.61±0.9μs
               512    III      9.52±0.7μs     7.48±0.2μs
               512     IV      10.2±0.9μs     8.58±0.8μs
               675     I       10.8±0.5μs     13.4±0.8μs
               675     II      10.8±0.2μs      10.2±1μs
               675    III      11.1±0.2μs     9.13±0.4μs
               675     IV      11.9±0.2μs     10.1±0.7μs
               1024    I        13.0±1μs      16.7±0.8μs
               1024    II      13.5±0.8μs      13.5±1μs
               1024   III       15.6±1μs      11.1±0.3μs
               1024    IV      16.4±0.9μs      14.6±1μs
               2025    I       24.1±0.4μs      37.4±4μs
               2025    II       24.8±2μs       22.0±1μs
               2025   III      31.4±10μs       26.5±4μs
               2025    IV      28.2±0.7μs      21.8±2μs
               2048    I       20.6±0.6μs      33.3±2μs
               2048    II      21.2±0.2μs      21.8±3μs
               2048   III      22.1±0.9μs      19.6±2μs
               2048    IV       28.3±1μs       23.4±2μs
              ====== ====== =============== =============

[ 87.50%] ··· fft_basic.RealTransformsND.time_dctn                            ok
[ 87.50%] ··· ========== ====== =============== =============
              --                            module
              ----------------- -----------------------------
                 size     type   scipy.fftpack    scipy.fft
              ========== ====== =============== =============
                75x75      I        94.6±4μs       93.8±4μs
                75x75      II       103±5μs        54.1±1μs
                75x75     III       91.7±2μs       56.4±2μs
                75x75      IV       110±5μs        65.6±2μs
               100x100     I        135±2μs        140±1μs
               100x100     II       161±10μs       96.7±3μs
               100x100    III       135±2μs        92.2±3μs
               100x100     IV       166±10μs       99.9±5μs
               135x135     I        222±8μs        254±40μs
               135x135     II       248±10μs       173±10μs
               135x135    III       215±10μs       177±10μs
               135x135     IV       333±40μs       239±20μs
               313x363     I        29.0±2ms      8.82±0.4ms
               313x363     II       30.1±2ms     3.43±0.05ms
               313x363    III      27.7±0.7ms     3.44±0.1ms
               313x363     IV       27.8±1ms       4.17±2ms
               1000x100    I       2.27±0.6ms    1.59±0.07ms
               1000x100    II     1.25±0.02ms      889±50μs
               1000x100   III     1.15±0.09ms    1.11±0.05ms
               1000x100    IV      1.63±0.2ms     1.01±0.2ms
               256x256     I       960±400μs       818±20μs
               256x256     II      872±100μs       585±30μs
               256x256    III      923±100μs       682±20μs
               256x256     IV      1.06±0.1ms      610±30μs
              ========== ====== =============== =============

[100.00%] ··· fft_basic.RealTransformsND.time_dstn                            ok
[100.00%] ··· ========== ====== =============== ============
              --                           module
              ----------------- ----------------------------
                 size     type   scipy.fftpack   scipy.fft
              ========== ====== =============== ============
                75x75      I        112±10μs      103±20μs
                75x75      II       125±10μs      67.1±4μs
                75x75     III       103±6μs       65.1±3μs
                75x75      IV       132±60μs      69.7±4μs
               100x100     I        136±3μs       139±20μs
               100x100     II       160±10μs      122±10μs
               100x100    III       137±2μs       95.3±4μs
               100x100     IV       184±20μs      106±10μs
               135x135     I        403±90μs      280±20μs
               135x135     II       304±60μs      200±20μs
               135x135    III       296±60μs      187±10μs
               135x135     IV       305±20μs      223±20μs
               313x363     I        30.0±1ms     7.60±0.3ms
               313x363     II      27.7±0.7ms    3.59±0.1ms
               313x363    III       28.1±1ms     3.44±0.3ms
               313x363     IV      27.4±0.7ms    3.79±0.9ms
               1000x100    I       1.35±0.2ms    1.60±0.1ms
               1000x100    II     1.29±0.05ms     972±20μs
               1000x100   III      1.28±0.2ms     909±30μs
               1000x100    IV     1.61±0.03ms     940±20μs
               256x256     I        644±20μs      840±40μs
               256x256     II       861±30μs      592±70μs
               256x256    III       799±40μs      569±30μs
               256x256     IV     1.01±0.01ms     593±7μs
              ========== ====== =============== ============

Running benchmarks for Scipy version 1.4.0.dev0+cf9335b at /mnt/d/git/scipy/build/testenv/lib/python3.7/site-packages/scipy/__init__.py

I've updated the benchmarks to make the FFT sizes smooth (could you check that for me?).

Looks fine to me!

It's still not entirely one sided but it does definitely tip the numbers in fftpack's favour.

Hmm, apparently fftpack's calling overhead is larger at the moment, so that pocketfft`s slowness for type I transforms only shows at larger lengths.

I could symlink them in from the fftpack test directory. IIRC git can handle that correctly.

I don't love adding 250 kB of npz files for testing against, but I can't think of a better alternative.

I hope that their copies in fftpack/ may soon be removed.

Otherwise LGTM. @mreineck are you also happy?

Perfectly happy!

I removed the copied files from the git history so it shouldn't even increase the .git directory size.

Thanks for pointing that out @peterbell10. FYI GitHub does not notify about force pushes, so if you had not commented, I wouldn't have realized you actually made changes!

Thanks @peterbell10


Integrate Go cases into build scripts

I think basically we can treat Go cases as a standalone module and write a Makefile to it for running linters and tests.
Then we wire it into the x.py script.

I am thinking whether the makefile is necessary, since it is just a test module and we already have x.py which is a general script for developers.
If there are two scripts that need to be maintained, I think it might cause some confusion.

@PragmaTwice you're right. Previously I'm thinking of adding linters for Go code and then build/test can depend on the lint task. Makefile does a good job for task dependencies while implementing it inside a script is awkward. However, now I believe we can run lint in another x.py command and there's no dependency between tasks to handle. So your patch is great.

Closed by #832.


Improve performance by spawning a separate thread to write to the pipe

Overview: What does this pull request change?

Add a thread-safe queue and a listener thread that takes frames and outputs them to a file.

Motivation and Explanation: Why and how do your changes improve the library?

Since optimizations are heavily encouraged by the community, the changes here improve performance by up to 15% (in limited benchmarks/use cases).
Steps to reproduce the results are as follows:
Example script:
import time

from manim import *

class MovingFrameBox(Scene):
    def construct(self):
        start = time.time_ns()
        text=MathTex(
            "\\frac{d}{dx}f(x)g(x)=","f(x)\\frac{d}{dx}g(x)","+",
            "g(x)\\frac{d}{dx}f(x)"
        )
        self.play(Write(text))
        framebox1 = SurroundingRectangle(text[1], buff = .1)
        framebox2 = SurroundingRectangle(text[3], buff = .1)
        self.play(
            Create(framebox1),
        )
        self.wait()
        self.play(
            ReplacementTransform(framebox1,framebox2),
        )
        self.wait()
        end = time.time_ns()
        print(f"{end - start} ns")

Run
manim -p example.py MovingFrameBox

Machine info:
CPU: 12th Gen Intel i7-1280P (14) @ 4.700GHz
MEM: 64 GB
OS: EndeavourOS Linux x86_64
Kernel: 6.3.9-arch1-1

Links to added or changed documentation pages

Further Information and Comments

This PR will need to be reviewed for correctness. It was implemented in haste, so I may have overlooked some details. Please let me know if any additional figures or information is needed.
Thank you!

Reviewer Checklist

[ ] The PR title is descriptive enough for the changelog, and the PR is labeled correctly
[ ] If applicable: newly added non-private functions and classes have a docstring including a short summary and a PARAMETERS section
[ ] If applicable: newly added functions and classes are tested

I would first try to fix the issues that currently exist, because parallel rendering for manim is currently really difficult to implement in a working way because there are too many side effects happening in the rendering pipeline
We will take a look at this when it is in a working state that is reviewable aka. the pipeline runs through all the tests, you can also run them locally with pytest to see if your changes actually work!
But this change also might make the current rendering too complex to change later on because we are currently rewriting the rendering system which also includes the file writing and all that which will be parallel anyway after the rewrite.

So if parallel things are implemented they should have a clean interface which maybe lives in a seperate class so we can actually reuse it later on instead of having the changes spread through the codebase because for now there is not really any parallel code or even just concurrent.
So if you have the capabilities to work on it that would be great but also implementing things in a hassle is never a good idea.

Hi @MrDiver
Thank you for taking the time to look over my changes, and for your suggestions.
I will start by fixing the test pipeline, and we can go from there. It's been a little while since I made these changes; however, from what I recall, this only adds concurrency to the file writing. If you have suggestions for other areas that can benefit from concurrency (or parallelism), please feel free to let me know, and I will take a look.
Lastly, I'm glad to work on this particular issue. I love parallelism/concurrency related issues, so it should be a lot of fun! I agree that rushing the implementation is a bad idea, so I'll aim for a quality solution. If you have the time do some code reviews as I push up code, that would be great!
Thank you!


Added 'provides' section

EDIT: check out pull #17 ; it fills out more info in META.info, along with the provides
=================================
This is to play nice with panda 11 that needs provides section.
==> Installing Template::Mojo
===WARNING!===
The distribution Template::Mojo does not seem to have a "provides" section in its META.info file,
and so the packages will not be installed in the correct location.
Please ask the author to add a "provides" section, mapping every exposed namespace to a
file location in the distribution.
See http://design.perl6.org/S22.html#provides for more information.

This fixes #14

@zoffixznet ++
This fixes one of the tutorials for perl6 that recommends Task::Star as part of 'getting perl6'

That's covered by #17, which I merged first, so that's done now. Thanks :)


refactor: add Log Out icon and change button

What type of PR is this? (check applicable) 🚧

[x] ♻ Refactor
[ ] ✨ Feature
[ ] 🐛 Bug Fix
[ ] 👷 Optimization
[ ] 📝 Documentation Update
[ ] 🔖 Release
[ ] 🚩 Other

Describe your PR request 🔥

refactor: add Log Out Icon and recapitalize text

Closes #53


No result coming on clicking enter

What happened?
I enter a number and click the button. No response.
What device are you seeing the problem on?
Desktop
What browsers are you seeing the problem on?
Microsoft Edge
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Assign this issue to me. I will fix it.


Fix mindeps-distributed CI build to handle numpy/pandas not being installed

Okay, so this is an interesting one. On main we've started getting
FAILED dask/tests/test_layers.py::test_dataframe_cull_key_dependencies[_shuffle_op] - ImportError: Dask dataframe requirements are not installed.

Please either conda or pip install as follows:

  conda install dask                     # either conda install
  python -m pip install "dask[dataframe]" --upgrade  # or python -m pip install

in our mindep-distributed build due to numpy/pandas not being installed. Looking closer at test_dataframe_cull_key_dependencies, we are in fact missing a pytest.importorskip("dask.dataframe") (which has been added in this PR). Though I was confused by we weren't seeing this before. It turns out that previously we were skipping this test (and all the test in test_layers.py) due to the following issue when attempting to import distributed:
dask/tests/test_layers.py:5: could not import 'distributed': tlz.peekn

Which was recently fixed in https://github.com/dask/distributed/pull/7309. So we had been skipping this test, but for the wrong reasons -- this PR makes it so we skip/run tests appropriately
cc @charlesbluca @ncclementi in case either of you have a moment to look at this

Will plan to merge this after CI finishes in order to get CI passing ASAP. Happy to handle any related changes in a follow-up PR


add some examples

Add Jose's ShearBeam example for Pisano plastic materials.
Add one dynamic example.
Add examples for different Poisson's ratios and plot the errors both in 2D and 3D.
Add one compressed file (*.tgz) to each model. The corresponding document has a downloadable link to the *.tgz file for each model.

Add 1D and 3D DRM examples.
Add script to generate HDF5-based DRM input.
Add documents.
Update the figures according to comments...


Add new Rails/ArelStar cop

This is a new cop that checks for usage of arel_table["*"] and MyModel["*"] calls. arel_table quotes anything a user enters in the brackets and this can lead to some unexpected consequences:
class MyModel < ApplicationRecord
  scope :my_arel_scope, -> {
    select(arel_table["*"])
  }
end

puts MyModel.my_arel_scope.to_sql

# (Assuming MySQL)
#=> SELECT `my_models`.`*` FROM `my_models`

This is correct-looking but actually incorrect SQL because a quoted asterisk is treated as a literal column name by MySQL and Postgres. I have not tried this in SQLite but I imagine it does the same thing.
Thanks for taking a look!

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] If this is a new cop, consider making a corresponding update to the Rails Style Guide.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Might this be considered as a bug in Arel?

@andyw8 Any further thoughts on this?

I'm still on the fence, but if another maintainer gives it a 👍 then let's go ahead.

Cool, thank you.

I'm happy to close this if there's no interest but any chance I can get another set of eyes on this?

Thank you!


to fix cross-repo [[insert]] gaff

the [[insert]] was trying to pull in a test vector from the pres-ex repo
i also noticed image files would be broken, so I duplicated the ../resources directory in the v0.1 directory
i then copied the external test vector to the new resources directory (adding a descriptive filename with the date of "snapshot") and updated the [[insert]] rel-ref

@bumblefudge the errors that weren't the fault of Spec-Up aside, what could make the process of cutting versions easier for you? Obviously I can't do magic, but maybe there's something the tool could do to help?


One question per page

Add get_next_edit_questions_section_id() and new parameter edit_questions to get_next_section_id.
Bump version to 2.8.1

This adds a new feature (if only a small one), so I'd go for a middle number bump to 2.9.0.

Bumped to 2.9.0

I think this is good to go :+1:


Export react select components

What's the problem this PR addressing?

What's the problem this PR addressing?

This will solve version incompatibilities if the consumer is already using a different version of react-select. In SS we are using an older version and if we try to pass a custom component to twigs select, its not working properly. @justin-ss


Add docs on storage engine WAL failover

Fixes:

DOC-9709
DOC-9916
DOC-9925
DOC-10149

Summary of changes:

Add a new section to cockroach start describing the WAL failover feature, how to enable/disable, and the related logging config changes that are needed if you enable the feature

Add a new section to 'Monitoring and Alerting' docs describing the store status endpoint at _status/stores

Update logging docs to add some anchor links so we can refer to specific config settings from the WAL failover docs

Update v24.1 alpha release notes to link to the WAL failover docs

Hi folks, I added each of you to review for the following reasons/areas, but please feel free to comment on anything you see that is missing/incorrect/etc as well:

@jbowens for WAL failover stuff
@abarganier for async file logging stuff
@mwang1026 for overall vibe

A few minor edit suggestions -- notably a few more places I think we should mention that feature is in PREVIEW

Updated in the places where you mentioned.  Do we also want to add this to the list of Features in Preview for v24.1?  My assumption is yes but figured I'd ask while you're here

I think we should also open a separate PR for how to monitor for failover -- the metrics to watch, how to inspect them, etc. -- thoughts?

That makes sense, I can make a followup PR - @jbowens I found the following metrics on the custom chart debug page of a 24.1 RC cluster.  Which ones do you think make sense to monitor and what values should one alert on?  Based on looking I'd guess switch.count could be a starting point? followed by the durations? but I'm just guessing :-)

storage.wal.failover.primary.duration
storage.wal.failover.secondary.duration
storage.wal.failover.switch.count
storage.wal.failover.write_and_sync.latency-avg
storage.wal.failover.write_and_sync.latency-count
storage.wal.failover.write_and_sync.latency-max
storage.wal.failover.write_and_sync.latency-sum
storage.wal.failover.write_and_sync.latency-p50
storage.wal.failover.write_and_sync.latency-p75
storage.wal.failover.write_and_sync.latency-p90
storage.wal.failover.write_and_sync.latency-p99
storage.wal.failover.write_and_sync.latency-p99.9
storage.wal.failover.write_and_sync.latency-p99.99
storage.wal.failover.write_and_sync.latency-p99.999

Which ones do you think make sense to monitor and what values should one alert on? Based on looking I'd guess switch.count could be a starting point? followed by the durations? but I'm just guessing :-)

Yeah, I think it makes sense to document those first three metrics:
storage.wal.failover.primary.duration
storage.wal.failover.secondary.duration
storage.wal.failover.switch.count

The storage.wal.failover.secondary.duration is probably the most interesting. Customers will generally expect this to be zero unless there's a failover. Then they might care about how long it remains non-zero because it provides indication into the health of the primary.

Yeah, I think it makes sense to document those first three metrics:
storage.wal.failover.primary.duration
storage.wal.failover.secondary.duration
storage.wal.failover.switch.count

The storage.wal.failover.secondary.duration is probably the most interesting. Customers will generally expect this to be zero unless there's a failover. Then they might care about how long it remains non-zero because it provides indication into the health of the primary.

Thanks @jbowens - I've filed https://cockroachlabs.atlassian.net/browse/DOC-10268 and will do that as a followup after we get this PR in
@mwang1026 are you good with this given the recent updates based on your feedback?   Remaining open question is if you also want a blurb in https://www.cockroachlabs.com/docs/v24.1/cockroachdb-feature-availability.html#features-in-preview or if you'd rather this feature did not show up there.  I believe our practice is to also list it there but maybe you don't want it there, idk

@mwang1026 I went ahead and added WAL failover to the list of preview features since AFAICT we do that for everything else in Preview
Let me know if you're good with the other changes and I'll send this along for docs team review so I can get it merged ASAP
Thanks!

@florence-crl this is RFAL from a docs POV now
in terms of sequencing, this should go in first, then #18548

@florence-crl thanks for the helpful review. I've incorporated everything from your first pass AFAICT - PTAL!


Enhancement: recursively add parent folders. Closes #5887

Closes #5887

Thanks @Saurabh7019, we'll try to review it ASAP!

Hi Mathijs, Thank you for taking the time to review. I have made the suggested changes. Please let me know if you have any further comments.

Thanks for the contribution @Saurabh7019 . Manually merged.


Update to standard-library 1.6

stdlib 1.6-rc1 is just out. No source changes are necessary! :-)
If you cut a new release, we can upgrade agda-categories in Nix.

The CI failed.
Could you please update the stdlib version in travis.yml too, @iblech?
https://github.com/agda/agda-categories/blob/8cd50f92ce3c91dbb325b4b0a66d231df809d0a9/.travis.yml#L19

Ah, I see, thank you; fixed in force-push. However, the CI will probably still fail because the new stdlib has not yet been released.

However, the CI will probably still fail because the new stdlib has not yet been released.
Indeed.

I'll let @JacquesCarette comment on whether the library is otherwise ready for a new release.

I might want to wait just a little for a new release.  And I want to start documenting what's new in the new releases now.

Closing and reopening the PR to trigger a new Travis build (now that stdlib 1.6 has been released).

Ping @sstucki I think this can be merged?

Do I have to make a release too?

That would be great, not the least from a packaging perspective in nixpkgs.

Done.


Add bias-shades

This task implements the bias-shades dataset for French and English. I'll add the other languages, when the respective prompts are available.

Hello!

[ ] Is shades merged into promptsource? (I didn't see it by name in promptsource or in the tracking sheet.)
[ ] Does this run locally with, say, gpt2?

This PR depends on the French and English prompts to be added in promptsource.


Add 4844 transaction fields to the CallParameter

currently eth_call does not work for blob transactions.
Spec for eth_call params
https://github.com/ethereum/execution-apis/blob/main/src/schemas/transaction.yaml

Branch https://github.com/macfarla/besu/tree/eth-call-4844


Make tests more resilient by reusing user in db

The hope is, that by not recreating a new user for each test, the loginAs method will work more robustly.
Bug: T311506

Cool! Let's see how this approach works :) Thanks.


Not compatible with Gnome 45

i pushed an update. im on 44, and cant test 45, but i dont see any issue with support for 45

Got this message after trying to enable it:
SyntaxError: import declarations may only appear at top level of a module @ resource:///org/gnome/shell/ui/main.js:3:0


bump(deps): go.opentelemetry.io/otel/metric from v0.26.0 to v1.16.0

What this PR does
Bump go.opentelemetry.io/otel/metric from v0.26.0 to v1.16.0.

Current dependencies on/for this PR:

main

PR #493   👈

PR #494

This comment was auto-generated by Graphite.

Codecov Report
Patch coverage: 60.00% and project coverage change: +0.01 :tada:

Comparison is base (9d37b96) 62.11% compared to head (8f4b941) 62.13%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #493      +/-   ##
==========================================
+ Coverage   62.11%   62.13%   +0.01%
==========================================
  Files         133      133
  Lines       18490    18502      +12
==========================================
+ Hits        11485    11496      +11
- Misses       6425     6426       +1
  Partials      580      580

Impacted Files
Coverage Δ

internal/metarepos/metadata_repository_metrics.go
50.00% <30.00%> (-13.16%)
:arrow_down:

internal/storagenode/telemetry/metrics.go
57.91% <61.53%> (-0.02%)
:arrow_down:

internal/metarepos/telemetry.go
70.37% <75.00%> (ø)

internal/storagenode/storagenode.go
72.83% <100.00%> (ø)

... and 5 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

@ijsong started a stack merge that includes this pull request via Graphite.

@ijsong merged this pull request with Graphite.


Ratio 16:10 or presentation whithout borders to the left and right

Hi,
is it  somehow possible to change the ratio to something else than 4:3 and 16:9?
I plan to give a presentation with a resolution of 1280 x 800 / 16:10.
I tried following options:

ratio: '16:10' (presentation was not rendered at all and only 4:3 and 16:9 seem to be supported
remark-slide-content: { width: inherit; height: inherit; ] as well as the values set to 1280 and 800 instead of inherit but I still had the borders to the left and right

Any solution, workaround or just hint is highly appreciated!

This can be closed. It was my fault and actually works fine.


Document this.hot in the loader API

Feature to document
this.hot in the loader context, see: https://github.com/webpack/webpack/issues/6496
Author(s)
@sokra
Additional information

It looks like you just deleted our lovely crafted issue template. It was there for good reasons. Please help us solving your issue by answering the questions asked in this template. I'm closing this. Please either update the issue with the template and reopen, or open a new issue.

@webpack-bot move to webpack/webpack.js.org

How is that issue related to this.hot?

Oops, I linked to the wrong issue. Here's the correct one: https://github.com/webpack/webpack/pull/6228

I see, feel free to send a PR


local gctl: auto set namespace selector against watch

Problem Statement: I would like to use Metac as a library that is packaged as my own binary. This binary will get deployed as a Kubernetes Deployment or STS. I would like to use MetaController CR as a config that should only consider watches in this Deployment/STS namespace.

@AmitKumarDas maybe could you elaborate what is needed to achieve this behavior ?

I will try to explain via a sample config for GenericController.
NOTE: A config has schema that matches one or more GenericController(s). This config is typically placed at /etc/config/metac/ path of the container. This container hosts the metac binary that reads this config & starts informers (one informer per watch mentioned in GenericController).
NOTE: In my case this binary is a golang project that uses metac as a library
Following is a sample config that makes use of 2 GenericControllers. On the whole, we want the metac binary to start informers for watches with kind CStorClusterPlan & CStorClusterStorageSet
apiVersion: metac.openebs.io/v1alpha1
kind: GenericController
metadata:
  name: sync-plan
  namespace: cspauto
spec:
  # i would like to watch kind CStorClusterPlan created under namespace
  # that matches the namespace of the binary that has this config file
  watch:
    apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterplans
  attachments:
  - apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterstoragesets
    updateStrategy:
      method: InPlace
  - apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterconfigs
  hooks:
    sync:
      inline:
        funcName: sync/cstorclusterplan
---
apiVersion: metac.openebs.io/v1alpha1
kind: GenericController
metadata:
  name: sync-storageset
  namespace: cspauto
spec:
  watch:
    apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterstoragesets
  attachments:
  - apiVersion: dao.mayadata.io/v1alpha1
    resource: storages
    updateStrategy:
      method: InPlace
  hooks:
    sync:
      inline:
        funcName: sync/cstorclusterstorageset

Coming back to the problem statement, I would like to spin up multiple metac binaries in dedicated namespaces. I want each binary to lookup up the watches that belong to its own namespace & proceed with its reconciliation.
Possible solution:
We can define following new environment variables e.g.:

MATCH_WATCH_WITH_SELF_NAMESPACE: <true/false>
SELF_NAMESPACE: (this will be a K8s downward API that will hold the POD's namespace)

Now when metac starts, the resulting GenericControllers will be like following:
apiVersion: metac.openebs.io/v1alpha1
kind: GenericController
metadata:
  name: sync-plan
  namespace: cspauto
spec:
  # i would like to watch kind CStorClusterPlan created under namespace
  # that matches the namespace of the binary that has this config file
  watch:
    apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterplans
    advancedSelector:
      selectorTerms:
      - matchFields:
          metadata.namespace: <POD's namesapce>
  attachments:
  - apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterstoragesets
    updateStrategy:
      method: InPlace
  - apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterconfigs
  hooks:
    sync:
      inline:
        funcName: sync/cstorclusterplan
---
apiVersion: metac.openebs.io/v1alpha1
kind: GenericController
metadata:
  name: sync-storageset
  namespace: cspauto
spec:
  watch:
    apiVersion: dao.mayadata.io/v1alpha1
    resource: cstorclusterstoragesets
    advancedSelector:
      selectorTerms:
      - matchFields:
          metadata.namespace: <POD's namesapce>
  attachments:
  - apiVersion: dao.mayadata.io/v1alpha1
    resource: storages
    updateStrategy:
      method: InPlace
  hooks:
    sync:
      inline:
        funcName: sync/cstorclusterstorageset

Above will need to work when metac binary is meant to be run by looking up its config file (described above). In other words metac needs to be started with following args:
 args:
        - --run-as-local


Recursive rest-client call to fetch all schools and kindergarten pages at once

PT-1234 PT-1298 PT-1312.

[ ] Tests are still to come!


bad reference to moment.js

Remove stray reference to moment.js in mlUtil requirejs dependencies --
this module has already been loaded during bootstrap time and the path
used under mlUtil is wrong (only firefox seems to notice).
Closes #633

This became apparent when looking at #601 but is a different problem than what was reported there.


WIP: Client-side of gRPC proxy

This works with the out-of-tree gRPC server middleware here: https://github.com/infobloxopen/coredns-grpc.
Still need to do something about the context - would prefer to see that passed to Exchange()

Codecov Report

Merging #511 into master will decrease coverage by -0.26%.

@@            Coverage Diff             @@
##           master     #511      +/-   ##
==========================================
- Coverage   38.16%   37.91%   -0.26%
==========================================
  Files         117      118       +1
  Lines        5809     5848      +39
==========================================
  Hits         2217     2217
- Misses       3411     3450      +39
  Partials      181      181

Impacted Files
Coverage Δ

middleware/proxy/grpc.go
0% <ø> (ø)

middleware/proxy/upstream.go
61.14% <ø> (-2.93%)
:x:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 353c1c6...ac2a44b. Read the comment docs.

Some moaning. Love how short this PR has become since the recent changes in proxy :)

lgtm.

re: context, yes makes sense to pass that.

[ Quoting<EMAIL_ADDRESS>in "Re: [miekg/coredns] WIP: Client-sid..." ]

johnbelamaric commented on this pull request.
2017/02/10 15:55:00 [INFO] Reloading
2017/02/10 15:55:00 [INFO] Creating gRPC connection 0 to <IP_ADDRESS>:4443.
2017/02/10 15:55:00 [INFO] Creating gRPC connection 1 to <IP_ADDRESS>:4444.
2017/02/10 15:55:00 [ERROR] SIGUSR1: listen tcp :1053: bind: address already in use

It seems the issue stems from this statement here: https://github.com/mholt/caddy/blob/master/caddy.go#L619 - it seems dnsserver.Server is NOT implementing the GracefulServer interface: https://godoc.org/github.com/mholt/caddy#GracefulServer

Ah :( Should fix then. Thought I did actually, but things might have moved on.
Thanks for finding. Also: tests (blame me)

@miekg please take a look, I think it's ready

oh, except the README

lgtm
prolly will polish the README.md a bit, post merge.


undefined is not a valid state value

Hi,
after updating I get following messages that are spamming my log ..
tesla-motors.0 | 2021-05-07 16:05:27.188 | info | (26991) undefined is not a valid state value for tesla-motors.0.vehicle.homelink_device_count
tesla-motors.0 | 2021-05-07 16:05:27.188 | info | (26991) undefined is not a valid state value for tesla-motors.0.vehicle.homelink_nearby
For your information, I do not have Homelink on my Tesla Model 3.
Thanks ,
xengosam

Thanks for the report. It seems that homelink-states do not exist when the car has no autopilot. I've made a hotfix for this and pushed it to the latest version on github, if you want you can update from there.

Thanks for the fix! Everything works like expected.