bin: adds cleanup scripts

What this PR does / why we need it: Adds some simple scripts that can clean up your clusters.
Which issue this PR fixes (use the format fixes #<issue number>(, fixes #<issue_number>, ...) to automatically close the issue when PR gets merged): fixes #213
Public facing documentation PR (if applicable)

https://github.com/elastisys/compliantkubernetes/pull/72
Special notes for reviewer:
This should remove everything except storage classes and any labels on the namespaces that are not deleted (e.g. kube-system). Do you think that I should remove that as well?
Checklist:

[x] Added relevant notes to WIP-CHANGELOG.md
[x] Proper commit message prefix on all commits
[x] Updated the public facing documentation
Is this changeset backwards compatible for existing clusters? Applying:

[x] is completely transparent, will not impact the workload in any way.
[ ] requires running a migration script.
[ ] will create noticeable cluster degradation.
E.g. logs or metrics are not being collected or Kubernetes API server
will not be responding while upgrading.
[ ] requires draining and/or replacing nodes.
[ ] will change any APIs.
E.g. removes or changes any CK8S config options or Kubernetes APIs.
[ ] will break the cluster.
I.e. full cluster migration is required.

I get:
Error from server (NotFound): error when deleting "bootstrap/crds/dex/authcodes.yaml": customresourcedefinitions.apiextensions.k8s.io "authcodes.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/authrequests.yaml": customresourcedefinitions.apiextensions.k8s.io "authrequests.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/connectors.yaml": customresourcedefinitions.apiextensions.k8s.io "connectors.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/oauth2clients.yaml": customresourcedefinitions.apiextensions.k8s.io "oauth2clients.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/offlinesessionses.yaml": customresourcedefinitions.apiextensions.k8s.io "offlinesessionses.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/passwords.yaml": customresourcedefinitions.apiextensions.k8s.io "passwords.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/refreshtokens.yaml": customresourcedefinitions.apiextensions.k8s.io "refreshtokens.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/signingkeies.yaml": customresourcedefinitions.apiextensions.k8s.io "signingkeies.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/gatekeeper/gatekeeper-constraints.yaml": customresourcedefinitions.apiextensions.k8s.io "k8sallowedrepos.constraints.gatekeeper.sh" not found
Error from server (NotFound): error when deleting "bootstrap/crds/gatekeeper/gatekeeper-constraints.yaml": customresourcedefinitions.apiextensions.k8s.io "k8srequirenetworkpolicy.constraints.gatekeeper.sh" not found
Error from server (NotFound): error when deleting "bootstrap/crds/gatekeeper/gatekeeper-constraints.yaml": customresourcedefinitions.apiextensions.k8s.io "k8sresourcerequests.constraints.gatekeeper.sh" not found
exit status 1

Also, on a rather "sick" cluster
$ kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io,mutatingwebhookconfigurations.admissionregistration.k8s.io
NAME                                                                                WEBHOOKS   AGE
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook      1          2d21h
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission   1          2d21h

NAME                                                                             WEBHOOKS   AGE
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook   1          2d21h

Would it make sense to start by removing webhooks first?

I get:
Error from server (NotFound): error when deleting "bootstrap/crds/dex/authcodes.yaml": customresourcedefinitions.apiextensions.k8s.io "authcodes.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/authrequests.yaml": customresourcedefinitions.apiextensions.k8s.io "authrequests.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/connectors.yaml": customresourcedefinitions.apiextensions.k8s.io "connectors.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/oauth2clients.yaml": customresourcedefinitions.apiextensions.k8s.io "oauth2clients.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/offlinesessionses.yaml": customresourcedefinitions.apiextensions.k8s.io "offlinesessionses.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/passwords.yaml": customresourcedefinitions.apiextensions.k8s.io "passwords.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/refreshtokens.yaml": customresourcedefinitions.apiextensions.k8s.io "refreshtokens.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/dex/signingkeies.yaml": customresourcedefinitions.apiextensions.k8s.io "signingkeies.dex.coreos.com" not found
Error from server (NotFound): error when deleting "bootstrap/crds/gatekeeper/gatekeeper-constraints.yaml": customresourcedefinitions.apiextensions.k8s.io "k8sallowedrepos.constraints.gatekeeper.sh" not found
Error from server (NotFound): error when deleting "bootstrap/crds/gatekeeper/gatekeeper-constraints.yaml": customresourcedefinitions.apiextensions.k8s.io "k8srequirenetworkpolicy.constraints.gatekeeper.sh" not found
Error from server (NotFound): error when deleting "bootstrap/crds/gatekeeper/gatekeeper-constraints.yaml": customresourcedefinitions.apiextensions.k8s.io "k8sresourcerequests.constraints.gatekeeper.sh" not found
exit status 1

I just try to remove all crds that we could install without checking which are actually present. So some errors are expected if say gatekeeper isn't installed.

Also, on a rather "sick" cluster
$ kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io,mutatingwebhookconfigurations.admissionregistration.k8s.io
NAME                                                                                WEBHOOKS   AGE
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook      1          2d21h
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission   1          2d21h

NAME                                                                             WEBHOOKS   AGE
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook   1          2d21h

Would it make sense to start by removing webhooks first?

The webhooks should be removed when the helmcharts are removed. Did you get any errors when it tried to uninstall the charts?

I just try to remove all crds that we could install without checking which are actually present. So some errors are expected if say gatekeeper isn't installed.

Sounds good. Should the script exit with status 0, so when used in CI/CD it does not make the pipeline go red? Or should it be the pipeline itself that ignores a non-zero cleanup.

The webhooks should be removed when the helmcharts are removed. Did you get any errors when it tried to uninstall the charts?

I think in normal situations the webhooks are removed properly. However, I think there is value in making this script work in abnormal situations too. For example, if we have a "permanent" testing cluster, then the cleanup script should bring the cluster to a known good state, no matter what crap was applied beforehand. :smile:

Sounds good. Should the script exit with status 0, so when used in CI/CD it does not make the pipeline go red? Or should it be the pipeline itself that ignores a non-zero cleanup.

Since this script does not care about the exit status of its commands it should already always return 0, right?

I think in normal situations the webhooks are removed properly. However, I think there is value in making this script work in abnormal situations too. For example, if we have a "permanent" testing cluster, then the cleanup script should bring the cluster to a known good state, no matter what crap was applied beforehand. smile

Hmm, not sure about if it's worth trying to do that now. I see your point, but I think we might need to do a lot more in order to make that work "no matter what was applied". If I was going to do that and be really thorough, then wouldn't I have to go through every kind of kubernetes resource and check that we don't have anything unexpected there?

I have a few concerns though. Did you try to install apps again after running this? Does it handle the existing resources well?

Yes I tried to install apps and that seemed to work fine :slightly_smiling_face:

Will this work also when there are things running in the user namespaces? I think there could potentially be issues there if the user for example creates Certificates or a Prometheus instance. Helmfile won't know about those resources so it may remove cert-manager and kube-prometheus-stack before the CRs are gone. That would block them from deletion.
I suggest adding a note in the readme telling the user to make sure they remove all CRs before they use this. And maybe also mention that admins are responsible for cleaning up things installed outside of compliantkubernetes-apps, e.g. extra namespaces not handled by apps.

Good point. I have not tested with anything in the user namespaces, but I could check that. My hope is that the user-rbac chart is deleted early and that it removes everything (since it should delete the namespaces).

Will this work also when there are things running in the user namespaces? I think there could potentially be issues there if the user for example creates Certificates or a Prometheus instance. Helmfile won't know about those resources so it may remove cert-manager and kube-prometheus-stack before the CRs are gone. That would block them from deletion.
I suggest adding a note in the readme telling the user to make sure they remove all CRs before they use this. And maybe also mention that admins are responsible for cleaning up things installed outside of compliantkubernetes-apps, e.g. extra namespaces not handled by apps.

Good point. I have not tested with anything in the user namespaces, but I could check that. My hope is that the user-rbac chart is deleted early and that it removes everything (since it should delete the namespaces).

Did a quick test with cert-manager issuer and cert in a user namespace.
Destroying user-rbac chart worked fine to delete the namespace even though there were pods, issuers, and certs in a namespace. But to be sure that e.g. cert-manager isn't deleted before the resource, the script now starts with destroying user-rbac and then goes on with the rest.
However I can't guarantee anything if a user adds their own controllers, then some race condition could come up.

Some of ,y namespaces are not terminating due to the finalizers being taken down too earl.
$ kubectl get ns monitoring -o json
{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "annotations": {
            "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{},\"labels\":{\"owner\":\"operator\"},\"name\":\"monitoring\"}}\n"
        },
        "creationTimestamp": "2021-03-01T13:14:02Z",
        "deletionTimestamp": "2021-03-02T10:08:15Z",
        "labels": {
            "owner": "operator"
        },
        "managedFields": [
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:metadata": {
                        "f:annotations": {
                            ".": {},
                            "f:kubectl.kubernetes.io/last-applied-configuration": {}
                        },
                        "f:labels": {
                            ".": {},
                            "f:owner": {}
                        }
                    },
                    "f:status": {
                        "f:phase": {}
                    }
                },
                "manager": "kubectl-client-side-apply",
                "operation": "Update",
                "time": "2021-03-01T13:14:02Z"
            },
            {
                "apiVersion": "v1",
                "fieldsType": "FieldsV1",
                "fieldsV1": {
                    "f:status": {
                        "f:conditions": {
                            ".": {},
                            "k:{\"type\":\"NamespaceContentRemaining\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceDeletionContentFailure\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceDeletionDiscoveryFailure\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceDeletionGroupVersionParsingFailure\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            },
                            "k:{\"type\":\"NamespaceFinalizersRemaining\"}": {
                                ".": {},
                                "f:lastTransitionTime": {},
                                "f:message": {},
                                "f:reason": {},
                                "f:status": {},
                                "f:type": {}
                            }
                        }
                    }
                },
                "manager": "kube-controller-manager",
                "operation": "Update",
                "time": "2021-03-02T10:08:21Z"
            }
        ],
        "name": "monitoring",
        "resourceVersion": "2351271",
        "uid": "9709b34a-2266-4dfe-af51-b539000fc4bb"
    },
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },
    "status": {
        "conditions": [
            {
                "lastTransitionTime": "2021-03-02T10:08:21Z",
                "message": "All resources successfully discovered",
                "reason": "ResourcesDiscovered",
                "status": "False",
                "type": "NamespaceDeletionDiscoveryFailure"
            },
            {
                "lastTransitionTime": "2021-03-02T10:08:21Z",
                "message": "All legacy kube types successfully parsed",
                "reason": "ParsedGroupVersions",
                "status": "False",
                "type": "NamespaceDeletionGroupVersionParsingFailure"
            },
            {
                "lastTransitionTime": "2021-03-02T10:08:21Z",
                "message": "All content successfully deleted, may be waiting on finalization",
                "reason": "ContentDeleted",
                "status": "False",
                "type": "NamespaceDeletionContentFailure"
            },
            {
                "lastTransitionTime": "2021-03-02T10:08:21Z",
                "message": "Some resources are remaining: challenges.acme.cert-manager.io has 2 resource instances",
                "reason": "SomeResourcesRemain",
                "status": "True",
                "type": "NamespaceContentRemaining"
            },
            {
                "lastTransitionTime": "2021-03-02T10:08:21Z",
                "message": "Some content in the namespace has finalizers remaining: finalizer.acme.cert-manager.io in 2 resource instances",
                "reason": "SomeFinalizersRemain",
                "status": "True",
                "type": "NamespaceFinalizersRemaining"
            }
        ],
        "phase": "Terminating"
    }
}

More info, seems like the challenges.acme.cert-manager.io CRD did not get deleted for me.

More info, seems like the challenges.acme.cert-manager.io CRD did not get deleted for me.

Hmm, that didn't happen when I tested it. Maybe it's some race condition.
My suggestion would be that I change the scripts to this:
...

# Destroy all helm releases
"${here}/.././bin/ck8s" ops helmfile sc app!=cert-manager destroy

# Clean up namespaces and any other resources left behind by the apps
"${here}/.././bin/ck8s" ops kubectl sc delete ns **everything except cert-manager**

# Destroy cert-manager helm release
"${here}/.././bin/ck8s" ops helmfile sc app=cert-manager destroy

# Clean up cert-manager namespace
"${here}/.././bin/ck8s" ops kubectl sc delete ns cert-manager

...

What do you think about that?

@viktor-f Sounds good to me, but please test.
Also, call me a dreamer, but couldn't helmfile itself deal with this serialization? I have cleaned up and re-applied apps a few times on the SC, and it seems to me that there might be a race condition when applying too. Something like Issuer becomes ready approximately when Order is being created, instead of first waiting for Issuer to be fully ready and then creating Order(s). Would be cool if apply order was "correctly" and destroy order was the exact reverse.

@viktor-f Sounds good to me, but please test.

Ok, will do

Also, call me a dreamer, but couldn't helmfile itself deal with this serialization? I have cleaned up and re-applied apps a few times on the SC, and it seems to me that there might be a race condition when applying too. Something like Issuer becomes ready approximately when Order is being created, instead of first waiting for Issuer to be fully ready and then creating Order(s). Would be cool if apply order was "correctly" and destroy order was the exact reverse.

We have ordering that makes several applications wait until cert-manager is installed. But I see now that they do not wait for our issuers, so you might be right that there is some sort of race condition there. Examples:
https://github.com/elastisys/compliantkubernetes-apps/blob/08757291d806b7cc6f5552cae6fe2d9fc0765335/helmfile/01-applications.yaml#L55-L63
https://github.com/elastisys/compliantkubernetes-apps/blob/08757291d806b7cc6f5552cae6fe2d9fc0765335/helmfile/01-applications.yaml#L92-L100
Helmfile should also follow this order backwards when deleting. The fear I do have is that some resources might be left in namespaces because they are not directly tied to charts. E.g. we generate certificates through annotations on ingresses, so the helmchart will just destroy the ingress and let cert-manager clean up the rest. If cert-manager is deleted directly afterwards then I think there is a race condition where cert-manager have not managed to clean up everything before it's deleted.
So that's why I suggested to delete cert-manager after all namespaces have been deleted, then all resources should be gone and cert-manager should be safe to delete. Hopefully that makes sense.

Sounds good to me too @viktor-f ! We may have to give the same special treatment to other operators in the future if they start using more finalizers and then it may get ugly, but that problem can wait.
Regarding your question, @cristiklein, I think helmfile cannot handle it since it is not the owner of all resources unfortunately. :slightly_frowning_face: We install cert-manager using helm(file), but a lot of cert-manager resources are created "automatically" as @viktor-f pointed out. The Certificates and Issuers could be handled as part of helm charts, but it is not realistic to handle the Challenges, Orders and CertificateRequest, not to mention the Pods that belong to the Challenges.
As a side note, I think Challenges are an excellent example on why finalizers are needed: Cert-manager creates a Pod to serve each Challenge, so when the Challenge is deleted it is also necessary to clean up the Pod or it would stay running!

@lentzi90 I just checked and erasing and Ingress also takes down all resources created by cert-manager (see "evidence" below). So what I suggest for the cleanup script:

Nice path: Cleanup "other" Releases, cleanup issuers, cleanup cert-manager.
Brute path (for clusters that are FUBAR): Cleanup webhooks, nuke finalizers, delete all namespaces.

Would that work?
cklein@minitrue:~/compliantkubernetes-apps$ kubectl get ingress,secret,certificate,order,certificaterequest,challenges
NAME                                                  CLASS    HOSTS              ADDRESS                                PORTS     AGE
ingress.networking.k8s.io/cm-acme-http-solver-vmldw   <none>   dex.cksc.a1ck.io   <IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>   80        131m
ingress.networking.k8s.io/dex                         <none>   dex.cksc.a1ck.io   <IP_ADDRESS>,<IP_ADDRESS>,<IP_ADDRESS>   80, 443   132m

NAME                               TYPE                                  DATA   AGE
secret/default-token-scq5j         kubernetes.io/service-account-token   3      134m
secret/dex                         Opaque                                1      132m
secret/dex-grpc-ca                 kubernetes.io/tls                     2      132m
secret/dex-grpc-client-tls         kubernetes.io/tls                     2      132m
secret/dex-grpc-server-tls         kubernetes.io/tls                     2      132m
secret/dex-tls-sc9dn               Opaque                                1      131m
secret/dex-token-4pfml             kubernetes.io/service-account-token   3      132m
secret/dex-web-server-ca           kubernetes.io/tls                     2      132m
secret/dex-web-server-tls          kubernetes.io/tls                     2      132m
secret/letsencrypt-prod            Opaque                                1      131m
secret/letsencrypt-staging         Opaque                                1      131m
secret/sh.helm.release.v1.dex.v1   helm.sh/release.v1                    1      132m

NAME                                  READY   SECRET    AGE
certificate.cert-manager.io/dex-tls   False   dex-tls   131m

NAME                                                STATE     AGE
order.acme.cert-manager.io/dex-tls-6knnf-16094029   pending   131m

NAME                                               READY   AGE
certificaterequest.cert-manager.io/dex-tls-6knnf   False   131m

NAME                                                               STATE     DOMAIN             AGE
challenge.acme.cert-manager.io/dex-tls-6knnf-16094029-4270009683   pending   dex.cksc.a1ck.io   131m
cklein@minitrue:~/compliantkubernetes-apps$ kubectl delete ingress dex
ingress.networking.k8s.io "dex" deleted
cklein@minitrue:~/compliantkubernetes-apps$ kubectl get ingress,secret,certificate,order,certificaterequest,challenges
NAME                               TYPE                                  DATA   AGE
secret/default-token-scq5j         kubernetes.io/service-account-token   3      135m
secret/dex                         Opaque                                1      132m
secret/dex-grpc-ca                 kubernetes.io/tls                     2      132m
secret/dex-grpc-client-tls         kubernetes.io/tls                     2      132m
secret/dex-grpc-server-tls         kubernetes.io/tls                     2      132m
secret/dex-token-4pfml             kubernetes.io/service-account-token   3      132m
secret/dex-web-server-ca           kubernetes.io/tls                     2      132m
secret/dex-web-server-tls          kubernetes.io/tls                     2      132m
secret/letsencrypt-prod            Opaque                                1      132m
secret/letsencrypt-staging         Opaque                                1      132m
secret/sh.helm.release.v1.dex.v1   helm.sh/release.v1                    1      132m

@lentzi90 I just checked and erasing and Ingress also takes down all resources created by cert-manager (see "evidence" below).

Isn't that only true as long as cert-manager is still there? This is the whole problem AFAIK, operators (like cert-manager) must be deleted after they have already taken care of all their resources. So we cannot delete cert-manager until all Certificates, Challenges, etc are gone.
And yes, a good way to get rid of them is to delete the Ingresses, if you use annotations on the Ingresses to create them to begin with. :slightly_smiling_face: If you create the Certificate in some other way it won't go away when you delete the Ingress though.

@lentzi90 I just checked and erasing and Ingress also takes down all resources created by cert-manager (see "evidence" below). So what I suggest for the cleanup script:
Just echoing what @lentzi90 said:
Helmfile should guarantee that the ingress is deleted before cert-manager, but I don't think it can guarantee that cert-manager has time to clean up the resources connected to the ingress. That's what we were trying to say.

Thanks @lentzi90 and @viktor-f! Now I think I finally get it.
So we would need to do:

Remove Ingresses
Wait for cert-manager to reconcile
Remove cert-manager

However, 2 cannot be (easily) implemented.

Cert-manager seems to encourage this approach: https://cert-manager.io/docs/installation/uninstall/kubernetes/#namespace-stuck-in-terminating-state

I'm not sure we read it the same way. They tell you first to do this:

Before continuing, ensure that all cert-manager resources that have been created by users have been deleted.

The "namespace stuck in terminating" thing seems like some help "if you screwed up".
But maybe that is the best we can get? Anyway, wouldn't the helmfile destroy be the problematic step for us? I guess it could get stuck and then we need to timeout and try to find the problem in order to delete it, right?


Removed dependency on System.IdentityModel.Tokens.Jwt from WPS

This PR includes a custom implementation of JWT builder/writer.
The JwtBuilder is now used instead of external dependency on System.IdentityModel.Tokens.Jwt
I wanted to make the code fast and non-allocating. The System.IdentityModel.Tokens.Jwt library caches tokens. This new implementation does not. The cache has been identified as problematic in preview 1.

@GrabYourPitchforks, you might want to look into the NS2Bridge.cs for BCL text transformation APIs we would love to have in a package.


Update to depend on rom 5.0

Updated rom dependency. Took as example rom-yaml
Also updated rubocop and applied corresponding changes (outdated cops and so on). Some of decisions are quite subjective, so I'm open to revert them if it does not match preferable style on the project

@ivleonov thanks man, this is very helpful. I'm gonna update this project to use our devtools setup which will provide an up-to-date rubocop.yml, ci setup etc. so we can deal with style fixes in separate PRs.


Make Rect.size const

Tiny change to match everything else. (... and fix my build after const-ing some things)

Thank you!


Complete tour of fundamental classes in Persephone

When this is done, I can point the README.md to readthedocs.


Docs suggestion: Document support of IWeakReferenceSource

Proposal: Document support of IWeakReferenceSource
This is a very minor suggestion. Please can you document somewhere which WinRT classes support IWeakReferenceSource. Maybe it's all of them in the Windows SDK? I couldn't find it documented anywhere (via a Google search), but it's quite important (in order to avoid reference cycles) to know this when writing C++ code which uses WinRT classes.

I see that both IWeakReference and IWeakReferenceSource docs have broken methods links, should fix that as well.

Hi, @benstevens48, and thanks for filing this issue.
The subject matter experts say that IWeakReference support is on by default for Windows Runtime types in C++/WinRT, WRL, and C++/CX. For example, you can find more info in Weak references in C++/WinRT. So, yes, you're right that it's our intent for all Windows Runtime types to support IWeakReferenceSource.
I'll see whether I can find a good place in the docs to make this clearer, so you can leave that with me.
@stevewri, I made some changes yesterday, and it looks like those broken links are now resolved (the links were redundant, and they're now gone).
-Steve

@stevewhims - OK, thanks for that. Technically, that means types I author myself will support it, but that doesn't guarantee that types in the Windows SDK do. In practice I suppose it means they probably do, but it would be nice if it was guaranteed.

@benstevens48, yes, I'd say that's an accurate assessment at present. We realize it would be better if it were guaranteed, or if there were a way to determine support at build time. Incidentally, Windows.UI.Compositon and Windows.Devices.Input.PenDevice are examples of exceptions: that is, namespaces where weak reference support is not on for those types. For such types, it's possible to run into a winrt::hresult_no_interface exception in rare cases--for example, when attempting to register an auto-revoke event handler. So it'd be necessary to register and revoke manually.

Closing this for now, let us know if anything else is needed!


Add instances of CanCatch, CanHandleError, CanRecover, FromFuture, Fx, FxCtor and ToFuture with Sync, Async and Effect.

Summary
Add instances of CanCatch, CanHandleError, CanRecover, FromFuture, Fx, FxCtor and ToFuture with Sync, Async and Effect.
Version
2.0.0-beta11
Description
No response

Remove ToFuture from this ticket because

in Cats Effect 3, it's hard to do it without having a specific effect (e.g. IO) or hard to test
It was done with cats.effect.std.Dispatcher, but the test failed with a timeout.

ToFuture is not so important when using Cats Effect as it could be handled like

foo() takes Int and returns F[A]
bar() takes Future[A] and returns Future[B]

val result = for {
    a <- foo(n) // F[A]
    b <- FromFuture[F].toEffect(
           bar(Future(a)) // Future[B]
         ) // F[B]
  } yield b
result // F[B]


Add icon for PowerShell folders

Description

Added an icon for folders with PowerShell scripts (powershell/, ps/, and also ps1/).
Closes #620

Contribution Guidelines

[X] By creating this pull request, I acknowledge that I have read the Contribution Guidelines for this project.
[X] I have read the Code Of Conduct and promise to abide by these rules

Yeah, I noticed that before.
I think it would make more sense to move it to the left so it keeps 1px to the left and 1px to right.

Moving the PS logo to the right would look a bit too uneven (in my opinion).

But hey, whatever you find best I'll change right now.

It might also be worth to make the motif a bit bigger and try to use 1px (at 16x16) witdh for the negative-spaced lines inside the motif (and align those to the grid.
Since each white "square" here represents 1 pixel at 16x16, this will render blurry or even almost invisible:

Perhaps it works better this way? Now the PS1 icon is aligned with the folder.

Perhaps it works better this way? Now the PS1 icon is aligned with the folder.

Read https://github.com/material-extensions/vscode-material-icon-theme/blob/main/CONTRIBUTING.md#tips-for-designing-pixel-perfect-icons

Perhaps it works better this way? Now the PS1 icon is aligned with the folder.

Read https://github.com/material-extensions/vscode-material-icon-theme/blob/main/CONTRIBUTING.md#tips-for-designing-pixel-perfect-icons

Got it, I forgot about that part. I'll adjust the icon right now.

I would make the powershell logo a bit bigger, to be able to fit the ">_" into the grid, like:

I think this way the icon is a bit easier to recognize when looking it at 16x16 pixels:
(yours left, mine right)

Let me know what you guys think

I would make the powershell logo a bit bigger, to be able to fit the ">_" into the grid, like:

I think this way the icon is a bit easier to recognize when looking it at 16x16 pixels:
(yours left, mine right)

Let me know what you guys think

Thanks for the suggestions. I would prefer that 🙂

I would make the powershell logo a bit bigger, to be able to fit the ">_" into the grid, like:

I think this way the icon is a bit easier to recognize when looking it at 16x16 pixels:
(yours left, mine right)
Let me know what you guys think

Yeah, to be fair that looks better.

Hi!
 Your last version was good enough already! Nice work!

Made the ">" sign a bit thicker (same width as the "_", for consistency and visibility)
Removed the background behind the logo, to generate more contrast with the the ">_" (and to make the icon a few kb smaller)
Optimized the files by running bunx svgo ...

I think we are ready to go now :)

@lucas-labs 🗿

This transparent area does not give me peace...

This transparent area does not give me peace...

Yeah, I noticed it after pushing 😅. But although my OCD went crazy, truth is it's like 1/5 of a pixel, it won't change anything when rendered at 16 pixels haha.

This transparent area does not give me peace...

Yeah, I noticed it after pushing 😅. But although my OCD went crazy, truth is it's like 1/5 of a pixel, it won't change anything when rendered at 16 pixels haha.

It will change.
I recommend filling that transparent area with blue color

It will change.

Can you see the difference?

It will change.

Can you see the difference?

Of course, the black pixel is visible here

Of course, the black pixel is visible here

I was wondering if you can you see the difference in VS Code's file explorer?

Of course, the black pixel is visible here

I was wondering if you can you see the difference in VS Code's file explorer?

Yes, the black pixel is really visible there

You have great eyes haha, I can barely see the whole icon hahah
I'll send a fix later this week

You have great eyes

*display


Error: Cannot send commands in PubSub mode after unsubscribing

Issue
Given the following test scenario.
(async function(){
    const redis = require('redis').createClient();
    await redis.connect();
    await redis.subscribe('abc', Function.prototype);
    await redis.unsubscribe();
    await redis.quit();
})();

I get this error:
Error: Cannot send commands in PubSub mode
      at RedisCommandsQueue.addEncodedCommand (node_modules\redis\dist\lib\commands-queue.js:82:35)
      at C:\Users\Guilherme\Projetos\nodecaf\redis\node_modules\redis\dist\lib\client.js:168:83
      at RedisSocket.quit (node_modules\redis\dist\lib\socket.js:80:19)
      at RedisClient.QUIT (node_modules\redis\dist\lib\client.js:167:71)
      at Context.<anonymous> (test\spec.js:15:21)
      at processTicksAndRejections (internal/process/task_queues.js:93:5)

Apparently, the client is only leaving the PubSub mode some time after the Promise returned from redis.unsubscribe() resolves.
Is this a bug, or should I rely on some event to know when I'm out of Pub/Sub mode?

Environment

Node.js Version: v14.5.0
Redis Server Version: Redis server v=6.0.9 malloc=jemalloc-5.1.0 bits=64 build=12c354e6793cb936
Platform: Windows 10
NodeRedis Version: 4.0.0-rc.1

The problem is now closed, but how can we write without reporting an error ?

I still have this error with version 4.3.1
I used promises how described in the migration guide but I have to set legacyMode=true to avoid the error.
this is the stack:
(node:54620) UnhandledPromiseRejectionWarning: Error: Cannot send commands in PubSub mode
at RedisCommandsQueue.addCommand (/developer/github/YouNeed/yn-ws/node_modules/@redis/client/dist/lib/client/commands-queue.js:72:35)
at RedisSocket.socketInitiator (/developer/github/YouNeed/yn-ws/node_modules/@redis/client/dist/lib/client/index.js:333:81)
at RedisSocket._RedisSocket_connect (/developer/github/YouNeed/yn-ws/node_modules/@redis/client/dist/lib/client/socket.js:138:77)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async Commander.connect (/developer/github/YouNeed/yn-ws/node_modules/@redis/client/dist/lib/client/index.js:166:9)


sbt 1.0

[ ] Cross-build seals-plugin for sbt 1.0.
[ ] Figure out if we can upgrade the whole build to use sbt 1.0.

According to https://github.com/sbt/sbt/wiki/sbt-1.x-plugin-migration, all the plugins we use are available for sbt 1.0.

Fixed in 0a0906e and e31a305.


Update kcp/README

The new changes in ./kcp/README.md allow users to be able to use "login.sh"

@Roming22 please see if it's good to go.

@Roming22 please see it's good to go.

We should not reference the infra repository (an internal RH repository) in this repository. Please create/update the README.md in the infra repository if necessary.

I see. That's my fault for forgetting to remove the reference to that script when I moved it to another repo. https://github.com/openshift-pipelines/pipelines-service/pull/60 should take care of it.
If you agree you can close this PR.


Asking for help, not issue in the product itself

I am currently serving my website using Nginx and Gunicorn. In particular, Nginx is serving static files and Gunicorn is serving rest-api. This is my current Nginx configuration:
worker_processes 2;

user nobody nogroup;
# 'user nobody nobody;' for systems with 'nobody' as a group instead
error_log  /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
  worker_connections 1024; # increase if you have lots of clients
  accept_mutex on; # set to 'on' if nginx worker_processes > 1
  # 'use epoll;' to enable for Linux 2.6+
  # 'use kqueue;' to enable for FreeBSD, OSX
}

http {
  include mime.types;
  # fallback in case we can't determine a type
  default_type application/octet-stream;
  access_log /var/log/nginx/access.log combined;
  sendfile on;
  proxy_connect_timeout       300;
  proxy_send_timeout          300;
  proxy_read_timeout          300;
  send_timeout                300;
  upstream app_server {
    # fail_timeout=0 means we always retry an upstream even if it failed
    # to return a good HTTP response

    # for UNIX domain socket setups
    # server unix:/tmp/gunicorn.sock fail_timeout=0;

    # for a TCP configuration
    server <IP_ADDRESS>:8181 fail_timeout=0;
  }

  server {
    listen 80;
    listen [::]:80;
    server_name  www.miralytics.social;
    return 301 https://www.miralytics.social$request_uri;
}

  server {
    # if no Host match, close the connection to prevent host spoofing
    listen 443 default ssl;
    ssl_certificate      /certificates/fullchain1.pem;
    ssl_certificate_key  /certificates/privkey1.pem;
    server_name www.miralytics.social;
    gzip on;
gzip_vary on;
gzip_types text/plain text/html text/xml text/css application/x-javascript image/png image/jpeg application/javascript application/octet-stream application/json;
gzip_proxied any;
gzip_http_version 1.1;
gzip_min_length 0;
gzip_comp_level 9;
gzip_buffers 16 8k;
    proxy_connect_timeout       600;
  proxy_send_timeout          600;
  proxy_read_timeout          600;
  send_timeout                600;
    keepalive_timeout 5;

    # path for static files
    root /home/edge7/UIBackend/dist;

    location ~ ^/(images|javascript|js|css|flash|media|static)/  {
      root    /home/edge7/UIBackend/dist;
      expires 1d;
    }

    location /auth/register {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;
      # we don't want nginx trying to do something clever with
      # redirects, we set the Host: header above already.
      proxy_redirect off;
      proxy_pass http://localhost:8181;
    }
    location /auth/login {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header Host $http_host;
      # we don't want nginx trying to do something clever with
      # redirects, we set the Host: header above already.
      proxy_redirect off;
      proxy_pass http://localhost:8181;
    }
    location / {
      # checks for static file, if not found proxy to app
      try_files $uri @proxy_to_app;
    }

    location @proxy_to_app {
      proxy_pass http://localhost:8181;
    }

  add_header Cache-Control no-cache; #(no cache for testing reasons)

  }

How can I adjust it to use prerender?

It looks like you'd just want to add the prerender config to your location / like:
    location / {
        proxy_set_header X-Prerender-Token YOUR_TOKEN;

        set $prerender 0;
        if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest|slackbot|vkShare|W3C_Validator") {
            set $prerender 1;
        }
        if ($args ~ "_escaped_fragment_") {
            set $prerender 1;
        }
        if ($http_user_agent ~ "Prerender") {
            set $prerender 0;
        }
        if ($uri ~* "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") {
            set $prerender 0;
        }

        #resolve using Google's DNS server to force DNS resolution and prevent caching of IPs
        resolver <IP_ADDRESS>;

        if ($prerender = 1) {

            #setting prerender as a variable forces DNS resolution since nginx caches IPs and doesnt play well with load balancing
            set $prerender "service.prerender.io";
            rewrite .* /$scheme://$host$request_uri? break;
            proxy_pass http://$prerender;
        }

        # checks for static file, if not found proxy to app
        try_files $uri @proxy_to_app;
    }


Doc fixes. Closes #3815

Closes #3815

Thanks for the PR @Conor0Callaghan! Awesome to have you with us! We'll review the PR ASAP.


javascript lesson 102:

please there is an issue with that lesson,  i can't continue to the next challenge .. It seems there's a problem with it..

Thank you for reporting this issue.
This is a standard message notifying you that this issue seems to be a request for help. Instead of asking for help here, please click the "Get Help" button on the challenge on freeCodeCamp and choose the "Ask for help" option, which will help you create a question in the right part of the forum. Volunteers on the forum usually respond to questions within a few hours and can help determine if there is an issue with your code or the challenge's tests.
If the forum members determine there is nothing wrong with your code, you can request this issue to be reopened.
Thank you and happy coding.


The div is trapped in endless loop and the slider is always jumping at certain position when I use Infinite Scroll

Describe the bug
I'm trying to test Infinite Scroll example with my data, but it behavior is unexpected, the blow GIF should show a explicit case. It's trapped in endless loop and the slider is always jumping, but I can't find problem via debug info. I guess it may I wrongly use it or edge case problem.
Your minimal, reproducible example
https://stackblitz.com/edit/stackblitz-starters-eldpph?file=src%2FApp.tsx
Steps to reproduce
Just drag slider
Expected behavior
normal display
How often does this bug happen?
Every time
Screenshots or Videos

Platform
OS: Linux
Browser: Chrome
tanstack-virtual version
3.0.0-beta.54
TypeScript version
5.1.6
Additional context
No response
Terms & Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I understand that if my bug cannot be reliable reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

@MJRT have you found a solution for this or the cause of it? I have the same thing.

Thanks @piecyk , it got rid of the issue above, but still have some jumping of the scrollbar when I scroll all the day down and keep it there. The biggest problem though now is that the scroll is jittery,  especially when it stops. Probably because it isn't "anchoring" now... I'll try to implement it in another way, have one of my scrolls working properly. Also I believe 'Alpha'  didn't  have this issue, might just downgrade to that  as well.

Looks like the parent of the items need the height to be set, I using a tutorial/code from a V2 and implemented the changes for V3. Somewhere in there I lost that. So for the parent of the items:
height: ${rowVirtualizer.getTotalSize()}px,

Thanks @piecyk , it got rid of the issue above, but still have some jumping of the scrollbar when I scroll all the day down and keep it there. The biggest problem though now is that the scroll is jittery, especially when it stops.

That sound not right, it would mean that the height changes when you scroll down, are your rows dynamic? If they have different size from what you estimated, did you add data-index ref={virtualizer.measureElement} ?

Probably because it isn't "anchoring" now... I'll try to implement it in another way, have one of my scrolls working properly. Also I believe 'Alpha' didn't have this issue, might just downgrade to that as well.

hmm don't quite understand what do you mean by anchoring, but we never had this feature.

@piecyk fixed it, comment above. Thank you for your help!


Azurite doesn't support filter parameter when use query table operation on table.

Error Description:
Azurite doesn't support filter parameter and return all tables when use query table operation on table.
To Reproduce:
The demo code is shown as following:
import * as assert from "assert"
import { TableServiceClient, odata } from "@azure/data-tables";

async function main() {
  const tableName = "mytable";
  const tableServiceClient = TableServiceClient.fromConnectionString("<Azurite-https-connection-string>");
  await tableServiceClient.createTable(tableName);
  var queryTableResults = tableServiceClient.listTables({
    queryOptions: { filter: odata`TableName eq ${tableName}` }
  });
  for await (const table of queryTableResults) {
    assert.ok(table.tableName);
  }

  await tableServiceClient.deleteTable(tableName);
  queryTableResults = tableServiceClient.listTables({
    queryOptions: { filter: odata`TableName eq ${tableName}` }
  });
  for await (const table of queryTableResults) {
    assert.ok(!table.tableName);
  }
}

main().catch((err) => {
  console.error(err);
});

Error Track:
In Azurite, the statement of the method queryTable lacks filter parameter. the code is shown as following:
https://github.com/Azure/Azurite/blob/table/src/table/persistence/LokiTableMetadataStore.ts#L153

Expected Behavior:
Azurite support filter parameter when use query table operation on table.
@jongio for notification.

Thanks for reporting. That's a missing feature for Table.

Thanks for reporting. That's a missing feature for Table.

QueryEntities already supports filter parameter, refer to https://github.com/Azure/Azurite/blob/7ffe161cdb8be3976f1a64c1eb0c12e59304b77d/src/table/handlers/TableHandler.ts#L466

azsdk-azurite

Additional info - Filter parameters still not working correctly.  Consider the following image - filtering by PartitionKey works, filtering by string data works, filtering by INT does not work.  To reproduce, simply create a table record with an Int32 and attempt to retrieve just that record.  Attached image, filter URL looks correct in all cases.  Works in classic azure storage emulator.  I'm unable to migrate to Azurite for local dev.  On version 3.14

@jroselle I didn't reproduce the question. I create a table record with an Int32 and retrieve the record by Int successfully. Can you tell me the detailed steps to reproduce the question？

Interesting. I will build a demo app with smallest repro I can and upload it.
Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: Ziheng Zhou(MSFT) @.>
Sent: Sunday, August 1, 2021 11:27:12 PM
To: Azure/Azurite @.>
Cc: Jonathan Roselle @.>; Mention @.>
Subject: Re: [Azure/Azurite] Azurite doesn't support filter parameter when use query table operation on table. (#675)
@jrosellehttps://github.com/jroselle I didn't reproduce the question. I create a table record with an Int32 and retrieve the record by Int successfully. Can you tell me the detailed steps to reproduce the question？
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/Azure/Azurite/issues/675#issuecomment-890757948, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHYM6ZYECPR34QGLQ42FNLLT2Y3EBANCNFSM4WCHQENQ.

Program.txt
Repro.zip
I've attached a very simple repro.  Works in classic storage emulator, does not work in azurite.  I've tried the docker container, the vs plugin and npm -g azurite.  all return same error, perhaps you have a newer secret version?  I'm using 3.14.0.
Here's the log from Azurite docker image:

@jroselle Ok, I have reproduced your question. The reason is you have a lower Azurite version. You can get the latest version Azurite v3.14.1 and have a try.

Thank you, I've confirmed that cloning the repository and building works (npm ci failed but I was able to work around that.)  Is there a timeline on when the official release will come through so I can get the full vscode/docker integration?

@XiaoningLiu something is wrong when use npm ci. What's more, do you know when the official release will come through？


add flix registry smart contract, to store interaction templates on chain

This PR is to add flix registry smart contract, to store interaction templates on chain.
Storing Flow interaction templates on-chain ensures immutability, transparency, and direct access within the blockchain ecosystem, enhancing trust and efficiency.

Based on @bjartek's suggestion removed enums from the smart contract, and changed them to plain strings to help deserialization.
Also deployed the new version to testnet.


How did you train the RRDB_PSNR_x4 ?

Hi

Modeds
Short Description

RRDB_ESRGAN_x4.pth
final ESRGAN model we used in our paper

RRDB_PSNR_x4.pth
model with high PSNR performance

As far as I know, you use the RRDB_PSNR_x4 as the pretrained model for your ESRGAN. May I ask how did you get this pretrained model? Is it the same architecture as the ESRGAN and the only difference is using different pretrained model?
Best Regards,

same arch as the ESRGAN but with the L1 loss to train the model.

May I ask what's the pretrained model for PSNR-oriented ESRGAN?

May I ask what's the pretrained model for PSNR-oriented ESRGAN?

I think I know what's the pretrained model. But how can I choose the model, because the PSNR is going down when training the model.

RRDB_PSNR_x4.pth is the PSNR-oriented model.
Did you mean how to choose the model when fine-tuning with gan loss?
Just from the image visual quality of the results. It is a pity that there is no good way.

How to modify the original loss to L1 loss for training the RRDB_PSNR_x4?  @xinntao
Much thx.

@xiaozhi2015
Using L1 loss is simple with configuration "pixel_criterion": "l1".
L1 loss is in https://github.com/xinntao/BasicSR/blob/c69bf49f0185f9d02f379dffc7b3b88265010b31/codes/models/SR_model.py#L29-L30

Thx xintao. And what value should I set "pixel_criterion" when training RRDB_ESRGAN_x4? It seems that it only include "l1" and "l2".  @xinntao

Take a look at this file

@xinntao
In this file, "pixel_criterion" is already set to "l1". When training "RRDB_PSNR_x4", "pretrain_model_G" shoud be set to "null"; when training "RRDB_ESRGAN_x4", "pretrain_model_G" shoud point to the previous "RRDB_PSNR_x4" model. Anything else should I change, especially the "pixel_criterion" value?

pixel_criterion l1
Using the default value is OK.

@xinntao
Does training RRDB_PSNR_x4 include generator only, not include discriminator? Like "python train.py -opt options/train/train_SRResNet.yml", but need set "which_model_G：RRDBNet" , "nb: 23" and so on.

Yes, training RRDB_PSNR_x4 only includes the generator.

Hi，Do you try train the  pretrained model (RRDB_PSNR_x4 ) with l2 loss ? Which is better?

In practice, can we use just the following settings?
  pixel_criterion: l1
  pixel_weight: !!float 1e-2
  feature_criterion: l1
  feature_weight: 0
  gan_type: ragan  # gan | ragan
  gan_weight: 0

Should we set the pixel_weight as 1? @xinntao


Index out of range error

I'm using 0.7.2 and every so often I see the following error being logged:
Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/__init__.py", line 251, in wait
    self._process_packets()
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/__init__.py", line 276, in _process_packets
    for engineIO_packet in self._transport.recv_packet():
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/transports.py", line 158, in recv_packet
    packet_text)
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/parsers.py", line 96, in parse_packet_text
    packet_type = int(get_character(packet_text, 0))
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/symmetries.py", line 33, in get_character
    return chr(get_byte(x, index))
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/symmetries.py", line 29, in get_byte
    return indexbytes(x, index)
IndexError: index out of range

It then is disconnected (but the events for disconnection never fire) and I then have to restart the service.
It looks kinda similar to #129 but not exactly the same.

Any ideas here?
Still seeing this issue intermittently...

Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/init.py", line 251, in wait
self._process_packets()
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/init.py", line 276, in _process_packets
for engineIO_packet in self._transport.recv_packet():
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/transports.py", line 158, in recv_packet
packet_text)
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/parsers.py", line 96, in parse_packet_text
packet_type = int(get_character(packet_text, 0))
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/symmetries.py", line 33, in get_character
return chr(get_byte(x, index))
File "/usr/local/lib/python3.4/dist-packages/socketIO_client/symmetries.py", line 29, in get_byte
return indexbytes(x, index)
IndexError: index out of range

@l33klin @robertlevy11 do you guys know if there is some other library to use? Looks like this one is no longer active. :(

I don't konw, i have the same error. I need a connection on the localhost from my server. On the Raspberry Pi it works without problems

See here:
Install the pip package: socketIO-client-nexus
For me it worked!
https://github.com/nexus-devs/socketIO-client-2.0.3

+1, I have the same problem with this client. After a few hours this error shows up.

I am facing the same issue with Centos 6.9, ain't there a solution yet?
this works fine with windows, only the error comes when testing with the centos

@Nixellion how did you fix it?

@nm3594 As I mention in my previous post, I set it to only listen for 1 hour, then reconnect.
RESTART_DELAY = 60  # in minutes
socketIO.wait(seconds=RESTART_DELAY * 60)  # conversion for minutes

@Nixellion Will that code restart it or did you have to add extra?

@Nixellion the application I use must response less than 1 second. :(

@Roledenez Oh no, this code makes the client RESTART and RECONNECT each hour. It listens for an hour, responding instantly for any incoming message, then it restarts itself and listens for another hour. Or whatever time you set. It responds instantly.

I'm also having this issue:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/socketlib.py", line 71, in _receive_events_thread
    self.socket_io.wait()
  File "/usr/local/lib/python3.7/site-packages/socketIO_client/__init__.py", line 251, in wait
    self._process_packets()
  File "/usr/local/lib/python3.7/site-packages/socketIO_client/__init__.py", line 276, in _process_packets
    for engineIO_packet in self._transport.recv_packet():
  File "/usr/local/lib/python3.7/site-packages/socketIO_client/transports.py", line 158, in recv_packet
    packet_text)
  File "/usr/local/lib/python3.7/site-packages/socketIO_client/parsers.py", line 96, in parse_packet_text
    packet_type = int(get_character(packet_text, 0))
  File "/usr/local/lib/python3.7/site-packages/socketIO_client/symmetries.py", line 33, in get_character
    return chr(get_byte(x, index))
  File "/usr/local/lib/python3.7/site-packages/socketIO_client/symmetries.py", line 29, in get_byte
    return indexbytes(x, index)
IndexError: index out of range

I managed to quickly reproduce it by spawning a lot (1000) threads, all connecting to the same server, and for each connection go into a loop and do emit.

At the end my problem was that I was using AWS. Now with the same library and the same code I don't have any problem using Google Cloud Compute Engine

@nicolamarinello what kind of problem you had with AWS? exactly same than in description? Did you use ALB/ELB? Have you seen any 502 rest failures? We have seen also problems with AWS with ALB but mostly with REST..

I have the same problem with socketIO_client_nexus 0.7.6.
I can happen as early as 5 seconds after the connection is open.
The server is a nodeJS server using some implementation of socketIO 2.
One thing I did change is putting the server behind a load balancer. I don't recall having this issue with a single instance of the server.


Fix handling of rejected promises when attempting to close the server

Current:
function bail(e?: Error) {
  try {
    returnserver.close();
  } catch (err) {
    console.warn('unable to close vite server');
  }

  throw e;
}

If this call to server.close() throws, then it does not get caught by the catch here as the promise is not awaited. For this catch to handle the rejection of this promise, the server.close() call needs to be awaited

Thanks!


Update to v<IP_ADDRESS>

Entity parameters are now sorted by type in the script editor GUI.
Fixed an issue where proxies would become corrupted and unresolvable.
Updated Newtonsoft JSON version to resolve a security alert.

Bugfix via #69


[HUDI-3445] Supporting Clustering Command Based on Call Procedure Command for Spark SQL

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

What is the purpose of the pull request
Supporting clustering command for SparkSQL
Brief change log

Add RunClusteringProcedure and ShowClusteringProcedure
Add test case in TestCallProcedure

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@XuQianJin-Stars  do you have some time to review this pr, thanks,  may be we can add this to HUDI-3161

@XuQianJin-Stars  do you have some time to review this pr, thanks,  may be we can add this to HUDI-3161

@huberylee Thank you very much for contributing， pls fixed the CI build

@XuQianJin-Stars do you have some time to review this pr, thanks, may be we can add this to HUDI-3161

well, Let me  review this pr.

hi @huberylee pls fixed the CI build.

@xiarixiaoyao @XuQianJin-Stars Sorry for the late reply, I will fix it wright now.

@hudi-bot run azure

This PR looks good overall. Thanks @huberylee

@huberylee can you please add a description for this PR?


failed to connect to socket after reboot

Hi, guys, yabai works perfectly before I reboot my laptop
but now as following
➜  ~ brew services restart yabai

Stopping `yabai`... (might take a while)
==> Successfully stopped `yabai` (label: homebrew.mxcl.yabai)
==> Successfully started `yabai` (label: homebrew.mxcl.yabai)
➜  ~ yabai -m window --toggle split
yabai-msg: failed to connect to socket..

and the top bar won't show now
What I have tried, but wont work:

reinstall yabai from brew

killall Dock

rm /tmp/yabai_${USER}.lock file

any ideas ?

I found the problem, in mission control if "displays have separate spaces" is disabled, the problem above will happen


Update outdated guidance for adding on-premises MDM application

Customers were confused by previous guidance, as it was outdated/incorrect.

Thank you @illfated :D.

Just as FYI, we also have a CI item open (116208) for an entire review of this page, but this one paragraph couldn't wait, as it what caused some customer grief. @JarrettRenshaw is aware of the CI item.

Very well, thanks for the heads up. Based on your comment, I will leave it to you and your colleagues to decide how fast you want to merge this content. The change in itself is good to go, as far as I am concerned.

#sign-off

Perfect, thanks again for this help. This one should be merged as soon as we can.


Handle bad sad diagnostic from compiler in tooling

Description:
Introduced a separate diagnostic for unhandled exceptions from the compiler in https://github.com/ballerina-platform/ballerina-lang/pull/31934
Steps to reproduce:
Affected Versions:
OS, DB, other environment details and versions:
Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

We are closing this issue due to inactivity. If you need further assistance or have additional input, feel free to reopen it. Thank you for helping us maintain a relevant and focused issue list!


Keyboard manager fails every time you reboot the computer

Microsoft PowerToys version
0.78.0
Installation method
PowerToys auto-update
Running as admin
Yes
Area(s) with issue?
Keyboard Manager
Steps to reproduce
Keyboard manager fails every time you reboot the computer,But the Keyboard manager switch is on,must be manually turned off and on again for customized key mapping to take effect.

✔ Expected Behavior
Key mapping will automatically take effect every time you boot up your computer.
❌ Actual Behavior
Key mapping does not take effect every time you turn on your computer, you need to manually turn off and then turn on the function to take effect.
Other Software
No response

/dup #24494


chore(protocol): index more events

Based on a report from code4rena - https://github.com/code-423n4/2024-03-taiko-findings/blob/main/data/0x11singh99-Q.md#n-11-event-is-missing-indexed-fields

Index event fields make the field more quickly accessible to off-chain tools that parse events. However, note that each index field costs extra gas during emission, so it's not necessarily best to index the maximum allowed per event (three fields). Each event should use three indexed fields if there are three or more fields, and gas usage is not particularly of concern for the events in question. If there are fewer than three fields, all of the fields should be indexed.

after a second thought, maybe we don't need this PR :D


flow-bin 0.71.0 fails to start

$ npx flow version
Flow, a static type checker for JavaScript, version 0.71.0

$ npx flow
Launching Flow server for /Users/craigbeck/dev/premise-frontend-v3
Spawned flow server (pid=5171)
Logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.log
Monitor logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.monitor_log
Lost connection to the flow server (3 retries remaining)
Launching Flow server for /Users/craigbeck/dev/premise-frontend-v3
Spawned flow server (pid=5219)
Logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.log
Monitor logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.monitor_log
Lost connection to the flow server (2 retries remaining)
Launching Flow server for /Users/craigbeck/dev/premise-frontend-v3
Spawned flow server (pid=5237)
Logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.log
Monitor logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.monitor_log
Lost connection to the flow server (1 retry remaining)
Launching Flow server for /Users/craigbeck/dev/premise-frontend-v3
Spawned flow server (pid=5282)
Logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.log
Monitor logs will go to /private/tmp/flow/zSUserszScraigbeckzSdevzSpremise-frontend-v3.monitor_log
Lost connection to the flow server (0 retries remaining)
Out of retries, exiting!

and contents of logs:
[2018-04-26 22:18:33.136] argv=/Users/craigbeck/dev/premise-frontend-v3/node_modules/flow-bin/flow-osx-v0.71.0/flow start --temp-dir /tmp/flow /Users/craigbeck/dev/premise-frontend-v3
[2018-04-26 22:18:33.192] Watching paths:
	/Users/craigbeck/dev/premise-frontend-v3
	/Users/craigbeck/dev/premise-frontend-v3/packages
[2018-04-26 22:18:33.197] Initializing Server (This might take some time)
[2018-04-26 22:18:33.197] executable=/Users/craigbeck/dev/premise-frontend-v3/node_modules/flow-bin/flow-osx-v0.71.0/flow
[2018-04-26 22:18:33.197] version=0.71.0
[2018-04-26 22:18:33.201] Parsing
Unhandled exception: Unix.Unix_error(Unix.EPIPE, "write", "")
Raised at file "src/core/lwt.ml", line 3008, characters 20-29
Called from file "src/unix/lwt_main.ml", line 42, characters 8-18
Called from file "src/server/server.ml", line 257, characters 6-54

Exception: Unix.Unix_error(Unix.EPIPE, "write", "")
Potential backtrace:
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/procs/worker.ml", line 96, characters 4-37
Unix.Unix_error(Unix.EPIPE, "write", "")
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/utils/sys/timeout.ml", line 28, characters 12-18
Exception: Unix.Unix_error(Unix.EPIPE, "write", "")
Potential backtrace:
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/procs/worker.ml", line 96, characters 4-37
Unix.Unix_error(Unix.EPIPE, "write", "")
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/utils/sys/timeout.ml", line 28, characters 12-18
Exception: Unix.Unix_error(Unix.EPIPE, "write", "")
Potential backtrace:
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/procs/worker.ml", line 96, characters 4-37
Unix.Unix_error(Unix.EPIPE, "write", "")
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/utils/sys/timeout.ml", line 28, characters 12-18
Exception: Unix.Unix_error(Unix.EPIPE, "write", "")
Potential backtrace:
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/procs/worker.ml", line 96, characters 4-37
Unix.Unix_error(Unix.EPIPE, "write", "")
Raised by primitive operation at file "unix.ml", line 257, characters 7-34
Called from file "hack/utils/marshal_tools.ml", line 144, characters 6-67
Called from file "hack/utils/marshal_tools.ml", line 157, characters 4-63
Called from file "hack/utils/sys/timeout.ml", line 28, characters 12-18

monitor log:
Apr 26 22:18:33.091 [info] argv=/Users/craigbeck/dev/premise-frontend-v3/node_modules/flow-bin/flow-osx-v0.71.0/flow start --temp-dir /tmp/flow /Users/craigbeck/dev/premise-frontend-v3
Apr 26 22:18:33.091 [info] Creating a new Flow server
Client and server are different builds. Flow server is out of date. Exiting

@gabelevi rip

Is it possible that you have something else competing to try to launch a different Flow version? Maybe you have Nuclide or VSCode running and one of them is trying to start a different version?

The issue was fixed a while ago.


Added live preview from dynajoe/vscode-pegjs-live

I would want @dynajoe to sign off on this before I even start looking at it.  vscode-pegjs-live does not have a LICENSE file.

His packag.json states MIT license, but I'm all in with getting him on board. Just needed this live functionality for Peggy, so sharing the integration was quite an afterthought.
Not too experienced with this corner of GitHub...will he be informed automatically by the mention, or should I get in contact separately?

The @ mention should have notified.  Give them a few days.

Seeing no comment from @dynajoe, we're going to go ahead.  Here are a list of things that need to get fixed, which I'm happy to do if you don't have time:

Rebase on top of #48, which I'm about to land.  This will cause a few issues and fix a few issues.
Several of the references to NodeJS.Timer need to be NodeJS.Timeout.
Ensure lint is clean.  Probably means adding at least one more ignore to memFs.
Look at debounceExecution and see if we need two debounce implementations.
Add copyright notice to preview.ts
Move CHANGELOG to same format as peggy (I'll do this one for sure)
Add more info to README on how live preview works.  At least talk about right-clicking, and maybe cmd-shift-p to launch
Add screenshot of preview to README

@hildjj I added you to my repo. Alas, I'm pretty busy with a new job and life currently, trying to stay open for possibly upcoming questions, though.


Android player not playing MP3 that is <1sec

Having trouble  playing a sub 1 second MP3 file on android. The file is from a zip containing iOS system sounds and is called Tink.mp3 (zip file linked below)
This file will play fine in iOS however does not play in Android. I opened the file in Audacity and extended the length to 1 second by adding silence to the end and then it plays as expected on Android. This makes me think it has to do with the length.
Link to iOS sounds zip file (link is in first post attachment)
https://forum.xda-developers.com/showthread.php?t=1165751

me too

Same.

Have you tried using LOW_LATENCY mode? Please provide some code so I can see the exact configurations you are using. Also a link directly to the exact mp3 file you are talking about is helpful.

Have you tried using LOW_LATENCY mode? Please provide some code so I can see the exact configurations you are using. Also a link directly to the exact mp3 file you are talking about is helpful.

Orginal - Tink.mp3
Modified - Tink.mp3
playMp3('assets/sounds/Tink.mp3').whenComplete(() {});
Future playMp3(String asset) async {   AudioPlayer _audioPlayer = AudioPlayer();   AudioPlayer.logEnabled = logEnabled;   final file = new File('${(await getTemporaryDirectory()).path}/temp.mp3');   await file.writeAsBytes((await loadAsset(asset)).buffer.asUint8List());   await _audioPlayer.play(file.path, isLocal: true);   _audioPlayer.onPlayerCompletion.listen((event) {     print('audio complete');     _audioPlayer.release();   }); }

This issue is outdated or uses a very old version of the project, could you please test if this happens on newer versions? And in case that this still happens, open a new issue?

same issue on 0.19.1, use LOW_LATENCY mode cause double sound .


reset error on going back

Details

Fixed Issues

$ https://github.com/Expensify/App/issues/49266
PROPOSAL: https://github.com/Expensify/App/issues/49266#issuecomment-2353430540
Tests

Go to Expensify
Log in with a new Gmail account
Click on FAB - New workspace
Enable "Accounting" in the "More features" page.
Navigate to "Accounting"
Connect to NetSuite and upgrade the workspace to Control when asked
Wait for the sync to finish
Navigate to Accounting - Import - Custom Segments/records
Click on the "Add custom segments/record
Choose segment or record and click on the "Next" button
Type in any custom record name and click on the "Next" button
Click on the "Next" button without filling the ID
Click on the "<" button on the RHP
The error message doesn't appear

Offline tests

QA Steps

Go to Expensify
Log in with a new Gmail account
Click on FAB - New workspace
Enable "Accounting" in the "More features" page.
Navigate to "Accounting"
Connect to NetSuite and upgrade the workspace to Control when asked
Wait for the sync to finish
Navigate to Accounting - Import - Custom Segments/records
Click on the "Add custom segments/record
Choose segment or record and click on the "Next" button
Type in any custom record name and click on the "Next" button
Click on the "Next" button without filling the ID
Click on the "<" button on the RHP
The error message doesn't appear

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
https://github.com/user-attachments/assets/453a1d64-053a-4eca-9a24-1f4af2f3d487

Android: mWeb Chrome
https://github.com/user-attachments/assets/7c7add2f-59be-4875-88f4-e77ada6dd6d6

iOS: Native

https://github.com/user-attachments/assets/c9478a67-d22b-43a4-b2c4-54ec772a76d7

iOS: mWeb Safari

https://github.com/user-attachments/assets/8a6eec32-15ad-430e-90eb-c2e88cc67f22

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/01fc88bd-aa74-4896-8b2c-454845e12bfe

MacOS: Desktop

https://github.com/user-attachments/assets/21de26b7-2b2e-4963-86c7-c6e9464a3f3b

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@cretadn22 I see one more page where we can make this change. Can you plz apply the same in MissingPersonalDetails
https://github.com/Expensify/App/blob/main/src/pages/MissingPersonalDetails/index.tsx

I have shown MissingPersonalDetails page in the desktop recording.

@cretadn22 Great job getting your first Expensify/App pull request over the finish line! I'll merge it assuming all the tests pass.
I know there's a lot of information in our contributing guidelines, so some points to take note of:
Once your PR is merged, you can be hired for another issue. Once you've completed a few issues, we may start hiring you for more than one issue at a time.
Once your PR is deployed to our staging servers, it will undergo quality assurance (QA) testing. If we find that this doesn't work as expected or causes a regression, you'll be responsible for fixing it. Typically we would revert this PR and give you another chance to create a similar PR without causing a regression. (I don't imagine this will happen with this PR, but it's something to be aware of)
Once your PR is deployed to production, we start a 7-day timer. After it has been on production for 7 days without causing any regressions, then we pay out the Upwork job.
So it might take a while before you're paid for your work, but we typically post multiple new jobs every day, so there's plenty of opportunity. I hope you've had a positive experience contributing to this repo!

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@cretadn22 Great job getting your first Expensify/App pull request over the finish line! I'll merge it assuming all the tests pass.

@MarioExpensify just to inform this isn't the first PR from @cretadn22, I see more in the prev issues list here

Whoops, sorry @Pujan92. Thank you.


Package-private types are not accessible from child loader

The following classloader hierarchy...
 main @73a8e994
   classes/
   [...] // 3rd-party jars
 test @5b5ac798
   test-classes/
   junit-jupiter-api-5.3.2.jar
   [...] // more test-related jars
 launcher @42383cb0
   junit-platform-launcher-1.3.2.jar
   junit-platform-engine-1.3.2.jar
   junit-jupiter-engine-5.3.2.jar
 isolator @217bf99e
   junit-platform-isolator-worker-1.0.0-M3.jar
   junit-platform-isolator-1.0.0-M3.jar

...leads to IllegalAccessError exceptions like:
java.lang.IllegalAccessError:
failed to access class isolation.Main from class isolation.MainTest
  - isolation.Main is in unnamed module of loader main @73a8e994;
  - isolation.MainTests is in unnamed module of loader test @5b5ac798

...in the underlying integration test project src/it/classloader-isolation.

Since https://github.com/sormuras/junit-platform-maven-plugin/commit/717c2d88462534c7e27f3f397d319a903b8ef8c3 the level of isolation can be selected out of four fixtures:

ABSOLUTE - 4 class loader layers with strict separation of main, test, platform and isolator deps.
ALMOST - same 4 layers as in ABSOLUTE except the target/classes is moved to the test layer.
MERGED - 3 class loader layers with merged test + main, platform and isolator deps.
NONE - 1 class loader that contains all project deps -- but still prevents access to the Maven loader.

For details, see new Isolation enum.
/cc @rfscholte


Merged Broker/Server services into one

Merged server and broker services into one service in docker-compose.
Fixes https://github.com/nemtech/symbol-bootstrap/issues/108
Although broker and server are 2 individual processes, there are not really independent. If one crashes, the other one must be stoped and the recovery must run. It is really hard to have these 2 services working nicely in compose as both services need to communicate that the other's process has crashed and one must take the lead when restarting and running the recovery process. Recovery can only run in one of these services while the other is stopped. This makes the services synchronization harder.
This PR simplifies the services, scripts, exception handling, and recovery by joining both services.
One related preset config is dockerComposeServiceRestart: 'on-failure:2'. This value fills the services' restart attribute. Possible values are https://docs.docker.com/config/containers/start-containers-automatically/
This value could be changed to unless-stopped via custom prest so if any service crashes unexpectedly, compose will restart it. This is especially important with the new broker/server service. When Docker restarts it automatically, the recovery will be executed before launching both processes.

Superseded by https://github.com/nemtech/symbol-bootstrap/pull/166


Training with UbuntuCorpusTrainer fails

It seems I can't train the chatbot with the UbuntuCorpusTrainer.
I tried multiple times and, after the first time, in which the trainer downloaded the TGZ file and extracted the TSV files, from thereon I was always receiving the following info:
INFO:chatterbot.chatterbot:File is already downloaded
INFO:chatterbot.chatterbot:File is already extracted
Then the script was stopped responding.
Questions:

What could the problem be?
The file 'Ubuntu_dialogs.tgz' , which I managed to download myself, contains thousands of TSV files. Where have they been extracted to or converted to ands stored as YML? They can't be found under the 'chatterbot_corpus' folder or anywhere  else.

I have debugged the training process of UbuntuCorpusTrainer and found out that it had extracted the TSV files into 'C:\Users\user\ubuntu_data\ubuntu_dialogs\dialogs folder' (in Windows) . This is totally unacceptable, i.e. using a folder in the main disk of the user instead of the folder in which ChatterBot has been created, as with the other corpus data!  And it is more unacceptable if one has installed ChatterBot in movable disk, like a flash drive. Because there is a very obvious reason why someone has chosen this a kind of installation: to be able to be used as stand-alone!
Then I realized that the trainer tries to create a DB from exactly 23251 files stored in that folder. And of course the process gets "stuck" and it looks like the script has crashed. No one can know why until one debugs the training process!
Totally UNACCEPTABLE!


Add cell type workflow into main workflow

Right now we have a separate workflow specifically for cell typing. We will need to add the workflow into the main workflow.

Just noting that when we do add the cell typing into the main workflow, we should include a parameter for whether or not cell typing should be performed. By default we don't want to do cell typing on every sample that's run through the workflow.

Question for discussion: What's our preferred approach for passing in cell type metadata? We've been using a project-specific file, but as we don't require a project id to be provided to the overall workflow, I think it would make sense to do one of the following:

Modify scpca-library-metadata.tsv to have expected cell type columns. This way everything just kind of works, but we'd have to have more (optional!) columns.
Retain a separate metadata file for cell typing, but also toss library ids and/or run ids into that file, so file would have more rows.

We could also do something fancier and map up which run ids are associated with the project ids in the cell type file, but I do think it's worth it to be more explicit

You shouldn't have to do either. I had previously had it in the workflow and all we need to do is tell the main workflow to run the add_celltypes.nf workflow. In add_celltypes.nf we add in all the library/ project/sample info that we need and filter to the relevant samples based on what's provided.
I also would still keep add_celltypes.nf so we can run it independently like it is.
This is what I had in there before:
https://github.com/AlexsLemonade/scpca-nf/commit/da47bd7d34f1437470466fbad804afecb7061669

This is what I had in there before:

I had forgotten it was once there..! Thanks for the reminder

Link to fig jam: https://www.figma.com/file/9zH1gXlz9ZlxVrndN4Wy54/cell-type-workflow?type=whiteboard&node-id=0-1&t=fKgV0qjgpXfOkmFo-0


[Feat]Show a message in the Categories section after submitting the Contact Us form and validate input entries.

Description
Display a confirmation message in the Categories section after form submission.
Validate input formats to ensure correct user entries.
Screenshots

The Name input field accepts invalid characters, allowing entries that are not alphabets.
The Email input field accepts capital letters and entries starting with numbers, which do not comply with proper email formatting.
Input validation needs to be implemented to restrict both fields to their respective valid formats.
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I'm a GSSoC'24-Extd contributor
[ ] I'm a Hacktoberfest'24 contributor
[X] I am willing to work on this issue (optional)

@mdazfar2 sir, Kindly assign me this issue under GSSOC'24 ext. I'm eager to contribute actively.

@NIKITABARANWAL890 Thank you for raising the issue.
As of now, we do not need it. We will let you know in the future if there is a need. Please search for another issue and raise it, which I'm happy to assign to you.

Please don't forget to give our repository a star and let me know once you've done it.

@NIKITABARANWAL890 Thank you for raising the issue.
As of now, we do not need it. We will let you know in the future if there is a need. Please search for another issue and raise it, which I'm happy to assign to you.

Please don't forget to give our repository a star and let me know once you've done it.

Okay sir


manpage: update verbose synopsis

Verbose is mutually exclusive with Body, Error and Output.

Already implemented in b09258e


Unable to import any 'rxbinding3.widgets' in kotlin file

Hi Jake,
While using latest 'com.jakewharton.rxbinding3:rxbinding:3.0.0-alpha2' library, I am unable to consume any android UI widget of your library in my kotlin file.
For example PFB widget that I want to consume:
RxView, RxTextView
PFB Screenshot for reference

Can you please look into it.

They are defined as extensions for Kotlin. Change your code to view.textChanges().

Ok, Got it
So they are defined as Extension Functions.
Thanks a lot! Jake

@JakeWharton
Same problem here.
I have this: implementation "com.jakewharton.rxbinding3:rxbinding:3.0.0-alpha2
And this:
import com.jakewharton.rxbinding3.widget.*

But:

Kotlin Version: 1.3.31
Android Studio Version: 3.4.1

@Drjacky
As JakeWharton mentioned above view.textChanges()

Here 'view' can be your 'EditText'

Therefore, your code should look like this:

locationEditText.textChanges()


Designing API to get candidate list for a job

API to get list of all candidates who applied for a job based on job ID.

Done.