KCP Sync Resource

Created on 2023-10-30 by Xin Ruan (@ruanxin)
Decision log

Name
Description

Title
KCP Sync Resource

Due date
2023-11-30

Status
Proposed on 2023-10-30

Decision type
Binary

Affected decisions
-

Context
Several modules need to create some resources in managed Kyma Runtimes based on the control plane contexts. However, in order to avoid introducing more central components to directly connect to remote clusters, optimize the number of connections, and maintain the principle of least privilege, where only a few central components have admin permissions to access remote clusters, we should provide a way for relies on one centralized component to push those resources to Kyma Runtimes.
Decision

Introduce a Custom Resource (CR) to persist resources' content as Kubernetes manifest YAML.
The central component (lifecycle manager) needs to watch this CR. When a new CR is created or updated, the lifecycle manager should apply it to the relevant remote Kyma Runtime.
Introduce a new command for the Kyma CLI to create this CR, enabling module teams to easily provide these resources.

KCP Sync Resource example:
apiVersion: operator.kyma-project.io/v1alpha1
kind: SyncResource
metadata:
  labels:
    "operator.kyma-project.io/kyma-name": [related kyma name]
    "operator.kyma-project.io/module-name": [related module name]
  name: [unique name indicate the resource to be synced]
  namespace: kcp-system
spec:
  channel: fast
  module: [related module name]
  kyma: [related kyma name]
  resource:
    configmap.yaml: |
        apiVersion: v1
        kind: Configmap
        metadata:
            name: config-update
        data:
            update.properties |
                foo=bar
    secret.yaml: |
        apiVersion: v1
        kind: Secret
        data:
            .dockerconfigjson: xxx
        kind: Secret
        metadata:
            name: some-credential
Status:
  conditions:
  - lastTransitionTime: "2023-10-30T15:56:31Z"
    message: resources are synced
    reason: Ready
    status: True
    type: Resources
  lastOperation:
    lastUpdateTime: "2023-10-30T15:56:31Z"
    operation: 'all resources successfully synced to skr'
  state: Ready

KCP Sync Resource explain:

operator.kyma-project.io/kyma-name label enables the lifecycle manager to identify the target Kyma Runtime for deploying the resource.
operator.kyma-project.io/module-name label indicates the module to which the resource belongs.
spec.channel specifies the channel to which the resources belong within the module.
spec.resource is defined similarly to a ConfigMap and uses the resource file name as the key, with the resource YAML content as plain text (as map[string][]byte), which support multiple resources.
status.conditions and status.lastOperation indicate the synchronization status and provide information about the last operation performed by the lifecycle manager.
status.state reflects the current state of the sync resource.

Consequences

Establishes a centralized mechanism and utilizes an existing component (lifecycle manager) for managing and syncing resources to remote Kyma Runtimes, reducing the need for direct connections and enhancing security.
Provides a standardized format (SyncResource) for defining resources as Kubernetes manifests.
Simplifies resource creation and testing for module teams using the Kyma CLI.

Risks:
Misconfigurations in the SyncResource could lead to synchronization problems or resource mismanagement.

Some feedback:

We need some identifier of SKR. Is it operator.kyma-project.io/kyma-name label?
What is spec.channel for? That resource should be independent of the channel.

Some feedback:

We need some identifier of SKR. Is it operator.kyma-project.io/kyma-name label?
Depends on the purpose of the identifier, if you want to indicate something like SKR shoot name, then it's not. This label is the Kyma CR name, which is one-to-one mapped with SKR cluster, it's also used for LM to identify the SKR cluster kubeconfig secret.

In current Kyma CR, we have following labels for indicate SKR cluster, but they are not create or used by LM, if you need similar label used for other services, we can also add them.
  labels:
    kyma-project.io/runtime-id: 6ca8a6ea-37b3-47f4-a2c8-fe1b6013db23
    kyma-project.io/shoot-name: f1b6307
    kyma-project.io/subaccount-id: 19ea2000-865d-42fd-8d6f-3e3e2dcce3c8

What is spec.channel for? That resource should be independent of the channel.
The purpose of this channel is just want to consistent with module template. For instance, do we have the use case that some module introduces this kind of resources in new version, and this version first released in fast channel, then released to regular channel. In this case, this resource should have an identifier for channel, or it's completely independent?

@ruanxin How would the submission process of manifest content look like in case of mandatory modules that need to synced across all remote runtimes?

@ruanxin If the resource manifests are removed from the content of Sync Object, would lifecycle manager remove the resources on the remote clusters?

@janmedrek @ruanxin
Lets clarify if the mandatory module case would fit into the implementation of the KCP Sync Object

@ruanxin How would the submission process of manifest content look like in case of mandatory modules that need to synced across all remote runtimes?

Currently, in my mind, this idea is more suitable for serving some dynamic content which based on each individual skr cluster, for example, some resources contain skr specific data. So it requires a component in KCP which can generate this resource on the fly, LM just take responsibility to sync it. But for your case, if all resources are identical, it's more relavent to put it in the module template. If you just submit it once, there is no such component recreate it when module disabled and enabled again.

@ruanxin If the resource manifests are removed from the content of Sync Object, would lifecycle manager remove the resources on the remote clusters?

Yes, LM take care the synchronize of this object.

Use case

[ ] KEB installs secret for BTP service operator
[ ] compass manager needs to propagate a secret for the runtime agent
[ ]The NFS storage controller needs to create a storage class and persistent volume
[ ]The warden needs to create a deployment and service account and role

Details about: "KEB installs secret for BTP service operator:

KEB receives credentials from ERS during Kyma provisioning or update
KEB applies a secret with credentials from ERS + generated by KEB clusterID. Example
some KEB k8s jobs manage that secret. That means once a day it checks if all SKRs have a secret in, if not apply it again.

Hi @PK85,

KEB applies a secret with credentials from ERS + generated by KEB clusterID. Example

What was the contract data from KEB to ERS to fetch this credential?  is it something you can get from current Kyma CR?
we have some labels defined in Kyma CR contains some data, e.g:
    labels:
      kyma-project.io/broker-plan-id: xxx
      kyma-project.io/broker-plan-name: azure
      kyma-project.io/global-account-id: xxx
      kyma-project.io/instance-id: xxx
      kyma-project.io/platform-region: cf-eu20
      kyma-project.io/region: northeurope
      kyma-project.io/runtime-id: xxx
      kyma-project.io/shoot-name: xxx
      kyma-project.io/subaccount-id: ""

are they contain the required data?

KEB is not fetching credentials, KEB receives credentials when ERS is calling KEB to spin a Kyma cluster. Nope, those labels are not related to those credentials.

@ruanxin, Based on our understanding, the Kyma CR implementation will also be enhanced to include the contents of Module CR spec in the status. We had following questions related to SyncResource CR and Kyma CR changes:

How LM will learn which Module CR(s) to be monitored and included in Kyma CR.status?
Will the Module CR spec in the KymaCR.status single object containing the last modified CR or a list containing all the changes?
2a.    If it is a single object, is there a possibility of losing concurrent updates to multiple watched CRs in SKR?
2b.    If it is a list, how and when the objects will get removed from the list?
2c.    If they don’t get removed from the list, should the KCP modules maintain some kind of state to track what has changed from previous reconcile?
Can the SyncResource be used to update the contents of the Module CR in SKR?
Assuming the SyncResource example above is already deployed, is the changes to resource.spec tracked only by the changes to the keys (resource file name) or by the changes to the contents as well? Answers to below questions may help:
4a.   If the KCP Module Manager updates the SyncResource CR and removes the secret.yaml from it, will the secret some-credential be deleted from SKR?
4b.   If the KCP Module Manager updates the SyncResource CR and changes the secret name to some-credential-1, will the secret some-credential be deleted and some-credential-1 be created?

In your example you outline that transporting a secret is a common use case. The kind Secret is special in kubernetes in regards on how the data gets stored in the APIServer. Embedding a secret inside a CR will not fullfil these requirements coming with secrets, as the APIServer will not treat the data special anymore. So it must be checked if the solution is acceptable for secret data at all, or what kind of limitations that will imply.
Maybe the secret data can be stored as a real secret in KCP and just referenced by the SyncResource? Is it really needed to embed the data?

As discussed today the sample scenario can look like this.

User installs Infrastructure module with default CR that will have empty spec. Next, the user will
add VPC peering and NFS volumes to the spec.

apiVersion: operator.kyma-project.io/v1alpha1
kind: Infrastructure
metadata:
  name: default
  namespace: kyma-system
spec:
  vpcPeerings:
    - name: peering-1
      description: peering-1
      remoteVpcId: vpc-1
      remoteRegion: eu-central-1
      remoteCidrRange:
    - name: peering-2
      description: peering-2
      remoteVpcId: vpc-2
      remoteRegion: eu-central-1
      remoteCidrRange:
  nfsVolumes:
    - name: nfs-1
      description: nfs-1
      size: 100Gi
    - name: nfs-2
      description: nfs-2
      size: 100Gi

The Kyma Lifecycle Manager will include the Infrastructure CR in the Kyma CR status in the Kyma Control Plane.

Infrastructure Manager controllers deployed in the KCP are watching for the Kyma CR status changes and create
the VPC peering and NFS volumes custom resources in the Kyma Control Plane. These resources will be reconciled and
will create the corresponding resources in the cloud provider account. As a result, SyncResource CRs will be created
to create the corresponding resources in the SKR cluster (storage class, persistent volume etc).

apiVersion: operator.kyma-project.io/v1alpha1
kind: SyncResource
metadata:
  namespace: nfs-manager
  name: kyma-1233445345345
spec:
  kyma: kyma-1233445345345
  resource:
  - strategy: CreateAndSync
    name: nfs-1-pv
    resource: |
      apiVersion: v1
      kind: PersistentVolume
      metadata:
        annotations:
          pv.kubernetes.io/provisioned-by: nfs.csi.k8s.io
        name: pv-nfs
      spec:
        capacity:
          storage: 10Gi
        accessModes:
          - ReadWriteMany
        persistentVolumeReclaimPolicy: Retain
        storageClassName: nfs-csi
        mountOptions:
          - nfsvers=3
        csi:
          driver: nfs.csi.k8s.io
          readOnly: false
          # volumeHandle format: {nfs-server-address}#{sub-dir-name}#{share-name}
          # make sure this value is unique for every share in the cluster
          volumeHandle: <IP_ADDRESS>#ssd#pvc1##
          volumeAttributes:
            server: <IP_ADDRESS>
            share: /ssd
  - strategy: CreateAndSync
    name: nfs-1-pvc
    resource: |
      kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: pvc-nfs-static
      spec:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 10Gi
        volumeName: pv-nfs
        storageClassName: nfs-csi
  - strategy: CreateAndSync
    name: nfs-1-sc
    resource: |
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
        name: nfs-csi
      provisioner: nfs.csi.k8s.io
      parameters:
        server: <IP_ADDRESS>
        share: /ssd
      reclaimPolicy: Delete
      volumeBindingMode: Immediate
      mountOptions:
        - nfsvers=3

@ruanxin Some additional aspect that came from discussion with the interested teams:
There is a case where module does not have an active component in the SKR. The manager is running in the KCP and wants to set the status of the module CR. The status is a subresource in kubernetes so it can be updated independently from spec, so I think it would be nice to provide a syntax to update status subresource.

Hi @pbochynski ,

There is a case where module does not have an active component in the SKR. The manager is running in the KCP and wants to set the status of the module CR. The status is a subresource in kubernetes so it can be updated independently from spec, so I think it would be nice to provide a syntax to update status subresource.

Is the usage just for updating Module CR status? or it need to support all resources subresource? The thing is status is just a common name for subresource, but kubernate allows to define subresource with any name.
But the SyncResource definition can be extended as follows:
  resources:
  - strategy: CreateAndSync
     name: module-cr
     resource: |
         apiVersion: operator.kyma-project.io/v1alpha1
         kind: SomeModule
         metadata:
            name: module-cr-name
         spec:
            ...
    subresource: |
        status:
          ...
          state: Ready

One thing I want to address here is if we allow update subresources, it really needs the Module team to take care of the frequency of updating the subresources, it should prevent updating SyncResource if the status is not changed, otherwise, it will significantly increase the amount of traffic for sync module status.

Hi @tmilos77,
Thanks for the feedback, regarding this UpdateAndDelete strategy, I can add it to the definition.
But I need to clarify this definition a bit,

this strategy means, LM only update the existing resource, and it can't combine with any Create strategies, otherwise will bring conflict.
In your use case diagram, LM still update the status of Peering CR even if the deletiontimestamp configured, but in your comment, I quote "but once destination resource is marked for deletion the sync is stopped, ", which is expected behaviour here? By principle, if resources are marked as deleted, the spec value should not be altered, but only subresource - status can be updated.
Who triggered the deletion? is it only from SKR user? if in this case, I only see Update responsible from this strategy, do LM really need to do Delete? what does this AndDelete mean here?

And I also want to make it clear by current design, LM only treats SyncResource.spec as read-only, it will not update the spec, only ModuleManager take care of remove/update entries to it, and the indicates to remove the entry (or the complete SyncResource) can get from status field.

@ruanxin I wasn't much concentrated on the SyncResource as much I was on the CloudResources flow. My point was to seek for a way to simplify the flow and not to have to watch yet another resource, SyncResources, to clean up orphaned (deleted) items. My idea was that preferably it would function as helm - removing an item would delete it from the SKR, w/out a need to prior marking item for deleting, then checking if it was deleted, and eventually removing the item from the chart/SyncResource. I understand that requires an additional state that will be compared with SyncResource's state. Have thought afterwards a bit more on how it could be achieved in the LM and tried with using helm lib. That would solve the removal but it has other drawbacks like updating an existing resource not created by helm. So, to keep it simple, I agree that this SyncResource feature should not automatically change the spec, treating it as read-only as you said, but interested parties will behave responsible and watch it's status and remove orphaned items on it's own, hopefully w/out bugs, removing the noise on LM and improving it's performance. Still some monitoring possibility would be great to detect orphaned items. I might seem pressing on the orphaned items aspect but predict that dynamic nature in the CloudResources implementation.
On the other hand, after these additional considerations, I'm unable to understand the need to distinguish create and update operations. If I'm wrong please correct me, but wouldn't server side apply create new resource if it doesn't exist, and just patch the existing object? If that stands, wouldn't just a simple Sync strategy be sufficient?
Following the automatic removal of deleted items considerations, then also CreateAndIgnore strategy seems as an overspill, and it's insurance/implementation can be transferred to the clients watching the SyncResource status and just removing that item once it's been created, same way as Delete responsibility would be shared.
Having all that, I would say two strategies are enough: Sync and Delete.
In the base line, just regarding the CloudResources functionality, following the mentioned option 2 with a helper Binding resource that will be synced, the only important thing in SyncResource functionality is to be able to change strategy of an item, from CreateAndSync (or as now proposed just Sync), to Delete, with additional reconciliation on it's status and eventual removal from the SyncResource.

@jeremyharisch I have no direct interests atm for it regarding the CloudResources, but since already included in the SyncResource proposal discussion... I missed the decision reasons and explanation on sensitive data, is there some record on it? I'm not particularly convinced that preventing certain resource kinds would solve the issue. One can always bypass the constraints and transfer the sensitive data trough other kinds, ie instead in a Secret in a ConfigMap, or even as a Deployment env var.
On the other hand, not sure what encryption at rest has to do with this SyncResource feature and it doesn't have with for example v1/Secret resources we already have stored in KCP, or a db password clients have in SKR. It's exposed as readable to k8s principals having appropriate roles, and it's up to the underlaying infrastructure to ensure encryption at rest on the physical levels to prevent it's leakage trough other channels. Isn't the situation same with SyncResource kind in KCP as it is with v1/Secret?

@ruanxin I wasn't much concentrated on the SyncResource as much as I was on the CloudResources flow. My point was to seek for a way to simplify the flow and not to have to watch yet another resource, SyncResources, to clean up orphaned (deleted) items. My idea was that preferably it would function as helm - removing an item would delete it from the SKR, w/out a need to prior marking item for deleting, then checking if it was deleted, and eventually removing the item from the chart/SyncResource. I understand that requires an additional state that will be compared with SyncResource's state. Have thought afterwards a bit more on how it could be achieved in the LM and tried with using helm lib. That would solve the removal but it has other drawbacks like updating an existing resource not created by helm. So, to keep it simple, I agree that this SyncResource feature should not automatically change the spec, treating it as read-only as you said, but interested parties will behave responsible and watch it's status and remove orphaned items on it's own, hopefully w/out bugs, removing the noise on LM and improving it's performance. Still some monitoring possibility would be great to detect orphaned items. I might seem pressing on the orphaned items aspect but predict that dynamic nature in the CloudResources implementation.

Hi @tmilos77, yes, I agree that has resources synced automatically handled by LM will be easier for module manager, actually I introduced a synced list in status field in my first draft to tracking the diff between resources,  but this concept challenged by @pbochynski, it introduced complexity to LM, also may bring inconsistency if the status messed up. That's why we bring this Delete strategy, but you don't have to implement a very consistent solution, basically the purpose of this strategy is just to let LM delete some outdated resources, LM will not report error if the configured resources with Delete strategy not found in cluster.

On the other hand, after these additional considerations, I'm unable to understand the need to distinguish create and update operations. If I'm wrong please correct me, but wouldn't server side apply create new resource if it doesn't exist, and just patch the existing object? If that stands, wouldn't just a simple Sync strategy be sufficient?

That's the thing I need to verify with you by your use case. As I said in the previous comment, do you use this strategy for only update the existing resource or not? If you have such use case, then introducing this Update make sense, we can distinguish it with Create, as you said, the server-side apply will create new resource if it doesn't exists, so if some resource should not be recreated after it's removed, then it make sense we add this new UpdateAndDelete strategy.

Following the automatic removal of deleted items considerations, then also CreateAndIgnore strategy seems as an overspill, and it's insurance/implementation can be transferred to the clients watching the SyncResource status and just removing that item once it's been created, same way as Delete responsibility would be shared.

CreateAndIgnore may not required for your operator, but it fit for other scenarios, this sync resource concept is not only designed for your use case.


MongoServerError: Cannot use Hint for this Query on Document DB

Link to reproduction
No response
Describe the Bug
We updated these deps.

Works locally with the Mongo dev container, but we get this message when using DocumentDB on AWS.
MongoServerError: Cannot use Hint for this Query. Index is multi key index , partial index or sparse index and query is not optimized to use this index.

To Reproduce
Didn't have time to track down the exact cause since we needed to revert fast.
Must be caused by one of the difference between mongo and DocumentsDB
https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html#functional-differences.multi-key-indexes
We're very much hoping we can continue to use the managed Document DB AWS service for payload.
Payload Version
2.4.0
Adapters and Plugins
"@payloadcms/bundler-webpack": "^1.0.5",     "@payloadcms/db-mongodb": "^1.1.0",     "@payloadcms/plugin-cloud-storage": "^1.1.1",     "@payloadcms/plugin-sentry": "^0.0.6",     "@payloadcms/richtext-slate": "^1.3.0",

Must be caused by one of the difference between mongo and DocumentsDB

This is exactly the problem.
There is a flag for disabling the use of hints already disableIndexHints already for DocumentDB compatibility. You can find it in the docs: https://payloadcms.com/docs/database/mongodb#options
I'm going to close this but @ message me if you have any trouble.

Much appreciated!
Can I suggest you also add this to the production Document DB docs too? https://payloadcms.com/docs/production/deployment#documentdb


fsolve can't find proper solution

Hi!
Today I tried solve easy equation:
import numpy as np
from scipy.optimize import fsolve

def func(U):
    return 9.889*(1-np.exp((U/60-1)*(-2.403)/(-0.167)))-(4.964*(1-np.exp((U/80-1)*(-2.369)/(-0.125))))

sol=fsolve(func,np.array([50]),xtol=1e-6,maxfev=100)

print sol

If there the starting estimate for the roots is near to real root (for example 50) then solution is right. But if I input another near starting estimate (for example 45) there absolutely wrong solution.
Additionally I test this equation in Matlab by using similar function "fsolve" and got absolutely right answer for all starting estimates in range of 0 to 60.

Under scipy==1.6.0 the original code snippet (with minor adjustments for Python 3) gives the expected answer 57.11177009, considering this issue closed

Under scipy==1.6.0 the original code snippet (with minor adjustments for Python 3) gives the expected answer 57.11177009, considering this issue closed

Try is with the guess as 45 instead of 50. I think the original code snipped always worked; the different initial guess is what was causing problems.

Try is with the guess as 45 instead of 50. I think the original code snipped always worked; the different initial guess is what was causing problems.

Good catch, I misread.  Can confirm this is still an issue.

Good catch, I misread.  Can confirm this is still an issue.


fix(bspwm): Update when focusing urgent desktops

When focusing a desktop with the urgent flag, two events are received
from bspc simultaneously, separated by a newline character.
This was not handled correctly and the second event was discarded causing
the urgent style to be removed, but the focused style would remain on the
previously focused desktop.
This fixes the problem by handling any number of events that arrive at the
same time (separated by newlines).

Thank you


Last updated broken for data files

They all seem to point to this commits on the 4th April for a metadata file not matter what the real change was.
It's always https://github.com/ONSdigital/sdg-data/commit/f4ce428b53e79ca9b3e32ab14eac1031d7bc6652

Worryingly I can't reproduce locally so maybe it's a Travis thing? I wonder if the git code got too complicated. All is fine for metadata files.

I have now reproduced. It's because of the --depth=50 flag in the git clone. So this actually needs fixing in sdg-data.

Fixed in ONSdigital/sdg-data#32


added issue templates

Fixes #23

Thanks, @Vineet-Sharma29. A quick fix needed here for DCO - https://github.com/layer5io/service-mesh-performance-specification/pull/25/checks?check_run_id=689635332

@leecalcote I have made the required changes


Handling a dataset upgraded to v4 in CBS source

Currently, a dataset ID given to statline-bq is first checked to see if a v4 version of it exists. If it does, it hands it to cbsodatav4_to_gcs, and if not to cbsodatav3_to_gcs.
In the future, when a certain dataset which was previously ingested as V3 is now available as V4, the old V3 linked datasets in bq should be removed, to avoid confusion.
In order to this:

A check should be made to see if this dataset id exists as a linked V3 table in BQ (for example, checking whether {dataset_id}_TypedDataSet exists)
If it does, collect all table names related to this dataset id -easiest might be go into appropriate GCS folder and get all file names
Then, erase all this tables from BQ
Finally, continue with V4 upload

This might be still relevant. Current implementation creates a (sort-of) duplicated dataset in BQ:

cbs_v3_83765NED
cbs_v4_83765NED

I would argue this is not the expected behaviour and that only the v4 should be exist in BQ (we can keep the old versions in GCS).
To fix, maybe add the following to check_bq_dataset(id, source, odata_version, gcp):
if odata_version == 'v4':
    if check_bq_dataset(id=id, source=source, odata_version='v3', gcp=gcp):
        add_some_flag_to_return_value

Then in gcs_to_gbq() where check_bq_dataset is currently called, delete the v3 dataset after succesfully creating the v4 one.


Privacy Policy not found

Is there an existing issue for this?

[x]  I have searched the existing issues

What happened?
BUG: Requirement to add a gene

ral privacy policy !
Record

[x]  I agree to follow this project's Code of Conduct
[x]  I'm a GSSOC'24 contributor
[x]  I want to work on this issue

@Priyaaa1 assign me this under gssoc label


not realy a issue but more a question....

i wanted to use 2 sdr-receivers on 2 diffferent pi's, to split some capcodes for different home assistant automations.
once up in the air, i couldnt get it to work as i wanted... they both spit out the same sensor name.  "sensor.p2000_capcode"
is there a way to change this?
or a other workaround?

Yes there is.
In the current code, it can create multiple sensors.
And for both sensors, you can give different capcodes or regions or keywords or....

Just want for every capcode a different entity if possible.
And how do I do that?

What are you trying to create?
A sensor for 1 capcode might not be the best option.
Yes, you can change te parameters that every capcode creates its own HASS sensor, but HASS will be flooded with errors if you create automations on a perticular sensor and it has not triggered once after a HASS reboot (sensor in HAAS will be created the first time when the sensor is send to hass).

the reason i want to split it is simple....
i'm a firefighter and when i'm home i want to create a automation to trigger lights and get spoken word for 4 capcodes.
the other capcode is 1 and that only speaks out the data.
sensor that way wont be floodded.
something like this?
[sensor_p2000_cacode]
friendlyname = P2000 capcode MON
searchcapcode = 000806240
[sensor_p2000_capcode]
friendlyname = P2000 capcode B2
searchcapcode =<PHONE_NUMBER>,000804047

Than the option with the new code is a good solution.
I would suggest changing the sensor names a bit.
[sensor_p2000_capcode_mon]
friendlyname = P2000 capcode MON
searchcapcode = 000806240
[sensor_p2000_capcode_b2]
friendlyname = P2000 capcode B2
searchcapcode =<PHONE_NUMBER>,000804047

first make it word then make pretty 😄


Create forkkkkkkkk

Description
New tool, Bug fixing, or Improvement?
Please include a summary of the change and which issue is fixed. Also include relevant motivation and context.
For new tools, please provide total size and installation time.

Related issue:
Check list

[ ] Related issue / work item is attached
[ ] Tests are written (if applicable)
[ ] Documentation is updated (if applicable)
[ ] Changes are tested and related VM images are successfully generated

@mikhailkoliada Help me please igot some issues with these
[Error loading webview: Error: Could not register service workers: NotSupportedError: Failed to register a ServiceWorker for scope ('https://07jui2prh42dtve2cdaattv4g8hf0g6baatn5dci2n7p3v3cs3b9.vscode-cdn.net/stable/6261075646f055b99068d3688932416f2346dd3b/out/vs/workbench/contrib/webview/browser/pre/') with script ('https://07jui2prh42dtve2cdaattv4g8hf0g6baatn5dci2n7p3v3cs3b9.vscode-cdn.net/stable/6261075646f055b99068d3688932416f2346dd3b/out/vs/workbench/contrib/webview/browser/pre/service-worker.js?v=4&vscode-resource-base-authority=vscode-resource.vscode-cdn.net&remoteAuthority=codespaces+ah-ha-admin-special-bassoon-qrvvv55v9wrcgj9'): The user denied permission to use Service Worker..
what does  that means and how can i resolve this bug :( maybe i just turn off something i should have left activated ???? thanks !
repository :
refactored-memory
codespace working in deccontsiner


Ma/add query context

Fix #523
Fix #522

Can we add it as an extension method AsNoTracking or WithNoTracking instead of passing a context in each method

@hishamco I thought about it originally. The problem if we do that, we would have to track the state whether to track the entities per session not per query. The idea is to be able to not track the results per-query not per-session. Like what if the users wants to first query non-trackable documents, and in the same scope, they want to query other trackable documents? If they don't remember to set the state before the second query, both queries will not be tracked.
I simplified things by adding WithNoTracking() method of the IQuery<T> so if you are building a query, you can use this method to pass instruction to not track the entries.
But, in the Session, I think it is better to provide QueryContext that would change the behavior of the query on demand.

Now looks much better for me, waiting for Seb comment

If you spent time to review a PR, you may want to consider approving what you think is ready. This way, any changes you request will be to eventually to approve the PR which is more helpful than just leaving comment :)

Just fixing a small comment then I will approve
LGTM

Internal caching can be very expensive when dealing with a large data set.

I don't think it adds much more overhead since it's only adding the items to a collection, and "dealing with a large dataset" means you will also do that. You should just get rid of the session when you are done with it and you don't want to keep the references around.
However I can see why you would not want to have to create a new Session for every page of data you load, by convenience, and you came up with the notion of QueryContext. I think it should still be in the Session and not at the query level. This way you create a session that doesn't save the state and can reuse it to do all your queries. So I would suggest to just add a property to the session to disable tracking altogether. And another advantage is that the object won't even be added to the identity map at all, so potentially even faster processing. This would be an optional bool in CreateSession. I don't think it matter to be able to change the state of the session to enable/disable tracking dynamically. Just create a new session if you need different characteristics.
Also can you rename DetachRange to Detach(IEnumerable<object>) instead? Unless you have an argument that makes it better.

@sebastienros I made the requested changes.


Fix initialPage prop of Tabs component for IOS

Motivation
The initialPage prop of Tabs component is not working in IOS devices.
Test Plan
For this code:
...
        <Tabs initialPage={2}>
          <Tab heading="Tab1">
            <Tab1 />
          </Tab>
          <Tab heading="Tab2">
            <Tab2 />
          </Tab>
          <Tab heading="Tab3">
            <Tab3 />
          </Tab>
        </Tabs>
...

Before fixing this issue, the content of the initial tab (in this case Tab3) will render but the first tab is showing and then nothing will be showing in the tab. After this fixing, the right tab will show in the page. In this way the initialPage prop will work properly in IOS devices.
This commit will also fix #1318 and #1010.

@VahidBo tried your fix. Don't think the issue is fixed.
Gif

This is because InteractionManager.runAfterInteractions(f) happen to sometimes never call f.
I've added a PR here: #1376

@VahidBo Are you planning to fix this PR?

@SupriyaKalghatgi I fixed this in #1376. It's basically the same fix as @VahidBo provided, only that is included a fix for IteractionManager which happens to never call the callback sent to runAfterInteractions.
And because of this, InteractionManager.runAfterInteractions(scrollFn); never run.
You can check the whole fix in #1376.


Style basket page

User Story: As a customer, I want the basket page to be visually appealing and easy to read so that I can easily review my items before checking out.
Acceptance Criteria:
The basket page must use a consistent style with the rest of the website.
The item names and prices must be clearly visible and easy to read.
The quantities of each item must be clearly displayed.
The total cost of the order must be clearly visible and easy to read.
The checkout button must be clearly visible and easy to locate.
When an item is added or removed from the basket, the page must update the total cost and item quantities dynamically without requiring a page refresh.
The basket page must be responsive and usable on all screen sizes.


root_scalar: a new method that improves upon the Newton's method

What does this implement/fix?

An "extended-Newton" method is implemented in root_scalar function based on https://doi.org/10.3390/a13040078
This extension improves the domain of convergence, especially for non Lipschitz functions without any extra computation required (still uses only one function and first derivative evaluation per iteration). The only extra is a second guess x1, which if not supplied is assigned as a perturbation of x0. The new method can also be seen as a combination of Newton and secant methods, and somewhere between Newton's and Halley's methods.
Additional information

I have tried to maintain the backward compatability. The default options remain the same and the standard Newton continues to work as is. I have also added the necessary documentation for the new method.

@ankushaggarwal thanks for proposing your method as an addition to our collection of root finders.
The paper doesn't seem to have any citations, and there's no benchmarks here either that show this is clearly an improvement on what we already have. http://scipy.github.io/devdocs/dev/core-dev/index.html#deciding-on-new-features gives some guidance on how we'd decide on this. For something as fundamental as root finding, I'd expect 50-100+ citations and/or convincing benchmarks. Do you have benchmarks (accuracy, not speed)?
gh-12620 may be related. I'll also add @ev-br's comment to that on the mailing list, which is relevant here:
FWIW I'd always recommend brentq for scalar root-finding.
Trying to make newton-based methods more robust seems to be more or less reinventing that wheel, which has already been implemented (don't know who implemented it; hope you read this thread so that you see my thank you!) and is as robust  as it gets.

@rgommers Thanks for your comment. This is my first time contributing to scipy. How can I add benchmarks? And what do you mean accuracy, not speed?

How can I add benchmarks? And what do you mean accuracy, not speed?

 And then seeing how many problems get solved with the existing methods in scipy.optimize and with your proposed new method.

How can I add benchmarks? And what do you mean accuracy, not speed?

 And then seeing how many problems get solved with the existing methods in scipy.optimize and with your proposed new method.

OK thanks! I will put that together. I suppose that will not go into the repository, and I can just share a script here in the comments?

I suppose that will not go into the repository, and I can just share a script here in the comments?

You can just start with a script. We may want to turn it into regression tests or benchmarks (see benchmarks/benchmarks/optimize_zeros.py) afterwards.

Regression testing for exnewton should be added to optimize/tests/test_zeros.py.  You can easily add test_exnewton_collections() by just cloning test_newton_collections, replacing newton with exnewton, and possibly setting known_fail.

@ankushaggarwal thanks for the PR. Were you interested in finishing this up?

Yes, I am interested in finishing this up. Really sorry, I got distracted with other things and lost track of this one. I will work on getting the benchmark tests and get back.

Ok! Let me know if you'd like this to be re-opened once you've merged main and developed benchmarks.


Does WebGL work for layers?

cornerstone-core version: 2.6.0
I've faced the issue that images added with addLayer don't use WebGL. There is a code in the internal function drawImageSync which adds conditional branching:
if (layers && layers.length) {
  // render when layers specified
  drawCompositeImage(enabledElement, invalidated);
} else if (image) { // render when layers not specified

Currently, WebGL works only for displayImage function. But it isn't possible to combine multiple images on the same canvas with  displayImage.

Is there any workaround to use multiple layers with WebGL?
Do you have plans to add WebGL support for layers in the nearest future?

I don't have an answer to your question, but I am curious what you are trying to accomplish with the layersApi, perhaps there is a different workaround

@shanebenlolo
I want to display a CT scan and can perform windowing, panning, and zooming operations. Also, I want to put a png image overlay on top of the CT scan. CT scan and overlay should be synchronized during panning and zooming.
I am considering three solutions:

Use layers API. Cons: cornerstone doesn't use WebGL while windowing, so FPS becomes lower
Use two cornerstone canvases. One for CT scan and one for overlay image. Cons: cornerstone adds black background to the overlay image, but the overlay background should be transparent. I didn't find any flag to make a background transparent, so the only solution I found is to set a background transparent on each rendering frame.
Use two canvases. Cornerstone canvas for CT scan and custom canvas for overlay image. Cons: I have to write additional code to resize, translate and synchronize the overlay image.

Maybe there is another approach that mitigates all cons?

Have you considered using the react-vtkjs-viewport? https://react-vtkjs-viewport.netlify.app/


Create MapContent Component

Right now the code inside the MapWithLocationsAndPath component is a lot to handle + it won't let us insert our hooks to the MapContainer scope.
The solution here will be to create a new component (possibly named MapContent) which will contain all the JSX that is currently inside the MapContainer component (all the code in the photo attached) - enabling us write all the hooks we want

Hey, I'll take it.

@menachem95 Good luck!


OPAM packages in dune-project

Remove .opam in src/, and list OPAM packages in src/dune-project.
Closes #12928.

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me

I fail to see a src/dune-project file modified in the commits. Am I wrong?

https://github.com/MinaProtocol/mina/pull/12930/files#diff-91c11ff6d2b2e272d247eadb73cf15c633b33d10ae6915b536ca72265734f1d0

!ci-build-me

I guess we need to add (generate_opam_files true) and it might generate the opam.export.

I tried adding that clause. If I then run make build, I don't see any files generated. If I run dune build in src/, it generates the .opam files that this PR deletes.

CI passed here: https://buildkite.com/o-1-labs-2/mina-o-1-labs/builds/27600


Update to .NET 7

Update to .NET 7.

Codecov Report

Merging #402 (496f530) into main (0a6b494) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 496f530 differs from pull request most recent head 7f0e5ee. Consider uploading reports for the commit 7f0e5ee to get more accurate results

@@            Coverage Diff            @@
##              main      #402   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           16        16
  Lines          699       699
=========================================
  Hits           699       699

Flag
Coverage Δ

linux
100.00% <ø> (ø)

macos
100.00% <ø> (ø)

windows
100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d783fe5...7f0e5ee. Read the comment docs.

Once #495 is merged and I publish 3.2.0, I'll merge this and #444 and then publish 4.0.0.


Use the old URL if env variables are missing.

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

Is this WIP or ready for review?

Ready for review


Update Add API wizard

Update Add API wizard to add an API without proxy settings. (Proxy should be added - or in the future selected - in separate UI.) This UI will add an API with basic details to Catalog.
Definition of done

[ ] User can enter all details related purely to  API backend such as name, description, base URL with Add API wizard

[ ] Add API wizard does not contain any details or settings related to proxy or selecting proxy

[ ] When user presses Publish API backend, the API will be added to API backends collection

[ ] User get redirected to single API view for the newly created API

@bajiat : need wireframe for this?

Discussed with @brylie about deprecating wizard and using simple form approach.
Base for the form will be new API Backend schema.
Fields for API backend schema:

name
description
base URL for API
isPublic checkbox

We decided to deprecate the Add API Wizard, since we are simplifying the basic API fields.
@jykae would you be interested in improving the Add API form after we have merged these initial changes?


Callback typedefs missing from Doxygen documentation

Slightly tangential to #71 , I noticed that there a 6 callback typedefs defined in the SDK:
src/common/pico_time/include/pico/time.h:typedef int64_t (*alarm_callback_t)(alarm_id_t id, void *user_data);
src/rp2_common/hardware_gpio/include/hardware/gpio.h:typedef void (*gpio_irq_callback_t)(uint gpio, uint32_t events);
src/rp2_common/hardware_timer/include/hardware/timer.h:typedef void (*hardware_alarm_callback_t)(uint alarm_num);
src/common/pico_time/include/pico/time.h:typedef bool (*repeating_timer_callback_t)(repeating_timer_t *rt);
src/rp2_common/hardware_clocks/include/hardware/clocks.h:typedef void (*resus_callback_t)(void);
src/rp2_common/hardware_rtc/include/hardware/rtc.h:typedef void (*rtc_callback_t)(void);

4 of these are documented in the Doxygen output:

alarm_callback_t
hardware_alarm_callback_t
repeating_timer_callback_t
resus_callback_t

But these two are missing from the Doxygen output:

gpio_irq_callback_t
rtc_callback_t

Fixed in #212


Add a change log, please.

Hello,
Would it be possible to create and keep a changelog in this project, please?

I have filled in the previous 4 releases changes. you can find now the change log in https://github.com/Microsoft/tslib/releases

Thanks. 👍


Add LIKE filter support

This change adds the ability to search on fields using the LIKE function provided by ArangoDB. There were no functions for precedent but I took the approach of named parameters in the condition. The text is assumed to be a document field. This approach may not work for other functions such as CONCAT but usually it makes sense that a parameter is either a field or a string.

Merged. Thanks a lot for your work.


redirect wiki profiles to profiles.ihe.net

The ITI published profiles should minimally be redirected to the profiles.ihe.net rather than the PDF publications
Possibly the duplication of the profile summary, actors/transactions, options, etc.. should be removed as the new html page is better maintained.
Possibly the wiki page for profiles might link to the html higher on the page?
Possibly the wiki page for profiles might auto-redirect to the html page?
There is current information on the wiki page that is not part of the technical framework. Such as pointers to tests, examples, etc.

Google Sheet to track progress of fixes for this issue is at https://docs.google.com/spreadsheets/d/1eM8OgzO8qGbc9Q8lEyBtwdALFOaYZN9W-AsyV6EnXWA/edit#gid=2021735577

done


Icons of playlist not displaying correctly

is there a possibility, of how to fix this problem. cause it seems like i am the only person with this issue

I've done some research and i found a hot fix for the time being, for anyone that might have the same issue in the future, here's a quick step

on the ~/.config/spicetify/Themes/Bloom/user.css ive added max-width:100% into the following line

here's the final result of applying new changes

@Crushoverride007 sorry for the incontinence and thanks for the fix
it seems we can merge this as a fix
can you open a pr?

is there a possibility, of how to fix this problem. cause it seems like i am the only person with this issue

Ur not the only one. But mine is kinda different in nature.

When i scroll down it looks like this.

the problem is mine work fine🥲

Can't reproduce this issue here either. ;w;

@windowz414 I think, its an installation problem. Ive an m1 so i think somethings gone wrong while installing spicetify

the problem is mine work fine🥲

can you leave out your source folder of spicetify?

the problem is mine work fine🥲

@nimsandu can you leave out your source folder of spicetify?

I am not quite sure I understood what you said

i meant your configuration of the spicetify in ~/.config/spicetify

@nimsandu thank you, dont worry about it 👍🏻


proposition for alternative property for InProgressDataDate column in MARCO-BOLO_Metadata_Dataset_Record_WP5

MARCO-BOLO_Metadata_Dataset_Record_WP5
Duplicate keys are not allowed in json(-ld)
--> meaning that the schema.org equivalent property of 'additionalProperty' for the columns "InProgressDataDate" and "EVDescription" needs to change for one these columns;
--> propose to change 'additionalProperty' into 'datePublished' for  "InProgressDataDate"
Is this okay?

fine by me - would need to change for all the WP spreadsheets, mind

except ... these googlesheets do not need to be transformed, laurian. only the URIs they refer to.

only the value in cell D8 of the 'Column descriptions' tab would have to be changed (no other values need changing) - and this would only be to be aligned with what is actually used in the triples (but not required for things to work/not break)

Are we OK here? In that, the "in progress date" is not something you need to read into your templates. Can I close this?


4.Schema for Admin User and Client User

Admin User Table:
Name
Email
Phone Number
Password
Address
Admin User Id(Auto generate Primary key)
User Table:
Name
Email
Phone Number
Password
Client Use Id(Auto generate Primary key)
Case Management:
Client Use Id
Admin User Id
Notes
last_update

Table 1 and 2 (admin and client users tables) created on local server and need review for datatypes and constraints;
Table 3 (cases table): last_update column needs to be further confirmed regarding datatype

Opening this issues for Yvonne!


Missing Link for Map documentation

Hi !
The links for Map , and Map modules are broken.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a0694330-a4e4-5a07-2b6b-8259b57e319a
Version Independent ID: 71e9da59-27a1-6194-3a69-09b9d3a383d4
Content: Collection Types - F#
Content Source: docs/fsharp/language-reference/fsharp-collection-types.md
Product: dotnet-fsharp
GitHub Login: @cartermp
Microsoft Alias: phcart

Sorry about that. i'm updating links right now. In the meantime, you can browse here: https://fsharp.github.io/fsharp-core-docs/

Thanks a lot for your reactivity.
The doc is great by the way, thanks !
De : Phillip Carter<EMAIL_ADDRESS>Envoyé : vendredi 14 août 2020 08:07
À : dotnet/docs
Cc : TRAIN Michel TGI/OLS; Author
Objet : Re: [dotnet/docs] Missing Link for Map documentation (#20030)
Sorry about that. i'm updating links right now. In the meantime, you can browse here: https://fsharp.github.io/fsharp-core-docs/
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/dotnet/docs/issues/20030#issuecomment-673904538, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACRENJILJSHGVHKUEXLQVT3SATIBVANCNFSM4P6EYBMA.

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.


ReadMany api fix on partition split

Issue - During partition split readMany api hangs , it goes in continuous cycle.
Root cause - When parent document producer spawn new document producers, it passed the stale createRequest function to its children, which causes each child creates RxDocumentServiceRequest with parent feedRange, due to which FeedRangeEpkImp going in continuous loop and throwing pk range gone exception  (410/1002) here
Fix - During ParallelDocumentQueryExecutionContextBase when we initialize read many , we are now creating document producer with correct createRequest function which dynamically takes child feed range instead of parent during splits.

/azp run java - cosmos - tests


Can't get PWM1 to work

There seem to be an issue with PWM generation on "tim1_ch3_pa10" (sixtron name "PWM1").
Can't get any PWM output.
It seems to be necessary to setup the prescaler for Timer 1 (timer counter is "only" 16bits), but it doesn't seem to be enough.
Example of a overlay configuration using a Zest Core STM32L4A6RG:
 / {
     test_pwm {
        compatible = "pwm-leds";
        status = "okay";

        backlight_lcd: backlight_lcd {
            pwms = <&pwm1 3 100000 PWM_POLARITY_NORMAL>; /* PWM controller phandle */
        };
    };
};

&pwm1 {
    status = "okay";
};

&timers1 {
    status = "okay";
    st,prescaler = <1000>;
};

This could be a Zephyr problem and not a sixtron dts configuration problem.

ping @Sokario @andiaye33

Disabling "pwm2" seems to work correctly in the zest dts:
&timers1 {
	st,prescaler = <1000>;
	status = "okay";

	pwm1: pwm {
		pinctrl-0 = <&tim1_ch3_pa10>;
		pinctrl-names = "default";
		status = "okay";
	};

//	pwm2: pwm {
//		pinctrl-0 = <&tim1_ch2_pa9>;
//		pinctrl-names = "default";
//		status = "okay";
//	};
};

Pwm 1 is now working. Possible problem with pinctrl-names = "default" ? Should it be only one?

Looking back at what's generated in the build folder:
With the current dts target file, in build\zephyr\zephyr.dts file:
		timers1: timers@40012c00 {
			compatible = "st,stm32-timers";
			reg = < 0x40012c00 0x400 >;
			clocks = < &rcc 0x60 0x800 >;
			resets = < &rctl 0x80b >;
			interrupts = < 0x18 0x0 >, < 0x19 0x0 >, < 0x1a 0x0 >, < 0x1b 0x0 >;
			interrupt-names = "brk", "up", "trgcom", "cc";
			st,prescaler = < 0x3e8 >;
			status = "okay";
			pwm1: pwm2: pwm {
				compatible = "st,stm32-pwm";
				status = "okay";
				#pwm-cells = < 0x3 >;
				pinctrl-0 = < &tim1_ch2_pa9 >;
				pinctrl-names = "default";
				phandle = < 0x24 >;
			};
		};

The generated Timer 1 configuration is incorrect: Pwm1 and Pwm2 are sharing the same pin.
When disabling "pwm2", the generated Timer 1 configuration in the build\zephyr\zephyr.dts file is now correct:
		timers1: timers@40012c00 {
			compatible = "st,stm32-timers";
			reg = < 0x40012c00 0x400 >;
			clocks = < &rcc 0x60 0x800 >;
			resets = < &rctl 0x80b >;
			interrupts = < 0x18 0x0 >, < 0x19 0x0 >, < 0x1a 0x0 >, < 0x1b 0x0 >;
			interrupt-names = "brk", "up", "trgcom", "cc";
			st,prescaler = < 0x3e8 >;
			status = "okay";
			pwm1: pwm {
				compatible = "st,stm32-pwm";
				status = "okay";
				#pwm-cells = < 0x3 >;
				pinctrl-0 = < &tim1_ch3_pa10 >;
				pinctrl-names = "default";
				phandle = < 0x24 >;
			};
		};

From the nucleo_l496zg.dts file, in Timer1, all channel output are setup in the same pwm object:
&timers1 {
	st,prescaler = <10000>;
	status = "okay";

	pwm1: pwm {
		status = "okay";
		pinctrl-0 = <&tim1_ch1_pe9
			     &tim1_ch2_pe11
			     &tim1_ch3_pe13>;
		pinctrl-names = "default";
	};
};

Source file: https://github.com/zephyrproject-rtos/zephyr/blob/main/boards/st/nucleo_l496zg/nucleo_l496zg.dts#L130-L132


Incompatibility with kombu 4.0

When running the pub/sub example from the docs, I get the following error:
AttributeError: 'Connection' object has no attribute 'exchange_declare'
Full traceback is here:
https://gist.github.com/adwhit/5007a304d68891ead581e4c9035db205
Nameko version 2.4.2, Kombu version 4.0.0
If I drop down to Kombu 3.0.37, it works. Therefore I assume the problem is either with Kombu or the Nameko-Kombu interface.
Edit - I see you have recently updated setup.py to use a version < 4. Feel free to close this issue! I note that every other feature seemed to be working correctly so hopefully there will not be too much work to get 4.0 working.

Nameko 2.4.3 has now been released, which pins kombu back down to a compatible version.
I'll leave this issue open until we fix the compatibility issue. Thanks for reporting!

started looking at kombu 4 compatibility. in praticular, it switches to (py-)amqp 2.0 (kombu 3 pins amqp below 2). 2.0 is pretty nice, e.g. has much better error messages for connection issues, and nicer basic return support.
it might be a bit tricky to support both versions, at least without version and/or feature sniffing, which makes coverage tricky (or requires suppressing cov requirements) [should we consider running a single travis job so that we can combine coverage reports e.g. from different python versions]
i guess the argument can be made that (unlike other requirements like 'requests'), nameko "owns" the kombu requirement (i.e. we wouldn't expect user code to depend directly on kombu), so we could decide to (bump our version and) just require kombu 4. not sure how i feel about that, esp. given how recently it came out. i don't really want to have to start maintaining multiple minor versions concurrently (i.e. nameko 3.x with kombu 4 and nameko 2.x with kombu 3)
thoughts and other considerations welcome

I agree that we should avoid supporting multiple versions of kombu concurrently, and with the argument that nameko "owns" kombu as a dependency. I would be fine upgrading to support kombu 4.0 only after it's shown itself to be reasonably stable.
However, I don't think we really need kombu at all. We only use the AMQP component, and we've had discussions in the past about how kombu restricts us in some ways.
I wonder how much work it'd be to change the AMQP components to use py-amqp (2.0) or another library directly instead of kombu, at least compared to the work required to upgrade to 4.x.

Also I guess there's no great urgency here. If nameko "owns" the kombu dependency it's fine for us to keep it pinned down to 3.x for the foreseeable future.

agree that we are in no rush, though it's nice to be compatible with the latest versions of libraries, to be sure to get security updates and bug fixes (not sure what kombu's policy is, e.g. if 3.x will stay maintained at all)

I don't know if my problem deserves separate issue, but I have a problem now, which seems to be related to this topic. I have latest celery (4.1.0) in project which depends on kombu 4.1.0. Since I have also nameko, I can't use pytest (and probably code that invokes calls to the nameko microservice, haven't tested yet), it just fails with:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pluggy/__init__.py", line 397, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2404, in load
    self.require(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2427, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 875, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (kombu 4.1.0 (/usr/lib/python3.6/site-packages), Requirement.parse('kombu<4,>=3.0.1'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/pytest", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python3.6/site-packages/_pytest/config.py", line 50, in main
    config = _prepareconfig(args, plugins)
  File "/usr/lib/python3.6/site-packages/_pytest/config.py", line 160, in _prepareconfig
    pluginmanager=pluginmanager, args=args)
  File "/usr/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/usr/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/usr/lib/python3.6/site-packages/pluggy/callers.py", line 196, in _multicall
    gen.send(outcome)
  File "/usr/lib/python3.6/site-packages/_pytest/helpconfig.py", line 68, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/usr/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.6/site-packages/_pytest/config.py", line 943, in pytest_cmdline_parse
    self.parse(args)
  File "/usr/lib/python3.6/site-packages/_pytest/config.py", line 1108, in parse
    self._preparse(args, addopts=addopts)
  File "/usr/lib/python3.6/site-packages/_pytest/config.py", line 1071, in _preparse
    self.pluginmanager.load_setuptools_entrypoints('pytest11')
  File "/usr/lib/python3.6/site-packages/pluggy/__init__.py", line 402, in load_setuptools_entrypoints
    "Plugin %r could not be loaded: %s!" % (ep.name, e))
pluggy.PluginValidationError: Plugin 'pytest_nameko' could not be loaded: (kombu 4.1.0 (/usr/lib/python3.6/site-packages), Requirement.parse('kombu<4,>=3.0.1'))!

When lowering down kombu to 3.0.17 i get error from celery (something about imports and other errors as well).
Am I missing something obvious or I just can't use nameko and celery in one project due to kombu versions mismatch (can you suggest something in this case)?

To the continuation of my previous post, actual code running nameko instruction doing just fine. Successfully sent an event to my microservice from django. Pytest's still an issue though.

Can we get an update on this?
We're trying to use amqps, which is not supported in kombu <4.0.0, which we've sort of gotten around by using amqp with ?ssl=true at the end. It mostly works, but it spits out a bunch of errors and it's causing issues in our automated deployments (connections aren't being reliably closed when our containers are replaced, causing our RabbitMQ to slowly run out of memory). We've forked nameko and upgraded the dependency to use kombu >=4.0.0 and switched our uris to use amqps and everything seems to be working great (but of course the nameko tests don't pass).
We need SSL at every level due to HIPAA requirements. We might just run a parallel nameko repository and periodically merge official changes into it, with the only difference being the kombu version. I don't love that solution, but this is a pretty significant issue for us.

I would still like to ditch kombu in favour of a pure AMQP library, and am getting closer to that point in a couple of branches on my fork.
In the mean time though, I think we should look at upgrading to kombu 4 and dropping support for older versions. If anyone here has the time and inclination to experiment with that I'd be receptive to a pull request.
@bweaver-cxp does the change in #524 help you?

It doesn't seem to, but I might be using it wrong. What should I have under AMQP_SSL in my config?
The error we fixed by upgrading to kombu 4.0.0 is this:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/amqp-1.4.9-py3.6.egg/amqp/transport.py", line 116, in __del__
    self.close()
  File "/usr/local/lib/python3.6/site-packages/amqp-1.4.9-py3.6.egg/amqp/transport.py", line 141, in close
    self._shutdown_transport()
  File "/usr/local/lib/python3.6/site-packages/amqp-1.4.9-py3.6.egg/amqp/transport.py", line 217, in _shutdown_transport
    self.sock = unwrap()
  File "/usr/local/lib/python3.6/site-packages/eventlet-0.21.0-py3.6.egg/eventlet/green/ssl.py", line 265, in unwrap
    super(GreenSSLSocket, self).unwrap))
  File "/usr/local/lib/python3.6/site-packages/eventlet-0.21.0-py3.6.egg/eventlet/green/ssl.py", line 109, in _call_trampolining
    return func(*a, **kw)
  File "/usr/local/lib/python3.6/ssl.py", line 1055, in unwrap
    raise ValueError("No SSL wrapper around " + str(self))
ValueError: No SSL wrapper around <eventlet.green.ssl.GreenSSLSocket fd=6, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('<IP_ADDRESS>', 50800)>

Oh interesting. I see this same error in the branch in #524, but hadn't noticed the negative side-effects. I guess there's a bug in Eventlet's green SSL implementation, but it may also be in pyamqp. Kombu 4 jumps to pyamqp 2.0, either fixing or avoiding this problem.
If we can identify and work around this bug perhaps the 4.0 upgrade isn't required.
#524 assumes you want to do certificate verification of the client, which I will relax. For now, the AMQP_SSL params need to be:
AMQP_SSL:
    ca_certs: <certificate-authority-cert-location>
    certfile: <client-cert-location>
    keyfile: <client-key-location>

Actually the branch at #524 doesn't require you to provide client certificates. You can specify AMQP_SSL  as just:
AMQP_SSL: True

The "No SSL wrapper" error can be reproduced by running the following a few times, even without eventlet:
from kombu import Connection

conn = Connection("amqp://guest:guest@localhost:5671/", ssl=True)
conn.connect()
assert conn.connected
conn.release()

So the bug is plainly in pyamqp. I looked for an issue there but haven't found one.

We don't know for sure that that particular error is causing our unwanted behavior, we just know that when we don't use SSL (or when we used the most recent Kombu) we don't see our issue, and that correlates with not getting that error. And our problem is only in a very specific environment; it doesn't happen locally or in manual starting and stopping of (Docker) containers on AWS, just in automated deployments on AWS. It's hard to pin down exactly what/where/why the problem is.
I tried both of those configuration options, not realizing that this branch wasn't meant to fix the "No SSL wrapper" error.

This traceback is fixed in pyamqp 2.0.3 by this commit: https://github.com/celery/py-amqp/commit/7db1b2d4a5cfc546ab941697ae7035ff34645a5a#diff-e3b09014ce12269dff5eece1f9f7c082
I think that __del__ is sometimes called by the garbage collector after the connection has already been unwrapped. If I comment out the conn.release() I can no longer reproduce the error.
Unfortunately I think this means it's unlikely to be the cause of whatever is leaving your connections open. I have just backported the fix into #524 so you can try and see?

#524 does fix the SSL wrapper error and our connections are still not closing. So I have no idea what the issue or fix is, except that SOME commit between py-amqp v1.4.9 and v2.2.2 must do the trick. I might try a deployment at each version to narrow down the search.
It's nice to have all those logs cleaned up though. Makes it easier to see what's happening.

Can someone clarify is there are still major incompatibility with kombu>4.0? We are using celery, which depends on kombu 4.x. This issue hardly influence the architectural choices, because it can not be used concurrently with celery.
Thanks.

We are currently in a situation where monolithic application is using celery and in a process of migration to soa/microservices architecture. We had two requirements:

use flask-nameko or
share a codebase in one environment

We can not split environments due to current architecture. Anyway, this approach introduces unnecessary complexity to support and deploy.
I think it would be great to freeze development of 2.x version and start working on new 3.x that will be backwards incompatible (possibly).

Fixed in #564

for future reference i wonder if we want to hold off closing issues until we release a version with the fix


fix(typo): fix then/than in rounded error messages

Got the error yesterday and spotted the typo
See also #1327
All Submissions:

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x] Does your submission pass tests?

@nmattia thanks much!


auto-animate breaks after scroll

https://codesandbox.io/s/hopeful-mayer-q5zz9o

I had the same problem.
The way I reproduce it is to modify the structure of  A quick demo through DevTools on the official website.
Give it a fixed height. Then set the y-axis overflow hide.

Then add a lot of items, then scroll to the middle, click to delete one, and this problem will occur.

same issue here... is there any progress?

I'm having this issue as well - for me, it appears to be related to the way scroll position is calculated. It does not correctly compensate for elements within a nested scroll element. A solution to this would be introducing an option to define which element the scroll offset should be calculated from


Include SASL functionality in a new release

I noticed the most recent release (0.0.14) is missing some functions on the master branch like irc_set_sasl and lsi_ut_sasl_mkplauth. Could you create a new release please?

Hi, sorry for not getting back to you earlier.
0.0.14 did not support SASL yet, so that's why those functions aren't included.  Unfortunately there's a couple loose ends right now so I don't feel like making a new release.
Please use the devel branch instead, at least for now.  It's stable.  If SASL turns out not to work in the devel branch, please tell me about it.


Clarify expectations upfront , provide deadlines where possible

This action item came from Hawaii Retro.

Plan to do so w/ sprint planning and releases - to open new card if this is still an issue.


Portal GUI changes -- Issue #1295

Fixes #1295.
Changes in this pull request:

Made GUI changes stated in the issue.
Fixed group changes pagination.

One observation:
For Group Change History changes, the pagination seems to have some issue.

One observation: For Group Change History changes, the pagination seems to have some issue.

Fixed it in the latest commit. Please have a look.

One observation: For Group Change History changes, the pagination seems to have some issue.

Fixed it in the latest commit. Please have a look.

Looks good.


DS-4264 item-submission.xml doesn't match the dtd

This is a duplicate of DS-4264
The submission-definitions.submission-process.step doesn't match the dtd definition. E.g. https://github.com/DSpace/DSpace/blob/b37d0e2/dspace/config/item-submission.xml#L170 is missing processing-class, type based on the dtd.

Closing, this duplicates https://github.com/DSpace/DSpace/issues/2786


Option to disallow multiple parallel unpacks of the same layer

What is the problem you're trying to solve
With the way things work right now, there's nothing stopping a parallel unpack of the exact same layer to a snapshot. The first one to get committed will live on while the other(s) get garbage collected so in the end things work out, but regardless of this it seems like wasted work. The real issue is that while unpack should be pretty cheap on Linux, the opposite is true for the Windows and lcow formats. Kicking off 10 parallel pulls of the same image brings my 6 core machine to a halt and pushes 100% cpu utilization. What all of this ends up causing is exponentially slower parallel pull times for images that either share layers, or just pulling the same image.
Fetch also becomes quite slow as you ramp up the pulls but @fuweid has a PR out to remedy this and it's quite an improvement from local testing:  https://github.com/fuweid/containerd/commit/5fd51782d87751f729ebf2540660a4df86532f3d
After doing some digging it seems like there was an issue years ago where parallel snapshots for the same layer would fail as the key format wasn't really designed to get around collisions, it was simply fmt.Sprintf("extract-%s", key), where key was the layer sha. https://github.com/containerd/containerd/issues/1240. @dmcgowan Had remedied this by adding a bit of uniqueness to the key format for active extraction snapshots with this PR https://github.com/containerd/containerd/pull/1480, but the discussion in the PR is what was interesting. The notion of waiting for completion of a different snapshot was brought up. This is truthfully the behavior we want on Windows (and I don't see a reason not to strive for it for any platform, doesn't feel right to waste cycles doing the same thing if it's avoidable).
I'd taken a stab at solving this, but the layer at which I'd added the work is likely non-ideal so I'm looking for feedback. It's currently housed in the metadata snapshotter that will eventually plumb the call down to whatever snapshotter the client requested. The logic is if a new RemoteContext option is specified, we revert to the old key format to intentionally cause a collision on the snapshot key, and the metadata snapshotter keeps track of any "in progress" active snapshots using the key. If snapshot 1 is somewhere on its lifecycle from active -> diff applied to snapshot -> committed (or something went wrong and Remove is called), and snapshot 2 (3, 4 or any N) being made for the same layer comes around, it will check if there's an entry for a snapshot of the same layer and simply wait on the result instead of creating a new snapshot and proceeding onwards to unpack the same layer.
Link of the WIP (There's likely races and concurrency issues here but gives you an idea of where I was headed).
https://github.com/dcantah/containerd/commit/72afe250a918f59f44209f1f72fa74145f134939
Below are some numbers from testing my hacky fix mentioned + @fuweid's work vs. Containerd built from main.:
Single Pull With Two Fixes
PS C:\Users\dcanter\Desktop\ctrd> $a=1..1 | %{start-job {C:\Users\dcanter\Desktop\ctrd\crictl.exe pull cplatpublic.azurecr.io/nanoserver_many_layers:latest}}; $a | wait-job | receive-job; $a | %{$.psendtime-$.psbegintime} | % totalseconds
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
23.2647532
PS C:\Users\dcanter\Desktop\ctrd> .\crictl.exe rmi cplatpublic.azurecr.io/nanoserver_many_layers:latest
Deleted: cplatpublic.azurecr.io/nanoserver_many_layers:latest
10 Parallel Pulls With Two Fixes
PS C:\Users\dcanter\Desktop\ctrd> $a=1..10 | %{start-job {C:\Users\dcanter\Desktop\ctrd\crictl.exe pull cplatpublic.azurecr.io/nanoserver_many_layers:latest}}; $a | wait-job | receive-job; $a | %{$.psendtime-$.psbegintime} | % totalseconds
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
25.3406896
25.1486873
25.0266887
24.8586885
24.7116929
24.5726943
24.4816912
24.375693
24.250694
24.1176895
PS C:\Users\dcanter\Desktop\ctrd> .\crictl.exe rmi cplatpublic.azurecr.io/nanoserver_many_layers:latest
Deleted: cplatpublic.azurecr.io/nanoserver_many_layers:latest
10 Parallel Pulls With Containerd Built Off Main (No Windows defender exclusion on data directory):
PS C:\Users\dcanter\Desktop\ctrd> $a=1..10 | %{start-job {C:\Users\dcanter\Desktop\ctrd\crictl.exe pull cplatpublic.azurecr.io/nanoserver_many_layers:latest}}; $a | wait-job | receive-job; $a | %{$.psendtime-$.psbegintime} | % totalseconds
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
134.416318
134.2893209
134.1643238
134.027323
133.8813211
133.7333303
133.5593271
133.4253269
133.2963282
133.1643288
10 Parallel Pulls With Containerd Built Off Main (With Windows defender exclusion on data directory):
PS C:\Users\dcanter\Desktop\ctrd> $a=1..10 | %{start-job {C:\Users\dcanter\Desktop\ctrd\crictl.exe pull cplatpublic.azurecr.io/nanoserver_many_layers:latest}}; $a | wait-job | receive-job; $a | %{$.psendtime-$.psbegintime} | % totalseconds
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
Image is up to date for sha256:f86afbf779d83cf5aa2cc8377ea0b9f1cea140de22e1189270634d3a8ca4cf0e
69.4417718
69.2987744
69.1677747
69.0297754
68.8907764
68.7397799
68.5937793
68.4547814
68.3187812
68.1597841
Describe the solution you'd like
A clean way to try and keep the client interface for pulling unchanged if possible, while avoiding the need to unpack the exact same layer in parallel if a fully committed snapshot with the layer is not complete.

@dmcgowan @stevvooe Curious on your thoughts about this as you seem to have a lot of background and were present/made the original fix for parallel extractions.

Closing as https://github.com/containerd/containerd/pull/6702 remedied this :)


run-01 naming (when only one run)

It would be helpful to have an option to label functional runs with run-01 even if there is only one run (right now, doesn't include run indication)
This would ease looping over data in scripts, e.g. creating GLMs for task data or post-processing resting state, where currently the naming is inconsistent depending on how many runs are acquired
sub-ID_ses-SES_task-rest_run-01_bold.nii.gz
vs.
sub-ID_ses-SES_task-rest_bold.nii.gz

If you add the run label to your BIDS data, it should carry over to the fMRIPrep derivatives. I would lean towards leaving this up to the user rather than have fMRIPrep rename files

Also note that because run is an index, zero padding will be lost.

@dclb just ran into this causing an upstream mismatch with the source dataset. it would be nice if fmriprep produced the same naming convention as the original run.

What version? IIRC we did make some changes so that only entities that are combined across are removed, but I don't recall which version.


Does not work for ansible 2.x

The only change I had to make was to explicitly define the type for 'options':
            options    = dict(type='list',
                              default=['daily',
                                       'missingok',
                                        ...

Hi @memelet.  I believe you are using an old version.  Please pull master, and let me know if that works.

Yes, you are correct. I was using an old version. I really wish there some better way to manage dependencies than manually deleting and the re-running requirements.txt.
Sorry for the noise.

all good


k8s plugin - kubectl cluster-info is hard to read

The output is formatted strangely and hard for a user to read, when using the same command in a mac terminal the output is formatted normal.
KUI returning these appended to the output strings:
[0;32m
[0m

Duplicate of https://github.com/IBM/kui/issues/1420


Save files with lowercase filenames

Coverage remained the same at 7.84% when pulling cb414b543d66c1bb87770649ab540021dad5d997 on rvanlaarhoven-patch-1-1 into 334bf230dcf3d85ba96beeb9c6b820324e732474 on master.


Fix bug on Datatables javascript assets

The Datatables scripts is not rendered

@enniosousa  we are already fixed in the latest version and we are using @push and @stack not @section. see here


New option list_filter_sheet does not seem to work

What version of Unfold are you using?
v0.41.0
What version of Django are you using?
5.1.2
What browser are you using?
Firefox latest
Did you checked changelog/commit history, if the bug is not already fixed?
Yes
Did you searched other issues, if the bug is not already fixed?
Yes
Did you checked documentation?
Yes. The new option list_filter_sheet is not documented.
Are you able to replicate the bug in the demo site?
The demo site shows the new filter UI, but when I cloned the demo site repo, old filter UI is shown. Does the repo contain the latest deployed code of the demo site?
Repository with reproduced bug
Simply clone the demo site repo and follow the instructions. visit the "drivers" change list page and see the filter UI and compare with the demo site.
Screenshot
Describe your issue
Hi there, I am trying to make the new filter UI work, but I am unable to. The demo site shows the new filter UI, but on my local (formula repo clone) the default filter UI is shown (see the drivers admin changelist for example). The default setting of list_filter_sheet=True doesn't seem to work either.
Screenshot
Am I missing something?
sample code:
from unfold.admin import ModelAdmin
from unfold.contrib.filters.admin import RelatedDropdownFilter

class LocationAdmin(ModelAdmin):
    list_display = (
        "country",
        "name",
        "subdivision",
    )
    list_filter = (
        ("country", RelatedDropdownFilter),
    )
    ordering = ("-created_at",)
    list_fullwidth = True
    list_filter_submit = True
    list_filter_sheet = True

Yes, it was missing in the docs which I already updated. You are supposed to set list_filter_sheet to False to enable "sidebar" version.

@lukasvinclav  Thanks for the clarification. I should have tried setting it to False, lol.


Add support for prompt_vars

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[Issue 7517][pulsar-broker] Reestablish namespace bundle ownership from false negative releasing and false positive acquiring

Fixes #7517
Motivation
In acquiring/releasing namespace bundle ownership, zk may disconnected before or after these operations persisted in zk cluster. This introduce inconsistency between local ownership cache and zk cluster.
Modifications
This commit does two things:

In ownership releasing, don't retain ownership in failure.
In ownership checking, querying and acquiring, reestablish lost ownership in false negative releasing and false positive acquiring.

i.e. this commit admits that the inconsistency between local ownership cache and zk cluster could not be avoided totally, thus performs ownership correction in checking, querying and acquiring.
Verifying this change

[ ] Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

Add tests to mock zk disconnection in namespace bundle unloading and acquiring. Existing code fails in these tests, this pr pass.
Add test to verify whether namespace bundle ownership could be reestablish after cache invalidated.

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

Documentation

Does this pull request introduce a new feature? (no)

/pulsarbot run-failure-checks

@codelipenghui @jiazhai can you review it?


Fftw registration warnings

These two commits fix the warnings of the dashboard results here and here.

Thanks for the review and sorry for not checking the style before hand. It seems to be fixed now.


Exit code for --download-archive

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.07.02. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.07.02

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Output
$ youtube-dl -v --download-archive archive https://www.youtube.com/watch?v=tPEE9ZwTmy0
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--download-archive', u'archive', u'https://www.youtube.com/watch?v=tPEE9ZwTmy0']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.07.02
[debug] Python version 2.7.12 - Linux-4.4.0-83-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 2.8.11-0ubuntu<IP_ADDRESS>, ffprobe 2.8.11-0ubuntu<IP_ADDRESS>, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] tPEE9ZwTmy0: Downloading webpage
[youtube] tPEE9ZwTmy0: Downloading video info webpage
[youtube] tPEE9ZwTmy0: Extracting video information
[youtube] tPEE9ZwTmy0: Downloading MPD manifest
[download] Shortest Video on Youtube has already been recorded in archive
$ echo $?
0

Issue
The exit code is 0 even though no video is downloaded. This makes it impossible in a script to determine if youtube-dl actually downloaded something. Could we have an appropriate exit code, like with --max-downloads? See also https://github.com/rg3/youtube-dl/issues/10388

IMO an exit case of 0 in this case is correct; there was no failure, everything worked as expected, so a nonzero exit code would not be convention. There are better ways to do what you want.
--max-downloads should arguably not be abusing the exit code in such a way either.

I'm having the opposite problem.
youtube-dl ytuser:${channel} --console-title -f '298+bestaudio[ext=m4a]/137+bestaudio[ext=m4a]/best[ext=mp4]/best' --youtube-include-dash-manifest --output "%(upload_date)s-%(playlist_index)s-%(title)s-%(id)s.%(ext)s" --restrict-filenames --download-archive ${channel}-youtube --ignore-errors --sleep-interval 3 --playlist-reverse --socket-timeout 11 --write-sub --sub-format 'ass/srt/best' --sub-lang en --embed-subs --rate-limit 7M --dateafter 20171022 --datebefore $(date --date='3 days ago' '+%Y%m%_d')
...
[download] MY HANGOVER MAKEUP ROUTINE // Grace Helbig has already been recorded in archive
[download] Downloading video 753 of 754
[download] MTV EMAs FASHION REVIEW w/KRISTEN MCATEE // Grace Helbig has already been recorded in archive
[download] Downloading video 754 of 754
[youtube] QdOVWsqPp2E: Downloading webpage
[youtube] QdOVWsqPp2E: Downloading video info webpage
[youtube] QdOVWsqPp2E: Extracting video information
WARNING: video doesn't have subtitles
[youtube] QdOVWsqPp2E: Downloading MPD manifest
[download] 2017-11-17 upload date is not in range 2017-10-22 - 2017-11-16
[download] Finished downloading playlist: Uploads from Grace Helbig
Error code: 1
[youtube:user] graciehinabox: Downloading channel page
[youtube:playlist] UUIiBf-JbtCazHSFqXV4JgoA: Downloading webpage
[download] Downloading playlist: Uploads from Grace Helbig

youtube-dl has completed what I consider a successful run but isn't exiting with a 0.
Was this changed since this bug was created? It doesn't even happen on every channel.

... what I consider a successful run

You asked for subtitles that weren't there. Probably the only way to analyze the result of a run intelligently would be to d/l and check the info-json files for each item, but that sounds like more work than any normal person would do (or program).
For OP's original problem, wrap the command in a script with this logic:

Copy archive to a temporary file, or make the file empty if archive doesn't yet exist.
Run the yt-dl command including the option --download-archive archive.
Compare archive with archive.tmp (or whatever it was called): if different, something was downloaded.
Delete the temporary file.


Bug: pressing keys faster

Pressing keys faster, like D and S faster, results in a collision.
Don't know why yet, only hypothesis.
Probably checkCollision() or direction().

So, here is the solution:
Indeed, it was the pressing the key faster than window.pollEvent(event).
Using this particular case of the issue, it happens (in a curt period of time):
Press D -> update direction from 0 (up) to 1 (right).
Press S -> Update direction from 1 (right) to 2 (down).
So, according to snake.direction() this is fine... But our snake hasn't been updated yet, we haven't moved yet! So this make the snake go down, making the collision (an internal one).
The code solution was simply to add a break statement after the switch, in this way, we read the key and close the loop.


Enable HIDE_SYMBOLS_BY_DEFAULT + patches (take II)

🎉 New feature
Part of https://github.com/gazebosim/gz-cmake/pull/392 and https://github.com/gazebosim/gz-cmake/issues/166.
Replaces #600.
Summary
The PR enables the HIDE_SYMBOLS_BY_DEFAULT option and patch the failures found during the build on Linux. Mainly by adding the default visibility behaviour for the custom define on each plugin.
Checklist

[x] Signed all commits for DCO
[ ] Added tests
[ ] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] Consider updating Python bindings (if the library has them)
[ ] codecheck passed (See contributing)
[ ] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

I think that the failures on the doxygen checks are related to https://github.com/doxygen/doxygen/issues/9142. I'm unable to remove them if we add the following to gz-cmake:
diff --git a/doc/doxygen/api.in b/doc/doxygen/api.in
index b3d72b2..cf6b9c4 100644
--- a/doc/doxygen/api.in
+++ b/doc/doxygen/api.in
@@ -2138,7 +2138,7 @@ INCLUDE_FILE_PATTERNS  =
 # recursively expanded use the := operator instead of the = operator.
 # This tag requires that the tag ENABLE_PREPROCESSING is set to YES.

-PREDEFINED             =
+PREDEFINED             = __attribute__(x)=

 # If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then this
 # tag can be used to specify a list of macro names that should be expanded. The

It might be quicker to change the names of the defines to be GZ_GUI_VISIBLE since those are filter out https://github.com/gazebosim/gz-cmake/blob/200de1b26edbad9ad1d7efd7c29dd4f2080edefa/doc/doxygen/api.in#L953

It might be quicker to change the names of the defines to be GZ_GUI_VISIBLE since those are filter out https://github.com/gazebosim/gz-cmake/blob/200de1b26edbad9ad1d7efd7c29dd4f2080edefa/doc/doxygen/api.in#L953

I'm not sure that we are able to do it, I tried in #600 and stopped that approach. The GZ_GUI_VISIBLE is used for gui libraries to control export/import of the symbols when building but in the PR there are plugins involved that need a different keyword to always import the library symbols.

My suggestion is not to use the GZ_GUI_VISIBLE that is defined in Export.hh, but to use the name GZ_GUI_VISIBLE as the name of the define. Eg. instead of
#ifndef _WIN32
#  define ImageDisplay_EXPORTS_API __attribute__ ((visibility ("default")))
#else
#  if (defined(ImageDisplay_EXPORTS))
#    define ImageDisplay_EXPORTS_API __declspec(dllexport)
#  else
#    define ImageDisplay_EXPORTS_API __declspec(dllimport)
#  endif
#endif

do
#ifndef _WIN32
#  define GZ_GUI_VISIBLE __attribute__ ((visibility ("default")))
#else
#  if (defined(ImageDisplay_EXPORTS))
#    define GZ_GUI_VISIBLE __declspec(dllexport)
#  else
#    define GZ_GUI_VISIBLE __declspec(dllimport)
#  endif
#endif

Would that work?

Would that work?

Ah I see. That would not work out of the box since the buildsystem is generating the -D*_EXPORTS_API definition when building the plugins and not using GZ_GUI_VISIBLE as it is currently. We could add it but given that we are going to mix in the compilation plugins and library headers it would be better than them have different keywords to control visibility. Another reason not to reuse the name is the confusion that it can generate in the developers or users since they are going to think that plugin visibility is handle the same way than library/component visibility with the same GZ_GUI_VISIBLE keyword, which is clearly not the case.

Would that work?

Ah I see. That would not work out of the box since the buildsystem is generating the -D*_EXPORTS_API definition when building the plugins and not using GZ_GUI_VISIBLE as it is currently. We could add it but given that we are going to mix in the compilation plugins and library headers it would be better than them have different keywords to control visibility. Another reason not to reuse the name is the confusion that it can generate in the developers or users since they are going to think that plugin visibility is handle the same way than library/component visibility with the same GZ_GUI_VISIBLE keyword, which is clearly not the case.

Okay, let's go with the gz-cmake changes then.


Additional Context in JsonReaderException

Thanks for a great library - use it every day.
Would it be possible to add in some additional context in a JsonReaderException's error message? Right now it looks like:
Unexpected character encountered while parsing value: P. Path '', line 0, position 0.

Could it add some additional characters around the parsing error? For example:
Unexpected character encountered while parsing value: P. Path '', line 0, position 0. Text in context: 'Parsing this will err[...]'
--------------------------------------------------------------------------------------------------------^

or
Unexpected character encountered while parsing value: t. Path '', line 0, position 22. Text in context: '[...]ata": "" this is incorrect JSO[...]'
-----------------------------------------------------------------------------------------------------------------------^

A third-party library we're using is throwing this exception up to us after deleting the JSON it was trying to parse, so any hints as to what JSON they were trying to parse is helpful.

The problem is Json.NET is a forward only reader. Reading forward to get more text for the error message prevents a user being able to recover from the error and do something else.
Also, with an error the reader doesn't know what state it is in. How does it know how far ahead to read?


Public constructors for Request and Reply types

It would be nice to be able to unit test my filesystem's FUSE interface.  Right now the only way to test it is to mount an actual filesystem and use the command line.  But if rust-fuse did the following, then I would be able to write tests that hook into my filesystem's rust-fuse interface and exercise the whole API.  This would require:

Public constructor (possibly Default for Request)
Public constructors for all the various Reply types.
Public accessors for the various Reply types, for example to see the data stored by ReplyData::data.

So much this.
Trying to use rust-fuse as a library crate is all but impossible atm. Would this be a better suited task for implementing a high-level API?
New to the library, and to Rust, and would really love to use rust-fuse in a project. Happy to contribute where I can.
Many thanks in advance.

Back then when rust-fuse was created, we intentionally chose to have no public constructors to ensure that implementations comply with what the kernel protocol requires. Nowadays the interface on the Rust side still works the same but looks a bit unusual. There's ongoing effort (e.g. in #90 and the modernize branch) to make rust-fuse more idiomatic and especially add async and testing capability. I suppose that request and reply types will completely change so that this issue won't apply anymore. But afaik there is no issue specific to testability yet, so if you edit this issue to be more generic towards the new implementation, we could use it for collecting ideas and tracking progress of making fuse implementation testing easier.

I've been working on a new approach to testing crates that use rust-fuse.  I'm using mockall to create mock versions of each Request and Reply type.  Then I have to redefine FileSystem to take these mock types instead of the real ones.  It's a lot of boilerplate, but it seems to work.


Fix after_scenario error when before_feature is not used

Coverage remained the same at 90.841% when pulling cc476292c062639aff2819b7b1f5adc8b4f4e3d6 on fix/reuse_tag into 52661703b4ba1854d867a82df6e8ef3f4c5ca36c on master.