Allow users to create operators from existing functions

Describe your changes and why you are making these changes
In this PR, I added a method call to_operator() in SDK. When we have an existing python function, we can call on to_operator(func) to covert it to Aqueduct Operator instead of redefining using @op. The integration test checks whether there is any difference between @op defined operator and to_operator defined operator

Update 1.0
After Wei's PR on enabling passing in imported function, Now the to_operator() can takes in locally defined function and imported function and convert them to an Aqueduct operator for use. I have also added an integration test to make sure it is working.
Related issue number (if any)
Eng 1318
https://linear.app/aqueducthq/issue/ENG-1318/allow-users-to-create-operators-from-existing-functions
Checklist before requesting a review

[x] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
[x] I have performed a self-review of my code.
[x] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
[x] If this is a new feature, I have added unit tests and integration tests.
[x] I have run the integration tests locally and they are passing.
[x] I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
[x] All features on the UI continue to work correctly.
[x] Added one of the following CI labels:

run_integration_test: Runs integration tests
skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

@Fanjia-Yan, can you test if this works with imported functions? i.e., I have a file called test.py with a function test_fn, and I want to do import test_fn; to_operator(test_fn) and then use it?

@Fanjia-Yan, can you test if this works with imported functions? i.e., I have a file called test.py with a function test_fn, and I want to do import test_fn; to_operator(test_fn) and then use it?

It seems like the imported function does not work. I am looking into how to enable imported function be converted to an operator.

This might be because of the package issue that @cw75 was talking about last week. What is the error you're getting @Fanjia-Yan?

This might be because of the package issue that @cw75 was talking about last week. What is the error you're getting @Fanjia-Yan?
Let me try to reproduce it on jupyter notebook

This might be because of the package issue that @cw75 was talking about last week. What is the error you're getting @Fanjia-Yan?
Let me try to reproduce it on jupyter notebook

This is the one I got from integration test

It seems like the function from other file is not successfully recorded by the DAG and when the aqueduct executor parse the function file, it fails.

Hm, the logs seem incomplete. I believe this is because when you call cp.load on the function, it's not able to find the package. Can you check the server logs to see what the full error message is?

Hm, the logs seem incomplete. I believe this is because when you call cp.load on the function, it's not able to find the package. Can you check the server logs to see what the full error message is?

The Complete Server log from terminal:
ERRO[0961] Unexpected error occurred while executing job function-operator-3b5cc6fe-e62b-4de2-b27f-8c30f7e8befd: exit status 1. Stdout:
Job Spec:
{"name": "function-operator-3b5cc6fe-e62b-4de2-b27f-8c30f7e8befd", "type": "function", "storage_config": {"type": "file", "file_config": {"directory": "/home/ubuntu/.aqueduct/server/storage"}, "s3_config": null}, "metadata_path": "7dc6c796-721a-4d2c-894f-8ee2d040de3e", "function_path": "operator-1b303499-99d7-4966-b4e3-e6b644772ecc", "function_extract_path": "/home/ubuntu/.aqueduct/server/storage/operators/735bd249-a846-4398-ae17-aeedf5205a19", "entry_point_file": "model.py", "entry_point_class": "Function", "entry_point_method": "predict", "custom_args": "", "input_content_paths": ["6a523d19-89da-4515-924e-a65eb02a44a8"], "input_metadata_paths": ["46517a93-91b5-4bf9-b68c-6d7f3d4925e0"], "output_content_paths": ["e20bd60c-610c-4008-a176-59a15478baad"], "output_metadata_paths": ["cb4610f1-e080-49cd-a046-993ccd448d9a"], "input_artifact_types": ["table"], "output_artifact_types": ["table"]}
reading from file: /home/ubuntu/.aqueduct/server/storage/operator-1b303499-99d7-4966-b4e3-e6b644772ecc

A bunch of requirement already satisfied statement

Invoking the function...
Failed with system error. Full Logs:
{"user_logs": {"stdout": "", "stderr": ""}, "status": "failed", "failure_type": 1, "error": {"tip": "Sorry, we've run into an unexpected error! Please create bug report in github: https://github.com/aqueducthq/aqueduct/issues/new?assignees=&labels=bug&template=bug_report.md&title=[BUG] . We will get back to you as soon as we can.", "context": "  File "/home/ubuntu/.local/lib/python3.8/site-packages/aqueduct_executor/operators/function_executor/execute_function.py", line 119, in run\n    results, system_metadata = _execute_function(spec, inputs, exec_state)\n  File "/home/ubuntu/.local/lib/python3.8/site-packages/aqueduct_executor/operators/function_executor/execute_function.py", line 82, in _execute_function\n    invoke = _import_invoke_method(spec)\n  File "/home/ubuntu/.local/lib/python3.8/site-packages/aqueduct_executor/operators/function_executor/execute_function.py", line 61, in _import_invoke_method\n    function = fn_class()\n  File "/home/ubuntu/.aqueduct/server/storage/operators/735bd249-a846-4398-ae17-aeedf5205a19/op/model.py", line 7, in init\n    self.func = cp.load(f)\n"}}
writing to file: /home/ubuntu/.aqueduct/server/storage/7dc6c796-721a-4d2c-894f-8ee2d040de3e
Stderr:
ERRO[0961] Unable to clean up storage file with key: cb4610f1-e080-49cd-a046-993ccd448d9a. remove /home/ubuntu/.aqueduct/server/storage/cb4610f1-e080-49cd-a046-993ccd448d9a: no such file or directory.
ERRO[0961] Unable to clean up storage file with key: e20bd60c-610c-4008-a176-59a15478baad. remove /home/ubuntu/.aqueduct/server/storage/e20bd60c-610c-4008-a176-59a15478baad: no such file or directory.

That's super weird. I wonder why we're cutting this off -- it doesn't actually say what the error is. @likawind @cw75 are there any other places we could check for logs?

I think the logs are off because it happens in a really unusual place - we use cloudpickle to load the pickled user function to our wrapper, and then this wrapper object get dynamically imported. I feel either we have something wrong with fetching the trace, or the dynamic import part might cut some stack trace for us

https://linear.app/aqueducthq/issue/ENG-1445/investigate-why-traces-are-off-for-dynamic-imports

@Fanjia-Yan we will look at this together but there are also thing you could try out:

try pull and update to the latest version. This shouldn't fix your issue but would bring us to the same page on where to look at.
try to manually unzip the operator, import the object, and initialize it as what's been done in _import_invoke_method. Hopefully this would reproduce the bug

Update 1.0
After Wei's PR on enabling passing in imported function, Now the to_operator() can takes in locally defined function and imported function and convert them to an Aqueduct operator for use. I have also added an integration test to make sure it is working.

@Fanjia-Yan, thanks for pushing through on this. As of your latest update, it sounds like the example test case I mentioned previously work? If so, I think this should be good to go from a product perspective!
I'll let someone who knows the SDK codebase better than me review the system internals.


Update curl dependency due to CVE-2024-9681

See https://curl.se/docs/CVE-2024-9681.html
Microsoft Reviewers: Open in CodeFlow

The build error could probably be fixed by updating the baseline.

Looks like the official repo has not been updated from 8.10.1 yet... https://vcpkg.io/en/package/curl

Guess this will have to wait then...
It isn't in any of the commits marked as "release" but the update is already in vcpkg's repo, so I thought setting the baseline to that commit would be enough. Weird that it worked locally but not on the pipeline

/azp run


about source of the table in the example

[Hello Sir, It would be really helpful if you could include the source of the table database to import and practice the example
as per the steps. Thank you for your generous support!]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6aa9fd82-71d1-0a3f-4034-f9b5f48f544c
Version Independent ID: 5ae72e59-d997-d53d-12ac-92285918acb8
Content: Promote or demote column headers
Content Source: powerquery-docs/table-promote-demote-headers.md
Service: powerquery
GitHub Login: @ptyx507
Microsoft Alias: v-miesco

Hi @ranganathgandikota -
Thanks for your feedback. The samples in the Power Query docs were written by the original author of the articles exclusively for each individual article. There are no current plans to remake the samples and supply them online.


How to get data with relationship

I want use like this code js, but i dont know how to implement in goprisma
const article = await fastify.prisma.article.findFirst({
  include: {
    Website: true
  }
});

const article = await fastify.prisma.article.findFirst({
  select: {
    id: true,
    Website: {
      select: {
        id: true,
        domain: true
      }
    }
  }
});

How to get count in table
const article = await fastify.prisma.article.count();

Please check the article Fetching additional data, I might have to improve the docs regarding that...
Select is currently not supported
Count is currently not supported
Let me know if this resolves your issue for now, please upvote and/or follow the linked issues if you want to have this feature!


[Windows] Can't deploy workloads on windows nodes after adding them to the Custom Azure cluster with external IP

What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):
Windows 1903 Azure image.

Add a Custom Cluster with Azure as Cloud Provider with support of Windows nodes.

I used 1 etcd/Control Plane, 1 linux worker, 2 windows workers.

In the Customize Node Run Command section, click in Show advanced options.
Provide the External IP only for each node.
Wait until all nodes are in Active state.

Result:
After the cluster is Active try deploying a workload in the windows workers. It will be permanently in Unavailable state.
Linux nodes internal IP was auto-detected.
Windows nodes internal IP was not auto detected.

Other details that may be helpful:
Environment information

Rancher version: master-head (v2.3) (09/03/2019)

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): Custom with Azure as cloud provider.

Machine type (cloud/VM/metal) and specifications (CPU/memory): cloud

Docker version (use docker version):

19.03.1

@sowmyav27 @sangeethah I was only able to create workloads successfully if I add both the External and Internal IPs in the Node Run Command.


Feature request \ Empty trash button

Hey,
As in many CMS, a clean trash button could be nice to delete content without cleaning the database directly !
Thank you !

Records can now be destroyed in bulk or individually from the trash thanks to #454.


Static mesh loading problem

CARLA version: cala-dev build from source
Platform/OS: ubuntu 22.04
Problem you have experienced:
I use RoadRunner build a new map and import it into carla. When I open carla in UE, I found that the map is empty. I checked the actor in the world, it is like this:

The static mesh of StaticMeshActor is None. Theoretically I can manually load each static mesh actor manually by myself. But actually there are thousands of Static Mesh files and it is impossible to do it one by one by hand.
What you expected to happen:
Is there any convenient tool that can load the static mesh file automatically? It is quite stange.
Thank you so much for your time!

You can try to import maps using RoadRunnerImportPlugin for UE.


a problem with druid jdbc support

when i use Avatica JDBC driver connected to druid router，i got an error:
org.apache.druid.java.util.common.IAE: Received an Avatica request without a connectionId.

The following is a detailed description：
Affected Version
0.14
Description
when i use Avatica JDBC driver connected to druid router，when avatica-remote send execute request to druid router, an error happend , i found the connectionId is under the key statementHandle  like this:
{"request":"execute","statementHandle":{"connectionId":"b1511e18-c6b8-4c79-a166-977c204511ef","id":1,"signature":{"columns":[{"ordinal":0,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":0,"signed":true,"displaySize":3,"label":"__time","columnName":"__time","schemaName":null,"precision":3,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":93,"name":"TIMESTAMP","rep":"NUMBER"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.sql.Timestamp"},{"ordinal":1,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":0,"signed":true,"displaySize":19,"label":"added","columnName":"added","schemaName":null,"precision":19,"scale":0,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":-5,"name":"BIGINT","rep":"NUMBER"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.Long"},{"ordinal":2,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"channel","columnName":"channel","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":3,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"cityName","columnName":"cityName","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":4,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"comment","columnName":"comment","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":5,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"countryIsoCode","columnName":"countryIsoCode","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":6,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"countryName","columnName":"countryName","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":7,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":0,"signed":true,"displaySize":19,"label":"deleted","columnName":"deleted","schemaName":null,"precision":19,"scale":0,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":-5,"name":"BIGINT","rep":"NUMBER"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.Long"},{"ordinal":8,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":0,"signed":true,"displaySize":19,"label":"delta","columnName":"delta","schemaName":null,"precision":19,"scale":0,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":-5,"name":"BIGINT","rep":"NUMBER"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.Long"},{"ordinal":9,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"isAnonymous","columnName":"isAnonymous","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":10,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"isMinor","columnName":"isMinor","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":11,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"isNew","columnName":"isNew","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":12,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"isRobot","columnName":"isRobot","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":13,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"isUnpatrolled","columnName":"isUnpatrolled","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":14,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"metroCode","columnName":"metroCode","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":15,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"namespace","columnName":"namespace","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":16,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"page","columnName":"page","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":17,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"regionIsoCode","columnName":"regionIsoCode","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":18,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"regionName","columnName":"regionName","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"},{"ordinal":19,"autoIncrement":false,"caseSensitive":true,"searchable":false,"currency":false,"nullable":1,"signed":true,"displaySize":-1,"label":"user","columnName":"user","schemaName":null,"precision":-1,"scale":-2147483648,"tableName":null,"catalogName":null,"type":{"type":"scalar","id":12,"name":"VARCHAR","rep":"STRING"},"readOnly":true,"writable":false,"definitelyWritable":false,"columnClassName":"java.lang.String"}],"sql":"select * from wikipedia where comment = 'added project'","parameters":[],"cursorFactory":{"style":"ARRAY","clazz":null,"fieldNames":null},"statementType":"SELECT"}},"parameterValues":[],"maxRowCount":100}

so, an error occured in org.apache.druid.server.AsyncQueryForwardingServlet#service:
if (isAvatica) {
     .....
     String connectionId = getAvaticaConnectionId(requestMap);
      ......
}
private static String getAvaticaConnectionId(Map<String, Object> requestMap)
  {
   // There should be an error here, because the connectionId is in `statementHandle`
    Object connectionIdObj = requestMap.get("connectionId");
    if (connectionIdObj == null) {
      throw new IAE("Received an Avatica request without a connectionId.");
    }
    if (!(connectionIdObj instanceof String)) {
      throw new IAE("Received an Avatica request with a non-String connectionId.");
    }

    return (String) connectionIdObj;
  }

So this method getAvaticaConnectionId  is not compatible with all requests from avatica-remote

@gianm

I got the same problem, when executing with jdbc connection, the request body is like
{"request":"prepareAndExecute","connectionId":"bc4d1171-60d9-409b-b350-8de8cf205efd","statementId":1,"sql":"SELECT dim4, SUM(metric1Sum)/COUNT(*) as va FROM apm_common_v1 WHERE \"__time\" > TIMESTAMP '2019-10-01 16:18:31' AND \"__time\" < TIMESTAMP '2019-10-25 17:18:31' AND dim2 = '1' AND dim3 = 'android'  AND dim1 in ('network') group by dim4 order by va desc limit 10","maxRowsInFirstFrame":-1,"maxRowCount":-1}
but when executing use mybastis, the request is like above with connectionId in the body of statementHandle, which can't be parsed by server

We come around this problem by setting @Options(statementType = StatementType.STATEMENT) on MyBatis Mapper, which force MyBatis to use Statement instead of PreparedStatement

Oh, interesting, we should fix this in the Router.
Does it work when you query directly against the Broker?

By the way, a patch against AsyncQueryForwardingServlet's getAvaticaConnectionId method to fix this would be greatly appreciated.

Oh, interesting, we should fix this in the Router.
Does it work when you query directly against the Broker?

it work when i query directly against the Broker.

Oh, interesting, we should fix this in the Router.
Does it work when you query directly against the Broker?

it work when i query directly against the Broker.
How to solve this problem , does it work properly in router mode?
@shizhengchao

Are there any plans to patch this in the next updates?

Fixed by #10272.


how do you release to Marketplace?

It's not clear how you release it to the marketplace. Would be great to have this procedure automated, maybe with the help of .rultor.yml (like we do in other projects in this org)

The release is already done with Rultor. We already have a .rultor.yml file in the root folder of the project. To make a new release, just call rultor from an issue with the message "release", followed by the release version


Delegate base directory path management from FileBlockchain to file system storage

Instead of FileBlockchain managing directories where blocks and account root file are stored let's delegate it to storages. So we pass directory to storage on creation and forget about it on FileBlockchain level. Like this:
block_directory = '...'
block_storage = PathOptimizedFileSystemStorage(path=block_directory)

account_root_files_directory = '...'
account_root_files_storage = PathOptimizedFileSystemStorage(path=account_root_files_directory)

This will also enhance security. Right now we pass absolute file paths to storages so you can overwrite something by a mistake. After we pass base directory to the storage it will operate only on files within this directory.

@dmugtasimov Please review https://github.com/thenewboston-developers/thenewboston-node/pull/118

merged


remove vote-program dep from rpc-client

Problem
solana-rpc-client is depending on solana-vote-program just for a const that it re-exports from solana_program
Summary of Changes
Don't do that

Coverage failure is unrelated

This needs a rebase on top of https://github.com/anza-xyz/agave/pull/3774 for CI to pass


postLogoutRedirectUri redirects user to Microsoft office website even when I have not mentioned it anywhere

Please follow the issue template below. Failure to do so will result in a delay in answering your question.
Library

[x]<EMAIL_ADDRESS>or<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>
Description
Please provide your question here, including as much relevant details as possible.
Examples:

"How do I use MSAL with Vue.js"
"How do I SSO between tabs?"
"How do I use MSAL to protect my custom Web API?"
"How can my app support multiple AAD tenants?"
"When will my scenario be supported?"
"When will this framework be supported"

After logging out, I am redirected to office.com with the following URL. It somehow overrides the configuration that I have given.
https://login.microsoftonline.com/common/oauth2/authorize?client_id=&redirect_uri=https%3A%2F%2Fwww.office.com%2Flanding&response_type=code%20id_token&scope=openid%20profile&response_mode=form_post&nonce=637242053377147792.ZmE1OTMwZGQtZmMxOS00ZjAzLWI0OTEtMGZkY2FjY2U2Nzk2ZmRjYjhjMzktYjNkNy00ZGRjLWIwMzMtYTIwMzFhZWVjYjk3&ui_locales=en-US&mkt=en-US&client-request-id=&state=MNJqtJpawyVxi-aka2dq0M2BYENLWDA0U5kaV7epKWrssA1fUa_-W_NLTIrfaGD5EhPOx3ijSEnK8S6TqK1nbeif57uXbUC4mpka7FudDc66W17zW36D-OY_elMsNFmk3JZLABdamTea9e1PJNaKRRQdzbpQ9qTnuLZ99ddmL2mpfDai4Nefuoyv01ii56U5Xcth-jC4PXNzrGvZu3ifiJS4eNHykb0voPIhUkHABjYl_8aQ5oUynn5syAaldW4oNBsm9KYM085FWABDn4fNxw&x-client-SKU=ID_NETSTANDARD2_0&x-client-ver=<IP_ADDRESS>
this is my configuration
auth: {
clientId: environment.clientId,
authority: environment.authority,
redirectUri: environment.redirectUri,
postLogoutRedirectUri : "http://localhost:4200/logout",
navigateToLoginRequestUrl: true,
},
and during my app registration, I have given

@sandesai I'm a bit confused, could you explain what you are expecting to occur when you logout vs what is actually happening? When you say you are "redirected to office.com with the following url", I'm not sure what you mean.
I see that the redirect URL in the url is the office page, which won't work given your registration. I also see response_type=code, which leads me to believe that you are using the @azure/msal-browser package (which is not what you have indicated in the template).
A Fiddler trace or some sort of network log may also help.

@sandesai I'm a bit confused, could you explain what you are expecting to occur when you logout vs what is actually happening? When you say you are "redirected to office.com with the following url", I'm not sure what you mean.
I see that the redirect URL in the url is the office page, which won't work given your registration. I also see response_type=code, which leads me to believe that you are using the @azure/msal-browser package (which is not what you have indicated in the template).
A Fiddler trace or some sort of network log may also help.

Hi Prithvi,
Thank you for the response.
I have not explicitly installed @azure/msal-browser package. It might have been added as part of @azure/msal-angular package if I am not wrong.
Coming back to your question,I would like my application to go back to http://localhost:4200 after logout as I have mentioned in postLogoutRedirectUri in msal configuration instead of redirecting me to log in to Microsoft office.
Step 1 : User clicks on log out
Here logout URL is my intended URL i.e; http://localhost:4200/

Step 2 : Once user select the account, this page is displayed

Step 3 : Next page is this where the redirect URL is suddenly changed to office.com

It should redirect me to localhost:4200 as mentioned in postLogoutRedirectUri is what I am expecting.
Regards,

@sandesai if you are using the @azure/msal-angular package, it is using the msal package, not the @azure/msal-browser package. Which is why I'm a bit confused why you are getting response_type=code in Step 3, as @azure/msal-angular only supports the implicit flow (where response_type=token or response_type=id_token). Are you using another service, like MSAL.NET, on the server side? I suspect what is happening is that the logout is working correctly, and upon returning to localhost:4200 another piece of code is kicking off an authentication request. I can't be sure without knowing more about the application scenario and the actual usage of the @azure/msal-angular package.

@pkanher617
I am using @azure/msal-angular package to integrate Azure Active Directory in my frontend Angular application. I have a node server running in the background.
I have registered both of my applications in the azure portal. The app successfully logs in and everything else works properly apart from the logout scenario.
The package: [(https://www.npmjs.com/package/@azure/msal-angular)]
The sample application that I have followed: [https://github.com/Azure-Samples/active-directory-javascript-singlepageapp-angular]
I have attached a fiddler screenshot which captures the logout scenario

Thank you,

@sandesai Could you send me the Fiddler trace in an email? My email is available on my profile.
I don't believe msal javascript is making the additional call with the office.com redirect uri, but I would like to take a more detailed look at the Fiddler trace to make sure.

@pkanher617
Unfortunately, I cannot share any more details than I already did due to my client's privacy concerns.
Regards,

Understood, unfortunately I can't really help unless I know where the call in Step 3 is originating from. Neither the @azure/msal-angular package nor the msal package craft URLs with response type=code.
If you can reproduce the issue in one of the dev samples in this repo, we may be able to identify the cause faster.

https://login.microsoftonline.com/common/oauth2/authorize?client_id=&redirect_uri=https%3A%2F%2Fwww.office.com%2Flanding&response_type=code id_token&scope=openid profile&response_mode=form_post&nonce=637242053377147792.ZmE1OTMwZGQtZmMxOS00ZjAzLWI0OTEtMGZkY2FjY2U2Nzk2ZmRjYjhjMzktYjNkNy00ZGRjLWIwMzMtYTIwMzFhZWVjYjk3&ui_locales=en-US&mkt=en-US&client-request-id=&state=MNJqtJpawyVxi-aka2dq0M2BYENLWDA0U5kaV7epKWrssA1fUa_-W_NLTIrfaGD5EhPOx3ijSEnK8S6TqK1nbeif57uXbUC4mpka7FudDc66W17zW36D-OY_elMsNFmk3JZLABdamTea9e1PJNaKRRQdzbpQ9qTnuLZ99ddmL2mpfDai4Nefuoyv01ii56U5Xcth-jC4PXNzrGvZu3ifiJS4eNHykb0voPIhUkHABjYl_8aQ5oUynn5syAaldW4oNBsm9KYM085FWABDn4fNxw&x-client-SKU=ID_NETSTANDARD2_0&x-client-ver=<IP_ADDRESS>

@sandesai Looking at the url given above, the x-client-SKU parameter shows this request coming from the MSAL.NET library. I suggest looking at their github repo here for more context: https://github.com/AzureAD/microsoft-authentication-library-for-dotnet


Testing speed is influenced by the total imgs?

Testing HTC with 16gpus in two nodes(--gres=gpu:8 --ntasks=16 --ntasks-per-node=8):
with 4K imgs loaded, the speed is about 20 task/s
but with 100K imgs loaded, the speed is about 11 task/s
Why the speed is influenced by the total imgs?

The inference time includes all preparing time including data and annotation loading. I guess there maybe two reasons. 1. Loading annotations of 100k images is slow, which happens at the beginning of the testing.  2. The multiprocessing communication is slower since the queue is larger.


add AnAsync{,Ext} traits to reuse {read,write}_with{,_mut}

This lets me dedupe some more code from rs-bwlim - specifically https://github.com/infinity0/rs-bwlim/commit/163717455b3d9cbd34ea4eb175da11601a486b16
However it is less trivial than #23 - we depend on an unstable nightly-only scary feature of rust - generic_associated_types. Also we need async_trait which itself a temporary thing, pending much-future advances in rust. So I can totally understand if you don't want it in just yet.
I tried various other strategies but couldn't get them working. The main problem being that due to instance coherence rules, rust does not let you impl ForeignTrait for T where T: LocalTrait but this is ideally what we want to do, i.e. impl AsyncRead for T where T: AnAsyncExt. So the current PR is the best I came up with.

Oh, another downside is async_trait carries a very minor runtime cost since it uses dyn Future instead of impl Future but this is probably not noticeable.

I don't think this fits into the overall goals of async-io - it is now at v1.0 so I'm hesitant of adding new API. And I like keeping things as simple as possible :)
Also, async_trait pulls in syn and quote, which was entirely eliminated from the dependency tree of smol because they make compilation considerably slower.
But also, I feel it probably doesn't make too much sense for async-io and rs-bwlim to share possible. At the end of the day, it's not that much code duplication and those two crates have fairly different goals.


Handle lack of keyring support and/or missing authentication token better

The Issue
A user recently reported a blank Explore page when accessing LBRY on Linux and not having gnome-keyring installed. We do check for errors when getting the auth token, but in this case it seems to be coming back as null (brief testing).  Doesn't look like the network call to our API occurs in this scenario.
We should warn the user and send them to the https://lbry.io/faq/startup-troubleshooting#auth
current check: https://github.com/lbryio/lbry-app/blob/14685f311dc906178e4b2124864fe2191d57c135/src/renderer/redux/actions/user.js#L68
Steps to Reproduce

disable gnome keyring: sudo chmod -x /usr/bin/gnome-keyring-daemon
reboot
load LBRY
Alternative: cancel request for password when prompted for keyring.

Expected Behaviour
Tell us what should happen.
Actual Behaviour
Tell us what happens instead.
Suggested Solutions
Tell us your suggested solutions if you have any.
System Configuration

LBRY Daemon version:
LBRY App version:
LBRY Installation ID:
Operating system:

Anything Else

Screenshots

Keyring has been removed.


Mapping of Orphanet:121715 results in 500 Internal Server Error

This issue has been logged via RT: https://helpdesk.ebi.ac.uk/Ticket/Display.html?id=370505
I just noticed that when I query OxO with a valid identifier
(Orphanet:121715), it responds with a 500 Internal Server Error. This
happens whether I use the API or the website (which obviously uses the
same API under the hood). Here's the request to reproduce this:
$ curl 'https://www.ebi.ac.uk/spot/oxo/api/search?size=500' -H
'Content-Type: application/json; charset=UTF-8' -H 'Accept:
application/json' --data-binary
'{"ids":["Orphanet:121715"],"mappingTarget":[],"distance":"3"}'
{"timestamp":1570787247760,"status":500,"error":"Internal Server
Error","exception":"org.neo4j.ogm.exception.MappingException","message":"Error
mapping GraphModel to instance of
uk.ac.ebi.spot.model.Term","path":"/spot/oxo/api/search"}
The mapping term which I'm requesting is unremarkable:
http://www.orpha.net/ORDO/Orphanet_121715, and OxO appears to be working
fine for other Orphanet terms.

Hi, I'm the original reporter. FYI this also happens for terms Orphanet:119117 and Orphanet:291794

I just ran into this bug, which seems to occur consistently when the same request is made multiple times: The first succeeds, then two fail, then the third succeeds, and repeat...


Error handling when fetching calendar events

There is no error handling when fetching calendar events. This means the card won't show anything if there are any problems. A proper error message should be shown.

Added in the dev branch. This will be included in the release for v1.1.0


Crashing

Just installed Ha version 2.2.5
Ha 2021.11.5
MQTT Current version: 6.0.1

Version
core-2021.11.5

Installation Type
Home Assistant OS

Development
false

Supervisor
true

Docker
true

User
root

Virtual Environment
false

Python Version
3.9.7

Operating System Family
Linux

Operating System Version
5.10.17-v8

CPU Architecture
aarch64

Timezone
America/New_York

Config default
INFO:weatherflow2mqtt.weatherflow_mqtt:SETTING UP TEMPEST Status SENSOR
Traceback (most recent call last):
File "/usr/local/bin/weatherflow2mqtt", line 33, in
sys.exit(load_entry_point('weatherflow2mqtt==2.2.5', 'console_scripts', 'weatherflow2mqtt')())
File "/usr/local/lib/python3.9/site-packages/weatherflow2mqtt-2.2.5-py3.9.egg/weatherflow2mqtt/main.py", line 10, in main
File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/site-packages/weatherflow2mqtt-2.2.5-py3.9.egg/weatherflow2mqtt/weatherflow_mqtt.py", line 843, in main
File "/usr/local/lib/python3.9/site-packages/weatherflow2mqtt-2.2.5-py3.9.egg/weatherflow2mqtt/weatherflow_mqtt.py", line 627, in listen
TypeError: cannot unpack non-iterable NoneType object

Update moved back to outside of HA on a standalone install  seems to running fine but reporting windspeed other sensors from hub seem to be a bit slower as per updating
but also getting a few of these on this installation
DEBUG:weatherflow2mqtt.weatherflow_mqtt:Device ST-00051410 has reported a sensor fault. Reason: ['lightning failed']

but reporting windspeed other sensors from hub seem to be a bit slower as per updating

If you, like me, live in a place where the days get shorter in the winter, then your device will not get enough sunlight to properly charge the batteries, and instead of shutting down, WeatherFlow is changing the reporting interval, as you can see below. That might be the reason for slower updates.
Power Save Modes
Here are details on how the Tempest may conserve power in order to operate for as long as possible. Most Tempests never go below Mode 0 and more than 90% will never go below Mode 1.

Mode 0: Voltage ≥ 2.455

All sensors enabled and operating at full performance
Wind sampling interval every 3 seconds

Mode 1: Voltage ≤ 2.415 from Mode 0 or ≥  2.41 from Mode 2
Wind sampling interval set to 6 seconds

Mode 2: Voltage ≤ 2.39 from Mode 1 or ≥  2.375 from Mode 3
Wind sampling interval set to one minute

Mode 3: Voltage ≤ 2.355
Wind sampling interval set to 5 minutes
All other sensors' sampling interval set to 5 minutes
Haptic Rain sensor disabled from active listening

DEBUG:weatherflow2mqtt.weatherflow_mqtt:Device ST-00051410 has reported a sensor fault. Reason: ['lightning failed']

This is not an error, but just telling you that there might be something close to your Weather Station that creates interference.


Add check before redrawing graph node slot stylebox

Fixes #65557
Ensure Graph Nodes have more than 0 children before redrawing slot styleboxes. There is already a check in place to ensure the slot's key is not greater than cache_y.size(), but it seems the cache isn't updated before redrawing occurs. Possibly a better fix would be to make sure the cache is updated before redrawing? Not sure, pretty new to the engine still.

Looks good! Could you squash the commits? See PR workflow for instructions.

Sorry, we apparently had another PR that fixed it, see https://github.com/godotengine/godot/pull/69284.


Improve User Experience When Terminal Content Is Insufficient for Meaningful Suggestions

When the current terminal content is insufficient to generate a meaningful suggestion, we should display a helper banner to the user, as shown in the attached image.
Additionally, to avoid overwhelming the user, if multiple requests result in a "more information required" response, we should display only one warning instead of multiple helper banners in sequence.

If possible:
If a user pauses when typing a question to shellmate and it get to this "more information required" state, don't show the banner

Mary's response:

I think for now, just making sure we dont display multiple in a row will be good. We can refine the experience later on if we think it needs it, or we will get some wins by adding in a pause button or settings that allow for the control of how fast suggestions in general come.

So no need for the feature to detect if user paused typing to avoid the banner for now


Prevents decimal conversion issues with certain locales

/cc @kristoffer-zliide
Addresses #84.

Uses Decimal to be consistent about the decimal separator (.)
Default to the final payment method type if not specified

@jamesblasco are you able to take a quick look to this one?


In landscape mode, burger menu overlaps content and page header is missing

Current behavior

^ title
Expected behavior
Should expected similar result as portrait mode.
How to reproduce it (as minimally and precisely as possible)
Rotate phone by 90'degree
Environment
Nuget Package:
Package Version(s):
Affected platform(s):

[ ] iOS (not sure)
[x] Android
[ ] WebAssembly
[ ] UWP

Issue not present on iOS


Composition vs Inheritance page is confusing as Inheritance is never talked about

Would be nice to see an example or two of what you're comparing composition against, or at least an external reference to a use case in the context of React.

can you provide more information about in which section or where the example would highly important

@kambleaa007 Here you go: https://reactjs.org/docs/composition-vs-inheritance.html

Yeah, its true, redux documentation is quite need more improvements.
lots of things are uncovered or partially written

What about containers, ,  arent they Inherited.
Our whole  wrapped inside them.


hey u still working on this ?

I'm not actually maintaining this. But if you would like to contribute or work on the project that would be cool!


[Bug] 官方qxerty打包的应用无法正常使用,输入正确无法继续

Search before asking

[X] 我在 issues 列表中搜索，没有找到类似的内容。
I searched in the issues and found nothing similar.

Pake version
1
System version
win11
Node.js version
11
Minimal reproduce step
1
What did you expect to see?
1
What did you see instead?
1
Anything else?
No response
Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

设置问题,无bug


Saved scene does not set correct brightness on Hue bulb

Home Assistant release with the issue:
0.102.2

Last working Home Assistant release (if known):
0.102.1
Operating environment (Hass.io/Docker/Windows/etc.):
Hassio

Rpi3 + Philips Hue hub
Integration:
https://www.home-assistant.io/integrations/scene/
https://www.home-assistant.io/integrations/hue/

Description of problem:
When using the new Scene editor in the frontend current state of my Hue bulbs seems to be saved to the YAML file, but when I activate the scene - wanted states are not set on some bulbs. Seems to affect brightness.
YAML from saved scene:
    light.lounge_rear_mid:
      brightness: 51
      effect: none
      hs_color:
      - 28.889
      - 52.941
      rgb_color:
      - 255
      - 185
      - 120
      state: 'on'
      xy_color:
      - 0.479
      - 0.384

Actual state when scene is activate
min_mireds: 153
max_mireds: 500
effect_list: colorloop,random
brightness: 159
hs_color: 28.889,52.941
rgb_color: 255,185,120
xy_color: 0.479,0.384
effect: none
friendly_name: Lounge Rear Mid
supported_features: 63

Additional information:

Just tested your scene, and seems to work fine here. What type of hub do you use? Any more information so I can reproduce this problem?

Just tested your scene, and seems to work fine here. What type of hub do you use? Any more information so I can reproduce this problem?

I have the latest hub, with the latest firmware not exactly sure of the version.

Do you get an error in the log from hue that the connection is reset maybe?

Do you get an error in the log from hue that the connection is reset maybe?

I was unaware, but I actually got an error message after activating a scene:
Unable to reach bridge <IP_ADDRESS> (Error requesting data from <IP_ADDRESS>: [Errno 104] Connection reset by peer)
Still, almost all my bulbs change to correct brightness except from 1 in the example above. I have certain other scenes that have other bulbs not changing to correct brightness. Colors are correct though.


Disallow dropping Classes that are referenced in the schema

Is this still relevant @moretea?
Temporarily assigning this to @etiennedi depending on the answer.

This sounds worth keeping. I need to research a bit what the implication is of deleting a referenced class, but I assume it's rather catastrophic ... so some sort of protection sounds good
cc @laura-ham

Relevant but trivial


Document ports used with kubectl and helm clients

We lack this documentation. Useful for people, and robots, who want to run the minimal set of ports open for these tools to function properly.

I believe these are 30000-32767

https://github.com/rancher/rancher.github.io/pull/798


Add sast flag if sarif - AM 1.9.8

[ ] The pull request is targeting the dev branch.
[ ] The code has been validated to compile successfully by running go vet ./....
[ ] The code has been formatted properly using go fmt ./....
[ ] All static analysis checks passed.
[ ] All tests have passed. If this feature is not already covered by the tests, new tests have been added.
[ ] All changes are detailed at the description. if not already covered at JFrog Documentation, new documentation have been added.

Based on PR by @attiasas  - added the AM to support the addition of "JFrog SAST" when using sarif format

Duplicate of: https://github.com/jfrog/jfrog-cli-security/pull/206


touch: if you tap next/prev link during a transition, it can disable those links semi-permanently

@scottjehl
https://github.com/filamentgroup/responsive-carousel/compare/155
Not ready to merge yet.
How should I set up the CSS, it's currently only on the slide-two.html functional test page. Wasn't sure about the best way to scope it.
I did my level best not to butcher the drag code.

Preview: https://155-origin-responsive-carousel.fgview.com/test/functional/slide-two.html


Alt+Up hotkey to go Up a folder doesn't work

Description
Alt+Up hotkey to go Up a folder doesn't work. Nothing happens when the hotkey is pressed. Both for the latest version as well as the preview version.
Steps To Reproduce

Open Files.
Go to a folder that has a parent folder.
Try the hotkey Alt+Up

Requirements
The expected behaviour is that the hotkey will navigate to the parent folder.
Files Version
<IP_ADDRESS> + <IP_ADDRESS> (preview)
Windows Version
10.0.19045.3324
Log File
Doesn't seem necessary.

Thanks for the report, I am unable to reproduce this

Can you see if the UI button works.
Can you please upload (Do not copy and paste content) the debug.log file directly to GitHub (email attachments don't work) . You can get it in Files settings > About > Open log folder.

The UI button does work. Alt+Right / Alt+Left do work fine. Alt+Up does work correctly in Windows Explorer.
Log file:
debug.log

Is this still an issue in the preview version? https://files.community/download/preview
If it's still an issue, can you share screen recording?


Add AckWait config option to nats_jetstream input

For NATS JetStream input, it would be nice to be able to specify custom AckWait configuration option.
It for configuring time that the NATS server will wait for an ack for any individual message. If an ack is not received in time, the message will be redelivered.
I found this useful for a pipeline that batches entries for extended time duration (above default consumer time limit of 30s).
Also, I have already implemented this change here, however following contributing guide I am creating this ticket (before possible PR).
Thank you for your consideration.

Hey @zbiljic, yeah absolutely I think that'd be great to add.

Great, then I will create a PR now.


feat: allow project installation

Closes #395
mypy requires dependencies and project to be installed. By default we will skip the installation of the library but if repos need it they can set
skip-install: false

I've given this a quick test against the jobqueue repo. It looks like it's failing because the pip install . line doesn't install the poetry runtime dependencies. I don't really know if this is a solvable problem with this approach: we don't want to have mypy as a main dependency or even an extra, but without using poetry to do the installation you won't pick up the dev dependencies.

No worries - let's do a system check in that case

@ansys/pyansys-core - a kind review?


Documentation

Hi,
I started using this plugin. I would like to help contribute in creating the documentation for this plugin.

OK, Thanks!   There's no time to write a document recently


删除主页的“返回”按钮

主页无需”返回”按钮，需要删除。

fixed by: https://github.com/boristown/TradeContestServer/commit/96705a0edc1fe9d0dcc8a86d1bef78662dc50689


xonsh: expose unwrapped

Currently it is impossible to import xonsh into python for e.g. language servers to be able to get completion for it, or otherwise treat it as a python lib. We aren't sure why the unwrapped derivation was made inaccessible, but it's definitely a problem.
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

If the idea is to make it a Python lib, then it should use toPythonModule.

If the idea is to make it a Python lib, then it should use toPythonModule.

It's not exactly; it's to just let you rewrite the wrapper, and a convenient side effect is being able to use it as a module as well.


Support GitHub App with OAuth

GitHub allows apps to optionally add OAuth during the app installation, see step 11 here. In this case, your callback URL is called directly after a user installs the app on GitHub.com with an auth code to exchange for a user token. However, this library fails to handle this because there is no state (relevant code). Below example url signature for this scenario.
Please add support for this auth flow!
http://localhost:3000/auth/github/callback?code=01234567890abcdef012&installation_id=01234567&setup_action=install

Debug error in console: OAuth2Strategy State from URL null

Latest version should work with GitHub Apps and OAuth apps as per the upstream client documentation.


Biofarm- inactive jobs on the company website

ENVIROMENT: production
URL: https://scrapers.peviitor.ro
Browser: Chrome
Device: laptop
OS: Windows 10
STEPS TO REPRODUCE:

Open URL in browser
Run scraper for Biofarm company https://scrapers.peviitor.ro/src/biofarm/index.html
Check the location for the jobs "Tehnician monitorizare" and "Fochist - Fabrica de medicina"

EXPECTED RESULTS:
The location of the jobs should be a locality, not a country
ACTUAL RESULTS:
The message "Romania is not a city in Romania" is displayed for each other
These two jobs are no longer active on the company website
Unable to apply to these jobs; the "apply" button is inactive

After retesting, Expected results matches with Actual results on https://scrapers.peviitor.ro/src/biofarm


multicheckbox flexbox colum

Is your feature request related to a problem? Please describe.
I'm am having issues overriding styles to allow multicheckbox's to stack vertically.
Describe the solution you'd like
A clear and concise description of what you want to happen.
When using CSS or Angular fxgrid, multicheckbox are unable to stack vertically, and do not space between items
Current display:

Desired display:

Having an option to align along a flex direction of row or column would be ideal for multicheckbox.

Having an option to align along a flex direction of row or column would be ideal for multicheckbox.

why not!
For the moment you could use a custom class that you may pass to className property https://github.com/ngx-formly/ngx-formly/issues/978#issuecomment-394832904

for anyone who might have been having the same issue. This example helped.
https://stackblitz.com/edit/multicheckbox?file=src%2Fstyles.scss
I found the only way that worked for me was adding the css override to the root stylesheet. Overrinding in app.component or the components stylesheets did not work.
My code:
formly-field-mat-multicheckbox {   .mat-checkbox {     display: flex;   } }


Use Spring CLI to install and run stubs locally

For the case where the consumer wants to prototype the API it would be good for them to immediately install and run stubs from the contracts.
E.g. installing
spring cloud contract install /Users/path/to/contracts/ 'groupid.a.b.c' 'artifactid.d.e.f' '1.0.0.BUILD-SNAPSHOT'

E.g. running
spring cloud contract run /Users/path/to/contracts/ 'groupid.a.b.c' 'artifactid.d.e.f' '1.0.0.BUILD-SNAPSHOT'

Currently you can run stubs using maven, even if you don't have pom.xml:
cd /tmp
mvn spring-cloud-contract-verifier:run -Dstubs="groupid.a.b.c"

It would be really awesome also to use spring cloud cli to download stub runner boot and pass some parameters to it

@jkubrynski  when I run "mvn spring-cloud-contract-verifier:run -Dstubs="groupid.a.b.c"" under such folder where there is no pom.xml and no contracts. Although the goal is successful, I found such log "Launching StubRunner with contracts from c:\empty_folder" and "Started stub server for project **** on port -1". as you see the port is -1, I can't access the stub runner via -1 port. Could you give me some suggestion to run stub runner via mvn?

Can you show how your contracts look like?

@marcingrzejszczak  Does it mean it is necessary to run "mvn spring-cloud-contract-verifier:run " under such folder it has to contain contracts?

@simonzhong1985 it's exactly how it works but you need to use a different property. As documented here you should pass -Dspring.cloud.contract.verifier.stubs=com.aaa:bb

@jkubrynski  thanks for your answer. It works, Now I can download stubs by -Dspring.cloud.contract.verifier.stubs=com.aaa:bb. However, It does not work by -Dspring.cloud.contract.verifier.http.port to run wiremock on assigned port.

You should pass the port in the ids -Dspring.cloud.contract.verifier.stubs=com.aaa:bb:+:stubs:1234. That way you'll rnu the stubs groupid com.aaa, artifact id bb , version + - latest, classifier stubs, port 1234

@marcingrzejszczak  thanks. It works as I expect. Thanks again.

@marcingrzejszczak  Is there any update about using spring cli to run stub?

Yeah, thanks for trying it out. Thanks to you, we've got these things that popped up - https://github.com/spring-cloud/spring-cloud-contract/issues/340 and https://github.com/spring-cloud/spring-cloud-contract/issues/341
As for the update - I haven't done any work on the spring cli. Come to think of it maybe we should stick to maven since we already have some implementation there. It's easier to download and install maven instead of installing spring cli + the cloud extension.

I think we should add the Stub Runner application to the Spring Cloud CLI project. The rest should be done via Maven

Done via https://github.com/spring-cloud/spring-cloud-cli/pull/65

Hi @marcingrzejszczak  @jkubrynski ,
if I run the  mvn spring-cloud-contract-verifier:run -Dspring.cloud.contract.verifier.stubs=com.example:SCC_producer:+:stubs:8080
In a directory which does not have any pom.xml file I am facing the issue of the plugin.
No plugin found for prefix 'spring-cloud-contract-verifier' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/home/gurushantha.rg/.m2/repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1


Error: Failed to find development corelib.

I've used your script to install cairo on my M1 macOS machine (Ventura 15.3.2).
curl -sL https://raw.githubusercontent.com/lambdaclass/cairo-by-example/main/build/installer.sh | bash -s 2.2.0

It installs correctly, but when I try to compile the hello world exercise, I get an error:
cairo-run --single-file hello-world.cairo
Error: Failed to find development corelib.

I am not able to understand why.
If I look for the core lib I find it, though. So it's installed:
ls<EMAIL_ADDRESS>Scarb.toml         cairo_project.toml src

hi @clacladev - two things that might be of help, done sequentially
have you set export CARGO_MANIFEST_DIR=<corelib-path> in your shell resource file?
<EMAIL_ADDRESS>in my case)
if you are working from within vscode, and are trying to run the cairo-run command from its cli, and already had the Cairo 1.0 extension installed, cloning https://github.com/starkware-libs/cairo and following the  vscode-cairo/README.md extension install instructions under the first subheader, down to and includingcode --install-extension cairo1*.vsix fixed the same issue for me

Thank you a lot @pfcrowe for the help. I thought the installer.sh was supposed to export the CARGO_MANIFEST_DIR. But now I manually did it and it just works.
Thanks for the help!


Don't use torii provider's fetch and close methods

as they don't actually exist as Cory just pointed out in the global Ember meetup.

actually it seems to be correct the way it is…


How to change the image width and height

Hi
How to change the image width and height, always the width and height of the image is 150 X 150.
There is any way to change the width and height. i want 230 X 138, maxWidth and maxHeight is not working. maxWidth and maxHeight these options works only when the values are less than the image width and height.
thanks in advance

Not in this library maybe react-native-image-crop-picker can do it since there is a different crop tool (I really don't know)


templateUrl used instead template

Related to https://github.com/NativeScript/nativescript-cli/pull/1874

Better match the template name with the TS file name. It should be app.component.html


bazel: replace tclap mirror. (#9072)

Description:
Change (non-existent) https://github.com/eile/tclap/ to https://github.com/mirror/tclap/. I didn't validate the new mirror in any way, but the downloded release file sha256sum stays the same. In the future the releases should probably be downloaded from the real upstream. However the release 1.2.1 tarball from https://sourceforge.net/projects/tclap/files/ did not have the same sha256sum as the release file which Envoy uses.
Risk Level: Medium
Testing: N/A
Docs Changes: N/A
Release Notes:
Fixes #9071
Signed-off-by: Ismo Puustinen<EMAIL_ADDRESS>Signed-off-by: Piotr Sikora<EMAIL_ADDRESS>

Cherry-picked https://github.com/envoyproxy/envoy/commit/648cfc3b6c1410d8b27aa33b6fc0392ab230d601

cc @bianpengyuan


MF6 bug: Package names should not exceed 16 characters

In GitLab by @JoerivanEngelen on Jul 6, 2022, 17:31
We should either truncate them and throw a warning, or throw an error in __setitem__ if a package key larger than 16 characters is provided.
  1. ERROR.  PACKAGENAME MUST NOT BE GREATER THAN 16 CHARACTERS.
  2. riv6 gwf1/riv-DETAILWATERLOPEN_25M.riv riv-DETAILWATERLOPEN_25M
  3. Error in PACKAGES block in file: gwf1/gwf1.nam

In GitLab by @Huite on Jul 11, 2022, 11:39
Truncation is not a good idea: since we're using a dictionary, it can easily result in naming conflicts which will then cause packages to be overwritten.
So a KeyError is best I think.

In GitLab by @Huite on Sep 1, 2022, 20:11
mentioned in commit d4c9e334ae6c264b86f5bf500c249039d86b4199


change the speaker configuration?

is it possible to change the speaker configuration?
For example change from 5.1 surround to stereo and vice versa or something like that for the default device.

please see Issue #32 for this, thank you


Problem with visibilty setting in menulist

Hi, I've been trying the templates and found out that the menu doesn't hide when onblur or even when selected. I've inspected and it seems to be a visibility setting is not setting to invisible. https://youtu.be/NMoyVfRvbkE here is a video of it.

I found in the chakra discord that this problem had to do with framer-motion  4.1.12. I updated the library and the problem is gone.


Vscode extension?

I like the connectivity and package management, but the editor is very basic.
Have you considered using parts of the solution to create an adding for vscode which would combine your great work, with a great and popular IDE with all kinds of other integrations.

Right now there are no such plans.
If there's someone willing to contribute it, I'd be happy to participate.

If that is your focus that is fine nand im sure if fills a need.
If I have time Ill take will remeber the good work,  and ping back here.


Bug fixes

Hi @danielhanchen 🤗, I'm excited about the idea of adding float8 + QLoRA finetuning support via Torch AO into Unsloth !
I'd like to contribute and work on this. Do you have any specific points to consider before I start?


New compile-time dependency added in Visual Studio 2017 15.3.x breaks current tools build

We are hitting the issue described in https://developercommunity.visualstudio.com/content/problem/94821/vs-153-undocumented-breaking-changes.html
In summary, the we need a new dependency when building the designer with newer versions of VS 2017:
    <Reference Include="Microsoft.VisualStudio.Shell.Interop.15.3.DesignTime, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
      <EmbedInteropTypes>True</EmbedInteropTypes>
    </Reference>

Without it you get:

"Cannot find the interop type that matches the embedded interop type 'Microsoft.VisualStudio.Shell.Interop.IVsToolboxItemProvider2'. Are you missing an assembly reference?"

Working on solving this by adding it to EFToolsVsReferences.settings.targets, conditional on VisualStudioVersion = 15.0. This means we will require 15.3.x at least to build.

Triage: We will continue to hack our way through deving (using VS 2013) and @bricelam will investigate getting a proper VS build done for 2017 using C.I. setup on unicorns420.


Add toggle to Motor and Robot

motor.toggle() would reverse the direction of the motor if moving (forward <=> backward).
I guess it would run motor.forward() if the motor is not in motion.
robot.toggle() would reverse the direction of the robot if moving (forward <=> backward; left <=> right).
I guess it would run robot.forward() if the robot is not in motion.

I'm not so sure about the robot.toggle() suggestion. To me, the concept of the toggle() method (which is so far true of all our implementations of it) is that running it twice (or any even number of times) should result in the object being left in the original state. However, under the specification given, if the robot is stopped, running it once starts it moving forward, then again moves it backward. No method is given for returning it to the original state via toggle().
Frankly, for things like motors I'm not sure toggle() makes much sense (we're not really switching between two distinct states). I do like explorer HAT's reverse() though. I could see that having a sensible definition in the case of motor via negation:
self.value = -self.value

In other words, it we're running at 1 (full speed forward), run at -1 (full speed backward). If we're at -0.5 (half speed backward), change to 0.5 (half speed forward). And if we're at 0, remain stopped.
A similar definition can also be applied to robot:
self._left.value = -self._left.value
self._right.value = -self._right.value

Thus, if the robot's running full speed forward, it'll run full speed backward. If it's turning left (left=-1, right=1) it'll turn right (left=1, right=-1), and so on. If it's stopped, it'll stay stopped which is why toggle() really isn't a good name for this - but reverse() makes good sense.

Yeah I wasn't sure. I don't like inconsistent toggle behaviour either, so I guess we should drop it. Do we still have toggle for PWM devices, or did you remove it? That was also inconsistent.
reverse does make sense for Motor and Robot.

Good question - I'll just check...

Hmmm, PWMLED inherits toggle from DigitalOutputDevice which simply queries is_active (which in the case of PWMOutputDevice returns true if value > 0.0, i.e. if the device has any duty cycle at all) and calls on() or off() in response. So, in the case of a PWMLED, if the device's duty cycle was, say, 0.1, a call to toggle() turns it off, then a second call turns it all the way up to 1.0. Not sure I like that.
I'm tempted to come up with a more reversible transform like 1.0 - value which would flip between 0 and 1 at the extremes but between 0.1 and 0.9 in between and would effectively do nothing at 0.5. Thoughts?

I think that's better than the current behaviour. Go for it.

Nice! Thanks


Can't render "DIN Condensed" font

The following code renders 26 chars as PNGs:
chars = %w{
  Courier
  DIN\ Condensed
  Verdana
}.flat_map do |font|
  FileUtils.mkdir_p "chars/#{font}"
  (?A..?Z).map do |char|
    filename = "chars/#{font}/#{char.ord}.png"
    Vips::Image.text(char, font: font, width: 100, height: 100).invert.write_to_file filename unless File.exist? filename
    [DHashVips::IDHash.fingerprint(filename), char]
  end
end

But for "DIN Condensed" it seems to fallback to Verdana or smth:

It should look like this:

Maybe it's because of " " in the font name?
UPD: the link "font : gchararray, font to render with" at https://libvips.github.io/libvips/API/current/libvips-create.html has interesting path -- https://libvips.github.io/usr/share/gtk-doc/html/gobject/gobject-Standard-Parameter-and-Value-Types.html#gchararray
UPD2: fontfile: "/Library/Fonts/DIN Condensed Bold.ttf" does not help.
BTW, other fonts have other types: /System/Library/Fonts/Courier.dfont, /System/Library/Fonts/Menlo.ttc. But Arial and Tahoma are .ttf too and they are fine.

I don't think it's spaces -- for example, I can do:
$ irb
irb(main):001:0> require 'vips'
=> true
irb(main):002:0> x = Vips::Image.text("hello!!", font: "Lovers Quarrel", dpi: 300)
=> #<Image 85x30 uchar, 1 bands, multiband>
irb(main):003:0> x.write_to_file("x.png")
=> nil

To make:

fontfile adds the font but does not select it. Have you tried:
Vips::Image.text "hello world!", fontfile: "/Library/Fonts/DIN Condensed Bold.ttf"", font: "DIN Condensed Bold", dpi: 300

chars = [
  ["DIN Condensed", "/Library/Fonts/DIN Condensed Bold.ttf"],
].flat_map do |font, fontfile|
...
    Vips::Image.text(char, font: font, fontfile: fontfile, width: 100, height: 100).invert.write_to_file filename unless File.exist? filename
...
end

didn't help.
Also tried to append "Bold" but it's looks like Verdana Bold.

Don't you have this font on your macOS?

Pango defaults to the CoreText rendering and mapping backend on macOS. You could try to use the Fontconfig backend with the PANGOCAIRO_BACKEND="fontconfig" environment variable.

PANGOCAIRO_BACKEND="fontconfig" environment variable

Should I try to use it while installing the gem or libvips? In runtime it does not seem to help. Both with :fontfile and without.

It's a runtime variable, so you don't need to reinstall libvips. Perhaps you need also to set the FONTCONFIG_PATH=/Library/Fonts environment variable. You could debug if it maps the font name to the correct font file with:
FONTCONFIG_PATH=/Library/Fonts fc-match 'DIN Condensed'

I guess it should output something like this:
DIN Condensed.ttf: "DIN Condensed" "Regular"

Or to find a list of candidate names, you could try this:
FONTCONFIG_PATH=/Library/Fonts fc-list

(Unfortunately, I don't currently have any macOS system to test it on)
You can also try to manually rebuild the font cache with:
fc-cache -v

But this is probably only needed when new fonts are added to the system (I guess DIN Condensed is already pre-installed on macOS).

I tried in nip2 on macos and it works fine, so I think it's something to do with the way fontconfig is being built by homebrew.
I had a look at gtk-osx and they just buyild with:
  <tarball id="fontconfig" version="2.8.0">
    <pkg-config>fontconfig.pc</pkg-config>
    <source href="http://www.fontconfig.org/release/fontconfig-2.8.0.tar.gz"
            hash="sha256:fa2a1c6eea654d9fce7a4b1220f10c99cdec848dccaf1625c01f076b31382335"
            md5sum="77e15a92006ddc2adbb06f840d591c0e" size="1548409"/>
    <dependencies>
      <dep package="expat"/>
    </dependencies>
  </tarball>

So no strange flags.

I spoke too soon -- while fonts appear in the interface, text in nip2 is still unable to render them.

This is what it says:
$ FONTCONFIG_PATH=/Library/Fonts fc-match 'DIN Condensed'
DIN Condensed Bold.ttf: "DIN Condensed" "Bold"

$ FONTCONFIG_PATH=/Library/Fonts fc-list | grep DIN
/Library/Fonts/DIN Alternate Bold.ttf: DIN Alternate:style=Bold
/Library/Fonts/DIN Condensed Bold.ttf: DIN Condensed:style=Bold`

And I don't remember to add any fonts to the system manually, and it's pretty fresh.
Tried right now both with and without fontfile and with both variables FONTCONFIG_PATH=/Library/Fonts PANGOCAIRO_BACKEND=fontconfig -- still looks like Verdana and Verdana Bold.

The output of those commands looks OK. I tried to reproduce this on my Fedora machine and noticed that there is also a pango-list and pango-view command. I installed the d-din-condensed-fonts package and ran:
$ pango-list | grep DIN
D-DIN Condensed
  Regular:           D-DIN Condensed, Condensed
  DINCondensed-Bold: D-DIN Condensed, Bold Condensed
  *Italic:           D-DIN Condensed, Italic
  *Bold Italic:      D-DIN Condensed, Bold Italic
$ pango-view --font 'D-DIN Condensed Bold' --no-display --text 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' --output out.png
$ pango-view --font 'D-DIN Condensed, Bold' --no-display --text 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' --output out2.png

Output:

So, perhaps you need to use DIN Condensed, Bold instead of DIN Condensed Bold.

So, perhaps you need to use DIN Condensed, Bold instead of DIN Condensed Bold

It works! Thanks.

Great, glad it's working now. Out of curiosity, does it also render with PANGOCAIRO_BACKEND=fontconfig? It might produce a crisper text according to Inkscape.

I don't see much difference.

I guess it can be closed unless you want to make something to make it easier to find the correct name for a font. Not sure if it would be possible or relevant to vips, though.


[BUG][MRG] Fix surface finding for mne.viz.Brain

Fixes #11360

Should be okay to merge when the CIs pass.


matrix interactions too large - cannot handle in memory

Hi - first of all, thanks for making your code available!
I'd like to use it on data from mouse, but am coming across issues with the larger chromosomes at high resolution, getting errors such as:
ERROR: matrix interactions too large - cannot handle in memory [19720 x 19720] (388,878,400 > 256,000,000 limit)
The actual memory usage of the code when running is low as far as I can tell, and I'm running it on a server with 512GB RAM, so I'm wondering if this matrix size limit can be adjusted? What would you recommend for using this code with mouse or human data?

Hi - I will remove that hard limit in the code.
For this code (matrix2insulation) the memory usage will scale with your selected insulation square size * nrow/ncol.
So memory usage should always be low [MBs] given normal usage.
We only load into memory N diagonals up to your specified insulation square size.
e.g.
1,000,000bp square size for a 5KB binned matrix would load into memory ~400 diagonals.
This script is more meant for matrics < 10,000 x 10,000.
For the higher resolution data and larger matrices, a more efficient data format (sparse, hdf5, etc) would be more ideal.
I tested the code on a ~50,000 x ~50,000 matrix, it takes quite a bit of time (15 minutes) to run.  Mainly to simply read the gzipped txt file, but it does complete AOK.
Script will work for any species, as long as the matrix is binned into equal sized intervals, and the intervals are formatted in the my5C format.
https://github.com/blajoie/crane-nature-2015/wiki
e.g. bin1|hg19|chr1:40000-80000
The code is updated now, give it another shot!

issue looks to be fixed now! closing this.


Provide a quick option to go back from the search results page to the project page

For example, when at this repo's search page, I would ideally like to be able to click a button to go back to the project home or maybe even just hit ESC.
One might also consider whether it makes more sense to go to the last viewed page instead of the project home page 🤔.
Example URL: https://github.com/<user>/<project>/search

The "last viewed page" is already in the browser’s back button.
The link to the project home is already there.

I realized that after, however I would still love to be able to press esc and have it go back.

esc doesn't make sense in this case because it's not a modal. You can press backspace if your browser supports it (Chrome needs an extension)


Enabled scroll when nested modal is closed

When closing a second level modal, the scroll is enabled (disableScroll) again, even though a first level modal still exists. There must be a check whether another modal exists before enabling it. Otherwise it won't support nested modals.

@julmot Appreciate you opening this PR.
However, in the interest of keeping this library lightweight and simple, we would not be adding support for nested modals. If over time demand for this feature grows, we could consider this again.

I'd also be interested in this, should you decide to re-open the issue. Thank you.


News not upating since 21st April

It's finally backed ;)))) I'm happy to see it working again.

It's finally backed ;)))) I'm happy to see it working again.

Old data was already there, but there is no new data after 21st April 2022

You're right, the bot seem to commit data to the repos regularly but no new data come in.

still not updating :(

We should build one, I'll start my own project next month.

We should build one, I'll start my own project next month.

https://github.com/win11react/NewsAPI we made one feel free to use urself (you can configure thing according to your need from the workflow file)

We should build one, I'll start my own project next month.

https://github.com/win11react/NewsAPI we made one feel free to use urself (you can configure thing according to your need from the workflow file)

Nice, but it's for unlimited use? I mean Saurav's NewsAPI have no limits. But in his API news refresh every hour.

@amitnarw we updated it to run every hour

Everyday update is fine for me, what I really need is a stable repository, that not easy shut down or be abandoned by the owner. I'll star your repos and working with your repos.

Everyday update is fine for me, what I really need is a stable repository, that not easy shut down or be abandoned by the owner. I'll star your repos and working with your repos.

it will be stable as we are using it on some quite large projects like blueedgetechno/win11React

That's what I mean, thank you so much for your work. I currently have nothing, but that's  must me a couple of cup of coffee for you when I'm in the right condition. So, a big thanks again.

https://github.com/win11react/NewsAPI  has stopped working it seems. Any other News API that's working currently?

If someone is still searchin for something similar to this, (only top-headlines tho)
I made a GitHub Actions based version at https://github.com/Ostabo/news-api
You could fork it and change the countries to your needs or use the 8 I used


The results in Antmaze

Hi,
I would like to ask the setting about the experiments in Antmaze. Should I need to tune the hyparameters for mujoco locomotion?
I find I cannot reproduce the results about Antmaze in the paper.
Best

I believe the only difference from the default hyperparameters for Antmaze was that normalization was not used. You should be able to obtain even better results than what was reported in the paper by using larger values of alpha.


Feature: Activity Selection

Hey @Ultron03 , I would like to work on this. Please assign this to me.

can you assign this issue to me ?


ListAllByDNSZoneComplete (services/dns/mgmt) doesn't return all records in case of pagination

Bug Report

related issue: https://github.com/StackExchange/dnscontrol/issues/770
import path of package in question, e.g. services/dns/mgmt/2018-05-01/dns
SDK version e.g. master (tested on v43.3.0): github.com/Azure/azure-sdk-for-go v42.3.0+incompatible
output of go version: go version go1.14.2 darwin/amd64

What happened?
I am using https://github.com/StackExchange/dnscontrol for managing a pretty large DNS zone (1000+ records) on Azure DNS. All these records were created by the tool on Azure DNS, but when I tried to repeat the same command, I received a set from 30 records that should be created one more time. Expected result — 0 changes. At the same time, all these records are present on Azure DNS.
I tried to troubleshout this case and found a cause - after some amount of records (generally speaking after some amount of text in API response from Azure DNS) API paginates a result with nextLink:
{"nextLink":"https:\/\/management.azure.com:443\/subscriptions\.../ALL?...","value":[{"id":"\/subscriptions\/...

And SDK call ListAllByDNSZoneComplete doesn't process this nextLink.
https://github.com/Azure/azure-sdk-for-go/blob/master/services/dns/mgmt/2018-05-01/dns/recordsets.go#L413
Tried to use azure-sdk-for-go v43.3.0 instead of v42.2.0 (it was latest for the moment of testing) - the same.
Tried to fix dnscontrol with some code like this — doesn't work, got an endless loop (6 seconds and a deferred cancel).
for recordsIterator, err := a.recordsClient.ListAllByDNSZone(ctx, *a.resourceGroup, zoneName, to.Int32Ptr(1000), ""); recordsIterator.NotDone(); err = recordsIterator.NextWithContext(ctx) {
        recordsResult := recordsIterator.Response()
        for _, r := range *recordsResult.Value {
            record := r
            records = append(records, &record)
        }
}

For the tesing purpose I changed a visibility of ListByDNSZoneNextResults from azure-sdk-for-go (it's private) and implemeted this function:
func (a *azureDnsProvider) fetchRecordSets(zoneName string) ([]*adns.RecordSet, error) {
	if zoneName == "" {
		return nil, nil
	}
	var records []*adns.RecordSet
	ctx, cancel := context.WithTimeout(context.Background(), 6000*time.Second)
	defer cancel()
	recordsIterator, recordsErr := a.recordsClient.ListAllByDNSZoneComplete(ctx, *a.resourceGroup, zoneName, to.Int32Ptr(1000), "")
	if recordsErr != nil {
		return nil, recordsErr
	}
	recordsResult := recordsIterator.Response()

	for _, r := range *recordsResult.Value {
		record := r
		records = append(records, &record)
	}

    // Process nextLink data
	for recordsResult.NextLink != nil {
		var err error
		recordsResult, err = a.recordsClient.ListByDNSZoneNextResults(ctx, recordsResult)
		if err != nil {
			fmt.Errorf("fetchRecordSets NextLink. loading Resource List: %+v", err)
			continue
		}

		for _, r := range *recordsResult.Value {
			fmt.Printf("fetchRecordSets NextLink FQDN: [%v]\n", *r.Fqdn)
			record := r
			records = append(records, &record)
		}
	}

	return records, nil
}

Please note for recordsResult.NextLink != nil and a call to ListByDNSZoneNextResults.
This variant is working perfectly. But I can't use it because of needed changes in azure-sdk-for-go.
Could you please check this case and recommend how to use properly azure-sdk-for-go and activate result.fn for fetching all DNS records?
https://github.com/Azure/azure-sdk-for-go/blob/master/services/dns/mgmt/2018-05-01/dns/recordsets.go#L325
func (client RecordSetsClient) ListAllByDNSZone
...
result.fn = client.listAllByDNSZoneNextResults
...

Or confirm a bug and fix it.
My case is not uniq, another persons have have running into this issue as well.
What did you expect or want to happen?
Comment from author of dnscontrol: I agree that the go module should either return all records (hide that there is any paging at all) or expose the paging and provide a way to get specific pages. I'd prefer it would hide the paging from the caller.
How can we reproduce it?
Use dnscontrol and create DNS records based on my example file from https://github.com/StackExchange/dnscontrol/issues/770:
https://github.com/StackExchange/dnscontrol/files/4804447/dnsconfig.txt
Anything we should know about your environment.
Nothing special, I tested this case by compiling dnscontrol natively on macOS Catalina, but it the same on Ubuntu 19.04.
Oleg

It seems like this may be related?
https://github.com/Azure/azure-sdk-for-go/issues/9510
There is a PR to fix that issue - https://github.com/Azure/autorest.go/pull/433
Not sure though, if these are the same problem.

Hi @run1et thanks for this issue! And thanks for @meowingtons for staying tuned!
I am not sure if this is exactly the same issue as #9510 since it is talking about issues raise when the service returns both empty value and non-empty next link, but this issue should be at least related.
Hi @jhendrixMSFT could you please have another review on the fix in autorest.go?

@run1et @ArcturusZhang,
It looks like the fix for #9510 was merged, what's the easiest way to test this to see if it's fixed or related?
Thanks!

@run1et @ArcturusZhang,
It looks like the fix for #9510 was merged, what's the easiest way to test this to see if it's fixed or related?
Thanks!

Those changes should be applied in this month's major release

I've fixed this issue in https://github.com/StackExchange/dnscontrol/pull/791
Looks like it's not related to the SDK and my fix is working with github.com/Azure/azure-sdk-for-go v43.3.0+incompatible. I moved from ListAllByDNSZoneComplete to ListAllByDNSZone and a proper iterator usage.
I checked ListAllByDNSZoneComplete with SDK from master branch — the same behavior as I described in this ticket. Will recheck with this month's major release.

@run1et, do you still have this issue now?


Add missing withResolvers method to PromiseWaiter

Promise.withResolvers() is missing on WrappedPromise leading to errors when code uses is in tests, for example pdfjs:

Thanks a lot for your PR. The responsible team will have a look. Internally we track this PR via DINC0355508.


try bearssl as backend provider

we can wait. let's finish things with boringssl first :)
moved to 0.9.6

ahem, it's actually done, just sitting in a separate branch

next steps:

update PR to enable all required tests
make sure that tests are passing

Status update (based on @mnaza input).
Changes are sitting here
https://github.com/mnaza/themis/tree/bearssl
Merge plan:

[ ] Merge this branch into your own fork.

[ ] Fix Keygen Interfaces
Check key generation functions, they are changed from two functions
themis_gen_ec_key_pair(...) and themis_gen_rsa_key_pair(...)
to one themis_gen_key_pair(THEMIS_SECURE_ENCRYPTED_MESSAGE_DEFAULT_ALG|THEMIS_SECURE_SIGNED_MESSAGE_DEFAULT_ALG, ...)

[ ] Make sure that themis_gen_ec_key_pair(...) and themis_gen_rsa_key_pair(...) are working correctly using new key_gen function (backward compatibility).
[ ] Make sure that all Themis wrappers are building/working correctly through old functions.

[ ] Handle Infrastructure issues
Currently we don't have any way to change 'backend providers' (OpenSSL/LibreSSL/BoringSSL/BearSSL). In fact, this switch is possible from C-core only, wrappers don't have this feature.
Our goal is to create simple "switcher" that allows to switch from different backend providers. Question: how to handle dependencies (for example, iOS wrapper is linked to OpenSSL directly through podspec).
Creating ./configure script might be an option.

[ ] Create switcher from one backend provider to another, allow wrappers to use different backend providers.

Update and Fix Tests
Merge tests with tests from base branch, check that tests are running successfully. Careful, CircleCI build usually takes ~30min

[ ] Make sure that all Themis wrappers are building/working correctly

as discussed with @mnaza, @vixentael: this barely adds any value for users in 0.9.5, so should be kept in a separate branch to illustrate research articles and that's it. When we have time after H/T releases, we'll definitely revisit ways for one build process for different backends, but it isn't worth it right now.

well, this is sitting in a separate branch
https://github.com/vixentael/themis/tree/bearssl
This thing is waiting its merge and blogpost

Well, almost 4 years after – if we decide to try bearssl, we will come back here, and reopen it.


Migrated slave client connections on the master node upon corresponding slave failure are not released after the slave node recovered

Hi Redisson Community,
While testing the client connection amount that redisson client establish, I notice three behaviors that I don't have an answer about. Might be a connection leak, but I don't know if there is any reason. Hope anyone could share some hint about the reason behind the scene.
Redisson client version
v3.17.3
Code I use for the behavior below
//Only using the initialization, no any query is performed

public class RedissonTest {

    public static void main(String[] args) throws Exception {

        File redissonConfigYaml = new File("<path>/redisson.yaml");
        Config redissonConfig = Config.fromYAML(redissonConfigYaml);

        if (Epoll.isAvailable()) {
            redissonConfig.setTransportMode(TransportMode.EPOLL);
        }

        RedissonClient redissonClient = Redisson.create(redissonConfig);
    }
}

Redisson configuration I use
clusterServersConfig:
  idleConnectionTimeout: 10000
  connectTimeout: 10000
  timeout: 3000
  retryAttempts: 3
  retryInterval: 1500
  password: null
  subscriptionsPerConnection: 5
  clientName: Test
  loadBalancer: !<org.redisson.connection.balancer.RoundRobinLoadBalancer> {}
  subscriptionConnectionMinimumIdleSize: 0
  subscriptionConnectionPoolSize: 0
  slaveConnectionMinimumIdleSize: 2
  slaveConnectionPoolSize: 2
  masterConnectionMinimumIdleSize: 2
  masterConnectionPoolSize: 2
  readMode: "MASTER_SLAVE"
  subscriptionMode: "SLAVE"
  nodeAddresses:
    - "redis://redis.master1:6359"
    - "redis://redis.master2:6359"
    - "redis://redis.master3:6359"
  scanInterval: 5000
  pingConnectionInterval: 30000
  keepAlive: false
  tcpNoDelay: false
threads: 2
nettyThreads: 32
codec: !<org.redisson.codec.FstCodec> {}

transportMode: "NIO"

1. Extra client connection in master nodes
When I initialize the redisson client, I ran the client list command to list all the client connections with name "Test" (defined in the configuration above). And I expect 2 connections to each master node, and 2 connections to each slave node. But instead, I saw 3 connections to each master node, and 2 connections to each slave node.
Below are the 3 client connections I captured in different time. And I noticed one of the connection is doing cmd "cluster" sometime.
id=190 addr=localhost:1852 laddr=redis.master1:6359 fd=20 name=Test age=12 idle=12 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=cluster user=default redir=-1
id=191 addr=localhost:1854 laddr=redis.master1:6359 fd=21 name=Test age=12 idle=12 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1
id=192 addr=localhost:1855 laddr=redis.master1:6359 fd=22 name=Test age=12 idle=12 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1

id=190 addr=localhost:1852 laddr=redis.master1:6359 fd=20 name=Test age=36 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1
id=191 addr=localhost:1854 laddr=redis.master1:6359 fd=21 name=Test age=36 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1
id=192 addr=localhost:1855 laddr=redis.master1:6359 fd=22 name=Test age=36 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1

id=190 addr=localhost:1852 laddr=redis.master1:6359 fd=20 name=Test age=126 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1
id=191 addr=localhost:1854 laddr=redis.master1:6359 fd=21 name=Test age=126 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1
id=192 addr=localhost:1855 laddr=redis.master1:6359 fd=22 name=Test age=126 idle=5 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20496 events=r cmd=ping user=default redir=-1

I am guessing that the extra connection is remained for resolving the topology, but I am not 100% sure. So may I ask what is the purpose of the extra connection on master node?
If there is a purpose for the extra connection, is this extra connection not limited by the pool size from the configuration (I set both connection pool size minimum and limit to 2)?
2. Upon slave node failure, the connections are migrated to master. But never released from the master node even the slave is recovered from failure.
After the initialization, I got 3 client connections for each master and 2 client connections for each slave.
Below is result of the script that I ran to count the client connection with name "Test" on each node.
Test has 3 client connection in redis.master1
Test has 3 client connection in redis.master2
Test has 3 client connection in redis.master3
Test has 2 client connection in redis.slave1
Test has 2 client connection in redis.slave2
Test has 2 client connection in redis.slave3

Then I stopped one of the slave node "redis.slave1" (I am using docker container, so I ran 'docker stop' which should equivalent to SIGTERM --> grace period --> SIGKILL ). Then I notice the 2 client connections on the stopped slave are migrated to the corresponding master.
(The master&slave relation of my local setup is like: redis.master1&redis.slave3, redis.master2&redis.slave1, redis.master3&redis.slave2)
Test has 3 client connection in redis.master1
Test has 5 client connection in redis.master2
Test has 3 client connection in redis.master3
Could not connect to Redis at redis.slave1: Connection refused
Test has 0 client connection in redis.slave1
Test has 2 client connection in redis.slave2
Test has 2 client connection in redis.slave3

Then I restart the "redis.slave1", the 2 client connections are reconnected. But after waiting about 20mins, the 2 extra connections that migrated to its master are never released.
Test has 3 client connection in redis.master1
Test has 5 client connection in redis.master2
Test has 3 client connection in redis.master3
Test has 2 client connection in redis.slave1
Test has 2 client connection in redis.slave2
Test has 2 client connection in redis.slave3

And here is the redisson log when during the stop and restart of the "redis.slave1"
[redisson-netty-2-5] INFO org.redisson.connection.MasterSlaveEntry - master redis.master2/redis.master2:6359 used as slave
[redisson-netty-2-5] WARN org.redisson.cluster.ClusterConnectionManager - slave: redis://redis.slave1:6360 has down for slot ranges: [[5461-10922]]
[redisson-netty-2-28] INFO org.redisson.connection.pool.SlaveConnectionPool - 2 connections initialized for redis.master2/redis.master2:6359
[redisson-netty-2-8] INFO org.redisson.cluster.ClusterConnectionManager - slave: redis://redis.slave1:6360 is up for slot ranges: [[5461-10922]]
[redisson-netty-2-21] INFO org.redisson.connection.pool.SlaveConnectionPool - 2 connections initialized for redis.slave1/redis.slave1:6360

When I test the situation when "master node is down", I notice the slave become master and take over the connections. And as soon as the master back alive, and fix the role by failover, the extra connections on the slave node are released. This is the expected behavior.
But the behavior is different when "slave node is down" , why the connections that been taken over by master are not released once the slave node is back alive? And this is causing the redisson client hold the connections more than the defined pool size of configuration.
Thanks for your time and patience.

Hi,

I am guessing that the extra connection is remained for resolving the topology, but I am not 100% sure. So may I ask what is the purpose of the extra connection on master node?

It's used for topology scanning.

But the behavior is different when "slave node is down" , why the connections that been taken over by master are not released once the slave node is back alive?

Because when there are no alive slaves for master node it's used as slave with additional connections.

Thank you for the quick response @mrniko !
I understand the master is also using these additional connection while slave is not alive. But after the slave is back alive, why these additional connection remains in master node not getting released? And the total connections are more than the configured master connection pool size?

Because readMode: "MASTER_SLAVE". Specify readMode: MASTER.

Hi @mrniko
I tried both "MASTER_SLAVE" and "MASTER", the behavior is same. After the slave is back, the addition connection on master node is not released.
Result:
After initializing the redisson client.
Test has 3 client connection in redis.master1
Test has 3 client connection in redis.master2
Test has 3 client connection in redis.master3
Test has 2 client connection in redis.slave1
Test has 2 client connection in redis.slave2
Test has 2 client connection in redis.slave3

After stopping one of the redis slave node.
Test has 3 client connection in redis.master1
Test has 5 client connection in redis.master2
Test has 3 client connection in redis.master3
Could not connect to Redis at redis.slave1: Connection refused
Test has 0 client connection in redis.slave1
Test has 2 client connection in redis.slave2
Test has 2 client connection in redis.slave3

After restarting that slave node.
Test has 3 client connection in redis.master1
Test has 5 client connection in redis.master2
Test has 3 client connection in redis.master3
Test has 2 client connection in redis.slave1
Test has 2 client connection in redis.slave2
Test has 2 client connection in redis.slave3

I am not sure if this behavior is designed. If yes, could you tell me the purpose of it? Because I am starting to considering the production environment, and this might leave 3 or 4 digits number of additional connection on the master node.
Current behavior of the master nodes client connection count:  3 -> - (slave down) -> 5 -> - (slave back alive) -> 5 -> 5 ...
(What I assume the behavior): 3 -> - (slave down) -> 5 -> - (slave back alive) -> 3 ...

@RunzhongHuang
What was the issue?

@mrniko Still not sure what was actually happening. And even with this observation, there weren't impact of the performance in our current product. So I shifted the priority and move on to other stuffs. Saw this was still opened, so I was thinking to close it for now and drill down to this issue later time with more knowledge of the behavior between Redisson client and Redis server.