Key Error ['items']

Hi, since I'm new to APIs, I'm trying to get accustomed to using Spotify API for creating a Flask Application.   I got a key error when i was trying to extract a user's recently played tracks.

I am not able to identify why this error occured. Could you please let me the the cause of the error?
Below is the code
'def get_user_last_played_tracks(self, auth_header, user_id, limit=25):
        """Get the last n tracks played by a user who logins
        :param auth_header: the authentication header
        :param user_id: the Spotify User ID when a user gets authenticated in Spotify
        :param limit: Number of tracks to get. Limit should be <= 50
        :return : List of last played n tracks from the user
        """
        last_played_api_endpoint = f"https://api.spotify.com/v1/users/{user_id}/player/recently-played?limit={limit}"
        tracks = json.loads(requests.get(last_played_api_endpoint, headers=auth_header).text)

        return [
             {
                'track_artist': track["track"]["artists"][0]["name"],
                'track_name': track["track"]["name"],
                'track_image': track["track"]["album"]["images"][0]["url"],
                'track_url': track["track"]["external_urls"]["spotify"],
                'track_id': track["track"]["id"]
             } for track in tracks["items"]
        ]`

Flask app
@app.route("/select_tracks", methods=['GET', 'POST'])
def select_tracks():
    authorization_header = session['authorization_header']

    def extract_letters(string):
        return ''.join([letter for letter in string if not letter.isdigit()])

    if request.method == 'GET':
        # -------- Get user's name, id, and set session --------
        profile_data = spotify_client.get_user_profile_data(authorization_header)
        user_display_name, user_id = profile_data['display_name'], profile_data['id']
        session['user_id'], session['user_display_name'] = user_id, user_display_name

        # -------- Get user recently played tracks data --------
        tracks_data = spotify_client.get_user_last_played_tracks(authorization_header, user_id, limit=25)

        return render_template('playlist_select_tracks.html',
                               user_display_name=user_display_name,
                               tracks=tracks_data,
                               func=extract_letters)

    return render_template('playlist_select_tracks.html')

https://api.spotify.com/v1/users/{user_id}/player/recently-played?limit={limit}

This endpoint does not exist.

Oh okay. Yes sure, i'll do that. Also, is there a way where I can retrieve any user's recently played songs when they login through my Flask application?

Use this method to retrieve the user's recently played tracks:
https://github.com/plamere/spotipy/blob/06551cd0553d6e030f3f4ee7da5fac12c424bac7/spotipy/client.py#L1432
See this example for how to use spotipy with flask.

Thanks alot!

Hi, is it possible to get recently played tracks from other user by using their user id when the login through my application?
Since this API endpoint is only for the current user https://api.spotify.com/v1/me/player/recently-played?limit={limit} ?

No, you can only get the recently played tracks for the current user. The spotipy documentation describes all of the available endpoints. You should also read the Spotify web API reference. If you don't see what you're looking for on those pages, then it's not possible.

Oh okay. Thanks


Broken links in the articles page

Most of the links pointing to Ondřej's blog are broken. Either we need to figure out if there's a path to retrieve the content (form archive.org or somesuch) or drop it.

/cc @HarryMichal

Links are up to date now!


Delayed aio infer during burst requests

Background
I have a simple FastAPI app that will call Triton server using Triton client. It works great for small, non-rapid requests. But the behavior starts to decline when I introduce a lot of concurrent requests, such as when using JMeter. Originally I thought the bottleneck was on my Triton server, but turns out it happened on the client (FastAPI app). Both Triton server and client are on my local PC.
Reproduce attempt
Below are all components required to reproduce this problem.
# Repo structure
.
├── model_repository
│   └── dummy
│       ├── 1
│       │   └── model.py
│       └── config.pbtxt
<snip>

# config.pbtxt

backend: "python"
max_batch_size: 10

dynamic_batching {}

input [
  {
    name: "image"
    data_type: TYPE_FP32
    dims: [3, 1000, 1000]
  }
]

output [
  {
    name: "result"
    data_type: TYPE_INT32
    dims: [5]
  }
]

# model.py
import time

import numpy as np
import triton_python_backend_utils as pb_utils

class TritonPythonModel:
    def execute(self, requests):
        n = len(requests)
        print(f"received {n} requests, at {time.time()}", flush=True)
        # always return dummy tensor
        responses = []
        for i in range(n):
            responses.append(
                pb_utils.InferenceResponse(
                    [pb_utils.Tensor("result", np.arange(5, dtype=np.int32))]
                )
            )
        return responses

# http_client_aio.py
import asyncio

import numpy as np
import tritonclient.http.aio as httpclient

async def main():
    client = httpclient.InferenceServerClient(url="localhost:8000")
    fake_image = np.random.random((1, 3, 1000, 1000)).astype(np.float32)

    tasks = []
    for i in range(500):
        fake_input = httpclient.InferInput("image", fake_image.shape, "FP32")
        fake_input.set_data_from_numpy(fake_image)
        task = asyncio.create_task(client.infer("dummy", [fake_input]))
        tasks.append(task)
    print("done sending")

    await asyncio.gather(*tasks)
    await client.close()

if __name__ == "__main__":
    asyncio.run(main())

Docker commands for Triton server.
docker run --shm-size=1gb --rm -p8000:8000 -p8001:8001 -p8002:8002 -v ${PWD}/model_repository:/models nvcr.io/nvidia/tritonserver:23.12-py3 tritonserver --model-repository=/models

I opened 2 terminals, run docker commands on the first and http_client_aio.py on the second window. Here is what happened:

Start run  http_client_aio.py
"done sending" printed after ~2 secs.
There are ~3 secs of delay which nothing seems to happen.
Logs in Triton server (TritonPythonModel print statements) starts rolling in.

Questions

Am I missing something?
What happened on those ~3 secs delay?
Can I saturate Triton server using just aio client with relatively big inference payload (I use 1000x1000 image resolution)?
What are the best practices to handle this bottleneck?

There was a fix for the HTTP client in async mode in 24.07 that resolved some unintended blocking that throttled requests (in most cases, close to the max throughput, but not always for some types of models).
Note that means once r24.07 goes out, requests get sent almost immediately, so it's possible to go beyond what your system/model can handle, thereby driving up latency (this will be published in the release notes). You can try building the client now or trying out 24.07 once it is released soon.

Have you tried adding a flush=true to the print statement? It's possible the delay is due to flushing not happening immediately.
CC: @Tabrizian

Just to make sure I'm understanding the problem clearly, the issue is that you're expecting the inferences to be executed as soon as asyncio.create_task is complete but it looks like all the tasks are created first, then there is a delay and after that logs are created is that correct?
I think it might be that you need release the current thread while you're creating tasks to unblock the threads to execute. For example, adding the line below every N requests might help with executing some of the client requests you're creating.
await asyncio.sleep(0)

For flush=True, I've added those and the behavior is identical.
Now, for the asyncio issue.

the issue is that you're expecting the inferences to be executed as soon as asyncio.create_task is complete but it looks like all the tasks are created first, then there is a delay and after that logs are created is that correct?

This is mostly correct, but the logs here belongs inside Triton model, not the client. Here are some examples:
500 tasks scenario:

[client] I schedule 500 aio inference calls.
[client] print("done sending") executed
[server] Wait long time
[server] Triton logs start showing up

30 tasks scenario:

[client] I schedule 30 aio inference calls.
[client] print("done sending") executed
[server] Wait short time
[server] Triton logs start showing up

Why does the number of tasks affect the initial data received by Triton server? It looks like triton client "bunches" up together requests, and then sends them all to Triton server. My original intuition tells me that the earlier inference requests should arrive at the server at around the same time, regardless of how many inference requests scheduled.

await asyncio.sleep(0) works! This has solved the delay problem, now the earliest request will enter Triton server with almost no delay. But after some quick tests, this method will only work if we control the task creation, e.g. the loop with asyncio.create_task(...) right? For web related tasks like using FastAPI, I still cannot inject the sleep(0) though. Maybe I'll find another solution for this. Thankyou.

Feel free to open a new Github issue, if you were still running into issues. The thread needs be released in order for the client to make progress and start actually sending the requests to the server.


Query Designer - Order by dropdownlist doesn't appear in Firefox

Query Designer - Order by dropdownlist doesn't appear in Firefox

Bug fixed - related to re-orderable properties plugin


Add types for optional options argument

Adds the missing types for the second argument

Based on that string[] is correct. We can't import properties from an interface as far as I'm aware, so we have to manually type it.

@leo no, they should be strings. process.argv is always strings, as with any other language.


Isolated Functions don't like my big blobs

We have a big Blob stored in Azure storage (testing with smaller blobs of a few Kb, with 50mb, and up to about 100mb).
Our ultimate goal is to react to a relatively big blob upload in Azure Storage and send it to an Azure Function I'm running as an out-of-process function with a BlobTrigger. As mentioned in the official Isolated Process Guide the binding support is scarcer in isolated functions, and "rich types like (...) and  CloudBlockBlobs are not supported", so when refactoring towards an isolated function, I tried to receive the Blob as string and as byte[], both working fine for smaller blobs. But!... the trigger doesn't pick up bigger blobs (50mb). The blob upload to storage is detected but doesn't trigger the function. Our function is using the correct worker dependencies also mentioned in the guide.
Here's the code:
[Function("Shipper")]
        [BlobOutput("clean/{name}", Connection = "AzureWebJobsStorage")]
        public async Task<List<ConstituentPatchRequestDto>> Run(
        [Microsoft.Azure.Functions.Worker.BlobTrigger("dirty/{name}", Connection = "AzureWebJobsStorage")]
        string inputBlob,
        string name = "")
        {
            Log.Information("New blob triggered function. Started processing blob with the following characteristics: \n Client: {name} \n Project GUID: {guid} ");

            var result = await _readService.Cleanse(inputBlob, name);

            Log.Information("Run successful!");

            return result;

        }

Does any one have any alternatives or possible solutions to achieve our goals stated above?

Any follow up on this? I was forced in the meantime to use HttpTriggers because neither EventGrid nor BlobTrigger would support big blobs due to their lack of support for the input Stream binding (which is advised to be used instead of just shoving the blob in a string). But it would defeat the purpose of having Blobtriggers in the first place. Anyone was able to trigger a function with big blob, either through EventGrid or BlobTrigger? Thanks

I have the same issues, with blob of approximately 4mb, the function never executes, I just get a tons of Microsoft.Azure.WebJobs.Host.FunctionTimeoutException and had to rollback to 3.1


Share your editor config

Hi Adrien,
You should share your editor config file, as this, dev is standardized

How do you usually do this?
(I use VSCode)

You've found^^

@MathRobin I used an extension for VSCode. Still curious of how you generate this file for your different projects, tho.

I don't know for all IDE, but on Jetbrains, you can export it natively.


Authentication for private pages

🚀 Feature
Add the authentication layer to support hosting the private pages in the same documentation site.
A simple example:

Let's say we have an "API integration guide" which is generic and open to anyone who wants to integrate it.

Let's say we have some premium partners for whom we have some custom APIs documented with some confidential parameters and values (on and above public ones), which are not to be exposed to the world.

We would want to enable auth for these premium partners for such pages.
I guess this would make a very reasonable addition to the product
Have you read the Contributing Guidelines on issues?
YES
Motivation
We have stumbled upon a request from one of our premium partners that, in addition to the public API documentation we are having, they want us to enable authentication based documentation which encapsulates custom APIs and Parameters implemented exclusively for them. Also, this is seemingly becoming a common ask among such premium partners.
Also, I don't think this is an uncommon request. Many people/organisations might just fly-by and not report this feature request and move on to something else to fulfil their requirements.
Pitch
This feature will enable a secure access to some private documents while still using all the goodness of the Docusaurus system. Basically, the private documentation should not become a reason for people to look for alternatives and eventually end up using a much complicated, paid service/product.
Supporting feedback from one of the users - https://v2.docusaurus.io/feedback/p/could-we-add-an-authentication

Docusaurus is a static site generator, it is not a server, so we cannot control access to pages. What you want, I think, can be done through a reverse proxy, for example, https://github.com/bitly/oauth2_proxy

Docusaurus is a static site generator, it is not a server, so we cannot control access to pages. What you want, I think, can be done through a reverse proxy, for example, https://github.com/bitly/oauth2_proxy

Thank you for quick reply. Seems like it is doable. Let me quickly check and confirm.

It appears that https://github.com/bitly/oauth2_proxy repo is not maintained anymore. It has also been marked read-only by its owner.
I am not sure if I want to use it for production purposes.

See other projects, there are links in that repo. Moreover if that project is no longer maintained, this does not mean that this is not a working solution.
In any case, that was only the first example I know, you can find other options. I mean, this issue still doesn't apply to Docusuarus directly.

See other projects, there are links in that repo. Moreover if that project is no longer maintained, this does not mean that this is not a working solution.
In any case, that was only the first example I know, you can find other options. I mean, this issue still doesn't apply to Docusuarus directly.

Whoops! Boy, did I miss that notice. Thanks again. I understand the static only nature of Docusaurus. May be we should let it be what it is.
Closing the ticket on that note.

You can make authentication work in a stateless manner like the APIs do. You can either use a 3rd party authenticator or use your own API authentication and do a stateless authentication. Docusaurus can be static but in the end every site is generating a static content. If your static site shows some data from API or backend you can authenticate yourself. But yes, not concern of Docusaurus.

You can make authentication work in a stateless manner like the APIs do. You can either use a 3rd party authenticator or use your own API authentication and do a stateless authentication. Docusaurus can be static but in the end every site is generating a static content. If your static site shows some data from API or backend you can authenticate yourself. But yes, not concern of Docusaurus.

I was thinking along the same lines and even have User Management System API built with Flask. It handles all the work of authentication. But now I wonder,

How do I make the request from docusaurus to my API?
Suppose I make this request and store the response data in my client via cookies or something, how do I use this data in my docusaurus app? Specifically, hide certain pages if the user does not have the required credentials or authorization to view the page?

Yes you can do anything authorized and authenticated using this way. If you are able to provide a hash to the user like JWT or any other authentication service token, since you can store in the cookie as you mentioned, you can store some basic user attributes like roles etc in the cookies and according to the user role you can show or hide content in docusaurus. And let's say user wanted to change in their cookie and revealed some menu items like another user role's menu item or page style. When they submit the altered cookie or user auth token to your API you still won't authorize to show the data they try to hack. Once you can authorize every request at the backend side you will never reveal sensitive data.
Let's revisit how you can customize the content according to the user role. Let's assume you get the user role from the cookie as it is taken from the token or your auth api and you know the current user is Role-A user. Show role-a pages, menu items to the user that is all. You can show different docusaurus top menu items using this method. If you don't want a user view a specific page even if they alter the cookie, just authorize the page at the begining using your Auth API and get the authorized or unauthorized response and show a message or redirect. That is all.

If you are asking how to send API requests from docusaurus, I suggest you to use axios module in docusaurus and use accordingly.

Yes you can do anything authorized and authenticated using this way. If you are able to provide a hash to the user like JWT or any other authentication service token, since you can store in the cookie as you mentioned, you can store some basic user attributes like roles etc in the cookies and according to the user role you can show or hide content in docusaurus. And let's say user wanted to change in their cookie and revealed some menu items like another user role's menu item or page style. When they submit the altered cookie or user auth token to your API you still won't authorize to show the data they try to hack. Once you can authorize every request at the backend side you will never reveal sensitive data.
Let's revisit how you can customize the content according to the user role. Let's assume you get the user role from the cookie as it is taken from the token or your auth api and you know the current user is Role-A user. Show role-a pages, menu items to the user that is all. You can show different docusaurus top menu items using this method. If you don't want a user view a specific page even if they alter the cookie, just authorize the page at the begining using your Auth API and get the authorized or unauthorized response and show a message or redirect. That is all.

Thank you for such a succinct high level overview. So now here lies the crunch point: specifics.

In which page/file do I make the request to my API?
In which page/file can I manipulate the pages to be shown on the sidebar based on my user details?

Thank you for any light you shed.

Not 100% sure I agree with everything here.
Docusaurus builds a static site. IE the content is exactly the same for all users.
If you want different content displayed for different user roles, you have to:

make that content dynamic for each user: ie render it only on the client side by swizzling React components and fetching your dynamic data yourself, like in any React app)
build one different static site per user role, each containing a different set of visible pages. You can use serverless edge functions (Cloudflare Workers for example) to serve one static deployment or another based on the role of the user and pages they can access.

Remember all these are concerns that are outside the scope of Docusaurus. In the end, we just build a static React site.

@LexxLuey well, in any page you can make the http (API) requests. If a page is supposed to be authorized by the API you should call your api from that page (Reactjs pages not markdown pages I am saying).
I was able to do this myself actually. I implemented a JWT authentication in the API side, created some pages for different roles. When a user is not logged you can identify it, when a user is logged in you can identify it too.
Finally, the site is static that's right but you don't have to show every content you have for instance if they are provided by API (in a DB for example). Ajax calls are used for this and you static side can pick data by utilizing fully authorized APIs.

Not 100% sure I agree with everything here.
Docusaurus builds a static site. IE the content is exactly the same for all users.
If you want different content displayed for different user roles, you have to:
* make that content dynamic for each user: ie render it only on the client side by swizzling React components and fetching your dynamic data yourself, like in any React app)

* build one different static site per user role, each containing a different set of visible pages. You can use serverless edge functions (Cloudflare Workers for example) to serve one static deployment or another based on the role of the user and pages they can access.

Remember all these are concerns that are outside the scope of Docusaurus. In the end, we just build a static React site.

I understand this is a static site generator. no doubt. i am only asking if it is possible. If it is not, then that's okay. I felt it was possible to limit what pages are displayed.

I am trying to add Keycloak identity provider to my docusaurus website through plugin configuration. can anyone suggest which package to use to create plugin?

As a side note: what does not work is to add http basic authentication to subdirectories of what Docusaurus builds.
That is because it doesn't behave like a true static app, it actually behaves like a single page app.
For instance, if you add HTTP authentication to /membersonly, then only invocations will hit the authentication, but it will not be hit upon navigation via a link from elsewhere in the site.
So it appears that protecting part of a site (as in one or more subdirectories) is fundamentally not possible with how Docusaurus is currently designed, is that correct?

@godmar the problem is that you don't only want to protect the html static pages, but also protect the JS code that permits to generate those pages dynamically on the client. That JS is code-split into the /assets/js folder and unfortunately it is not easy to do.
If you want to have a /membersonly/ section on your site, my recommendation would be to not use static pages for this part of your site but instead create a plugin with a dynamic route path like /membersonly/*, let it render an empty /membersonly/index.html static page shell, and then use React-Router in your <MembersOnly> route component to decide what to render using regular React code, eventually fetching protected API data or displaying a login screen. What I mean is: you can use Docusaurus in the same way as you would use Create-React-App, for a subset of your site.
Regarding dynamic client-side routes, one example to take inspiration from is our feature request page. The data you see on this page is not static, it is only loaded later, after the initial React hydration:
https://docusaurus.io/feature-requests/p/comments-in-documents-or-blogs
Plugin code: https://github.com/facebook/docusaurus/tree/main/website/src/plugins/featureRequests

@slorber
do you have document about how to use the [featureRequests] plugin? (https://github.com/facebook/docusaurus/tree/main/website/src/plugins/featureRequests)

The feature request plugin is not an officially distributed plugin. I mentioned it above only as an open-source example so that you can take a look at its code.

Since I recently succeeded in protecting (the whole) docusaurus deployment for private documentation sites with user login, I thought I would share my example. I might also add a blog post about it soonish.

@godmar the problem is that you don't only want to protect the html static pages, but also protect the JS code that permits to generate those pages dynamically on the client. That JS is code-split into the /assets/js folder and unfortunately it is not easy to do.
If you want to have a /membersonly/ section on your site, my recommendation would be to not use static pages for this part of your site but instead create a plugin with a dynamic route path like /membersonly/*, let it render an empty /membersonly/index.html static page shell, and then use React-Router in your <MembersOnly> route component to decide what to render using regular React code, eventually fetching protected API data or displaying a login screen. What I mean is: you can use Docusaurus in the same way as you would use Create-React-App, for a subset of your site.
Regarding dynamic client-side routes, one example to take inspiration from is our feature request page. The data you see on this page is not static, it is only loaded later, after the initial React hydration:
https://docusaurus.io/feature-requests/p/comments-in-documents-or-blogs
Plugin code: https://github.com/facebook/docusaurus/tree/main/website/src/plugins/featureRequests

It would be really useful if it was possible to only do SSG for a specific route while disabling SPA bundling for that route's content.
That way, any usage of <Link to="/membersonly"> would be forced to do an actual HTTP request, and then trigger the authentication gateway response.
For example by combining CF Zero Trust with Docusaurus.


fix: LSDV-5158: Hide prediction delete button as it doesn't actually allow prediction deletions at this time

PR fulfills these requirements

[X] Commit message(s) and PR title follows the format [fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made ex. fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errors
[ ] Tests for the changes have been added/updated (for bug fixes/features)
[ ] Docs have been added/updated (for bug fixes/features)
[X] Best efforts were made to ensure docs/code are concise and coherent (checked for spelling/grammatical errors, commented out code, debug logs etc.)
[X] Self-reviewed and ran all changes on a local instance (for bug fixes/features)

Change has impacts in these area(s)
(check all that apply)

[ ] Product design
[X] Frontend

Describe the reason for change
Currently predictions cannot be deleted, so there is no point in having a delete button. Removing this for consistency until such time that we have a proper method of deleting predictions.
Which logical domain(s) does this change affect?
AnnotationButton

Codecov Report
Patch coverage: 66.66% and project coverage change: -0.01 :warning:

Comparison is base (ee9cfa5) 65.35% compared to head (23bd99d) 65.35%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1427      +/-   ##
==========================================
- Coverage   65.35%   65.35%   -0.01%
==========================================
  Files         436      436
  Lines       27283    27285       +2
  Branches     7237     7239       +2
==========================================
+ Hits        17832    17833       +1
  Misses       8114     8114
- Partials     1337     1338       +1

Impacted Files
Coverage Δ

...components/SidePanels/TabPanels/SideTabsPanels.tsx
1.33% <0.00%> (-0.01%)
:arrow_down:

...omponents/AnnotationsCarousel/AnnotationButton.tsx
61.33% <100.00%> (ø)

src/components/SidePanels/SidePanels.tsx
93.64% <100.00%> (+0.03%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Upgrade JaCoCo when Android Gradle Plugin 4.2 is released

We're pinning an old version of JaCoCo because of a bug in the Android Gradle Plugin that's been fixed and will be available in the upcoming 4.2 release.

@gnarea I believe this one can also be closed.
https://github.com/relaycorp/awala-endpoint-android/pull/274
Dependabot already must have made the upgrade, because on that PR it's using the latest version

Thanks for looking into this, @Filmaluco!
Dependabot can't manage JaCoCo because it isn't managed as an actual Gradle dependency. See jacoco.gradle.
However, I just checked and you already upgraded it to the latest version in #235, so I'm closing this issue.


Flyway Integration into subproject adapter-persistence-jpa

Flyway Integration int subproject adapter-persistence-jpa:

Configured Root build.gradle for flyway
Added flyway into the subproject adapter-persistence-jpa
Configured build.gradle into adapter-persistence-jpa for flyway task
Offloaded hibernate schema creation to flyway, keeping hibernate only for schema validation whereas flyway for schema migration(naming convention: V1__.sql)
Updated build.gradle in adapter-persistence-jpa so that test depends on flyway migration first.
Updated ReadMe File for flyway operation.

Thanks @eamtalu!


Fix age-groups exceeded Transferlimit

Fix age-groups exceeded Transferlimit as described in #461
Fix #461

Hi Marlon, this is realy a bug, the response also contains the districts data which are not filtered out! as a result, the data output is a complete mismatch!


Put M1 incompatibility in the readme please

Saw the other issue about lack of M1 compatibility.  Saying "you're screwed due to your hardware choice" is becoming less of an issue as intel will slowly be phased out of Apple world AND when we get new macs in the corp environments they are shipping M1s (and now M2s)..  So if we need to avoid your package, that's fine.  At least give us a heads up to avoid you right on the front page.  Please and thank you.

Again someone who thinks open source is a free wish place where you can demand everything without giving anything back. If you need the support again: Push a PR for it. I wont do it for you, I don't have a Mac and will never have. So if you wish for that feature, do it yourself, its that easy and will be accepted as a PR if it fulfills the requirements and doesn't break anything.

And again, just b/c apple arrogantly states: "Well developers will have to adjust after us.", it doesn't all will, actually most wont. You opted for special hardware, then you will have to do your fair share to get it supported, not the other way around.

Asking for accurate documentation isn't the same as asking for a feature, or a bug fix...  Also, get over yourself...  This is why you're still <v1.0, this is why no one is sponsoring you..  This is why many will avoid your library, and ultimately anything you touch..

you started the tone, you get what you seed. This library is mostly funded by the projects that use it. Again you want something, then do it yourself.  And neither will you get your special treatment. I will not state as well this is incompatible with eventually powerpc or other architectures as well.
You're not willing to open a PR and participate on your own to open source, fine, it will stay as it is, but that is on you if you decide to change it and make it compatible.
Bye, then, still invited to do the right thing and open a PR fixing the issue without breaking anything, otherwise have a nice day.


Production Ready?

A few questions:

Is swarm production ready?
There appears to be no actual production ready Storages (MongoDB, Mysql, Postgres?)
How hard is it to make custom a Storage?
I don't see a clear strategy for query or collection syncing (Some adds a new item, removes one, etc from a query)

Congratulations, your issue is number 1000000 (binary).
Here are my (likely incomplete) answers:

No. Version 0.3 is abandoned due to some difficulties (the math was not good enough to make a production ready system). The upcoming release is very unstable (and backwards incompatible as it uses new awesome math).
2.The upcoming release works on top of LevelUp (practically, leveldb or IndexedDB). It may flush data to whatever other DB, theoretically.
If we talk about DB integration (not storage engine) then the answer is "easy". Unfortunately, "easy" integration works one way, i.e. Swarm to some DB (Kafka-like "db as a view" situation).
There are various CRDT collection types. 0.3 has Vector and Set. Queries are possible using the aforementioned DB integration (API published queries, not arbitrary queries from the client).

Thank you so much for your prompt and complete-enough answers! :+1:


Hire a Workshop Coordinator

Issue to discuss hiring a workshop coordinator (PR #286). Budget is 12k, funded from the Moore grant (please submit invoices here). Project Description.

This budget was increased from the #310 budget to 18k. And a hire has been made. Introductory announcements will go out soon.

This project was completed. :/


.composer folder not created fix

Make sure that .composer folder exist.

Why would it not be created after installing Composer?

For some reason now composer is uses .config/composer/ instead of .composer, I didn't looked it up but in my machine i use .config/composer/.

So what does this addition actually fix? Sorry, trying to understand. If Composer uses a new directory why do we create this one?

well you may don't have this issue on OSX but with ubuntu the default home for composer is .config/composer, and i used to have .composer as COMPOSER_HOME dir and most php guys still do.

I did these steps and was not able to reproduce this.

vagrant box add bento/ubuntu-14.04
Opened virtualbox and imported the box.
sudo apt-get update
sudo apt-get install curl php5-cli
curl -sS https://getcomposer.org/installer | php

Tried with php7.0-cli too.

How are you getting ${COMPOSER_HOME} to be ${HOME}/.config/.composer?

Yes, you are right with this way installing composer, but did you try to use the provision.sh?
In other words try to create a vagrant machine using the current provisioning scripts
here is the output of the console when it is downloading and installing composer:

==> default: All settings correct for using Composer
==> default: Downloading...
==> default:
==> default: Composer successfully installed to: /home/vagrant/composer.phar
==> default:
==> default: Use it: php composer.phar
==> default:
==> default: PATH="/home/vagrant/.composer/vendor/bin:$PATH"
==> default: Changed current directory to /home/vagrant/.config/composer

and after installing ssh to the new machine

and ls to your .config

Don't know if this worth checking or not but i saw a lot of my friends who are using homestead are facing a problem that their composer global are not working, but the thing they don't know that the home_dir path is set to .composer in .profile instead of .config/composer and thats why it seems to be broken to them.

what i think now is changing the line from
printf "\nPATH=\"/home/vagrant/.composer/vendor/bin:\$PATH\"\n" | tee -a /home/vagrant/.profile
to
printf "\nPATH=\"/home/vagrant/.config/composer/vendor/bin:\$PATH\"\n" | tee -a /home/vagrant/.profile
this is better i think !!

Hmm you're right. How about this:

printf "\nPATH=\"$(composer config -g home 2>/dev/null)/vendor/bin:\$PATH\"\n" | tee -a /home/vagrant/.profile

I just had this issue, where i had to change the path in the ~/.profile

@alexbowers in your .profile find the line
PATH="/home/vagrant/.composer/vendor/bin:$PATH"
replace it with
PATH="/home/vagrant/.config/composer/vendor/bin:$PATH"

Yeah i've fixed, just pointing out that it is an issue and needs merging in :)

Looks good to me.


feature request to add/tag clustername/id to rancher server logs

Rancher version:
v2.2.3
Type:
Enhancement
Description:
Currently in a HA configuration with 3 application servers (min), rancher server pod on any one of the node can handle the client request and the logs are written as expected based on how the pod handles the request.
in troubleshooting scenarios, it would be good if each of the logs are 'tagged' with the cluster name or cluster-id to better distinguish the logs, if multiple requests are being handled for the same type, the logs type and data for the same type of log will be the same across various clusters and extremely hard to distinguish which cluster a particular log belongs to.
for this example snippet, its very hard which particular node/pod these messages are related to?
2019/07/18 20:29:53 [INFO] stdout: Waiting for machine to be running, this may take a few minutes...
2019/07/18 20:29:53 [INFO] Waiting for machine to be running, this may take a few minutes...
prefixing the clustername/cluster ID will add value to the logs.
gz#5187
SURE-1033

During validation of this ticket on v2.7-head (Commit ID: 914f0f4), I was able to verify that when checking the logs for the "leader" node on an HA Rancher server, all downstream RKE1 cluster logs that are captured in the "leader" node for the local cluster will be prefaced by the Mgmt Cluster name for the downstream cluster as well as the node name for the downstream cluster's specific node (when applicable). I was also able to verify that modifying the loglevel to DEBUG or TRACE also works as intended. Therefore I am closing this ticket.


Kryo5Codec putAll() Number format value issue

Expected behavior
When the putAll() method is called, the numeric values should be correctly stored in the hash set, and when the addAndGet() method is called, each numeric value should accumulate correctly.
Actual behavior
When the putAll() method is called, the numeric values are stored as "," and when the addAndGet() method is called, an error occurs with the message "ERR hash value is not a float."
Steps to reproduce or test case
Map<Object, Object> countCacheMap = new HashMap<>();
countCacheMap .put("test", 1);
countCacheMap .put("test2", 1);

redissonClient.getMap(key).putAll(countCacheMap);

redissonClient.getMap(key).addAndGet("test", 1);

Redis version
5.0.7
Redisson version
3.21.1
Redisson configuration
           Config config = new Config();

           String redisAddress = "redis://<IP_ADDRESS>:52533";
           String redisMasterName = "test";

           String[] sentinelHosts = getSentinelAddresses(redisAddress);

           config.useSentinelServers()
                   .setMasterName(redisMasterName)
                   .addSentinelAddress(sentinelHosts)
                   .setReadMode(ReadMode.MASTER)
                   .setSubscriptionMode(SubscriptionMode.SLAVE)
                   .setCheckSentinelsList(false);

           RedissonClient client = Redisson.create(config);

This method uses HINCRBYFLOAT Redis command which supports only plain values. It works with JsonJacksonCodec, StringCodec or any NumberCodec.


Google 'Empty' not generated with a namespace

I've got a protobuf which returns an Empty e.g.
syntax = "proto3";
import 'google/protobuf/empty.proto';

service Subscription {
  rpc Subscribe(Subscriptions) returns (google.protobuf.Empty);
}

However, the generated code:
   class SubscriptionService < Twirp::Service
     rpc :Subscribe, Subscriptions, Empty, :ruby_method => :subscribe
   end

Whereas it should be:
   class SubscriptionService < Twirp::Service
     rpc :Subscribe, Subscriptions, Google::Protobuf::Empty, :ruby_method => :subscribe
   end

I'm calling the generator using:
grpc_tools_ruby_protoc --ruby_out=app/rpc --twirp_ruby_out=app/rpc app/proto/*.proto

root@11768bc0a972:/# grpc_tools_ruby_protoc --version
libprotoc 3.5.1

Any ideas what I'm doing wrong?

Nothing is wrong in your definitions. The code generation simply does not support importing other message types yet. I'll see if I can add support to it.


Updated key_management.md to fix #47

Proposal for fixing #47. Also added normative reference for IETF RFC 4107 / BCP 107.

This has two approvals and its a rather small wording change, so I am going to merge it.


Make RSS feeds discoverable

To allow folks to subscribe to this site using their feed readers, we
can add the rel=alternate link to the page metadata for the home,
ideas and subscribe pages.

TIL! ty


N_VRAD for VAD

The VAD of Payerne has 12 beams and is done every 30 min. I use averaging of 60 min, so two scans. To get any output, I need to set N_VRAD to 4. This seems very low. I would expect, that a value of 8 beams should be ok. Can you check, what is going on there?

Same happens with the new data set from Feb 9th. How is N_VRAD used for VAD?

N_VRAD is the minimal number of available line-of-sight directions that are used the retrieval. As an example, if you measure with a VAD with 12 fixed azimuth direction, dividing the circle into 30 degree steps, you need at least 3 directions to perform the retrieval. But the results are much more reliable, if you choose a bigger number, like 6, i.e. half of the maximal number of available directions. Before the retrieval is performed, the data is filtered for each of the fixed directions, but taking into accout the all the radial velocities matching theses directions. If I set the averaging time to 30 min and the instruments did 10 whole sweeps in this time window, the algorithm will choose the most likely radial velocities for each direction among 10 measurements. If some directions end-up without a valid radial velocity, the number of available directions is reduced by that amount and if this number turns out to be less than the specified N_VRAD threshold, the output will be NaN.

Thank you for clarifying.


Sharepoint files: Invalid URI: The hostname could not be parsed

Issue type bug

Azure Data Studio Version: 1.15.1 (user setup) / Windows 10

Steps to Reproduce:

Open T-SQL file created in SSMS (.sql filetype now associated with Azure Data Studio)
Click on Connect in the query window and select connection
Run query:

Execution failed due to an unexpected error:
Invalid URI: The hostname could not be parsed.

Note that opening a new query and pasting the same code in runs without error.
The error is not dissimilar to #7201 but the error message and potentially wider than initially thought(?)

I can't personally repro but from the error message it looks like it is in our backend service. Could you upload the logs from that session? {USER_FOLDER}\AppData\Roaming\azuredatastudio\logs\{DATE}\exthost1\Microsoft.mssql\sqltools.log

@sassora are you also using postgres like is being reported in https://github.com/microsoft/azuredatastudio/issues/7201?

@anthonydresser, I open Azure Data Studio and then a SQL file with the open dialog and ran "SELECT 1 as a" which brought up the connections to select from. Here is the log folder 20200221T054111.zip I found no files in the Microsoft.mssql folder.
@kburtram I'm using T-SQL, the SELECT statement I'd previously saved in SSMS produced the same error message so I simplified the code right down to "SELECT 1 as a" and had the same result.
I am going to trying creating the sql file within Azure Data Studio, save and reopen just in case it doesn't like what SSMS did for some reason.

It looks like the cause might be SharePoint, when I store the SQL file on the desktop it's absolutely fine.  SharePoint tends to break things so this may be expected. I am planning to move code from here into windows folder (and use git). Of course if there is a fix for working with SharePoint I'm sure it would be appreciated. Thanks!

Thanks, for the update. We will certainly continue to work to see if this is a problem we can solve.

I am also experiencing this, using a file hosted on WSL (not SharePoint). I don't know how SharePoint paths are constructed, but could it be that Azure Data Studio is having trouble with the $ in the path?

I just had the same issue with a file hosted on WSL2. It goes away if you map a network drive to WSL2 and use that from within ADS e.g. Z:\home\myUser\bin\myRepo

same problem here using WSL...
Is there any plan to make WSL integration on Azure Data Studio?

Npt sure if this is just adding more nosie, but I'm also experiencing this, and my configuration is:

WSL2/Ubuntu
Projects folder, and my sql project is underneath, named 'SBDReporting'
this folder is also a git repository, hosted in Azure DevOps
SQL files in this folder were saved locally by right clicking a view in Azure Data Studio, and selecting Script as Create, then saving locally
Opening this file, connected to the database, then clicking execute gives me the error:
Execution failed due to an unexpected error:    Invalid URI: The hostname could not be parsed.


Create Annual Emissions State Aggregation Endpoints

Definition of Ready

[x] Does this meet the INVEST criteria?
[x] Does this have a clearly defined user?
[x] Does this have an estimate?
[x] Does this have acceptance criteria?
[x] Does this have sufficient context for the specific functionality being developed (design, business rules, etc.)
[x] Does this have a dev link been provided in the design?
[x] Is this traceable to a feature, epic, or PI?

User Story
As a CAMPD user I want to aggregate monthly emissions data at the facility level  So that I can conduct analysis
Acceptance Criteria
Given that a user has entered one or more valid values for all the parameters (see parameters below)
When API endpoint is executed
Then the system displays the records that match the criteria (containing data elements below)
Given that a user has entered one or more valid values for all the parameters in an upper case or lower case format
When API endpoint is executed
Then the system displays the records that match the criteria
Given that a user has entered one or more valid values for all the parameters without white space(s) before or after the delimiter
When API endpoint is executed
Then the system displays the records that match the criteria
Given that a user has entered one or more valid values for all the parameters with white space(s) before or after the delimiter
When API endpoint is executed
Then the system displays the records that match the criteria
Given that a user has made a GET request with the media type set to json or csv
When request is executed
Then the system displays the records that match the criteria in the appropriate format (json or csv)
Given that a user has entered one or more invalid values for a parameter
When the API endpoint is executed
Then the system displays the appropriate error message (see error message repo)
Development Notes
The scope of this ticket includes creating the following endpoints:
/emissions-mgmt/apportioned/annual/state
Out of scope:
/emissions-mgmt/apportioned/annual/state/stream

Ensure data labels and descriptions are in alignment with the Master data dictionary
Ensure that you can query multiple years on swagger
address issue with querying multiple years with the annual facility endpoints

Does this Story require 508 Testing?

[ ] Implementing a new authoring tool that will produce code / content for the application
[ ] Implementing a 3rd party widget that will be part of the application
[ ] Adding new / updating custom functionality
[ ] Adding new / updating a page template
[ ] Adding new / replacing existing functionality (unless the replacement has already passed 508 testing)
[ ] Full Site Review prior to a scheduled EPA 508 review
[ ] Planning a video that will appear on the application
[ ] Producing a video that will appear on the application
[ ] Producing Training materials
[ ] Producing documents

Context
Error message repo

CAMPD Master Data Dictionary
Parameters
API Parameters:
Year: Required, multiple values allowed, Format: YYYY, Range: 1980, 1985, 1990, 1995-latest reporting quarter, Allow indiv. Years, ranges, and comma sep. list
Facility ID (ORIS code): Optional, multiple values allowed
State: Optional, multiple values allowed, 2 letter postal abbreviation for the 50 states plus Puerto Rico.
Unit Type: Optional, multiple values allowed
Unit Fuel Type: Optional, multiple values allowed, query will filter on both primary and secondary fuel type
Control Technologies: Optional, multiple values allowed
Program: Optional, multiple values allowed, excludes MATS
Validations
Year:
Validate that at least one value exists for the parameter
Validate that the year is in the YYYY format
Validate that the year is 1980, 1985, 1990 or greater than or equal to 1995
Error Message 1: should not be null or undefined FV-106
Error Message 2: One or more < insert property > are not in the < insert format >. Ensure all are in the < insert format > FV-102
Error Message 3: Update one or more < insert property > to < insert valid range > FV-105
State:
Validate that the value matches a valid 2 letter postal abbreviation for at least 1 of the 50 states, PR, or DC
Error Message: One or more states codes are not valid. Use the two letter postal abbreviation (use TX, not Texas). FV-101
Facility ID (ORIS Code):
Validate that the value is a number
If the value is a number, validate that the format is correct
positive integer (no negative numbers, not 0)
whole numeric value (no letters or special characters)
no more than 6 digits
Error Message: One or more < insert property > are not valid. Refer to the list of available < insert property > for valid values [placeholder for link to endpoint] FV-100
Unit Type:
Validate that the value is a valid unit type.
Error Message: One or more < insert property > are not valid. Refer to the list of available < insert property > for valid values [placeholder for link to endpoint] FV-100
Unit Fuel Type:
Validate that the value is a valid fuel type.
Error Message: One or more < insert property > are not valid. Refer to the list of available < insert property > for valid values [placeholder for link to endpoint] FV-100
Control Technologies:
Validate that the value is a valid control technology.
Error Message: One or more < insert property > are not valid. Refer to the list of available < insert property > for valid values [placeholder for link to endpoint] FV-100
Program:
Validate that the value matches a valid program code
Error Message: One or more < insert property > are not valid. Refer to the list of available < insert property > for valid values [placeholder for link to endpoint] FV-100
Data Elements:
State Aggregation
When the system identifies at least one record that matches the criteria, then for each record the system will display the data below and ordered by state> Year
stateCode
year
grossLoad
steamLoad
so2Mass
co2Mass
noxMass
heatInput
Testing
Test Case: TC704
Tester: @JanellC
Definition of Done

[x] Have all Acceptance Criteria been agreed to and validated by the scrum team?
[x] Has the code been unit tested and peer reviewed?
[x] Have the functional tests been executed?
[x] Have all defects been reviewed and dispositioned, resolved, or deferred?
[x] Is the user story ready to be deployed to the test/staging environments?

TC704 Passed @tlageman-epa
https://api.epa.gov/easey/dev/emissions-mgmt/apportioned/annual/by-state/?api_key=T8r7OW4f12XtWKLY7CPgKmKu1WoP3TVG0x4eqmlB&year=2019&page=1&perPage=100
https://api.epa.gov/easey/dev/emissions-mgmt/apportioned/annual/by-state/?api_key=T8r7OW4f12XtWKLY7CPgKmKu1WoP3TVG0x4eqmlB&facilityId=0000003   |   000612 &unitType=invalid &unitFuelType=invalid&controlTechnologies=invalid&programCodeInfo=mats


fix: handle multiple locales in relationship population

Description
#1444

[ ] I have read and understand the CONTRIBUTING.md document in this repository

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist:

[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Existing test suite passes locally with my changes
[ ] I have made corresponding changes to the documentation

This handles the originally reported issue, but it does not also handle a relationship field with hasMany: true being read with ?locale=all


A watch script

Wrote a watch bash script.
Hope it will be helpful to somebody ❤
Note that this is kind of CPU eager.
watch.sh
#!/bin/bash

# ANSI color codes
RED='\033[0;31m'
NC='\033[0m' # No Color

while true
do
    # Get the current time
    TIME=$(date +"%Y-%m-%d %H:%M:%S")

    # Execute shiroa build
    if shiroa build >/dev/null 2>&1; then
        echo "[shiroa watch @ ($TIME)] - Build Successful"
    else
        echo -e "${RED}[shiroa watch @ ($TIME)] - Build Failed${NC}"
    fi

    # Wait 1 seconds before the next iteration
    sleep 1
done

I guess you should write and preview your book document by existing editor integrations instead of relying on some shiroa watch. Please check https://github.com/Myriad-Dreamin/shiroa?tab=readme-ov-file#setup-for-writing-your-book.


hiredis hangs on redisBufferRead

We use hiredis to connect to redis server in block mode. Our application runs in RHEL 4 32 bit. It works fine and after some time while trying to do an operation, it hangs. On analysis, found it got hanged in the redisBufferRead. This application act like a switch which gets the data from TCP Socket and store the same in the Redis database for recoverability.
On netstat output, We found the SendQ of the our client has non-zero value which indicates that  request has not reached redis server. This behaviour happens with INCR & INCRBY mostly.
We opened a redis-client and did the same operation and it works fine.
gdb output
#0  0x008787a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x001de69b in __read_nocancel () from /lib/tls/libpthread.so.0
#2  0x08073a42 in redisBufferRead (c=0x89c2fb8) at hiredis.c:1142
#3  0x08073c59 in redisGetReply (c=0x89c2fb8, reply=0x590dcbc) at hiredis.c:1228
#4  0x08073e99 in redisvCommand (c=0x89c2fb8, format=0x8077f40 "INCRBY %s %ld", ap=0x590dcf8 "t\021\225\b\001") at hiredis.c:1332
#5  0x08073ed2 in redisCommand (c=0x89c2fb8, format=0x8077f40 "INCRBY %s %ld") at hiredis.c:1349

I met the same issue in 2012 and 2013.  And in these days I met them again.  Maybe it's due to the old lib, but why this issue is still keeping open status.
enviroment:

Linux tesla001 2.6.32-504.12.2.el6.x86_64 #1 SMP Wed Mar 11 22:03:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux.
libhiredis.so.0.10 (yum updated from default centos 6 repo)

thread stacks:
[Switching to thread 3 (Thread 0x7fdbee57b700 (LWP 7795))]#0  0x000000348c40e75d in read () fro                                                                                m /lib64/libpthread.so.0
(gdb) bt
#0  0x000000348c40e75d in read () from /lib64/libpthread.so.0
#1  0x00007fdbeeddd7bb in redisBufferRead () from /usr/lib64/libhiredis.so.0.10
#2  0x00007fdbeeddefcf in redisGetReply () from /usr/lib64/libhiredis.so.0.10
#3  0x00007fdbeeddf085 in redisvCommand () from /usr/lib64/libhiredis.so.0.10
#4  0x00007fdbeeddf118 in redisCommand () from /usr/lib64/libhiredis.so.0.10
#5  0x0000000000405843 in CRedisOperator::handleGeneratorInfo (this=0x7e5a30, ownerId=3, data=0                                                                                x7e59b8) at src/database/CRedisOperator.cpp:146
#6  0x00000000004038a2 in CTransaction::onGeneratorInfo (this=0x7e4d40, sequence=5112, data=0x7                                                                                e59b8) at src/transaction/CTransaction.cpp:161
#7  0x0000000000403367 in CTransaction::onMessage (this=0x7e4d40, msg=0x7e59a8) at src/transact                                                                                ion/CTransaction.cpp:58
#8  0x000000000040c802 in CNodeGroup::working (this=0x7e5858) at src/traffic/CNodeGroup.cpp:70
#9  0x000000000040b5b2 in CWorker::run (object=0x7e5860) at src/common/CWorker.cpp:68
#10 0x000000348c4079d1 in start_thread () from /lib64/libpthread.so.0
#11 0x000000348c0e88fd in clone () from /lib64/libc.so.6

No one investigated this in more detail yet, but as per the comments above this might be an issue with the network or the Redis contacting and not a bug in hiredis itself.

I used timeout on the connection, it's 1 second.  But I think the timeout only works with connect, not for redisCommand().
Yes, it's a blocking command.
Actually the redis-server and the application using hiredis are in the same machine.  netstat outputs related to redis-server below:

[rafael@tesla001 softwares]$ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 localhost:38836             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38839             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38838             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38842             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38845             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38852             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38854             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38851             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38847             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38849             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38837             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38848             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38840             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38841             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38844             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38843             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38855             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38850             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38853             localhost:6379              TIME_WAIT

So all connections to the Redis server are already half-closed, which might indicate a completely different problem.

I will try async APIs.

sorry bad boy, I run net status again, it looks I delete some established connections.  actually there are active connections. paste them all:
[rafael@tesla001 softwares]$ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 localhost:38836             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38839             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38838             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38842             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38845             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38852             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38854             localhost:6379              TIME_WAIT
tcp        0      0 <IP_ADDRESS>:10506         <IP_ADDRESS>:10174          ESTABLISHED
tcp        0      0 localhost:38851             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38847             localhost:6379              TIME_WAIT
tcp        0      0 localhost:6379              localhost:37590             ESTABLISHED
tcp        0      0 <IP_ADDRESS>:ssh           <IP_ADDRESS>:28720         ESTABLISHED
tcp        0      0 <IP_ADDRESS>:ssh           <IP_ADDRESS>:9326           ESTABLISHED
tcp        0      0 localhost:38849             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38837             localhost:6379              TIME_WAIT
tcp        0      0 localhost:35097             localhost:6379              ESTABLISHED
tcp        0      0 localhost:38848             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38840             localhost:6379              TIME_WAIT
tcp        0      0 localhost:6379              localhost:35097             ESTABLISHED
tcp        0      0 localhost:38841             localhost:6379              TIME_WAIT
tcp        0      0 localhost:37590             localhost:6379              ESTABLISHED
tcp        0      0 localhost:38844             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38843             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38855             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38850             localhost:6379              TIME_WAIT
tcp        0      0 localhost:38853             localhost:6379              TIME_WAIT
tcp        0      0 <IP_ADDRESS>:53500         <IP_ADDRESS>:http          ESTABLISHED
tcp        0      0 localhost:38846             localhost:6379              TIME_WAIT
tcp        0      0 <IP_ADDRESS>:ssh           <IP_ADDRESS>:10273          ESTABLISHED

your netstat output shows there is no none-zero sendq.
btw. you can use redisSetTimeout() to set read/write timeout value.

@echoma Thanks.
Actually I used redisSetTimeout() and  I found the reason finally.  I used multi-threads with the api, but it doesn't support.

So this can be closed?

@badboy Yes, I have solved the problem by move all api calls in the same thread.  But @saravanan-elangovan opened this issue, I am not sure he met the same situation like me.

shoutrain commented on 4 May 2015
@echoma Thanks.
Actually I used redisSetTimeout() and I found the reason finally. I used multi-threads with the api, but it doesn't support.
Question：hiredis doesn't support multi-threads?

shoutrain commented on 4 May 2015
@echoma Thanks.
Actually I used redisSetTimeout() and I found the reason finally. I used multi-threads with the api, but it doesn't support.
Question：hiredis doesn't support multi-threads? where i can see it? I used multi-threads also!

@abigbigbird Can see what?
I added a short note a while back, see the note under Connecting.
The context object has no synchronization of its internals, using it from multiple threads at once will most likely result in race conditions.

Oh,I saw the note !  thanks
I use hiredis in my project,It works fine in most of the time，but there are several times ，it hangs。
## the gdb core stack as below:：
#0  redisBufferRead (c=0x7f26c4013a10) at hiredis.c:800
#1  0x00007f27479e9cd0 in redisGetReply (c=0x7f26c4013a10, reply=0x7f26ea4e7ea0) at hiredis.c:903
#2  0x00007f27479ea02b in __redisBlockForReply (c=0x7f26c4013a10, argc=, argv=,
argvlen=) at hiredis.c:1010
#3  redisCommandArgv (c=0x7f26c4013a10, argc=, argv=, argvlen=)
at hiredis.c:1035
#4  0x00007f27479d31cc in bfd::codis::CodisClient::RedisCommand (this=0x200ceb0, command=..., tt=0) at src/CodisClient.cpp:995
#5  0x00007f27479d3662 in bfd::codis::CodisClient::RedisCommand (this=0x200ceb0, command=..., tt=0) at src/CodisClient.cpp:1035
#6  0x00007f27479c9f64 in bfd::codis::CodisClient::get (this=0x200ceb0, key=..., tt=0) at src/CodisClient.cpp:152
#7  0x00007f27479e5a18 in bfd::codis::BfdCodis::get (this=0x2130120, key=..., tt=0) at src/BfdCodis.cpp:60
#8  0x000000000050d89e in RedisClientProxy::get (this=0x20e34f0, key=...)
(gdb) p c
$1 = {err = 0, errstr = '\000' <repeats 127 times>, fd = 72, flags = 3, obuf = 0x7f26b40087c8 "", reader = 0x7f26c4013af0,
connection_type = REDIS_CONN_TCP, timeout = 0x0, tcp = {host = 0x7f26c4013d50 "192.168.*", source_addr = 0x0, port = 19000},
unix_sock = {path = 0x0}}
(gdb) p *buf
Cannot access memory at address 0x7f26ea4e3d80
os：Red Hat Enterprise Linux Server release 6.4 (Santiago)
gcc version:4.6.3
hiredis version:0.13.3

@abigbigbird Is this the same as #442? If so, please re-open there. This issue here seems to be another problem.

oh，sorry，i saw it core at the same code。my fault！

I am running into a similar backtrace on gdb. I am using hiredis for mainly get/set queries. It works fine most of the times but once in a while it hangs on RedisBufferRead().
I am using a connection pool which has Redis Contexts being created and given to threads as and when required. Could this be an issue?

The same issue here:
(gdb) bt
#0  0x00007f84e681a51d in read () at ../sysdeps/unix/syscall-template.S:84
#1  0x00007f84e5432f8e in redisBufferRead () from /usr/lib/x86_64-linux-gnu/libhiredis.so.0.13
#2  0x00007f84e5433280 in redisGetReply () from /usr/lib/x86_64-linux-gnu/libhiredis.so.0.13
#3  0x00007f84e54332b5 in ?? () from /usr/lib/x86_64-linux-gnu/libhiredis.so.0.13
#4  0x00007f84e5433634 in redisCommand () from /usr/lib/x86_64-linux-gnu/libhiredis.so.0.13
#5  0x00000000006bd86f in DisqueClient::AddJob (this=0x7f84dc0008c0, queueName="umbrella_api", job="device_connected:862670225013751", retry=5, ttl=2592000) at /root/device_manager/src/managerd/DisqueClient.cpp:17
#6  0x0000000000666c79 in WorldSocket::HandleAuth (this=0x7f84c0112f40, recvPacket=...) at /root/device_manager/src/managerd/WorldSocket.cpp:235

The server is not redis, but similar as redis: https://github.com/antirez/disque

same issue here if i run it in a multi- threads

sometimes we are even unaware of threading usage, like being in an callback routine, why dont we introduce a mutex inside redisCommand?  I believe there are many products in the market shipped with this bug and the authors have not noticed

Making hiredis thread safe would require a lot more than just adding a mutex to redisCommand.  Besides, if we added thread safety to hiredis proper it would reduce flexibility.
The solution, I think, is to handle synchronization in each individual application in the best way for that application.  There are also thread-safe wrappers to hiredis available but I have not used any of them personally.

Closing this old issue, since it's not clear there is actually a bug being described.


Assert keyword

Hi there !
I want to use "assert" keyword for write strong programs.
But, I've a problem: when the assertion state is false, exception isn't raised.
For example:
void main()
{
    String myString = null;
    assert (myString != null); //nothing
}

Have you same problem ?
Thanks for your help.

Are you running in --checked mode?

For native Dart program, it works, but when I compile dart program, it don't work.
I make an error ?

How do you compile the program?

Hi,
Like this:
dart2js -o mytest.dart.js mytest.dart

assert() is only executed in checked mode (development only). For dart2js you need to pass -c See also https://www.dartlang.org/tools/dart2js/

Ohw, ok. I didn't know.
Thanks for your help ! :)


modal present will change url hash

somecase when modal present will change url hash
is that a bug? how to fix it?

Hello! While we appreciate you opening an issue with us i am going to close this issue and ask that you open a new one that has the issue template filled out please. Thanks for using Ionic!


Allow setting MCELOG_MAX_CORRECTED_RATE, default to 100/24h

An alternative way would be to create a file /etc/sysconfig/nhc which nhc sources if it exists, and then set the MCELOG_MAX_CORRECTED_RATE env var there.
Either way is fine by me really, I just chose modifying nhc.conf here since we generate it from a template anyway.

Thanks! Making a new release after updating some testing things.

New version added in fgci-ansible in https://github.com/CSC-IT-Center-for-Science/fgci-ansible/commit/f4de6d8b1b8821d239244bfb38abda98e39077e7


detect lacking setRawMode support on stdin

related to https://github.com/zeit/now-cli/issues/24

Thanks everyone for the awesome discussion and fix


Any plans to turn this into a module?

Are there any plans to turn this into an external module? If not, I have resources I could apply to the task.

Yes Phillip. This is most definitely on the list too.  I'm hoping Jan will be quieter around here and I will have some time to work on a few things like this that are on the list. It may well be useful to have some help with this one.

Hi, are there any updates about turning this into a module? Thanks

Hi, are there any updates about turning this into a module? Thanks

We have our own variant of this module in production at the University of Florida. We're dealing with a nasty bug at the moment, but expect to fix it soon. Our work is at https://github.com/ctsit/record_autonumbering. @lsgs, if you don't want to publish this, we'd be happy to.

Hello Philip,
I hope all is well with you. Which do you mean? lsgs/redcap-longitudinal-reports or lsgs/redcap-record-autonumbering?
The answer is yes to both. Do you think the autonumbering one is complete enough? I would like to solve the problem with it with records created by the “Randomize” process (I have another idea to investigate for that at some point), and there is also the question of how to propagate any custom classes people have into new versions.
The longitudinal reports thing is very high on my list and I’ve made some progress on it. I’m basically integrating a “rehaping” option into the built-in reports page...
Regards,
Luke
From: Philip @.>
Sent: Wednesday, 3 March 2021 07:13
To: @.>
Cc: @.>; @.>
Subject: Re: [lsgs/redcap-longitudinal-reports] Any plans to turn this into a module? (#2)
Hi, are there any updates about turning this into a module? Thanks
We have our own variant of this module in production at the University of Florida. We're dealing with a nasty bug at the moment, but expect to fix it soon. Our work is at https://github.com/ctsit/record_autonumberinghttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fctsit%2Frecord_autonumbering&data=04|01||602e15a8eefe4bf5edfe08d8ddb7995e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637503127904454646|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=jYiP2mPonD8QX6Z3xV6brlnfyq9JDDhusJynepZ%2FA2I%3D&reserved=0. @lsgshttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flsgs&data=04|01||602e15a8eefe4bf5edfe08d8ddb7995e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637503127904464644|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=2FM9zcVUlXEjMoMFgZULeVN%2FZUK%2FUyT6MrqoazHdt50%3D&reserved=0, if you don't want to publish this, we'd be happy to.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flsgs%2Fredcap-longitudinal-reports%2Fissues%2F2%23issuecomment-789182763&data=04|01||602e15a8eefe4bf5edfe08d8ddb7995e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637503127904464644|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=WMfvjnwiT5Lxf21XGEEz1ZrbK7MYwYsJHUpZEuHWs%2F8%3D&reserved=0, or unsubscribehttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABYZPSSMBBM67NFLQV55IHLTBVBFJANCNFSM4EHCEVAQ&data=04|01||602e15a8eefe4bf5edfe08d8ddb7995e|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637503127904474634|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=Ygx2L9Rgu713FDSupsPZGnC%2B6HeGV0r02qyyZGESGso%3D&reserved=0.

I hope all is well with you. Which do you mean? lsgs/redcap-longitudinal-reports or lsgs/redcap-record-autonumbering?

I am commenting on the wrong module. Sorry about the confusion. I'll send you an email privately about our thoughts on record_autonumbering

Hi, just wondering if there's been any progress on turning this into an external module? Thank you

@pbchase, @skhalliday, @bfulcher-monash: https://github.com/lsgs/redcap-extended-reports/ is now released in the REDCap Repo


Not true CSS-styles declaration order

Styles do not apply because they are in front of 'MuiButtonBase' where default styles

if I manually remove these styles, then the buttons will appear as expected

Code of Buttons:
import {Button} from "@material-ui/core";
...
<Button variant="outlined"     color="secondary"      onClick={() => routerHistory.push(this.props.AppStore.modules.newCreateInterface ? "/issue/create/new" : "/issue/create")}>         Добавить </Button>
Package versions:
"@material-ui/core": "^4.9.4",
"@material-ui/icons": "^4.9.1",
"@material-ui/pickers": "^3.2.10",
"react": "^16.12.0",
"webpack": "^4.41.6",
"webpack-cli": "^3.3.11",

Please provide a minimal reproduction test case. This would help a lot 👷 .
A live example would be perfect. This codesandbox.io template may be a good starting point. Thank you!


fix(checkout): CHECKOUT-6969 Display cart summary at bottom

What?
Revert previous PR https://github.com/bigcommerce/checkout-js/pull/1036
Not scroll any more, display cart summary at bottom.
Why?
By default, modern browsers prevent cross-domain iframes from accessing anything in the parent window.
The previous PR is unable to work in a cross domain scenario, which is the most common use case for the embedded checkout.
As a result, this PR provides a pure CSS solution to the problem that a shopper cannot see the cart summary when clicking on the cart button.
Testing / Proof
Android - Chrome
https://user-images.githubusercontent.com/88361607/192906250-e95d8b48-2015-4ff8-a6e3-e010e4159a60.MP4
iOS - Safari
https://user-images.githubusercontent.com/88361607/192906254-eb203e66-109b-4ba6-ac61-d6d87c562059.mov
iOS - Chrome
https://user-images.githubusercontent.com/88361607/192906258-a5ea1d78-27a2-4762-8ec5-758841ecf846.mov
mac - Safari

This fix is good, but we need to look at the white space at top for safari screens.

This fix is good, but we need to look at the white space at top for safari screens.

I tried to remove the white background and shadow of the container, then it looked like this.

However, I'm not sure if this will break anything. So I'll stick to my original plan and leave this option for the future.


Update custom prompt tutorial w latest OMP steps

Docs Build status updates of commit 3e2f1dd:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

TerminalDocs/images/ohmyposh-terminal-prompt.png
:white_check_mark:Succeeded
View

TerminalDocs/tutorials/custom-prompt-setup.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


No debug info on Darwin

macOS / Darwin builds store debug info separate from the main binary, in a .dSYM folder. However our caching system does not support more than one build output.
I think we should fix this by expanding the caching system, but that can probably wait until we upstream the rest of the CPU code.

What do you think contributing this feature in the upstream first?

I mean it's largely a CPU-specific problem since the caching only affects CPU kernels. But as my task description was saying, I think we can wait until we upstream the entire CPU backend first before working on this, I think it's pretty low-pri


adding cluster timeout config

This implements a configuration for setting timeouts for requests to the k8s cluster. GKE has been consistently outside the default timeout threshold for us, so we need a way to set .withRequestTimeout(pluginSettings.getClusterRequestTimeout()) on the Kubernetes Client.

Thanks for the PR - seems like a sensible change to me and relatively straightforward to validate. Will take a look soon.

Is it possible for you to allow changes to the PR from collaborators? Would like to make minor tweaks that are easier to "just make" than PR back-and-forth :-)

For sure! Thanks for taking a look so quickly. I added you as a collaborator for the forked repo.

Ahh, sorry, didn't mean full collaborator on the forked repo - that's a bit extreme. Normally on PRs there is a check-box to the right hand side which effectively grants specific access to the branch associated with the PR while the PR is open. This one:

I think it's not ticked in this case?

@chadlwilson It's weird, I'm just not seeing that option on this PR. I totally don't mind that if you just accept the invite to collaborate on the repo. If we do end up making further changes, we'll be putting them back in your hands for PR anyway :)

Ahh no worries, I'll merge and tweak on master right prior to release, since there's no major blocker here.

Apologies for the delay, finally got around to sanity testing and releasing this in https://github.com/gocd/kubernetes-elastic-agents/releases/tag/v3.9.1-501


Utility, Get Combinations userId e itemId, from indexes

In our case, we found it really useful for handling prediction cases by limiting to indexes, avoiding further code changes.
// Get combinations from index 0 to 10
$combinations = $recommender->getUserItemCombinations(0, 10);
$recommender->predict($combinations);

or
// Get all combinations
$combinations = $recommender->getUserItemCombinations();
$recommender->predict($combinations);

or
// Get combinations for user with ID 1, from index 0 to 10
$specificUserId = 1;
$combinations = $recommender->getUserItemCombinations(0, 10, $specificUserId);
$recommender->predict($combinations);

or
// Get combinations for item with ID 1, from index 0 to 10
$specificItemId = 1;
$combinations = $recommender->getUserItemCombinations(0, 10, null, $specificItemId);
$recommender->predict($combinations);

Hi @marcellov7, thanks for another PR! However, I don't really understand the use case. Can you share more context about how you're using it?

We found it very useful to generate combinations directly from the recommendation system, instead of using the database and then doing the fit, it is just a utility, we use it to generate combinations for a user and then make predictions for all items.

Thanks. I'm not sure this pattern is very common (I think most users will use the userRecs method), so it's not something I'd like to include.


DPDV-2608: Introduce alias s1query

Jira Link: https://sentinelone.atlassian.net/browse/DPDV-2608
🥅 Goal
Explain why is this change needed
🛠 Solution
Introduce alias - https://docs.splunk.com/Documentation/Splunk/9.1.0/Admin/Searchbnfconf#searchbnf.conf.example
🏫 Testing
How the changes were tested

Although the alias is being recommended, it cannot be used. :/

@martin-majlis-s1 see https://community.splunk.com/t5/Building-for-the-Splunk-Platform/Aliased-Search-Command-Results-In-quot-Unknown-Search-Command-my/m-p/131721 hope it helps

Thanks. :) I was planning to search for the cause. :)


DJIMarshal+Private.h: CFStringEncoding vs. NSStringEncoding

I've hit a few compile errors in DJIMarshal+Private.h when trying to build Djinni-generated Obj-C interface code on Linux with GNUstep (yeah, yeah.) GNUStep provides NSStringEncoding, but not CFStringEncoding, which is the return type of getWCharEncoding in DJIMarshal+Private.h, so the compiler takes issue with that.
However, looking at this code, I think it's slightly broken: getWCharEncoding() declares a return value of CFStringEncoding, but the implementation actually returns NSStringEncoding values. That return value is used in two places, both also in DJIMarshal+Private.h, on lines 157 and 165. The first calls CFStringConvertEncodingToNSStringEncoding() to "convert" the value (even though it's already actually an NSStringEncoding) and passes it to a message which expects an NSStringEncoding. The second (on line 165) just passes the return value directly to a message expecting an NSStringEncoding, casting it implicitly.
So, the use of CFStringEncoding here seems superfluous. It seems like getWCharEncoding() should simply return NSStringEncoding, and the call to CFStringConvertEncodingToNSStringEncoding() should be removed?
(Removing the references to CFStringEncoding also has the added bonus of unblocking building this code with GNUstep, as it happens...)

That sounds plausible, and I agree from code inspection that it appears the enum types are being mixed (and not triggering any compiler warnings - sigh).  @konovalov-aleks is the author of the wchar feature, so I'd be interested in his opinion here.  @jgus go ahead and send a PR with a proposed fix and I'll give it a look.


Wasm Error

Describe the bug
The following appear each time I run netlify-cli:
wasm streaming compile failed: TypeError: Failed to parse URL from /home/user/.local/share/npm/lib/node_modules/netlify-cli/node_modules/netlify-redirector/lib/redirects.wasm
falling back to ArrayBuffer instantiation

Steps to reproduce
Run netlify status.
Configuration
None.
Environment
System:
OS: Linux 5.17 Arch Linux
CPU: (12) x64 AMD Ryzen 5 2600 Six-Core Processor
Memory: 6.53 GB / 15.61 GB
Container: Yes
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 18.1.0 - /bin/node
Yarn: 1.22.18 - /bin/yarn
npm: 8.5.5 - /bin/npm
npmGlobalPackages:
netlify-cli: 10.3.1

Same error on OSX 12.3.1 with Node 18.2.0 and neltify-cli 10.3.3.

I've been gettting the same error.

Same error here, is it recommended to downgrade node or netlify-cli?

@johnloy @arpit-agr I downgraded node to 16.0.0 and it now works!

@RossellaFer Downgrading might work but isn't a solution.

I was able to reproduce this issue by following the tutorial section of netlify: https://docs.netlify.com/get-started/  using node v18.  the function section would not work.
after downgrading to v16 the local dev would then work properly.
Agree with @danisztls  that fixing the issue with out the need to downgrade should be the right path forward

Related: https://github.com/netlify/cli/issues/4625#issuecomment-1137701649

This is already fixed on the main branch and the warning will disappear with the next version of the cli.

How was this fixed?

How I fixed it on my side:

double-checked that I do not have several global netlify-cli packages installed (NPM, yarn, PNPM, bun)
removed all of the installed ones + realized that I had some remaining in /usr/local/bin since I'm running Ubuntu (those were taking prio over other package managers)

there I deleted both ntl and netlify (double checking that I have nothing installed thanks to whereis)

used nvm to install node v20.10.0 (got some issues with v20.9)
npm install -g<EMAIL_ADDRESS>to have the same working version as on my Macbook

This went from failing to launch a local Nuxt2 app, to having a successful (even tho a bit slower) start of my Nuxt3 app (is actually a Nuxt3 one, the CLI was trying to launch both Nuxt v2 and v3 before probably because I got the v9 of the CLI haha).
Maybe this could be helpful. 😸


cmd/link: windows/arm64 binary output invalid with linkmode=external

What version of Go are you using (go version)?

$ go version
go version go1.18 windows/arm64

With fix from #51903 applied
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=arm64
set GOBIN=
set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
set GOENV=C:\Users\Administrator\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=arm64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\Administrator\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\Administrator\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\msys64\clangarm64\lib\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\msys64\clangarm64\lib\go\pkg\tool\windows_arm64
set GOVCS=
set GOVERSION=go1.18
set GCCGO=gccgo
set AR=ar
set CC=clang
set CXX=clang++
set CGO_ENABLED=1
set GOMOD=NUL
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\msys64\tmp\go-build1929041516=/tmp/go-build -gno-record-gcc-switches

Windows 11 ARM64, running MSYS2's package of clang/llvm/lld 13.0.1.
What did you do?
Download and extract https://downloads.rclone.org/v1.58.0/rclone-v1.58.0.tar.gz
  go build -ldflags -s
  (
    cd librclone
    go build --buildmode=c-shared -ldflags -s -o librclone.dll
    cat > test.c <<END
#define WIN32_LEAN_AND_MEAN
#include <stdio.h>
#include <windows.h>

int main()
{
        HMODULE hmod = LoadLibrary("librclone.dll");
        if (!hmod || hmod == INVALID_HANDLE_VALUE)
                printf("%lu\n", GetLastError());
        return 0;
}
END
    clang -o test.exe test.c
    ./test.exe
  )

What did you expect to see?
Successful run.
What did you see instead?
193

(this is ERROR_BAD_EXE_FORMAT)
I haven't been able to figure out what's wrong with the DLL that Windows won't load it.  I'm attaching it in case somebody has some idea of what to look for.
librclone.zip
Split from #51903 since this seems to be a different issue.
/cc @cherrymui and @mstorsjo in case he has some idea what's invalid about this DLL

Does it work with other DLL built from Go (e.g. just an empty main function)?

I have absolutely no Go experience, so I apologize in advance.
cat > test.go <<END
package main
function main() {}
END

go mod init test
go build --buildmode=c-shared -ldflags -s -o librclone.dll
./test.exe

test.exe from earlier steps.  no error reported.

Thanks. So the error seems specific to rclone? What if you build rclone with -ldflags="-s -w", or just -ldflags="-w"?

Same result with -ldflags -w, no additional message

Sorry, could you clarify if it still fails with error 193? Or no error reported?

it seems to have tried to link in my test.c that was sitting there at first

go build includes all Go and C (and a few others) source files in the current directory. You can do go build <flags> test.go.

Same result with -ldflags -w, no additional message

Sorry, could you clarify if it still fails with error 193? Or no error reported?

Linking had no error or warning.  Attempting to load the DLL gives error 193

It occurred to me that an lld "repro.tar" of linking the DLL might be helpful (especially for @mstorsjo)
go build --buildmode=c-shared "-ldflags=-extldflags -Wl,-Xlink=-linkrepro:$(pwd -W)" -o librclone.dll
repro.tar.gz

It occurred to me that an lld "repro.tar" of linking the DLL might be helpful (especially for @mstorsjo)
go build --buildmode=c-shared "-ldflags=-extldflags -Wl,-Xlink=-linkrepro:$(pwd -W)" -o librclone.dll
repro.tar.gz

Awesome! Thanks, yes that did help me pinpoint the issue further. I think I know the root cause of what's happening here.
TL;DR; the go.o object file contains relocations in nonmonotonic order at one place. If inspecting llvm-objdump -r C/Users/ADMINI~1/AppData/Local/Temp/go-link-173614113/go.o with the files that @jeremyd2019 so kindly provided, I find this:
RELOCATION RECORDS FOR [.rdata]:
[...]
000000000081cf70 IMAGE_REL_ARM64_ADDR64 go.string.*
000000000081cf80 IMAGE_REL_ARM64_ADDR64 go.string.*
000000000081cfa0 IMAGE_REL_ARM64_ADDR64 go.string.*
000000000081cf90 IMAGE_REL_ARM64_ADDR64 go.string.* <---- Note, this address is smaller than the previous one
000000000081cfc8 IMAGE_REL_ARM64_ADDR64 go.string.*
000000000081cfd8 IMAGE_REL_ARM64_ADDR64 go.string.*

What leads me to believe this is the root cause (and how I found it):
Trying to load this DLL with a small test program like in the original post here, I get a dialog box pop up saying that it was unable load it (with error status 0xc000007b). If I try to do the same on Wine on linux on aarch64, it loads successfully. So the DLL isn't entirely broken, but some more strict check in Windows bails out here.
One of the most common issues in failing to load an executable like this has in the past been if there are gaps in the section layout, which often can happen after stripping (but wouldn't happen if the linker omits debug info directly, unless the linker is broken). But that isn't an issue here, all the sections are correctly laid out. All the DLL flags and similar are set correctly. (On arm64, the dynamicbase flag is a must, contrary to x86.)
Next, I tried to look at what parts of the executable that the Windows loader might be interacting with. I tried modifying the executable to zero out some fields, to have less things for the Windows loader to interact with. (To modify the executable, I used llvm-objcopy where I manually edited the code to do the exact tweaks on the PE header that I wanted; there's probably other tools for tweaking the PE header that would have worked too.)
Looking at the file headers, it has a couple data directories defined; export table, import table, import address table, exception table, base relocation table, TLS table.
I tried zooming in on the base relocation table. (See https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#the-reloc-section-image-only for some docs on it.) As the executable has got dynamicbase set, having no base relocation table at all doesn't seem to fly, but we can truncate it to see if it contains bad data somewhere. For inspecting the relocation table, I used MSVC's dumpbin.exe; dumpbin -relocations librclone.dll. That produces a printout like this:
BASE RELOCATIONS #7
  EDB000 RVA,       1C SizeOfBlock
       0  DIR64<PHONE_NUMBER>1705D8
       8  DIR64<PHONE_NUMBER>28C178
      10  DIR64<PHONE_NUMBER>209768
      18  DIR64<PHONE_NUMBER>28C160
      20  DIR64<PHONE_NUMBER>28C158
      28  DIR64<PHONE_NUMBER>1707B0
      30  DIR64<PHONE_NUMBER>1707B8
      38  DIR64<PHONE_NUMBER>1707D0
      40  DIR64<PHONE_NUMBER>1707E0
       0  ABS
  EDD000 RVA,      224 SizeOfBlock
     790  DIR64<PHONE_NUMBER>06FDB0
     798  DIR64<PHONE_NUMBER>C8E960
...

This output contained 174k lines, so we can't sit down and inspect it by hand. We can't truncate it to zero. But we can truncate it to only contain entries for some pages. Here, the size 1C and 224 are sizes of individual pieces of the relocation table. If we truncate the relocation table size (in the data directory) to 0x1C bytes, we suddenly get a different error, ERROR_NOACCESS (998) instead of the previous dialog box and error 193. (I have no idea why this produces the error ERROR_NOACCESS here, but it's at least a difference from before, so I zoomed in on that.)
So somewhere, in the relocation table, there's something that makes the Windows loader bail out. So I bisected the table, leaving out more or less of it, until I managed to pinpoint the offending piece:
 16F7000 RVA,      190 SizeOfBlock
       8  DIR64<PHONE_NUMBER>358C74
      28  DIR64<PHONE_NUMBER>343004
      18  DIR64<PHONE_NUMBER>342167
      50  DIR64<PHONE_NUMBER>341FB7
      60  DIR64<PHONE_NUMBER>343AF8
      70  DIR64<PHONE_NUMBER>34B456
      90  DIR64<PHONE_NUMBER>343004
      80  DIR64<PHONE_NUMBER>342167
[...]
     F78  DIR64<PHONE_NUMBER>358977
     F98  DIR64<PHONE_NUMBER>343004
     F88  DIR64<PHONE_NUMBER>342167
     FC0  DIR64<PHONE_NUMBER>341FB7
     FD0  DIR64<PHONE_NUMBER>347341
     FE0  DIR64<PHONE_NUMBER>370BF6
 16F8000 RVA,        C SizeOfBlock
       0  DIR64<PHONE_NUMBER>343004
       0  ABS
 16F7000 RVA,        C SizeOfBlock
     FF0  DIR64<PHONE_NUMBER>342167
       0  ABS

So here, there's a discontinuity in the table (or whatever you'd call it - it's not correctly sorted). With some more debugging in lld, I was able to pinpoint the source of this discontinutiy down to the .rdata section in the go.o object file, and to the list of relocations I referenced above.

@mstorsjo thanks for the detailed investigation! So it sounds like some relocations are not sorted and the PE loader doesn't like them?
Could you, or @jeremyd2019 , try if this patch makes any difference? Thanks!
diff --git a/src/cmd/internal/obj/objfile.go b/src/cmd/internal/obj/objfile.go
index e7bc45ccdf..39838da4ad 100644
--- a/src/cmd/internal/obj/objfile.go
+++ b/src/cmd/internal/obj/objfile.go
@@ -173,6 +173,7 @@ func WriteObjFile(ctxt *Link, b *bio.Writer) {
        h.Offsets[goobj.BlkReloc] = w.Offset()
        for _, list := range lists {
                for _, s := range list {
+                       sort.Sort(relocByOff(s.R)) // XXX sort relocations
                        for i := range s.R {
                                w.Reloc(&s.R[i])
                        }

@mstorsjo thanks for the detailed investigation! So it sounds like some relocations are not sorted and the PE loader doesn't like them?

Yes; object file level relocations aren't sorted. For normal linking this should be mostly harmless, but the non-sortedness carries over to the DLL/EXE base relocations, which seem to need to be in strictly increasing order, and lld assumes the object file relocations are correctly ordered to begin with - which they seem to be so far - I haven't checked the spec if this is a strict requirement or just something that all toolchains practically do.
Technically it would be possible to add a check and warning for it in lld, but as lld generally is quite performance focused and sometimes quite blindly assume that the input files are correct, I'm not sure if a check for it (or even worse, a sorting pass) would be welcomed there.
I could check and see what MS link.exe does in such a situation - if it does resolve it nicely, one could argue that lld should get some fix too.

Could you, or @jeremyd2019 , try if this patch makes any difference?

I don't have any setup available for testing it unfortunately, but let's hope @jeremyd2019 can.

What leads me to believe this is the root cause (and how I found it):
Trying to load this DLL with a small test program like in the original post here, I get a dialog box pop up saying that it was unable load it (with error status 0xc000007b). If I try to do the same on Wine on linux on aarch64, it loads successfully. So the DLL isn't entirely broken, but some more strict check in Windows bails out here.
One of the most common issues in failing to load an executable like this has in the past been if there are gaps in the section layout, which often can happen after stripping (but wouldn't happen if the linker omits debug info directly, unless the linker is broken). But that isn't an issue here, all the sections are correctly laid out. All the DLL flags and similar are set correctly. (On arm64, the dynamicbase flag is a must, contrary to x86.)

This is as far as I was able to get with my knowledge.  I wish there were some sort of 'PE linter' that could detect issues that would make Windows reject an image.  I guess I could start one at least with what I've learned so far...

Could you, or @jeremyd2019 , try if this patch makes any difference?

I don't have any setup available for testing it unfortunately, but let's hope @jeremyd2019 can.

https://github.com/msys2/MINGW-packages/compare/master...jeremyd2019:test-go-patch :grin:
Rebuild go with those packages, and then rebuilt rclone/librclone.  The DLL loaded in my test exe, and librclone's included python/ctypes wrapper test also worked.  I also managed to get their C test for librclone working with minimal modifications for Windows.

It makes me wonder why Windows i686/x86_64 doesn't have this issue (ie, is there some earlier step sorting relocations for them?)