Websocket freezing

Hello!
I noticed that after some time my BinanceSocketManager stalls on the await ms.recv() line. The time it freezes is constantly between 60-65 seconds. After the time it starts working again for a short moment until it freezes again for the same period of time.
I can get rid of the error by removing the line await asyncio.sleep(0.1) in my code. But in my actual trading bot there is some math that needs to be done and the sleep is command is replacing it. I don't know if this is a bug or am I doing something wrong.
Here's my code to illustrate the issue:
import asyncio
import time

from binance import BinanceSocketManager, AsyncClient

async def socket():
    messages = 0
    streams = ["ethbtc@bookTicker", "btcusdt@trade", "ethusdt@bookTicker"]

    client = await AsyncClient.create()
    bm = BinanceSocketManager(client)

    multi_socket = bm.multiplex_socket(streams)

    async with multi_socket as ms:
        while True:
            messages += 1
            # Measure the time it takes to receive message
            begin = time.time()
            msg = await ms.recv()
            time_elapsed_receiving = time.time()-begin

            # If it took over 5 seconds to receive the message: print "Error" and the message
            if time_elapsed_receiving >= 5:
                message = msg["data"]
                print(f"\nError! Time taken:{time_elapsed_receiving} Message: {message}")

            await asyncio.sleep(0.1)  # This illustrates some math that would be done

            # print current message's stream and the total amount of messages received
            stream = msg["stream"]
            msg = f"\r{stream} Messages: {messages}"
            print(msg, end="")

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(socket())

Expected behavior
Expecting a continuous stream of messages. Instead stream freezes after around 45 second. The freeze lasts always a bit over a minute after which the stream starts again until another freeze.
Environment (please complete the following information):

Python version: [e.g. 3.9]
OS: Windowns
python-binance version 1.0.10

Hello,
Same problem for me with this new version :(
https://github.com/sammchardy/python-binance/issues/870

This has been fixed in v1.0.11

Sorry but it doesn't seem to fixed at least for me. Now instead of continuing receiving the program freezes permanently after around 42 seconds.
Note:
Switched the the following line in the main post
await asyncio.sleep(0.1) to time.sleep(0.1)
I don't know if this makes a difference but in my real application the thread is busy doing math so it isn't something that can be awaited if have understood coroutines correctly.


Differences from spf13/afero

I'm curious what this brings to the table that https://github.com/spf13/afero does not have?

Afero takes a holistic interface approach to file systems, for the most part. In other words, your own FS must implement all of the different file and filesystem operations in order to be used. HackpadFS gives you more flexibility by only requiring implementations use small, composable interfaces. The interop with io/fs is improved as a result.
It also lacked consistency in behavior and had a few game-breaking bugs when it was first pulled into Hackpad. I actually reached out with bug fixes and also offered to improve things, though they had a different plan for the project. HackpadFS adds the shared test suite that was missing from afero.

https://github.com/fclairamb/afero-s3 might be interesting you this project
It allows you to read and write to s3 from what i can see.


ErrorException in POST /sso/security/details

Error in VATSIM-UK/Core
ErrorException in POST /sso/security/details
Trying to get property of non-object
View on Bugsnag
Stacktrace
app/Http/Controllers/Sso/Security.php:87 - Illuminate\Foundation\Bootstrap\HandleExceptions::handleError
app/Http/Controllers/Sso/Security.php:87 - App\Http\Controllers\Sso\Security::postDetails
app/Http/Controllers/BaseController.php:50 - App\Http\Controllers\BaseController::App\Http\Controllers\{closure}
app/Http/Middleware/TrackInactivity.php:41 - App\Http\Middleware\TrackInactivity::handle

View full stacktrace

An error linked to this issue has been reopened in Bugsnag
ErrorException in POST /sso/security/details

An error linked to this issue has been reopened in Bugsnag
ErrorException in POST /sso/security/details

An error linked to this issue has been marked as fixed in Bugsnag
ErrorException in POST /sso/security/details


Question about demo_image.py

Thanks for your great work. There are several question about demo_image.py:

I find that you directly set 'focal=1000' and then 'focal = focal / 256 * bbox_xywh[2]'. Could you tell me why the focal length is calculated by this way?
The point of view of the demo is from the front of the image, I wonder how to change other views. I checked the documentation on the official website, but could not find the relevant explanation.
Maybe my question was very stupid, but I really hope for your help.

Hi @383029372,

We assume a fixed focal length in the cropped box during training and inference, i.e., focal=1000. Therefore, to project the result to the original image, we follow the perspective project to change the focal length.
To render the pose to other views, you can apply a global orientation to the predicted vertices.


create head branch in developer branch

Create a new feature/head branch (based on the develop branch).

Не без проблем, але виконано


Still doesn't work with Nuxt 3

The issue https://github.com/elmeet/vite-plugin-javascript-obfuscator/issues/3 was closed as fixed, but it "worked" because the example project didn't include any file to be obfuscated.
I forked the example project to fix the plugin settings, the same error happens
Check it at https://stackblitz.com/edit/github-ftj7nb-dedmuj?file=nuxt.config.ts
Run nuxt generate in the terminal:
ERROR  Identifier 'useRuntimeConfig' has already been declared (Note that you need plugins to import files that are not JavaScript)
file: virtual:nuxt:/home/projects/github-ftj7nb-dedmuj/.nuxt/paths.mjs:1:1205
1: import { joinURL } from '/home/projects/github-ftj7nb-dedmuj/node_modules/ufo/dist/index.mjs'
                                                                                ^
2: import { useRuntimeConfig } from '#internal/nitro'
3: const appConfig = useRuntimeConfig().app

ERROR  Identifier 'useRuntimeConfig' has already been declared (Note that you need plugins to import files that are not JavaScript)

  at error (node_modules/rollup/dist/es/shared/node-entry.js:2150:30)
  at Module.error (node_modules/rollup/dist/es/shared/node-entry.js:13463:16)
  at Module.tryParse (node_modules/rollup/dist/es/shared/node-entry.js:14170:25)
  at Module.setSource (node_modules/rollup/dist/es/shared/node-entry.js:13772:39)
  at ModuleLoader.addModuleSource (node_modules/rollup/dist/es/shared/node-entry.js:23737:20)

The plugin version is "vite-plugin-javascript-obfuscator": "^3.1.0"

+1, same error

Change exclude settings:
exclude: [/node_modules/, /paths\.mjs/, /components\.plugin\.mjs/]

tried your suggestion, but still not work, maybe I configured it wrong?
vite: {
    plugins: [
      obfuscatorPlugin({
        exclude: [/node_modules/, /.nuxt\.mjs/, /components\.plugin\.mjs/],
        debugger: true,
        options: {
          compact: true,
          debugProtection: true,
        },
      }),
    ],
  },

can you share the full defineNuxtConfig?

Modify your configuration.  @chz8494
- exclude: [/node_modules/, /.nuxt\.mjs/, /components\.plugin\.mjs/]
+ exclude: [/node_modules/, /paths\.mjs/, /components\.plugin\.mjs/]


Allow "Group" prefix to be configurable

From @summerville on July 17, 2014 15:24
Services that don't natively support "groups" (e.g. Customer.io) get the group traits appended the user traits when the group() function is used. Each of the group traits are copied over and the keys are prefixed with the string "Group ".
It would be awesome to configure the automatic prefix. For us, we would use "account_" or something similar b/c we'd rather see "account_id" instead of "Group id" in the different analytics tools we use.
Copied from original issue: segmentio/analytics.js-integrations#350

From @summerville on July 17, 2014 20:17
No problem! I know it would be useful to us and likely others as well.
Thanks for considering it and keep up the good work!

From @ianstormtaylor on July 17, 2014 18:14
I completely agree with this as a feature. I think this will take us a little while, because it requires a bit of an infrastructure change to be able to expose those settings, but I completely agree that this is something we should provide. Thanks for submitting the issue


[NETBEANS-491] Fix issue reporting

Updated URL in the exception report message - https://issues.apache.org/jira/projects/NETBEANS/issues
Review and Report Problem button opens https://issues.apache.org/jira/projects/NETBEANS,
as suggested by Geertjan.

Closed PR477 as these are the same now.

OK, merging this.

I'd have thought we might be better having a link to the website, from which we can redirect issues as we want? eg. something like https://netbeans.apache.org/ide-report-issue ?  We could set that up as a redirect to JIRA now without hardcoding it into the IDE, giving us the ability to change it easily later.

I don't see https://issues.apache.org/jira/projects/NETBEANS/issues changing...

There was talk elsewhere of providing a more user friendly gateway to reporting issues, providing a way for people to report issues without a JIRA account, etc.

Maybe what we could do in future is have a page on the website similar to https://netbeans.org/community/issues.html that provides information, search functionality etc... and redirects to JIRA.


Block producer search does not work

Whenever I type into the search bar and there would be at least one result, the screen turns blank.
Reproduceable on Windows 10 and Windows 8
react-dom.production.min.js:179 TypeError: Cannot read property 'supportedContracts' of undefined
    at Object.render (Row.js:47)
    at us (react-dom.production.min.js:163)
    at ls (react-dom.production.min.js:162)
    at ps (react-dom.production.min.js:169)
    at Vs (react-dom.production.min.js:204)
    at Ks (react-dom.production.min.js:205)
    at Ba (react-dom.production.min.js:220)
    at Ia (react-dom.production.min.js:219)
    at Ca (react-dom.production.min.js:216)
    at $s (react-dom.production.min.js:214)

Thanks for the report! We're able to reproduce and will get a fix out for this in the next release.


Fixes the Endpoints getting-started sample.

The Dockerfile and openapi.yaml cannot be in the appengine directory.
These have been moved out into their own directories.
Also, the jetty user needs access to the war file. The Dockerfile has
been updated accordingly.

Codecov Report

Merging #585 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             master   #585   +/-   ##
=======================================
  Coverage        51%    51%
  Complexity      245    245
=======================================
  Files            87     87
  Lines          2786   2786
  Branches        185    185
=======================================
  Hits           1421   1421
  Misses         1315   1315
  Partials         50     50

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 824adf3...46b52f2. Read the comment docs.

@frankyn PTAL
@wmwong - A bit surprised by this, but Frank should know for sure.

@lesv I agree with this change.
2 cents:
There are issues in the associated doc on cloudsite that need to be updated as well. Instructions expect files to be located in java-docs-samples/endpoints/getting-started/, but they're located in endpoints/getting-started/src/main/appengine/.
Action-item Paths need to be updated in the associated doc on cloudsite along with this PR.

@dpebot merge when green

Docs have been updated, waiting on review from Frank and Lawrence.

Hi @wmwong,
I have a question, why did you place the openapi.yaml in src/main/openapi instead of in the root of this sample endpoints/getting-started? I'm not sure if there were complications, but if not we should move this down to the root of the sample. I can commit a change for this.


Travis with MySQL 5.7

@LeanderK

mariadb 10.1 is compatible with mysql 5.6, not 5.7. I have an idea for an workaround

MariaDB 10.1 is the current stable (GA) release of MariaDB. It is built on MariaDB 10.0 with features from MySQL 5.6 & 5.7, and entirely new features not found anywhere else.

According to https://downloads.mariadb.org/

yeah, but for full support we need 10.2, and thats not released yet


Adjust code to support Ruby 2.7+

This PR adjusts implementation to avoid warnings in Ruby 2.7+
Also seems like stray_quotes error was introduced in latest version of CSV lib, so I've added support for that as well

Thanks for this! I'm pulling this in as part of getting the tests going on GitHub Actions :+1:

This is all working now, thanks! Your code definitely helped me work out what was breaking it, even though I've not merged it directly. :)


Pods unable to access each other, pod CIDR in cali-cidr-block

The ingress-nginx pod is not able to access the other pods in order to allow ingress to the applications running on the cluster. On closer inspection, all traffic inter-pod is not working.
Putting a manual rule at the top of the forward chain to allow inter-pod traffic works, but Calico quickly places its cali-FORWARD rule before it.
The iptables rule it is hitting is in the cali-cidr-block chain:
Chain cali-cidr-block (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       0    --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/18        /* cali:MBFnMC9HO-P01G4D */
   18  1080 DROP       0    --  *      *       <IP_ADDRESS>/0            <IP_ADDRESS>/18         /* cali:9vWkRqRgEwhV2A-K */

<IP_ADDRESS>/18 is the pod network
Expected Behavior
ingress-nginx should be able to reach pods in order to serve applications
Current Behavior
ingress-nginx is not able to reach the pods to serve the applications. no pods are able to communicate with each other.
Steps to Reproduce (for bugs)

Install with kubespray enabling BGP, disabling nat_outgoing, and peering with a BGP router
Deploy ingress-nginx
Deploy a web application and create an ingress resource for it
browse to the newly created ingress

Your Environment

Calico version: v3.26.4
ClusterType: kubespray,bgp,kubeadm,kdd,k8s
Orchestrator version kubernetes
Operating System and version: debian 12
Link to your project (optional):

IIUC, the cali-cidr-block chain drops traffic to advertised cluster IPs that didn't get resolved to Pod IPs by kube-proxy.  Perhaps you have misconfigured the service CIDR to overlap with your Pod CIDR range?

Definitely don't have overlapping subnets - I have <IP_ADDRESS>/18 for service, <IP_ADDRESS>/18 for load balancers, and <IP_ADDRESS>/18 for pods. It was stood up with kubespray, if that makes any difference?

I've got the same issue on a brand new cluster that I have set up. No pod is able to browse to any service. Browsing directly to the pod IP is fine.

Could you provide a bit more detail about how you are accessing the service? What type of traffic are you sending and which of the services IPs (or domain names?) are you using to reach the Service? (ClusterIP, LoadBalancer IP, etc).
It would also be helpful to see:

kubectl get bgpconfiguration default -o yaml
ipset list cali-cidr-block (on the node dropping packets)

@rf152 any updates on this? could you share the info @caseydavenport requested? I'm closing this as stale since the last comment was more than a month ago, feel free to reopen (with more details) if you're still seeing this issue


[microsite] Use GA4 as main site tag

Closes #12677

:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

I've cross-linked the UA and GA4 tags, hopefully this will allow us to use the GA4 tag as the main and only tag. I'm hopeful that it will work since I've done it with one other site, but GA has offered a few surprises recently. Fingers crossed 😄 If it doesn't then I'll revert the change (by submitting another PR).

Nice! Thanks for the help @chalin! Fingers crossed it works 🤞

Both the UA and GA4 site tags are still processing events. As of the time of writing this PR was published about 35 minutes a go, and events are still being received (screenshots taken at the time of writing):

GA4

UA


Support Symbolic Links on Windows

Right now we support synchronization of symbolic links if both endpoints are POSIX systems, but symbolic links are not propagated to or from Windows machines.  There are a variety of reasons behind this:

Symbolic links on Windows behave significantly differently than those on POSIX systems. Some information on the exact differences can be found here.  The typed nature of symbolic links on Windows will complicate things.
Symbolic links aren't supported before Windows Vista (though this isn't a huge issue since we're dropping support for Windows XP soon anyway).
Symbolic links require administrator permissions to create (or they require that the SeCreateSymbolicLinkPrivilege be given to the user, but in either case they don't work out of the box).  As of Windows 10, symbolic links can be created without these elevated permissions (more information available here), but they require that Developer Mode be enabled (which, again, it isn't by default), and they require that the SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag be provided to CreateSymbolicLinkW (a change which isn't going to land until Go 1.11).
Perhaps most importantly, symbolic link targets can't be round-tripped to disk on Windows systems.  The Go os package (necessarily) modifies target paths to adhere to the requirements of CreateSymbolicLinkW, which causes the replacement of / with \, potential conversion to a UNC path, etc.  The complete ritual can be found here.  We need to be able to generate a reasonable equivalence check between Windows and POSIX symbolic links to ensure that they haven't changed, which is impossible to do in the general case, and will be very hard to do even if we restrict ourselves to supporting a subset of symbolic link targets.

Given the complexities of symbolic links on Windows, the permissions required to create them, their relatively small user base, and the current lack of a safe and sound mechanism for checking their equality, I'm inclined to punt on this until further research and design can be performed.
Any comments on potential designs are welcome.

One idea for checking equivalency would be creating a Windows symbolic link with a POSIX symbolic link path and checking the result, but this equivalency check has to happen during the reconciliation phase, and that happens on the controller, which may not be running Windows.

We also need to remember that unless we have some sort of symbolic link normalization scheme, our rsync-based transfer of serialized snapshots is going to take a performance hit.  This was easy enough to work around in the executability case, but executability semantics are much easier to work with.

Closed by 2e82f53.
This has been implemented to the extent that I think is even theoretically possible.  Implementing it had the nice side effect of cleaning up symlink support in general.


[Bug]: The gleaning is not including the original input

Describe the bug

If you look at the below function:
https://github.com/microsoft/graphrag/blob/309abc982f158c38099c6098d30b35a20972d258/graphrag/index/graph/extractors/graph/graph_extractor.py#L148C5-L182C23
async def _process_document(
        self, text: str, prompt_variables: dict[str, str]
    ) -> str:
        response = await self._llm(
            self._extraction_prompt,
            variables={
                **prompt_variables,
                self._input_text_key: text,
            },
        )
        results = response.output or ""

        # Repeat to ensure we maximize entity count
        for i in range(self._max_gleanings):
            glean_response = await self._llm(
                CONTINUE_PROMPT,
                name=f"extract-continuation-{i}",
                history=response.history or [],
            )
            results += glean_response.output or ""

            # if this is the final glean, don't bother updating the continuation flag
            if i >= self._max_gleanings - 1:
                break

            continuation = await self._llm(
                LOOP_PROMPT,
                name=f"extract-loopcheck-{i}",
                history=glean_response.history or [],
                model_parameters=self._loop_args,
            )
            if continuation.output != "YES":
                break

        return results

The call to do gleaning does not include the original input
glean_response = await self._llm(
                CONTINUE_PROMPT,
                name=f"extract-continuation-{i}",
                history=response.history or [],
            )

response.history only includes the last output from the LLM i.e. it is missing the GRAPH_EXTRACTION_PROMPT prompt.
I have verified this by looking at the exchange in the debugger as well.
Is this expected implementation? I would have thought that in order to glean at minimum the original text (chunk) would be required. Here the gleaning seems to only use the last response
Please guide. Thanks.
Steps to reproduce
No response
Expected Behavior
No response
GraphRAG Config Used
No response
Logs and screenshots
No response
Additional Information

GraphRAG Version:
Operating System:
Python Version:
Related Issues:

https://github.com/microsoft/graphrag/pull/720

Thank you @ksachdeva for pointing out this important bug! Really appreciate you finding this so that we can make the core GraphRAG engine better.


Increase k8s rest client timeout for informers.

What does this PR do?
The rest.Config.Timeout is applied to http.Client requests from the client-go. This timeout also applies to a "watch" and and is causing client-go cache.Reflector used in MetadataController to recreate the "watch" every 2 seconds, which is having a noticeable impact on the apiserver.
This PR introduces ClientBuilder so that the SharedInformerFactory can use a clientset configured with a different timeout than the APIClient. The APIClient will still use the same hardcoded 2 second timeout and the SharedInformerFactory will use a timeout specified by kubernetes_informers_restclient_timeout (defaults to 1 minute).
Motivation
Bug fix

APIClient is in the critical path for the host metadata payload creation, because of node labels as tags. We need this request to timeout in a couple of seconds. We also use the same client for the kube_apiserver check, that must run in less than 15 seconds.
Is there a way to have different timeout values depending on what we are querying?

@xvello
That is a good point. Individual requests can always specify a server-side timeout using timeoutSeconds but this is the client-side timeout. That said, if the apiserver gets backed up then having this timeout at 1 minute could be a problem.

@xvello @CharlyF I took a different approach for this issue. Let me know what you think.


Small clarification/fix on Greater than behavior

This:
The expression (dialog.orderTotal + dialog.orderTax) > 50 adds the values of the properties dialog.orderTotal and dialog.orderTax, and evaluates to True if the sum is greater than 50 or False if the sum is less than 50.

Should be this:
The expression (dialog.orderTotal + dialog.orderTax) > 50 adds the values of the properties dialog.orderTotal and dialog.orderTax, and evaluates to True if the sum is greater than 50 or False if the sum is 50 or less.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 98fbb9f4-706c-e90d-bad8-bf14a80f838c
Version Independent ID: 06f4923f-e7aa-abf1-0b08-32b8513e69eb
Content: Adaptive expressions - Bot Service
Content Source: articles/v4sdk/bot-builder-concept-adaptive-expressions.md
Service: bot-service
GitHub Login: @Kamrani
Microsoft Alias: kamrani

Thanks for pointing this out @dmvtech, the fix should go live soon.

Fix should be live now.


Menu bar disappears

It is unclear what steps can reproduce this problem, but I encountered it twice after using it for an hour. I had to restart Intellij IDEA and the menu bar reappear after restarting.

If I just close the editor tab and reopen, the menu bar area shows up without the icons.

Right-click menu, open the console, check for errors。

I saw one error but I don't think it relates to this problem because the error is there as soon as I open the markdown file and the menu bar is showing:
2index.min.js?_ijt=bi9sfgng8p27749iirdk7jh9m2:1 Uncaught TypeError: Cannot read properties of null (reading 'classList')
    at index.min.js?_ijt=bi9sfgng8p27749iirdk7jh9m2:1:237981
    at NodeList.forEach (<anonymous>)
    at e.addCaret (index.min.js?_ijt=bi9sfgng8p27749iirdk7jh9m2:1:237939)
    at e.addToUndoStack (index.min.js?_ijt=bi9sfgng8p27749iirdk7jh9m2:1:235590)
    at index.min.js?_ijt=bi9sfgng8p27749iirdk7jh9m2:1:177274

however, I found easy to reproduce steps:

open a markdown file, seeing the menu bar
go to some other files such as .py files
go back to the markdown editor tab
now the menu bar is mostly hidden
close the markdown file
open the markdown file again and the menu bar reappears

This problem has been reproduced.
The status bar is obscured because the size of the editor has changed, trying to fix it.


fix get alerts api with default params to return chained alerts and skip audit alerts

fix get alerts api with default params to return chained alerts and skip audit alerts
adds metadata update fix for monitors running within workflows
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

CI failing because of breaking changes from core
https://github.com/opensearch-project/OpenSearch/commit/e3542011cd9584c354405ba6cbf9a31ced7bc5ce


Update<EMAIL_ADDRESS>

This PR updates readable-stream to 3.0.2 to work on fixing an NPM bug. See npm/cli#59

We can't do this due to dropping old node support which would be a semver major.  Sorry.


Allow configuring the app source folder for android

Motivation (required)
I have been working on integrating React Native into an existing app and realised that the source folder for android code is currently hardcoded as app. Being a legacy codebase we can't really change that. So I wanted to be able to customise this folder name so that I can integrate RN seamlessly and start it from the CLI without having to run it from Android Studio.
Test Plan (required)

Create a barebones RN app
react-native run-android works successfully
Refactor the name of the app folder inside the android directory to TestFolderName using Android Studio
react-native run-android now fails
Changing this command to react-native run-android --appFolder=TestFolderName works successfully

Merging because the motivation and test plan look good
Generally I would advise to use a different command than react-native run-android when launching a legacy codebase, but I think this is reasonable.

Thanks for this, @ericvicenti!

@ericvicenti Is there a reason why this was closed? Do you need me to add anything to get this in?


fix: chart-lint CI job

[x] I have read Pelorus Contributing guidelines

Linked Issues
No.
Description
chart-lint is failing in GitHub actions (locally is running fine).
Testing Instructions
Check CI logs.

/retest

/lgtm

/retest


UI language set to Russian

Title, I downloaded the latest windows exe and the UI language is set to Russian, is there any way to change that?

@woflmao Buzz will detect supported languages automatically, are you in Russia or is any of the supported languages on your computer Russian?
To get additional information, please see this https://github.com/chidiwilliams/buzz/blob/main/CONTRIBUTING.md#troubleshooting
And share the part from the logs that has language codes in it

Thank you that fixed the problem! I had Ukrainian as a language pack, I'm unsure why it used that though, no other application has ever used that as a default, is there a way to keep my language pack installed and have the UI remain in English?
Also apologies for labeling Ukrainian as Russian, I am not very familiar with the differences between the two languages!

@woflmao Next version will have option to override locale via an advanced preference environment variable BUZZ_LOCALE. For more info see https://chidiwilliams.github.io/buzz/docs/preferences#advanced-preferences
To get the latest development version with this feature:

Take latest development build from https://github.com/chidiwilliams/buzz/actions/workflows/ci.yml?query=branch%3Amain
Scroll down to artifact section
Download installation file. To see download button, you need to be logged in the github.


Proposal: Default gateway priority with few user define networks

When connect container to multiple networks is not possible to select required the default gateway. Is
it possible to specify the container to use a gateway, can be with option --use-network-gateway?!
The current behavior of the gateway is set in alphabetical order of connected networks from the list in "Networks" container option.

duplicate: https://github.com/docker/docker/issues/20179

Thanks @mort1k
Yes this issue is a duplicate of https://github.com/docker/docker/issues/20179.
Given the needed code changes are in docker/docker code only, I would close this issue and leave the other one open.
Thanks,


Recursive download seems broken

When running fs.download(rpath, lpath, recursive=True)
I now get the following error:
IsADirectoryError: [Errno 21] Is a directory:

Is lpath a directory, does it exist? Can you please find more details, perhaps make a test case.

Hi Martin,
so I have the same issue as Nicolas.
Our code was working on gcsfs 0.6.0 and it fails on 0.6.1, with Python 3.6 (for me).
fs = GCSFileSystem(project=PROJECT)
fs.download(GCS_URI, f'/tmp/{dataset_name}/', recursive=True)

where GCS_URI is a directory on a GCS bucket
and the lpath is indeed a directory on the local disk.
You can find where the error appears in the following trace:
  File "/home/xxx/.local/share/virtualenvs/private-learning-lab-iw_HFsYy/lib/python3.6/site-packages/fsspec/spec.py", line 977, in download
    return self.get(rpath, lpath, recursive=recursive, **kwargs)
  File "/home/xxx/.local/share/virtualenvs/private-learning-lab-iw_HFsYy/lib/python3.6/site-packages/fsspec/spec.py", line 610, in get
    with open(lpath, "wb") as f2:

So basically, ffspec tries to open a file which is actually a dir. I don't know why but it didn't happen with gcsfs 0.6.0.

Does that "dir" correspond to a real, existent key on in the bucket?
We could possibly add an exception in fsspec (or here, although the same is true for s3fs), that zero-length files should not be written, or explicitly prune out paths that look like they contain deeper nested things.

Yes, the URI points to an existing key in the bucket. Our use case is very generic, and the issue should arise for any case with recursive=True. The weird thing is that it was perfectly fine on 0.6.0.
I may spend some time tomorrow to understand which change produced that behavior.

If there is a key that has the same path as a directory, then this problem should be expected - but of course it would be better if it was worked around. I suspect in the previous version, the key-with-the-name-of-a-directory simply wasn't being returned by gcsfs, which is also wrong.


⚠ Bofrost.de has degraded performance

In 8b816d5, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 911 ms

Resolved: Bofrost.de performance has improved in 65bb9dd after 12 minutes.


[video_player] Can't play videos on HUAWEI and HONOR devices

Steps to reproduce

Initialize network video.
Try to play.

Expected results
Normally playback videos without any issues. It actually plays fine on all other devices. We have this app in production and only HUAWEI and HONOR devices are experimenting issues.
Actual results
Nothing happen, screen is black. Logs shows some codecs error but after trying to research about the error message we couldn't find out.
Code sample
Code sample
    _controller = VideoPlayerController.networkUrl(
            Uri.parse("https://url_to_video"),
            videoPlayerOptions: VideoPlayerOptions(
              mixWithOthers: true,
              allowBackgroundPlayback: true,
            ),
          )
            ..addListener(() {})
            ..setLooping(true)
            ..setVolume(0)
            ..initialize().then((_) {
              resumeVideo();
            });

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
10-08 12:14:50.354 23696 23696 I ExoPlayerImpl: Init 905e38e [AndroidXMedia3/1.4.1] [HWELE, ELE-L29, HUAWEI, 29]
10-08 12:14:50.382 23696 24857 I OMXClient: IOmx service obtained
10-08 12:14:50.383 23696 24857 I ACodec  : In onAllocateComponent create compenent, codec name: OMX.hisi.video.decoder.avc
10-08 12:14:50.386 23696 24856 I MediaCodec: [OMX.hisi.video.decoder.avc] setting surface generation to 24264714
10-08 12:14:50.387 23696 24857 E ACodec  : [OMX.hisi.video.decoder.avc] setPortMode on output to DynamicANWBuffer failed w/ err -2147483648
10-08 12:14:50.391 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
10-08 12:14:50.391 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
10-08 12:14:50.391 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] using color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) and dataspace 0x104
10-08 12:14:50.392 23696 24857 I ACodec  : onStart
10-08 12:14:50.424 23696 24857 I HwExtendedUtils: Set to window composer mode as 2
10-08 12:14:50.426 23696 24857 I ACodec  : gralloc usage: 0(OMX) => 0x2900(ACodec)
10-08 12:14:50.426 23696 24857 W ACodec  : [OMX.hisi.video.decoder.avc] setting nBufferCountActual to 10 failed: -1010
10-08 12:14:50.427 23696 24857 W ACodec  : [OMX.hisi.video.decoder.avc] setting nBufferCountActual to 9 failed: -1010
10-08 12:14:50.427 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] Allocating 8 buffers from a native window of size 1342464 on output port
10-08 12:14:50.461 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
10-08 12:14:50.461 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
10-08 12:14:50.461 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] using color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) and dataspace 0x104
10-08 12:14:50.467 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
10-08 12:14:50.467 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] got color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) err=0(NO_ERROR)
10-08 12:14:50.467 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] using color aspects (R:2(Limited), P:1(BT709_5), M:1(BT709_5), T:3(SMPTE170M)) and dataspace 0x104
10-08 12:14:50.468 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] Now handling output port settings change
10-08 12:14:50.469 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] Output port now disabled.
10-08 12:14:50.470 23696 24857 I HwExtendedUtils: Set to window composer mode as 2
10-08 12:14:50.470 23696 24857 I ACodec  : gralloc usage: 0(OMX) => 0x2900(ACodec)
10-08 12:14:50.473 23696 24857 W ACodec  : [OMX.hisi.video.decoder.avc] setting nBufferCountActual to 19 failed: -1010
10-08 12:14:50.474 23696 24857 W ACodec  : [OMX.hisi.video.decoder.avc] setting nBufferCountActual to 18 failed: -1010
10-08 12:14:50.474 23696 24857 W ACodec  : [OMX.hisi.video.decoder.avc] setting nBufferCountActual to 17 failed: -1010
10-08 12:14:50.474 23696 24857 W ACodec  : [OMX.hisi.video.decoder.avc] setting nBufferCountActual to 16 failed: -1010
10-08 12:14:50.474 23696 24857 E ACodec  : Failed to allocate output port buffers after port reconfiguration: (-1010)
10-08 12:14:50.474 23696 24857 E ACodec  : signalError(omxError 0x80001001, internalError -1010)
10-08 12:14:50.474 23696 24857 E ACodec  : Error occurred while disabling the output port
10-08 12:14:50.474 23696 24856 E MediaCodec: Codec reported err 0xfffffc0e, actionCode 0, while in state 6
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer: Video codec error
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:   java.lang.IllegalStateException
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:2789)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at q5.h0.i(SourceFile:1)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at q5.t.H0(SourceFile:1)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at q5.t.e(SourceFile:1)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at d6.k.e(SourceFile:1)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at h5.o1.w(SourceFile:1)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at h5.o1.handleMessage(SourceFile:1)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at android.os.Handler.dispatchMessage(Handler.java:106)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at android.os.Looper.loop(Looper.java:219)
10-08 12:14:50.481 23696 24854 E MediaCodecVideoRenderer:       at android.os.HandlerThread.run(HandlerThread.java:67)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal: Playback error
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:   h5.u: MediaCodecVideoRenderer error, index=0, format=Format(1, null, null, video/avc, avc1.64001F, -1, null, [1080, 720, 100.0, ColorInfo(BT709, Limited range, SDR SMPTE 170M, false, 8bit Luma, 8bit Chroma)], [-1, -1]), format_supported=YES
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at h5.o1.handleMessage(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at android.os.Handler.dispatchMessage(Handler.java:106)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at android.os.Looper.loop(Looper.java:219)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at android.os.HandlerThread.run(HandlerThread.java:67)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:   Caused by: d6.j: Decoder failed: OMX.hisi.video.decoder.avc
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at d6.k.C0(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at q5.t.e(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at d6.k.e(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at h5.o1.w(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       ... 4 more
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:   Caused by: java.lang.IllegalStateException
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:2789)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at q5.h0.i(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       at q5.t.H0(SourceFile:1)
10-08 12:14:50.481 23696 24854 E ExoPlayerImplInternal:       ... 7 more
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal: Disable failed.
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:   java.lang.IllegalStateException
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at android.media.MediaCodec.native_flush(Native Method)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at android.media.MediaCodec.flush(MediaCodec.java:2194)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at q5.h0.flush(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at q5.t.K0(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at q5.t.M0(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at q5.t.c0(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at d6.k.c0(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at h5.n.f(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at h5.o1.v(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at h5.o1.A0(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at h5.o1.s1(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at h5.o1.handleMessage(SourceFile:1)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at android.os.Handler.dispatchMessage(Handler.java:106)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at android.os.Looper.loop(Looper.java:219)
10-08 12:14:50.482 23696 24854 E ExoPlayerImplInternal:       at android.os.HandlerThread.run(HandlerThread.java:67)
10-08 12:14:53.469 23696 24857 E ACodec  : signalError(omxError 0x80001001, internalError -110)
10-08 12:14:53.469 23696 24856 E MediaCodec: Codec reported err 0xffffff92, actionCode 0, while in state 10
10-08 12:14:53.482 23696 24857 I ACodec  : [OMX.hisi.video.decoder.avc] forcing the release of codec
10-08 12:14:53.496 23696 24857 E ACodec  : OMX.hisi.video.decoder.avc
10-08 12:17:16.187 23696 23696 W HiTouch_PressGestureDetector: Touch pointer move a lot. The moving distance of X is:19.0, limit is:60The moving distance of Y is:74.0, limit is:60
10-08 12:17:17.351 23696 23696 I ExoPlayerImpl: Release 905e38e [AndroidXMedia3/1.4.1] [HWELE, ELE-L29, HUAWEI, 29] [media3.common, media3.exoplayer, media3.decoder, media3.datasource, media3.extractor]

Flutter Doctor output
Doctor output
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.24.3, on macOS 15.0 24A335 darwin-arm64, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2024.1)
[✓] VS Code (version 1.94.0)
[✓] Connected device (4 available)
[✓] Network resources

• No issues found!

Same Issue here for Huawei P20 Pro and P30 Pro
There was an attempted Fix (https://github.com/flutter/engine/pull/54879) but unfortunatly they only tested on P40 Pro, which seems to not have resolved the issues on P20 Pro nd P30 Pro.
Also there is some more information on this closed Issue: https://github.com/flutter/flutter/issues/154068
But it was not fixed unfortunatly

@Inakitajes @richert-addisca This is similar to https://github.com/flutter/flutter/issues/154068 and the PR https://github.com/flutter/engine/pull/54879 fixed for it is only available on Flutter beta and master channels, it has not landed on stable channel yet. Can you all try switching to one of these channels and confirm?

@richert-addisca Unfortunately I don't know the exact time for it. But you can read more about the build release channels here https://github.com/flutter/flutter/blob/master/docs/releases/Flutter-build-release-channels.md.

There is a release in process now. You have a choice of using an older version of Flutter (and waiting for the new stable), or the beta channel, at the moment. There won't be any more fine estimate than that.


fixed building on nightly

Hi!
Thank you for allocators!
Please consider merging fix for compilation on nightly-2017-06-08-x86_64
after the fix:
➜  allocators git:(7fb0ab3) rustup override add nightly-2017-06-08-x86_64-apple-darwin
info: using existing install for 'nightly-2017-06-08-x86_64-apple-darwin'
info: override toolchain for '/Users/vladimirshakhov/IdeaProjects/allocators' set to 'nightly-2017-06-08-x86_64-apple-darwin'

nightly-2017-06-08-x86_64-apple-darwin unchanged - rustc 1.19.0-nightly (f062832b2 2017-06-07)

➜  allocators git:(7fb0ab3) cargo test
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running target/debug/deps/allocators-f4b307a4ccdeed34
running 13 tests
test scoped::tests::owning ... ok
test scoped::tests::out_of_memory ... ok
test composable::tests::null_allocate ... ok
test scoped::tests::scope_scope ... ok
test tests::boxed_allocator ... ok
test tests::heap_lifetime ... ok
test tests::unsizing ... ok
test scoped::tests::use_outer ... ok
test tests::take_out ... ok
test scoped::tests::mutex_sharing ... ok
test freelist::tests::it_works ... ok
test scoped::tests::placement_in ... ok
test tests::heap_in_place ... ok

test result: ok. 13 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

running 3 tests
test src/lib.rs - Allocator::allocate (line 76) ... ok
test src/lib.rs - Allocator::make_place (line 102) ... ok
test src/lib.rs -  (line 4) ... ok

test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

before:

HEAD is now at e2c7582... Merge branch 'master' of https://github.com/rphmeier/allocators
➜  allocators git:(e2c7582) cargo test | pbcopy
   Compiling allocators v0.1.9 (file:///Users/vladimirshakhov/IdeaProjects/allocators)
error: type `std::ptr::Unique<u8>` cannot be dereferenced
   --> src/lib.rs:204:9
    |
204 |         *self.ptr
    |         ^^^^^^^^^

error: type `std::ptr::Unique<T>` cannot be dereferenced
  --> src/boxed.rs:21:45
   |
21 |         let val = unsafe { ::std::ptr::read(*self.item) };
   |                                             ^^^^^^^^^^

error: type `std::ptr::Unique<T>` cannot be dereferenced
  --> src/boxed.rs:22:32
   |
22 |         let block = Block::new(*self.item as *mut u8, self.size, self.align);
   |                                ^^^^^^^^^^

error: type `std::ptr::Unique<T>` cannot be dereferenced
  --> src/boxed.rs:30:20
   |
30 |         Block::new(*self.item as *mut u8, self.size, self.align)
   |                    ^^^^^^^^^^

error: type `std::ptr::Unique<u8>` cannot be dereferenced
   --> src/lib.rs:204:9
    |

Hi @bogdad
Thanks for contributing :) I'll push a new version shortly

0.2.0 pushed.
Thanks for reminding me about this project. I think I'll spend some time porting parts of it to be more like the accepted Allocators RFC in terms of a Layout type, error types, and semantics of Clone


ghc-mod check succeeds on error

ghc-mod check succeeded with:
GMEIOException /home/sio/src/borel/dist/setup-config: hGetContents: invalid argument (invalid byte sequence)
Checked 1 file(s)

Seems like this probably shouldn't result in a successful exit status?

That's "cabal and ghc and ghc-mod aren't talking right now". Probably shouldn't be a git-vogue error as it has nothing to do with the code in the repo (and everything to do with your cabal/cabal-install being too new).

Alright, if this is expected behavior I won't worry about it.

Should help the user out with the explain function.


update dygraph link

add dygraph link in 'recomputation.rst' and 'sharding.rst'.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Remove the email forwarding feature

Followup on https://github.com/opencollective/opencollective/issues/2847
We used to have an "email forwarding" feature that was deprecated 2 years ago in https://github.com/opencollective/opencollective/issues/2847 / https://github.com/opencollective/opencollective/issues/4131, mostly because of SPAM issues, low security levels and lack of maintenance.
This feature has been replaced by a contact form, and we're planning more improvements in https://github.com/opencollective/opencollective/issues/2263. There's also an open discussion about reviving this feature in a more secure way at some point: https://github.com/opencollective/opencollective/issues/2840.
When we did this change at the time, 3 collectives opted-in to keep it active. Only one of them, https://opencollective.com/peermaps, is still active today. Looking at the email logs, it doesn't seem like they're actively using it.
I'm therefore proposing to remove this feature entirely, following this plan:

[x] Today: Contact peermaps to let them now we'll remove the feature next month
[ ] (Unless we get blocking feedback): remove the feature entirely on 1st of October

Just replaced the word "kill" with "remove" in the proposal. Less violent language!


Typo in example

import normalizrMiddleware from 'redux-normalize-middleware';
First line...screwed me up haha.
Says normalize instead of normalizr

Thanks :)


Include CopyObject in our custom cloudtrail

We want to have a record of all writes into our s3 buckets. This allows us to monitor flow of data through the conformance zones.
Currently we log PutObject, but in the landing-to-raw lambda we copy files from landing to the data bucket, using a raw/ or fail/ prefix.
I think for these events to be logged, we need to include the CopyObject event name.

This is still not working as expected:
matt.moore@MJ004284 putObjectLogs % jq '.Records[] | .requestParameters.key' <<PHONE_NUMBER>02_CloudTrail_eu-west-2_20231122T1705Z_G0FcID8VlmfFRz48.json
"landing/example_prison_data_product/v1/testing/load_timestamp=20231122T170313Z/d7169be7-7a40-44c2-b83d-7ab2c8d26e30.csv"
"curated/example_prison_data_product/v1/testing/extraction_timestamp=20231122T170313Z/20231122_170326_00012_vg7ew_c0d448a1-9b08-42a4-b18f-98f44fe33b77"


Even when accounting for the fact that division by 0 isen't handled properly, by doing 1/(x+1).
It just gives a straight line at y=0.

This is just because of me sucking at math, and having no clue about how it is supposed to look. The function 1/(x+1) always lies between 1 and 0 from x=0 to x=infinity, therefore it is drawn on one line, since one line is one pixel on the y axis, as there is no scaling on the y axis yet.


Ws week picker

Pull request for issue: #30
Component Screenshot:

Description (short):
Component was broken because of some typos. + Added new styling by UX.

I don't know why, but there is some foreign commit in there ^^

looks great!


Create @bitsongjs/storage package

We should create a new package to manage the storage, such as:

NFT.Storage
Pinata

I suggest creating a class that defines the structure of a basic provider, then each storage service implements its methods.
This allows you to also define custom classes, when those basic offers are not sufficient

Tasklist:

[x] Create provider definition
[x] Implement NFT.storage provider
[x] Implement Pinata provider
[x] Implement file validations
[ ] Check Pinata upload folders
[ ] Implement IPFS provider using js-ipfs


New release

Hi,
Thanks a lot for this beautiful and legible theme.
Could you please release a new version? A released fix for #23 in PyPI would be great.
Thanks.

@mtdowling  Will there be a new release with the recent merged commits please? Thanks!

Ping!
Hello all, love guzzle, been using it for years in PHP.
Using python now at my current employer and would love to have this be the style of my internal sphinx docs. But we only import stable releases. Would it be possible to create a stable tag with the current set of commits?
Thanks!


Notes on plugin architecture, for discussion

Ideas on building an architecture for plugins. Comments and suggestions welcome!

Thought this would be automatic, but I guess I have to add it by hand: Rendered

Nice document, I didn't know the syntax highlighting line state technique, but it makes a lot of sense.
My gut feeling is that syntax highlighting might be worth putting into the core, but a reasonable step to experiment with plugin-based syntax highlighting would be to develop a Rust crate with most of the functionality and then expose that in a simple plugin binary. If later it was deemed that syntax highlighting is indeed better in the core, the crate could be used directly there (maybe with some small changes as needed.) One advantage of the plugin approach is that different methods of syntax highlighting could be tried (such as LPeg vs Sublime's YAML files with regexp) without having to change the core or add dependencies to Lua or whatever. This could also allow for different approaches to syntax themes if that was deemed interesting, with all syntax highlighting plugins just sending spans with color information that the front-end faithfully renders. The other option is to leave all theming in the front-end with some sort of established naming convention for syntax scopes (identifier, string, etc) which are mapped to colors.
As for the configuration language I would lean towards TOML but as you note the fact that YAML is also needed to parse Sublime's new syntax definitions does make things more complicated. Though I doubt supporting both would add too much bloat to the core.
Beyond this I need to do more research on RPC approaches and differential synchronization before commenting.
Thanks for writing this document!

@leavengood you'd want to decouple colour from the backend and as you said just send through scopes with the content, similar to CSS - the benefit obviously being theming which can take advantage of whatever niceties the platform has.

Loading latest commit…

#44 Breaking changes coming up

Major upside to TOML over YAML is its readability, of course. I'd also note that doing anything for the sake of compatibility with Sublime is… maybe not a great plan. ST is incredibly popular, of course, but I think you're probably better with doing whatever's actually best and providing a conversion mechanism. I like YAML in principle, but it is incredibly hard to write correctly in my experience.

Since posting the above, I've done a lot more reading, and am now convinced that OT's are too complicated, differential synchronization relies on a poorly defined "patch" operation, and CRDT's are the One True Way. I'll update the concrete proposal after I've had a time to refine the details a bit, but in the meantime here are some readings I highly recommend for those who want to follow along:
I recommend starting with Marc Shapiro's 2011 talk at Microsoft Research explaining the CRDT concept and the underlying mathematical concept of monotonic semi-lattice.
The concept of CRDT actually started with the collaborative editing use case, before they generalized it to more general cloud-flavored problems (it now powers eventual consistency in Riak, among other things). The WOOT paper (2006) is very good at comparing the new approach to operational transforms, but I'm definitely not going to adapt the details of that implementation.
Lastly, I recommend reading A comprehensive study of Convergent and Commutative Replicated Data Types (2011), which surveys many of the use cases and goes into more detail about the duality of the state-based and operation-based approaches to implementing CRDTs. It also provides a very useful retrospective on the application to collaborative text editing (WOOT and TreeDoc), placing the somewhat complex implementation details into a clean mathematical framework.
Lastly, yes, I'll definitely want to flesh out the security section. A situation like neovim is completely unacceptable imho.

As someone who has only had a limited need to develop for mobile platforms I'm wondering if the system of communicating between the view layer and the editor itself via stdin and stdout rather than ffi means that there might be issues with mobile platforms?

That's been a point of some curiosity on my part as well.. I don't believe you can do XPC this way on iOS, because of the sandboxing rules and restrictions on running things like Python.

Personally I think mobile platforms are the last target for something like a programmer's editor, which as far as I know is what Xi is meant to be.
Therefore I think for the moment mobile should not be a concern. But long term the core could probably be used as a library on mobile rather than through the stdin/stdout mechanism.

Define 'mobile platform', though: I'm typing this from an iPad Pro, which has astounded me with how capable it is even for programming tasks. I think the question of whether that's in line with @raphlinus' goals is a totally fair one, but distinct from whether the platform is capable of supporting it (ad that one has a clear answer: it is). here are of course plenty of reasons why mobile devices (whether Android, iOS, or other) are not a go-to yet, some of them technical and some of them cultural, but I wouldn't count on that remaining the case over even the medium term.
To be clear, I think opting not to support mobile devices is totally fair and reasonable! I nite it not because it's a worry for me with regard to this project—about which, honestly, I'm still just kind of giddy—but more just as a point of curiosity and consideration.
I'll leave my thoughts there, as I don't want to derail this particular issue, and like I said I'm mostly just curious.

@leavengood I don't know, like @chriskrycho I use a fully functioning bluetooth keyboard on an ipad pro and could easily see myself hacking on it while on the road. I've also used a keyboard with android in the past for writing.
The other thing is that while there is Sublime, Atom, Visual Studio Code, etc. on PC platforms there isn't a decent cross platform text editor that can work on mobile platforms which are increasingly becoming productivity tools. I think it's reasonable to think about it, and it could be that the core is able to communicate with a client both using stdin/out or embeddable as a library (although that would likely break all plugins and syntax highlighting as currently discussed).
So very likely the goal of this project is not to support mobile platforms at all, which as @chriskrycho mentioned is perfectly fine, it's just worth thinking about at this early stage.

Yes, I am thinking about mobile. If anything, I think asynchrony is even more important to get right because the input method is in a separate process and generally does more complex edits than a typical hardware keyboard. Figuring out the interaction between that and, say, undo, is not simple.
Subprocesses work great on Android, but (as I understand it) are not allowed on iOS App Store apps. Running the front-end in core in separate threads should be fine. The plugin story will be more restrictive, but I think at least some subset should still be doable (say, ones written in Rust and bundled with the app, probably also JavaScript). I believe that JSON for the IPC is still viable on mobile, but I can imagine replacing or augmenting it with a binary transport (maybe even using shared memory buffers to cut down on copies).
I definitely want to make xi good on desktops first before getting too distracted by mobile, but I totally agree it's worth thinking about and planning for.
To the RFC, I'm planning on updating it soon. I've explored CRDT-land and do believe I can come up with a design that's a lot simpler than the fully distributed peer-to-peer protocols, yet allows high degrees of asynchrony from the plugins and input method. More soon.

You make fair points guys. I'm still using my MacBook Pro as my development machine and my iPad Air as a consumption machine, but I can see how we are moving toward mobile devices being productivity machines too.
In the time I've thought about this project I think the safest and most flexible approach is to build all the various pieces as standalone parts that have clear interfaces and that can be used as either a library or through the stdin/stdout mechanism (which really would just be a sort of "frontend" on the library part.) If most of the main components are in Rust it would always be an option to compile them into the core. For example I'm starting work on a Rust syntax highlighting crate which hopefully can prove useful and it could serve as a testbed for trying out SH in a plugin. If that proves too slow or otherwise is not working well, the same code should be usable directly by the core.
And to bring this back to the topic of mobile, if a lot of the pieces of Xi are like that then they could be compiled into the core for mobile. But overall I think we just need to see what happens.

@raphlinus What use cases do you have in mind regarding the OT/CRDT stuff? Multiple frontends connected to one backend? Or also multiple backends connected together? Or is it mostly about asynchronous updates from plugins?

@rkusa What I plan on implementing in short order is a simple engine that evaluates the CRDT serially, for asynchronous updates from plugins. However, I'm imagining that the engine could be replaced with one that actually does distributed CRDT without affecting the rest of the editor.

@raphlinus Thanks for the info. Since in the plugin case, the backend is the single point all plugins report to, OT would work perfectly fine. I read the papers you linked some time ago, but if I remember correctly using CRDTs for collaborative/concurrent text updates introduces some overhead (e.g. in data structure) compared to OT. Anyway, I think most CRDTs are easier to understand/implement and since CRDTs are better with regards to additional future features: +1 for CRDTs

@raphlinus I just read through this and thought I might weigh in on syntax highlighting specifically:
There's a possible middle ground between including syntax highlighting in the core, and having it be just another plugin that uses a JSON RPC API to place edits on the rich text annotations.
As I calculated earlier, syntax highlighting on large files can be quick if the serialization overhead is low. But I worry about using the normal JSON API, especially if the API is adding edits to the rich text sounds like it would have high overhead. With parse state caching, streaming and incremental rendering you could make it work for common cases but there would still be laggy cases like changing the syntax highlighting mode at the bottom of a long file. Or scrolling quickly soon after opening a file.
I'm thinking that perhaps there could be a special API for plugins that want to annotate buffers with Sublime-style scopes that uses a custom binary protocol (perhaps with Cap'nProto/Thrift). This could have low enough overhead that RPC is not an issue.
Alternatively, just a special JSON protocol optimized for low overhead might work. Perhaps something like sending large batches of updates to text styles as JSON arrays to avoid re-sending key names a billion times.
Also I bet you'll have to have a special case for the CRDTs for syntax highlighting so as not to overload them with millions of edits per second. An easy way is not to store past highlighting and just re-highlight on undo. Also if you plan on heavily relying on caching to make undo work fast that you may also have to cache parse states in the past (or worse, rely on plugins to cache past parse states).
Another thing to think about (can be combined with any of the above) is representation of semantic annotations. I really like Sublime/Textmate scopes and that is what syntect outputs. The thing is without a good representation they're super costly because if you use strings that can be like a 50 character string every 5 characters of text. I use a string interning method that also allows fast prefix checks for super fast colouring, you might want to borrow that from syntect (or just use it as a library, it is public). When giving data to plugins you could can just send them the numbers, which are all you need to check equality and prefixes, and then give them an API to access the interning table so they can stringify them and intern their own scopes/selectors.
My only non-highlighting related comment is that 20 undos is way too few for me. I use undo all the time. I undo over 50 things more often than I open large files even, so I would take an editor that offered long undo history over large file support (although I rarely use both at the same time, so that might be an out).

@trishume Sorry for taking so long to respond. Life has been busy, it's been hard to find time for xi. I'm basically going to start coding on a prototype to get myself unstuck. I think I have a pretty good design. It's more line oriented than what I was thinking before (
To address some of the things you raised: I'll definitely send deltas, sending the whole file goes against the spirit of the project. I'll avoid difficulties with undo and the like by having syntax highlighting a transient layer, basically downstream of all edits to the buffer. There's a small amount of logic needed to handle deltas (you move the spans along with the underlying text they annotate, then the plugin may or may not replace the spans on any given line later). I'll have a "get" RPC from the plugin to the core which is about sending chunks of up to a megabyte or so (but aligned on line boundaries, unlike my previous ideas). This lets the plugin keep as much or as little state as it likes; in the latter extreme, it can just scan through the whole file, fetching chunks and replacing the spans line-by-line.
The prototype is an experiment. If performance is good, I'll stick with it, and this will make things very good for people wanting to experiment with alternative techniques for syntax highlighting (PEG parsers, etc). If performance is not awesome, we very much could move to a more optimized interprocess communication model, maybe with binary serialization, maybe with shared memory buffers. But at that point, I think it would be simpler (and more performant) to just bring it into the core and share ropes across thread boundaries. But we'll see.
I hear you regarding the undos. In fact, a large part of what's been distracting me has been researching Operational Transforms and CRDT's. The current design is very much n^2 in the number of undos, so increasing it too much is expensive. I have some ideas how to reduce that, but unfortunately it's pretty complicated, so I'd like to get back to implementing basic functionality (including highlighting!) before spending too much more time on it.

A heads-up for people following this issue: I've made considerable progress on the plugin protocol recently, to the point where it wouldn't be crazy for someone to start prototyping a plugin for syntax highlighting. I've written a bit more at a thread on the xi_editor subreddit.

I'm going to merge this stuff even though it leaves the docs in a far-from-perfect state. Having the content in the repo and iterating from there seems better than holding PR's open for a long time without activity.


Enhance FAQ entry on green-/low-risk encounters

closes #365
This PR aims to reduce ambiguity in the FAQ entries for green-/low-risk encounters (cf. #365).
I've focused on the English version of the FAQ entry for now, but once this version is in a state which makes everyone happy I'd o/c also translate those changes for the German version 🙂.
I'd be interested what you think of the proposed changes @svengabr @mtb77 @Ein-Tim 🙂.

@daimpi are you finished with this PR? At the moment it is still "work in progress"
Cheers,
Sascha

Hey @mtb77, I'm glad you like it 🙂.
In this case I (or @Ein-Tim) will add the correction for the German version of the FAQ to this PR as well and then I'll mark it "ready for review". Does this sound good to you?

@daimpi @Ein-Tim more then that, sounds great :) Waiting for your update.
Cheers,
Sascha

Added the translation 👍

@mtb77 It's ready for review/merge 🙂.


Dynamically changing editable and selectable properties

Thanks for the project!
I use the wrapper on one page in the following way:
<FullCalendar
	ref="calendar"
    :events="totalEvents"
    :editable="editable"
    :selectable="editable"
    :header="calendar.header"
    :config="calendar.config"
/>

editable is a computed property on the component that the calendar is in. The calendar doesn't update the editable or selectable status when the property changes unless the entire calendar is destroyed and rerendered (changing tabs in my case). How can I get it to automatically update?

Hey
You should be able to achieve this with a combination of setting options dynamically and this wrapper's fireMethod function
Off the top of my head, something like this should work.
this.$refs.calendar.fireMethod('option', 'editable', true)


Incorporate residual F0 prediction into acoustic model

https://arxiv.org/abs/2108.02776

Here's a prototype of Conv1dRenet + skip connection for the record:
class ResConv1dResnet(BaseModel):
    def __init__(
        self,
        in_dim, hidden_dim, out_dim, num_layers=4,
        in_lf0_idx=300,
        in_lf0_min=5.3936276,
        in_lf0_max=6.491111,
        out_lf0_idx=180,
        out_lf0_mean=5.953093881972361,
        out_lf0_scale=0.23435173188961034,
    ):
        super().__init__()
        self.in_lf0_idx = in_lf0_idx
        self.in_lf0_min = in_lf0_min
        self.in_lf0_max = in_lf0_max
        self.out_lf0_idx = out_lf0_idx
        self.out_lf0_mean = out_lf0_mean
        self.out_lf0_scale = out_lf0_scale

        model = [
            nn.ReflectionPad1d(3),
            WNConv1d(in_dim, hidden_dim, kernel_size=7, padding=0),
        ]
        for n in range(num_layers):
            model.append(ResnetBlock(hidden_dim, dilation=2 ** n))
        model += [
            nn.LeakyReLU(0.2),
            nn.ReflectionPad1d(3),
            WNConv1d(hidden_dim, out_dim, kernel_size=7, padding=0),
        ]
        self.model = nn.Sequential(*model)

    def forward(self, x, lengths=None):
        out = self.model(x.transpose(1, 2)).transpose(1, 2)

        # denormalized lf0 from the input musical score
        lf0_score = x[:, :, self.in_lf0_idx].unsqueeze(-1)
        lf0_score_denorm = lf0_score * (self.in_lf0_max - self.in_lf0_min) + self.in_lf0_min

        # TODO: must be careful about dynamic features
        # Residual connection in denormalized f0
        lf0_res = out[:, :, self.out_lf0_idx].unsqueeze(-1)
        lf0_res = 0.693 * torch.tanh(lf0_res)
        lf0_pred_denorm = lf0_res + lf0_score_denorm
        # Back to normalized f0
        lf0_pred = (lf0_pred_denorm - self.out_lf0_mean) / self.out_lf0_scale

        out[:, :, self.out_lf0_idx] = lf0_pred.squeeze(-1)

        return out

Now I merged #73. Next, I will revise my local implementation for the new acoustic model and make a PR soon.

PR is up https://github.com/r9y9/nnsvs/pull/79. A heuristic parameter 0.693 was replaced with a better value. Also added some more comments in the code. Will add the sinsy's acoustic model soon.
Here's the distribution of residual log-F0 for nit-song070 database:

The most of data (>99.7%) is in the range of [-0.35 ~ 0.35] (i.e. [-600, 600] (in cent))

fixed by #81


discover command not returning a list of available options

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/574
Originally created by @spinx on 2019-04-10 11:21:03

What is the current bug behavior?
meltano discover all does not return a list but <itertools._grouper object at 0x116a26c50>
What is the expected correct behavior?
It should return list of available options to use
Steps to reproduce
Run meltano discover all on version 0.19.0

View 8 previous comments from the original issue on GitLab


Integration of the data-i18n translation system - Cyberpunk Red character sheet

_Integration of the data-i18n translation system on an existing Cyberpunk Red character sheet
Based on the CyberpunkRED_raycw character sheet, I integrated the data-i18n translation system._

[x ] I created the translations folder and the files it should contain.

[ x] The data-i18n tags have been added to the base html file so that the character sheet can be translated into different languages.

[ x] Fully functional en.json version

[ x] Fully functional fr.json version

[ x] Json files for other languages are available for other contributors.

[x ] Character sheet tested with both languages and working like the base character sheet.

Submission Checklist
Required

[x ] The pull request title clearly contains the name of the sheet I am editing.
[ x] The pull request title clearly states the type of change I am submitting (New Sheet/New Feature/Bugfix/etc.).
[x ] The pull request makes changes to files in only one sub-folder.
[ x] The pull request does not contain changes to any json files in the translations folder (translation.json is permitted)

New Sheet Details

The name of this game is: <  >  (i.e. Dungeons & Dragons 5th Edition, The Dresden Files RPG)

The publisher of this game is: <   > (i.e. Wizards of the Coast, Evil Hat)

The name of this game system/family is: <   > (i.e. Dungeons & Dragons, FATE)

[x ] I have followed the Character Sheets Standards when building this sheet.

[x ] This sheet has been made on behalf of, or by, the game's publisher.
[ ] This game is not a traditionally published game, but a copy of the game rules can be purchased/downloaded/found at: <   >
[ ] This sheet is for an unofficial fan game, modification to an existing game, or a homebrew system.

Changes / Description

Character Sheet Info Roll20 Internal Use only.

Hi @ctotone !
Unfortunately, we are going to have to reject this Pull Request on the basis that it includes changes to translation files. Unfortunately, we no longer accept translations submitted outside of Crowdin.
In this instance, you should re-create the pull request excluding any changes to files, or new files, in the translations subfolder.
To learn more about Crowdin and how to submit translations through Crowdin, please see our article on How to Become a Translation Volunteer.
Many thanks!


Update index.md

A few minor copy edits, and some Americanizations for consistency.

Thanks cormullion! I don't like American spelling but I do like consistency :sweat_smile:.

Yes I struggle all the time with colours ... :) And don't get me started on the Oxford "-ize"! :)


Add canny detector, midas detector

Hi i was trying your tutorial https://huggingface.co/blog/controlnet. But this feature stop me for testing other models.
This is a copy of https://github.com/lllyasviel/ControlNet/tree/main/annotator with few modification to support PIL.Image.
All credit goes to https://github.com/lllyasviel .

Cool!


Docker improvements & Symfony Dep

:tada: This PR is included in version 3.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


fixed dependencies for rc6

I removed the rxjs & zonejs peer dependencies, as @angular already requires them for 2 reasons:

angular 2 is loose on the zone js dependency, which mean that we can use a newer version when needed (for example 0.6.17 has a serious bug with longtrace)
angular 2 requires rxjs beta 11 when your lib wanted beta 6

Thanks!


Let OCIO CMake support add_subdirectory for git submodule

Hi
And many thanks for your wonderful library.
It could be pleasant for git based application hosting OCIO library to benefit of submodule git feature.
This allowing OCIO library source code be directly a child folder of the host application git folder.
Then host application CMakeLists.txt will just have to add_subdirectory of the OCIO sub folder to let everything works.
For this we just have to modify CMAKE_SOURCE_DIR  by CMAKE_CURRENT_SOURCE_DIR in the OCIO CMakeLists.txt files to let CMake understand that OCIO may not be always located in the CMake root folder.
Here my patch proposal:
https://github.com/dlrd/OpenColorIO/commit/95194321306b9493e98c15792adfc3ca4585eeb9
Client application will also have to set CMAKE_MODULE_PATH manually or we may consider to patch this line too by using CMAKE_CURRENT_SOURCE_DIR
set(CMAKE_MODULE_PATH
    ${CMAKE_MODULE_PATH}
    ${CMAKE_SOURCE_DIR}/share/cmake/utils
    ${CMAKE_SOURCE_DIR}/share/cmake/macros
    ${CMAKE_SOURCE_DIR}/share/cmake/modules

Of course other CMAKE_BINARY_DIR replacement by CMAKE_CURRENT_BINARY_DIR may also be considered.
Thanks in advance for your interest about this.
All the best

Thanks for the report @Qlex42, would you be able to open a PR for this?


DO NOT MERGE PSK Support

** Work in progress, do not merge! **
Support user configured noise PSK mixin via an hkdf derived from a shared passphrase for all nodes. Currently only enforcing 8 character string for deriving this, but this may be considered insufficient, depending on how important we think this feature to be.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[server][client][test] Refactor pub-sub message deserialization

[server][client][test] Refactor pub-sub message deserialization
This commit introduces changes to improve the handling of pub-sub message deserialization. The following modifications were made:

PubSubMessageDeserializer has been transformed from an interface into a concrete implementation. This change ensures that all pub-sub clients utilize the same deserializer.

Additionally, the following enhancements were made to the integration tests:

VeniceTlsConfiguration is now available to configure and expose TLS for pub-sub brokers during integration testing.

How was this PR tested?

GHCI
Does this PR introduce any user-facing changes?

[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.

Thanks for the review, @haoxu07!


execute supports not loading outfiles in memory

Description

Changelog description

Status

[ ] Code base linted
[ ] Ready to go

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@43639f2). Click here to learn what that means.
The diff coverage is n/a.


Adding a new task "SciML/Physical Science/AI4Science: Atomistic Simulations"

Introduction
There have been an emerging trend and great interest in foundation machine learning interatomic potentials (MLIPs/MLFFs) for simulating atomistic systems close to density functional theory (DFT) accuracy, either for universal (supporting roughly >80 elements across ptable) or tailored chemical systems. Some of the best open-source universal MLIPs currently in the game are

MACE-MP (equivariant GNNs) [arxiv, code]
CHGNet (invariant GNNs) [paper, code]
M3GNet (invariant GNNs) [paper, code]

We have seen a surge in literature trying to benchmark these three models. However, the unified and fair benchmark is missing and people not familiar with the implementation and training details of these models oftentimes have been trying to publish misleading results. The community needs a new collections of tasks, datasets, and spaces for unified/fast inference and fair benchmark that is not subject to

Unfair/imbalanced training datasets for different models (e.g. some of the current best open-source dataset MP, MPtrj vs proprietary datasets used by GNoME, Matlantis)
Different training configurations/resources used for pretrained checkpoints. In particular, some people might try to beat other models by overfitting one model but underfitting others in their reports. Or in general not consider the design difference or dataset inconsistencies between the models.
Different machines used for inference and benchmark. One key factor of atomistic simulations, both single point calculations and molecular dynamics, is the inference speed. The faster a MLIP, the more/longer the simulations can be done. This is important for large-scale (both in time and length) simulations for materials physics and chemistry. To fairly judge the speed performance of each model, we need to run all the models on the same machine and GPUs.

Some existing example benchmarks that worth notice are:

matbench-discovery for materials discovery
OCP for catalysis
Example MD benchmarks and zero-shot prediction tasks implemented in MACE-MP

🤗 Task and Transformer Implementation
Input

Huggingface transformer input will only intake a batch of ASE Atoms (list[ase.Atoms]) or a pytorch geometric Data (torch_geometric.data.Data). Each model/checkpoint submission (in pytorch .pt or equivalent in flax, tensorflow) needs to implement data interconversion and interface with their core architecture, and take care of graph/edge/neighborlist generation using ase or matscipy. We won't allow any other dependencies or customed package installation. The submitted models/checkpoints will be converted into standalone hgf transformers.

Output

Commonly used outputs should obey consensus (probably in ASE convention) and be in a list of JSON/dict. For examples,

[
{"energy": -12.3,
"forces": [[0.1, -0.7, 0.23], [0.3, -0.27, 0.29], [-0.06, -0.457, 0.23], [0.1, -0.7, 0.23]],
"stress": [[1.11, 0.0, 0.0], [0.0, 1.12, 0.01], [0, 0.01, 1.10], [0.1, -0.7, 0.23]],
"charges": [1.1, 0, -1.1], "dipoles": [], "magmoms": []},
{"energy": -12.3,
"forces": [[0.1, -0.7, 0.23], [0.3, -0.27, 0.29], [-0.06, -0.457, 0.23], [0.1, -0.7, 0.23]],
"stress": [[1.11, 0.0, 0.0], [0.0, 1.12, 0.01], [0, 0.01, 1.10], [0.1, -0.7, 0.23]],
"charges": [1.1, 0, -1.1], "dipoles": [], "magmoms": []},
]

Hi @chiang-yuan, thanks for the detailed explanations! I have a few questions that comes to mind :

what is your expectations from a Hub perspective? and from a library/tools perspective?
you mentioned that atomistic simulations can fit in the transformers framework. Does that mean that could be a new pipeline?
benchmark-wise we have a template to host reproducible leaderboards as Spaces on the Hub. Would you be interested in that direction?
are you mentioning that you'd like to see it listed as a task on https://huggingface.co/tasks? We usually start to do that once there is a consistent ecosystem in the domain.

I'm asking this to figure out the scope you had in mind so that I can redirect you to resources and/or ping the correct people internally :hugs:

what are your expectations from a Hub perspective? and from a library/tools perspective?
We might need a new 🤗 task for this specialized input and output.
you mentioned that atomistic simulations can fit in the transformers framework. Does that mean that could be a new pipeline?
I am not familiar with pipeline but we might need a class that allow to inject saved model in torch.nn.Module or other relevant formats.
benchmark-wise we have a template to host reproducible leaderboards as Spaces on the Hub. Would you be interested in that direction?
Yes I am looking into building a Space for leaderboard, but realize the first step is we need a task
are you mentioning that you'd like to see it listed as a task on https://huggingface.co/tasks? We usually start to do that once there is a consistent ecosystem in the domain.
We already have clearly defined benchmarks and pretrained models and datasets but it seems very incompatible with current tasks available on hf. If you could point/help us how to build and deploy a leaderboard incorporated with hf dataset and hf models without adding a task first, then we could talk about hosting a small independent ecosystem.

Thanks so much for your help :)

Hi @chiang-yuan, sorry for the late reply. So I think there is a misunderstanding about what the Hugging Face Hub is and allows. The Hub is a place where anyone can host any ML model of there choice, no matter the task, architecture, size, etc. There is indeed a list of "official" tasks here but this list is not exhaustive and only based on what we/the community agree the most for what should be "a task".
If you want to start harmonization work for a new task atomistic simulations, you shouldn't need our help to get started :) For what I understand, you are planning to build a new library to define what is a atomistic simulation model and what should be its inputs and outputs, right? To do so, I would advice to use the PytorchModelHubMixin object. It's a class that you can inherit from in your own library and which adds built-in from_pretrained, save_pretrained and push_to_hub methods. Here is how it would look like in practice:

import torch
import torch.nn as nn
from huggingface_hub import PyTorchModelHubMixin

class AtomisticSimulationModel(
        nn.Module,
        PyTorchModelHubMixin,
        library_name="???",
        tags=["atomistic-simulation"],
        repo_url="???",
        docs_url="???",
    ):
    def __init__(self, hidden_size: int = 512, vocab_size: int = 30000, output_size: int = 4):
        super().__init__()
        self.param = nn.Parameter(torch.rand(hidden_size, vocab_size))
        self.linear = nn.Linear(output_size, vocab_size)
        # implement custom logic here

    def forward(self, x):
        return self.linear(x + self.param)
        # implement custom logic here

    # implement custom methods here

then you will be able to do:
model = AtomisticSimulationModel(hidden_size=256)

# Save model weights to local directory
model.save_pretrained("my-awesome-model")

# Push model weights to the Hub
model.push_to_hub("my-awesome-model")

# Download and initialize weights from the Hub
model = AtomisticSimulationModel.from_pretrained("username/my-awesome-model")
model.hidden_size # 256

All of this can be built with huggingface_hub and hosted on the Hub without custom backend implementation on our side. All models pushed with this class will be tagged as atomistic-simulation and therefore easily searchable on the Hub. You will also have download counts, model cards, etc. You can implement all the custom logic in AtomisticSimulationModel and especially work on the correct input/output format that you expect people to use. Once the project gets mature, we can think of integrating it more deeply in the platform, especially to officially support the library with integrated code snippets for instance.
What do you think? Is this something you are looking for? Happy to answer any further questions :)

Thank you @Wauplin for the exhaustive answer! Your explanation clears a lot of my confusion from the documentation.
Yes - I think PyTorchModelHubMixin is exactly what I would picture how the MLIP community archive and deploy models on huggingface. Being searchable on Hugging Face as tag sounds also very nice. I think at this stage this implementation would be sufficient :)
I guess this still not enough to become an inference endpoint? It will be nice to have it so people can run simulations as API calls. But of course we could talk about it later once we really see more demands.

Hi @Wauplin, is there any way to access the revision/version/commit hash for the uploaded model through push_to_hub or any file loaded through the following example code?
fpath = hf_hub_download(
            repo_id="organization/hf_repo",
            subfolder="pretrained",
            filename="pretrained.model",
            revision="some revision"
        )

I guess this is still not enough to become an inference endpoint? It will be nice to have it so people can run simulations as API calls. But of course we could talk about it later once we really see more demands.

Indeed, support for InferenceEndpoints would require some more work -still doable-. We can discuss this specific topic later :)

Thanks @Wauplin ! Sorry for not being clear. I understand it is easier to use mixin, but I actually want to get the revision from the retrieved model after AtomisticSimulationModel.from_pretrained("organization/hf_repo"). I looked into the doc and attribute suggestion but haven't found one that looks promising

Oh I see. Then this is not something doable unfortunately. What you can do however is:
from huggingface_hub import model_info

# Commit hash for latest revision on main
model_info("organization/hf_repo").sha

# Commit hash for revision "refs/pr/1" (pull request ref)
model_info("organization/hf_repo", revision="refs/pr/1").sha

# Commit hash for revision "my-custom-branch-or-tag"
model_info("organization/hf_repo", revision="my-custom-branch-or-tag").sha

This is not built-in the mixin but would definitely work. Out of curiosity, what's the purpose of getting the commit hash that has been loaded?

Hi @Wauplin sorry for late reply. I would like to use the commit hash as a uuid for pretrained checkpoints

Ok, then using model_info separately would work. I don't see a way of doing it out of the box  when loading with the mixin (users usually don't care about the commit hash^^). So would this solution we ok for you?

Yes this works great! Thanks!


Installation on CentOS 7 is broken by moby packages

As posted on stackoverflow
I have a docker image for CentOS 7 which installs docker-ce via the recommended instructions. i.e.
RUN yum-config-manager \
  --add-repo \
  https://download.docker.com/linux/centos/docker-ce.repo
RUN yum install -y yum-utils \
  device-mapper-persistent-data \
 lvm2
RUN yum update -y
RUN yum install -y docker-ce docker-ce-cli containerd.io

Recently this stopped working and now fails as follows:
--> Processing Conflict: moby-containerd-1.3.6+azure-1.x86_64 conflicts containerd
--> Processing Conflict: moby-runc-1.0.0~rc10+azure-2.x86_64 conflicts runc
--> Finished Dependency Resolution
Error: moby-containerd conflicts with containerd.io-1.2.13-3.2.el7.x86_64
Error: moby-runc conflicts with containerd.io-1.2.13-3.2.el7.x86_64
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
The command '/bin/sh -c yum install -y docker-ce docker-ce-cli containerd.io' returned a non-zero code: 1

If the install command is replaced with:
 yum install -y docker

I get a different error to do with an unsigned package:
Package runc is obsoleted by moby-runc, trying to install moby-runc-1.0.0~rc10+azure-2.x86_64 instead
...
Package moby-runc-1.0.0~rc10+azure-2.x86_64.rpm is not signed

I tried forcing use of a few old versions as below to no avail e.g.
RUN yum install -y docker-1.13.1-102.git7f2769b.el7.centos

As a workaround I have to exclude moby packages. As in:
RUN yum install -y docker --exclude=moby-\*

I think this is a defect in the repository however.
I don't think an unsigned package belongs in a repository named "stable".
I guess there is something wrong with how the existing packages are being obsoleted too.
The installation instructions (engine/install/centos.md) may need to be reviewed with this.

Hey. I've been having this issue for a few weeks and today got into doing some digging. The bit in the errors I received that got me poking was Package moby-engine-19.03.12+azure-2.x86_64.rpm is not signed
I have installed the DotNetCore libraries onto my centos box, so I had a dig into the Microsoft repositories, and found that if I disabled the packages-microsoft-com-prod repository, then installed docker, my install was successful
So it seems in my case that the moby-engine from Microsoft is causing the unsigned error, not sure if this will help you in your scenario, but it's something to try

This is affecting me as well --- I am using a Centos 7 on an Azure VM (OS Version: Linux (centos 7.8.2003)) -- I am using the the exclude workaround for the time being as well. It was stopping up an apt update.

Update --- my repo's needed to be updated --- the following resolved me:
curl https://packages.microsoft.com/config/rhel/7/prod.repo > ./microsoft-prod.repo
sudo cp ./microsoft-prod.repo /etc/yum.repos.d/
yum update -y

my mistake. This is not a bug in the docker repo at all, which I did find surprising. Its Azure's fault. Its Microsoft who've been mismanaging their repos. The moby packages come from "packages-microsoft-com-prod".
For completeness and anyone who encounters this issue and winds up here it seems the baseurl has changed.
The correct one is now:
baseurl=https://packages.microsoft.com/rhel/7/prod/

installed via:
RUN curl https://packages.microsoft.com/config/rhel/7/prod.repo >/etc/yum.repos.d/microsoft-prod.repo

The one with the dodgy packages is:
baseurl=https://packages.microsoft.com/centos/7/prod/

Installed via:
RUN rpm -Uvh https://packages.microsoft.com/config/centos/7/packages-microsoft-prod.rpm

The moby packages exist only in the CentOS repo which is possibly defunct, as Microsoft themselves have changed installation documentation in various places.

Thanks was having a similar issue automating some vm builds


Curriculum Guides missing

Although it exists in the repo (though out of date?), none of the information from the Curriculum Guides sections (NCEA & AP-CSP) have been refactored for the new system

I can't speak for the AP-CSP guides, but I do know that the NCEA guides are now out-of-date, as the achievement standards, they were written for changed substantially.

I have been told that the AP-CSP guides are current, so we should pull them across.


Start markdown

A markdown practice
1. A markdown practice
(2). A markdown practice
This is a markdown practice.

Learned different headers.


fix: fix cloudbuild.yaml

:tada: This PR is included in version 1.0.0-main.3 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


No equivalent to LEAST and GREATEST in SQL ?

What's up?
I have a table where each row has for example
temperature_a, temperature_b, etc
and I want to create a condition if min(temperature_a, temperature_b) < somethreshold OR max(temperature_a, temperature_b) > someOtherThreshold
in SQL I can do something like
    SELECT
        LEAST(value.temperature_front, value.temperature_back) as min_temp_cells,
        GREATEST(value.temperature_front, value.temperature_back) as max_temp_cells,
        GREATEST(value.temperature_mosfet, value.temperature_ic) as max_temp_bms
   FROM ...

but I don't see the equivalent in PRQL?

What would be a nicer way to do this than
from t = s"SELECT * FROM recent_cse(4)"
derive {
  min_temp_cells = s"LEAST(value.temperature_front, value.temperature_back)",
  max_temp_cells = s"GREATEST(value.temperature_front, value.temperature_back)",
  max_temp_bms = s"GREATEST(value.temperature_mosfet, value.temperature_ic)",
}
filter min_temp_cells < -10 || max_temp_cells > 40 || max_temp_bms > 70
group bms_id (
  aggregate {
    min_temp_cells = min min_temp_cells,
    max_temp_cells = max max_temp_cells,
    max_temp_bms = max max_temp_bms,
    count = count this,
  }
)

Yes we should probably add these functions. Using s-strings is the best approach for the moment.

For adding them — do we want to use LEAST & GREATEST? I always found these were slightly awkwardly named — they seem to use synonyms for MIN & MAX to to distinguish them from aggregation functions. But the words don't indicate whether they're aggregation functions or not.
We could instead use math.min & math.max given we have the advantage of modules. But possibly that's confusing...

I would probably keep least and greatest for consistency, or perhaps just have an "overload" of min which can be used when there are multiple params, eg min(someField) would be the aggregate of someField over the rows, and min(fieldA, fieldB, fieldC) would be the min of the field for each row (but might be confusing, don't know if there are contexts in which they could have both an "in-row" and "across row" meaning?)


Place markers

Hi, is it possible to create a functionality to place markers on the rendered map, based on lat,lng?

Yes it would be very interesting, I will try to work on it

@Heinoman i search and find something for this but none seemed to be enough for my case wishing to have overlays on my images but found this   https://github.com/mapbox/mapbox-sdk-js/blob/master/docs/services.md#getstaticimage /docs/services.md#getstaticimage from` mapbox, and provides everything we need.


Make custom PageManagers return the correct PageQuerySet subclass

As reported in https://github.com/wagtail/wagtail/issues/3250#issuecomment-284988695, custom page managers created with PageManager.from_queryset(CustomPageQuerySet) fail to return instances of CustomPageQuerySet. This breaks the EventPageQuerySet example given at http://docs.wagtail.io/en/v1.9/topics/pages.html#custom-page-managers.

Merged in be73f16e14f61b6d424c2ae7f25029ef3ba2e933


How about Animated.subtract?

I'm finding myself typing Animated.add(x, Animated.multiply(-1, y)) a lot.
Would you take a PR that adds an Animated.subtract?

cc @janicduplessis

Sure, we have Animated.divide so I guess substract does make sense too. If you want to make a PR for it I can review it, it should be relatively simple if you start from the add implementation.

Hey, @dgreensp are you working on it? Or is it something I could take on?

@avendiart Go for it!


Fix sending pull requests to non-owned repositories with github personal access tokens

We have the ability to use repos where we don't have write access by using the --fork-owner option in send_pull_request.py.
However, github personal access tokens don't work well with sending PRs to repositories that you don't own:
https://github.com/orgs/community/discussions/106661
Because of this, sending actual PRs will fail, and we can only use --pr-type branch. This can apparently be worked around by using classic access tokens on github, but it would be nice to be able to find a way to use personal access tokens.

I think we won't be able to fix this, maybe it'll be fixed by a github app.


tools/docker: adjust the syzbot dockerfile for arm64

The changes look good to me.
Let's wait with merging until I test whether syz-ci runs fine within the container.


Fix Mailbox Server solution

handle_call isn't used as part of requirement so replaced GenServer.call with GenServer.cast.
Fix the unused variable warning for pid.

Good catch!