High CPU and disk read/write, very large (2GB) state.db on k3s 1.22.9

We have many installations (around 200+) of k3s running on 4-CPU x86 hosts (single-node clusters) running Ubuntu 20.x, mostly at customer sites. Recently, one of the customers has been complaining of high CPU usage (around 80% consistently). I dove into it for a couple weeks, and have landed on some key symptoms:

High CPU, as mentioned. Also, running uptime yields load average typically around 8-9, vs. other customers' systems which usually sit between about 1.0 and 3.0. k3s as highest CPU user per top
Very high disk usage per dstat -d - reads of 30-60MB/s consistently, and write between 3-20MB consistently, vs. other systems consistently under 1 MB/s. k3s as highest disk user per top
/var/lib/rancher/k3s/server/db/state.db is > 2GB, vs. < 50 MB on other systems.
In journalctl -e -u k3s most of the logs are Trace, mixed with some errors such as "Compact failed" and "context deadline exceeded" (more detail on logs below)

I tried the vacuum steps mentioned in https://github.com/k3s-io/k3s/issues/1575#issuecomment-613078214 , which brought it from about 2.7GB down to 2.1GB, but otherwise didn't seem to benefit much.
I realize 1.22.9 is a few years old, but I'm hoping we can get it fixed without upgrading given the number of installations in the field, and I'm also hoping maybe this is just something simple I haven't realized yet.
Environmental Info:
K3s Version:
k3s version v1.22.9+k3s1 (8b0b50a5)
go version go1.16.10
Node(s) CPU architecture, OS, and Version:
Linux QX7500B5 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Cluster Configuration:
Single node
Describe the bug:
See above
Steps To Reproduce:
Unsure exact steps. I don't think we're doing too much crazy with the install/config of k3s, but LMK if there is anything relevant to this issue and I can include details.
Expected behavior:
System should run at a reasonable CPU utilization level (e.g. 20-30%) and load average of 2-3. Disk usage per dstat -d under 1MB/s most of the time. Reasonably unpolluted system logs.
Actual behavior:
See above
Additional context / logs:
From journalctl -e -u k3s:
Jun 06 21:45:38 hostnameredacted k3s[11771]: I0606 21:45:38.893725   11771 trace.go:205] Trace[636222759]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/hostnameredacted,user-agent:k3s/v1.22.9+k3s1 (linux/amd64) kubernetes/8b0b50a,audit-id:dc061c09-b316-4d84-8bec-5a44affb1a75,client:<IP_ADDRESS>,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/1.1 (06-Jun-2024 21:45:37.907) (total time: 986ms):
Jun 06 21:45:38 hostnameredacted k3s[11771]: Trace[636222759]: ---"Object stored in database" 985ms (21:45:38.893)
Jun 06 21:45:38 hostnameredacted k3s[11771]: Trace[636222759]: [986.213986ms] [986.213986ms] END
Jun 06 21:45:39 hostnameredacted k3s[11771]: I0606 21:45:39.002407   11771 trace.go:205] Trace[920478471]: "Get" url:/api/v1/namespaces/ingress-nginx/configmaps/ingress-controller-leader,user-agent:nginx-ingress-controller/v1.1.3 (linux/amd64) ingress-nginx/9d3a285f19a704524439c75b947e2189406565ab,audit-id:c23eb4c6-63e2-4023-b0a3-6f4ab7250f53,client:<IP_ADDRESS>,accept:application/json, */*,protocol:HTTP/1.1 (06-Jun-2024 21:45:38.487) (total time: 515ms):
Jun 06 21:45:39 hostnameredacted k3s[11771]: Trace[920478471]: ---"About to write a response" 515ms (21:45:39.002)
Jun 06 21:45:39 hostnameredacted k3s[11771]: Trace[920478471]: [515.265455ms] [515.265455ms] END
Jun 06 21:45:45 hostnameredacted k3s[11771]: I0606 21:45:45.104616   11771 trace.go:205] Trace[1785314302]: "GuaranteedUpdate etcd3" type:*v1.Endpoints (06-Jun-2024 21:45:44.261) (total time: 842ms):
Jun 06 21:45:45 hostnameredacted k3s[11771]: Trace[1785314302]: ---"initial value restored" 393ms (21:45:44.654)
Jun 06 21:45:45 hostnameredacted k3s[11771]: Trace[1785314302]: ---"Transaction committed" 449ms (21:45:45.104)
Jun 06 21:45:45 hostnameredacted k3s[11771]: Trace[1785314302]: [842.863573ms] [842.863573ms] END

From journalctl -e u k3s | grep -v Trace:
Jun 06 21:41:36 hostnameredacted k3s[11771]: time="2024-06-06T21:41:36+05:30" level=error msg="error in txn: context deadline exceeded"
Jun 06 21:41:40 hostnameredacted k3s[11771]: time="2024-06-06T21:41:40+05:30" level=error msg="Compact failed: failed to record compact revision: sql: transaction has already been committed or rolled back"
Jun 06 21:41:40 hostnameredacted k3s[11771]: time="2024-06-06T21:41:40+05:30" level=error msg="error in txn: context canceled"
Jun 06 21:46:09 hostnameredacted k3s[11771]: E0606 21:46:09.074134   11771 status.go:71] apiserver received an error that is not an metav1.Status: context.deadlineExceededError{}: context deadline exceeded
Jun 06 21:46:09 hostnameredacted k3s[11771]: time="2024-06-06T21:46:09+05:30" level=error msg="error while range on /registry/events/qumulex/agent-66c685c88-qz6rq.17c4f6b6012a0980\x00 /registry/events/qumulex/: context deadline exceeded"
Jun 06 21:46:09 hostnameredacted k3s[11771]: E0606 21:46:09.087808   11771 writers.go:117] apiserver was unable to write a JSON response: http: Handler timeout
Jun 06 21:46:09 hostnameredacted k3s[11771]: E0606 21:46:09.088341   11771 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"http: Handler timeout"}: http: Handler timeout
Jun 06 21:46:09 hostnameredacted k3s[11771]: E0606 21:46:09.091877   11771 writers.go:130] apiserver was unable to write a fallback JSON response: http: Handler timeout
Jun 06 21:46:09 hostnameredacted k3s[11771]: E0606 21:46:09.092988   11771 timeout.go:139] post-timeout activity - time-elapsed: 4.684528ms, GET "/api/v1/namespaces/qumulex/events" result: <nil>
Jun 06 21:46:32 hostnameredacted k3s[11771]: time="2024-06-06T21:46:32+05:30" level=error msg="Compact failed: failed to record compact revision: sql: transaction has already been committed or rolled back"

dstat -d:
root@hostnameredacted:/# dstat -d
-dsk/total-
 read  writ
  38M   18M
 191M 2668k
  73M  900k
  90M 1232k
  72M 1024k
  53M   74M
  71M 1164k
  68M 1692k
  33M   73M
  41M 1164k
  63M 2256k

As you noted, v1.22.9 is a quite old release. There have been a number of improvements to the compact logic made in newer releases, so I suspect that things might be improved by newer versions. However, the messages you're seeing indicate that a combination of slow disk IO and high datastore write volume has filled to the database to the point where compaction can no longer run in the allotted time, and old rows are not being purged.
Follow the steps from this comment to figure out what's filling your database: https://github.com/k3s-io/k3s/issues/1575#issuecomment-722674596
You can then run this query to manually compact the old rows:
delete from kine where id in (select id  from (select id,name from kine where id NOT IN (select max(id) as  id from kine group by name)) as t);


Next example doesn't build

Preflight checklist

[X] I could not find a solution in the existing issues, docs, nor discussions.
[X] I agree to follow this project's Code of Conduct.
[X] I have read and am following this repository's Contribution Guidelines.
[X] This issue affects my Ory Cloud project.
[X] I have joined the Ory Community Slack.
[X] I am signed up to the Ory Security Patch Newsletter.

Describe the bug
build throws the following errors:

@ory/client missing
axios missing
and a TS error:

./pages/login.tsx:104:21 at setFlow(err.response?.data)
Type error: Argument of type 'unknown' is not assignable to parameter of type 'SetStateAction<LoginFlow | null>'.

There might be more TS errors, it stops at the first one.
Reproducing the bug
run build
Relevant log output
No response
Relevant configuration
No response
Version
master
On which operating system are you observing this issue?
None
In which environment are you deploying?
None
Additional Context
No response

Hi @hyperknot
Please check if you have done the following:
cd elements
npm run initialize
cd examples/nextjs-spa
npm run dev

I'm using pnpm, I haven't invested into how to make it work with Lerna. But aren't the examples supposed to be "self-sufficient" that is, they can be moved outside the repo and work on their own? For example Next.js has over 300 examples, and all of them are self-sufficient.
https://github.com/vercel/next.js/tree/canary/examples

You can use normal npm.
The examples are linked to the latest elements build, which exists through the Lerna build process. This makes it easy for us to test and maintain within the same repository. You can of course move the nextjs example outside of this repositories scope and just install the missing packages manually. There shouldn't be that many since the examples are also quite lean.
To understand how this works inside of the elements repository, see here https://github.com/ory/elements/blob/main/examples/nextjs-spa/package.json#L17
Take note of the * for the package version. Lerna handles this for us by using a local link instead of a version on the npm registry.
We also documented this inside the readme https://github.com/ory/elements#contributing and https://github.com/ory/elements#understanding-ory-elements

I understand. It's an easy fix anyway, just 2-3 packages installed is no big deal.
Do you know why is the TS error? Is it related to something in the TS config or build also throws this on your side?

I don't get any ts errors :confused:
I would recommend you delete the node_modules folder and reinstall the packages. Below commands should do the trick:
cd elements
npm run clean
npm run initialize
cd examples/nextjs-spa
npm run dev

you can repace npm run dev with npm run build for a production build of nextjs.

unless you copied out the nextjs example and don't have eslint setup correctly :thinking:

It works with the lerna steps, no error. But are these using non-standard ESLint configuration? I mean I guess every user who encounters this repository will want to run / copy these into their personal projects. For example the Next.js one should compile with the standard ESLint coming from create-next-app, otherwise it'd be a lot of effort just to try to make it run.

I went through and integrated Ory Elements into my Next.js project. The example is basically copy-pasted, except for these errors, which all have to be @ts-ignore-ed, one-by-one.
I believe the Next.js setup is more strict then what you are using for developments. For a nicer DX I'd recommend using the same settings as Next.js so that users don't have to manually @ts-ignore them.
As an alternative, having a higher level wrapper for <Login oryClient={} config={}> would be a very nice DX. Only a single line import, everything handled internally.

Thank you for the feedback! I will take a look in the coming days on how we can improve this


Language service use of a whole notebook as single (concat) document

Develop an end-to-end sample that maps many cells to a single LS document, e.g have single concat document for a notebook that re-uses TypeScript features and/or features from a language service (LSP)

The joh/concat-branch has some changes that do the following

a ConcatDocument is also a TextDocument, and
concat documents appear as normal text documents, e.g are part of workspace.textDocument or onDidOpen/Close events etc
an extension registers language providers for notebook-cells but then "redirects" requests onto the concat document.

The snippet below does that for the definition provider. A request for a notebook cell is translated into a request for the concat document. Then executeCommand is being used to get the actual language service involved and finally the results is translated back.
const cellSelector = { language: 'css', scheme: 'vscode-notebook-cell', exclusive: true };

vscode.languages.registerDefinitionProvider(cellSelector, new class implements vscode.DefinitionProvider {
	async provideDefinition(document: vscode.TextDocument, position: vscode.Position, token: vscode.CancellationToken) {
		if (!notebook.cells.find(cell => cell.uri.toString() === document.uri.toString())) {
			console.log('NOPE')
			return;
		}
		const pos = concatDoc.positionAt(new vscode.Location(document.uri, position)); // translate: cell -> concat
		const result = await vscode.commands.executeCommand<vscode.Location[]>('vscode.executeDefinitionProvider', concatDoc.uri, pos);
		return result && result.map(loc => concatDoc.locationAt(loc.range)); // translate: concat -> cell
	}
});

:+1: This works! Variables defined in one CSS cell appear in other cells and thereby enable go-to-def, suggest etc
:-1: Because CSS already works for individual notebook cells, duplication of results is happening, e.g there are results based on cells and results based on the concat document. The current "solution" is to register providers as exclusive - something that we cannot make generally available.
:-1: This only works because CSS is already able to digest documents from arbitrary schemes and the added value is the introduction of symbols (variable defined in another cell). However, experience has shown that digesting arbitrary sources is hard, e.g TypeScript isn't as good as CSS
👎 This does require an extension that enables these languages features (e.g the snippet above) and isn't something that just works out of the box
👎 There are some hacks in the current exploration that need careful engineering, e.g currently the concat document only lives on the extension host side but must also exists in the renderer for language features to work.

Closing as the "a concat doc is a normal doc" approach didn't work so well (see above). We will add a language service guide for notebooks


[FEATURE] Minio as service

It would be great if it was possible to enable MinIO as a replacement for AWS storage

MinIO is a High Performance Object Storage released under Apache License v2.0. It is API compatible with Amazon S3 cloud storage service.

It will be available in the next release (today or tomorrow). 🎉


Use v3 compatibilityJSON for plural strings

https://github.com/stolostron/backlog/issues/23101
Plural/singular support restored:

/cc @RobDolares


Update alpine Docker tag to v3.17

This PR contains the following updates:

Package
Update
Change

alpine
minor
3.16 -> 3.17

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

@balena-ci I self-certify!


Add support for datatype float (Single) and obsolete double (Double) …

…usage.
Expanded all unit tests.
Addresses #132.

@mycroes
Thanks for your input and suggestions, do you have a S7-400 available for testing? Not only asking for this PR, in general.

No, I only have access to S7 1200 and S7 1500.


Add the render target for the client-side rendering example

The client-side rendering example with React lack the render target node so it can't be copy-pasted to test it.

Thank you!


x/tools/gopls: crash importing test variant of tools file

gopls version: v0.15.1/go1.22.0
gopls flags:
update flags: proxy
extension version: 0.41.1
environment: Visual Studio Code darwin
initialization error: undefined
issue timestamp: Fri, 01 Mar 2024 00:31:32 GMT
restart history:
Fri, 01 Mar 2024 00:30:44 GMT: activation (enabled: true)
Crashing gopls
panic: nil metadata for "github.com/mailru/easyjson/easyjson [command-line-arguments.test]"

goroutine 7085 [running]:
golang.org/x/tools/gopls/internal/cache.(*Snapshot).getPackageHandles.func1(0x1400d4f3950, {0x14009c2b040, 0x41})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:872 +0x53c
golang.org/x/tools/gopls/internal/cache.(*Snapshot).getPackageHandles.func1(0x0, {0x14009c54500, 0x7a})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:889 +0x468
golang.org/x/tools/gopls/internal/cache.(*Snapshot).getPackageHandles(0x140021d59e0, {0x10104c648, 0x14008365590}, {0x1400b2c9b08, 0x1, 0x0?})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:899 +0x17c
golang.org/x/tools/gopls/internal/cache.(*Snapshot).forEachPackage(0x140021d59e0, {0x10104c648, 0x14008365320}, {0x1400b2c9b08, 0x1, 0x1}, 0x1400a266c80, 0x1400cb41ca0)
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:343 +0x134
golang.org/x/tools/gopls/internal/cache.(*Snapshot).PackageDiagnostics(0x140021d59e0, {0x10104c680?, 0x1400582e6e0?}, {0x1400b2c9b08, 0x1, 0x1})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/snapshot.go:686 +0x19c
golang.org/x/tools/gopls/internal/cache.(*Snapshot).orphanedFileDiagnostics(0x140021d59e0, {0x10104c680, 0x1400582e6e0}, {0x1400a266c00, 0x3, 0x10104aaf8?})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/snapshot.go:1452 +0x1c8
golang.org/x/tools/gopls/internal/cache.(*Session).OrphanedFileDiagnostics(0x1400048e150, {0x10104c680, 0x1400582e6e0})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/session.go:1165 +0x474
golang.org/x/tools/gopls/internal/server.(*server).diagnoseChangedViews(0x14000190000, {0x10104c680, 0x1400582e6e0}, 0x3, 0x140086b0b40, 0x0)
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/server/diagnostics.go:157 +0x4f0
golang.org/x/tools/gopls/internal/server.(*server).didModifyFiles.func2()
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/server/text_synchronization.go:269 +0x3c
created by golang.org/x/tools/gopls/internal/server.(*server).didModifyFiles in goroutine 108
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/server/text_synchronization.go:268 +0x468
[Error - 01:31:32]

gopls stats -anon
{
  "DirStats": {
    "Files": 8067,
    "TestdataFiles": 52,
    "GoFiles": 6683,
    "ModFiles": 4,
    "Dirs": 1348
  },
  "GOARCH": "arm64",
  "GOOS": "darwin",
  "GOPACKAGESDRIVER": "",
  "GOPLSCACHE": "",
  "GoVersion": "go1.22.0",
  "GoplsVersion": "v0.15.1",
  "InitialWorkspaceLoadDuration": "1.65849025s",
  "MemStats": {
    "HeapAlloc": 108681552,
    "HeapInUse": 144924672,
    "TotalAlloc": 294147416
  },
  "WorkspaceStats": {
    "Files": {
      "Total": 6971,
      "Largest": 7361676,
      "Errs": 0
    },
    "Views": [
      {
        "GoCommandVersion": "go1.22.0",
        "AllPackages": {
          "Packages": 1426,
          "LargestPackage": 214,
          "CompiledGoFiles": 8363,
          "Modules": 177
        },
        "WorkspacePackages": {
          "Packages": 593,
          "LargestPackage": 214,
          "CompiledGoFiles": 4074,
          "Modules": 1
        },
        "Diagnostics": 0
      }
    ]
  }
}

@findleyr reproducable repo https://github.com/painhardcore/goplscrash . Just go to tools/tools_test.go and gopls will crash.

Indeed it does! Apparently the file must be in a nested directory. Thank you for the reproducer!

Yep, in our case file tools_test.go was in root folder, so every test file in a big repo was like a trigger 😄

@painhardcore and @amirrmonfared, huge thanks for your conscientious help resolving this bug.  The CL above should fix this crash. However, we have a couple other issues to sort out before we cut<EMAIL_ADDRESS>with this fix.

@painhardcore and @amirrmonfared, huge thanks for your conscientious help resolving this bug.  The CL above should fix this crash. However, we have a couple other issues to sort out before we cut<EMAIL_ADDRESS>with this fix.

Thanks.
I fixed the issue in our repo, so we can work on 0.15.1

Thanks a lot @findleyr for your support and thanks to @painhardcore for solving the issue in our repo ❤

This fix is included in our prerelease:
go install<EMAIL_ADDRESS>


Please describe where we store information

Where are we storing the information? For how long? How can people contact us to get their information removed? (Maybe we need to handle the parts besides storing information.)

Information is stored on TF private server, in TF datacenter. This is technical information. We keep the data as long as marketing desires. Data will be kept as long as campaign is running I suppose?


C++: Structs

Hackerrank problem link
Provide your hackerrank problem link here.
https://www.hackerrank.com/challenges/c-tutorial-struct/problem
Issue Checklist

[x] If you have previously contributed to this repo, you declare that you have not made a pull request for more than one issue in this repo till now.
[x] Your forked repo is up-to-date with the the origin remote.
[x] Have you checked that the hackerrank problem you are raising is not already present in the repo?
[x] Have you checked that the hackerrank problem you are raising is not raised by or assigned to another contributor?

Hacktoberfest Contributor

[x] Are you a Hacktoberfest Contributor?

Any other Changes?
If you are raising an issue with some more changes, please provide the details here.
N/A

I'd like to be asigned this problem please!

@R0merol
Hey, Thanks for raising an issue!
I've assigned you the issue and you may please go ahead!
Happy Coding😃!


InputManager crash because Unity EventSystem.current is null during initialization

Overview
I am hitting a crash in InputManager because the inputEventData.selectedObject can't be evaluated, resulting a null reference exception and I found that it happens because EventSystem.current is null during InputManager initialization. Any ideas why this occurs?
Expected Behavior
No crash
Actual Behavior
Crash
Steps to reproduce
Air tap anywhere results in the crash
Unity Editor Version
Unity 2017.3.1f1
Mixed Reality Toolkit Release Version
HoloToolkit-Unity-2<IP_ADDRESS>

Not sure I can follow the repro steps, I don't know where to begin to get the same result.

@StephenHodgson once you get into this state then the repro steps are straight forward and consistent - the air tap gesture always results in null reference exception because the InputManager instance is not valid.
Like I said, I debugged this and narrowed down to the fact that Unity EventSystem.current is null during InputManager Awake call. Do you know in what circumstances this can occur?

Do you not have an event system in your scene?

I do.
Funny enough, I have noticed this issue somehow gets fixed, if I clone the repo and build (/regenerate) the UWP project again, without any modifications to the code. I am a suspicious this could be bug in the result of the project generator that leads to different output between iterations.
Or maybe an issue with the execution order that results in InputManager being initialized before EventSystem? I wonder, can we safely assume that Unity initializes EventSystem before Awake is called for InputManager?

Sounds about right. You could make sure your script execution order is setup so the event system is executed and created first.


Update python.py

Delete a useless "print"

Who can review?
Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

@dev2049  we need to re-run tests on this, i'm a bit confused about what happened to the test runner. Concerned that there might be some test that relies on having output in stdout

tests look good 👍 https://github.com/hwchase17/langchain/actions/runs/5028185554

thanks @pengwork!


Crash (Fabric): Crash on initialize caused by compat with Tool Stats.

Minecraft 1.19.2
Fabric modloader
Mods:
Forgero 0.10.8
Tool Stats. 12.0.2
Fabric API 0.76.0
Bookshelf 16.2.18
Issue only occurs with Tool Stats present.
Crash log:
crash-2023-04-06_17.04.57-client.txt

It seems like this issue has already been patched, and will be updated in 0.10.9


Devices going into "unavailable" state

I noticed that my Tuya devices go into "unavailable" state. After reloading, integration is restored for some period.
Diagnostic for device that goes to unavailable state most often:
{
  "home_assistant": {
    "installation_type": "Home Assistant OS",
    "version": "2023.4.6",
    "dev": false,
    "hassio": true,
    "virtualenv": false,
    "python_version": "3.10.10",
    "docker": true,
    "arch": "x86_64",
    "timezone": "Europe/Vienna",
    "os_name": "Linux",
    "os_version": "6.1.24",
    "supervisor": "2023.04.1",
    "host_os": "Home Assistant OS 10.0",
    "docker_version": "23.0.3",
    "chassis": "vm",
    "run_as_root": true
  },
  "custom_components": {
    "alarmo": {
      "version": "v1.9.9",
      "requirements": []
    },
    "hacs": {
      "version": "1.32.1",
      "requirements": [
        "aiogithubapi>=22.10.1"
      ]
    },
    "multiscrape": {
      "version": "6.5.0",
      "requirements": [
        "lxml==4.9.1",
        "beautifulsoup4==4.11.1"
      ]
    },
    "panasonic_cc": {
      "version": "1.0.32",
      "requirements": []
    },
    "powercalc": {
      "version": "v1.5.1",
      "requirements": [
        "numpy>=1.21.1"
      ]
    },
    "tuya_local": {
      "version": "2022.4.1",
      "requirements": [
        "pycryptodome~=3.17",
        "tinytuya==1.12.4"
      ]
    },
    "webrtc": {
      "version": "v3.1.0",
      "requirements": []
    },
    "roborock": {
      "version": "0.1.2",
      "requirements": [
        "python-roborock==0.1.7"
      ]
    },
    "browser_mod": {
      "version": "2.2.1",
      "requirements": []
    },
    "scheduler": {
      "version": "v0.0.0",
      "requirements": []
    },
    "watchman": {
      "version": "0.5.1",
      "requirements": [
        "prettytable==3.0.0"
      ]
    }
  },
  "integration_manifest": {
    "domain": "tuya_local",
    "name": "Tuya Local",
    "codeowners": [
      "@make-all"
    ],
    "config_flow": true,
    "dependencies": [],
    "documentation": "https://github.com/make-all/tuya-local",
    "integration_type": "device",
    "iot_class": "local_push",
    "issue_tracker": "https://github.com/make-all/tuya-local/issues",
    "requirements": [
      "pycryptodome~=3.17",
      "tinytuya==1.12.4"
    ],
    "version": "2022.4.1",
    "is_built_in": false
  },
  "data": {
    "name": "Beok TOL47 Main thermostat",
    "type": "beok_tol47_thermostat",
    "device_id": "**REDACTED**",
    "local_key": "**REDACTED**",
    "host": "**REDACTED**",
    "protocol_version": "auto",
    "api_version_set": 3.3,
    "api_version_used": 3.3,
    "api_working": true,
    "status": {},
    "cached_state": {
      "updated_at":<PHONE_NUMBER>.0076742,
      "1": false,
      "2": 150,
      "3": 217,
      "4": "cold",
      "5": "0",
      "8": false,
      "13": true,
      "16": 0,
      "20": -18,
      "21": 30,
      "25": "1",
      "26": false,
      "31": false,
      "41": "1",
      "42": "1",
      "101": false,
      "105": 5,
      "107": 5
    },
    "pending_state": {},
    "connected": true,
    "force_dps": [],
    "home_assistant": {
      "name": "Beok TOL47 Main thermostat",
      "name_by_user": "Main Thermostat",
      "disabled": false,
      "disabled_by": null,
      "entities": [
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": null,
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "climate.main_thermostat",
            "state": "off",
            "attributes": {
              "hvac_modes": [
                "auto",
                "off",
                "heat"
              ],
              "min_temp": 5.0,
              "max_temp": 95.0,
              "target_temp_step": 0.5,
              "preset_modes": [
                "Program",
                "Temporary",
                "Manual"
              ],
              "current_temperature": 21.7,
              "temperature": 15.0,
              "hvac_action": "off",
              "preset_mode": "Manual",
              "fault_code": 0,
              "icon": "mdi:radiator-off",
              "friendly_name": "Main Thermostat",
              "supported_features": 17
            },
            "last_changed": "2023-04-26T09:56:54.371840+00:00",
            "last_updated": "2023-04-26T09:56:54.371840+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": "switch",
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "switch.main_thermostat_anti_frost",
            "state": "off",
            "attributes": {
              "device_class": "switch",
              "icon": "mdi:snowflake-melt",
              "friendly_name": "Main Thermostat Anti-frost"
            },
            "last_changed": "2023-04-26T09:56:54.370657+00:00",
            "last_updated": "2023-04-26T09:56:54.370657+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "select.main_thermostat_backlight_brightness",
            "state": "Low",
            "attributes": {
              "options": [
                "High",
                "Low",
                "None",
                "Middle"
              ],
              "icon": "mdi:lightbulb-on-40",
              "friendly_name": "Main Thermostat Backlight brightness"
            },
            "last_changed": "2023-04-26T09:56:54.372413+00:00",
            "last_updated": "2023-04-26T09:56:54.372413+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": "\u00b0C",
          "state": {
            "entity_id": "number.main_thermostat_calibration_offset",
            "state": "-1.8",
            "attributes": {
              "min": -9.0,
              "max": 9.0,
              "step": 0.1,
              "mode": "auto",
              "unit_of_measurement": "\u00b0C",
              "icon": "mdi:arrow-collapse-up",
              "friendly_name": "Main Thermostat Calibration offset"
            },
            "last_changed": "2023-04-26T09:56:54.372816+00:00",
            "last_updated": "2023-04-26T09:56:54.372816+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "lock.main_thermostat_child_lock",
            "state": "unlocked",
            "attributes": {
              "icon": "mdi:hand-back-right",
              "friendly_name": "Main Thermostat Child lock",
              "supported_features": 0
            },
            "last_changed": "2023-04-26T09:56:54.371102+00:00",
            "last_updated": "2023-04-26T09:56:54.371102+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": "temperature",
          "icon": null,
          "unit_of_measurement": "\u00b0C",
          "state": {
            "entity_id": "number.main_thermostat_external_sensor_temperature_limit",
            "state": "5",
            "attributes": {
              "min": 5,
              "max": 60,
              "step": 1.0,
              "mode": "auto",
              "unit_of_measurement": "\u00b0C",
              "device_class": "temperature",
              "icon": "mdi:arrow-collapse-up",
              "friendly_name": "Main Thermostat External sensor temperature limit"
            },
            "last_changed": "2023-04-26T09:56:54.373413+00:00",
            "last_updated": "2023-04-26T09:56:54.373413+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "diagnostic",
          "device_class": null,
          "original_device_class": "problem",
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "binary_sensor.main_thermostat_fault_alarm",
            "state": "off",
            "attributes": {
              "device_class": "problem",
              "friendly_name": "Main Thermostat Fault alarm"
            },
            "last_changed": "2023-04-26T09:56:54.370950+00:00",
            "last_updated": "2023-04-26T09:56:54.370950+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": "temperature",
          "icon": null,
          "unit_of_measurement": "\u00b0C",
          "state": {
            "entity_id": "number.main_thermostat_high_temperature_limit",
            "state": "30",
            "attributes": {
              "min": 15,
              "max": 95,
              "step": 1.0,
              "mode": "auto",
              "unit_of_measurement": "\u00b0C",
              "device_class": "temperature",
              "icon": "mdi:thermometer-chevron-up",
              "friendly_name": "Main Thermostat High temperature limit"
            },
            "last_changed": "2023-04-26T09:56:54.373007+00:00",
            "last_updated": "2023-04-26T09:56:54.373007+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": "switch",
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "switch.main_thermostat_output_main_reverse",
            "state": "off",
            "attributes": {
              "device_class": "switch",
              "icon": "mdi:swap-horizontal",
              "friendly_name": "Main Thermostat Output main reverse"
            },
            "last_changed": "2023-04-26T09:56:54.370792+00:00",
            "last_updated": "2023-04-26T09:56:54.370792+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "select.main_thermostat_schedule",
            "state": "Weekday+Weekend",
            "attributes": {
              "options": [
                "Weekday+Weekend",
                "Mon-Sat+Sun",
                "Daily"
              ],
              "icon": "mdi:calendar-clock",
              "friendly_name": "Main Thermostat Schedule"
            },
            "last_changed": "2023-04-26T09:56:54.372593+00:00",
            "last_updated": "2023-04-26T09:56:54.372593+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "select.main_thermostat_sensor_selection",
            "state": "Internal",
            "attributes": {
              "options": [
                "Both",
                "External",
                "Internal"
              ],
              "icon": "mdi:home-thermometer",
              "friendly_name": "Main Thermostat Sensor selection"
            },
            "last_changed": "2023-04-26T09:56:54.372069+00:00",
            "last_updated": "2023-04-26T09:56:54.372069+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": "switch",
          "icon": null,
          "unit_of_measurement": null,
          "state": {
            "entity_id": "switch.main_thermostat_sound",
            "state": "on",
            "attributes": {
              "device_class": "switch",
              "icon": "mdi:music-note",
              "friendly_name": "Main Thermostat Sound"
            },
            "last_changed": "2023-04-26T09:56:54.370347+00:00",
            "last_updated": "2023-04-26T09:56:54.370347+00:00"
          }
        },
        {
          "disabled": false,
          "disabled_by": null,
          "entity_category": "config",
          "device_class": null,
          "original_device_class": null,
          "icon": null,
          "unit_of_measurement": "\u00b0C",
          "state": {
            "entity_id": "number.main_thermostat_temperature_control_switch_difference",
            "state": "0.5",
            "attributes": {
              "min": 0.5,
              "max": 9.5,
              "step": 0.1,
              "mode": "auto",
              "unit_of_measurement": "\u00b0C",
              "icon": "mdi:thermometer-plus",
              "friendly_name": "Main Thermostat Temperature control switch difference"
            },
            "last_changed": "2023-04-26T09:56:54.373224+00:00",
            "last_updated": "2023-04-26T09:56:54.373224+00:00"
          }
        }
      ]
    }
  }
}

Same here, started some time ago.

I have six smart plugs, all of them have now state "unavailable".
State periodically return to normal but after some time go to unavailable
Cloud tuya integration works normally...

I also have the same issue

i also have the same issue, the device randomly changes status to unknown and then to unavailable, restarting temporally fix the issue

I think there may be a limit in the number of threads Home Assistant is letting an integration start.  Because the underlying tinytuya library is not async, this integration starts a thread per device to receive updates asynchronously from the device.

@make-all I only have 1 device, the pool heatpump.
Can we activate any additional logging ?

I see this as well, it only started happening recently (last few days), likely related to upgrading tuya-local to latest.
I only have a single pool heater connected and HA shows it as offline - using the Tuya app (Cloud) to change the pool heater state makes the entity in HA come online again but then shortly after it will go offline again.

I have the same issue with just one pool heat pump. @adamkoch  which version you have used before (which was stable for you)? I have switched from local tuya to tuya local due to the device is supported out of the box but if it is not working reliable it will not be that useful. Is there any idea what we can try to log? I'm experiencing this behavior just after a while. The wifi connection to the pump is not the best so my guess is, that is has something to do with a reconnect of the pump. Perhaps the loss of connection is bringing the integration somehow to the limit?

If your protocol version is set to auto, you will get faster recovery times after a network disconnection if you set the protocol version to the version actually used by your device.  If you don't know what that is, download Diagnostics for the device when it is working, and the version detected will be in the diagnostics file.

I have the same issue with just one pool heat pump. @adamkoch which version you have used before (which was stable for you)?

It may have been a red herring. I had never seen the device go "unavailable" for 1+ years and then I upgraded to the latest version and it went unavailable constantly all day. However, for whatever reason, it seems to be stable again today and no longer going offline. I will report back in case I see more issues going forward.

Same problem here. Tonight all my 16 devices went unavailable  :(
Only restarting HA did bring them back to live :(

If your protocol version is set to auto, you will get faster recovery times after a network disconnection if you set the protocol version to the version actually used by your device. If you don't know what that is, download Diagnostics for the device when it is working, and the version detected will be in the diagnostics file.

I've tried to set protocol to version 3.3 and this is much more stable. I still sometimes get an unavailable state, but it comes back very quick...

I'm new to HA and started to use tuya-local a few weeks ago and already "became" with this behave, which is very annoying, my automation fails constantly.
However, my devices are set with protocol = auto.
Changing to the correct protocol and crossing my fingers to reduce this behavior.

I noticed this as well, restarting home assistant seems to fix it until it happens again after a period of time.

@make-all  , just a note.
I'm trying since yesterday to change thru the interface the protocol version of my devices.
They are working and available, however, when I go to the configuration and change the protocol from "auto" to the correct version they give an error that aren't able to connect, the problem should be interment...
I restarted my HA, my S.O. and the problem persists...
To force the protocol I was forced to shutdown my HA and change manually the core.config_entries file.

Same issue here,
Slowly, all my devices becomes "unavailable",
I might corelate this behavior with the upgrade of HA to 2023.5

@georgef333 I have the issue for home assistant 2023.4.6 as well. Before 2023.4 I have not used the integration so I cannot say to much.

The workaround is as @make-all suggested, figure out the protocol and configure it directly, haven't had a device going unavailable in over a week.

The workaround is as @make-all suggested, figure out the protocol and configure it directly, haven't had a device going unavailable in over a week.

Can confirm my devices aren't going offline with this

Confirm working after updating tinytuya to 1.12.7,
Thanks!

Confirm working after updating tinytuya to 1.12.7, Thanks!

Hallo how to uodate tintuya? Whats tintuya?

If you use HACS, tinytuya is automatically updated to match the version specified in the manifest when you upgrade to the latest version of this integration.
If you install manually, you should understand how to also install the dependency libraries.

The logger documentation is here: https://www.home-assistant.io/integrations/logger/
Tuya local is under custom_components.tuya_local, for communication issues, probably just enabling debug on custom_components.tuya_local.device is enough.  The low level communication is handled by tinytuya, and you should be able to enable debug logging for that also if you cannot spot anything at the tuya-local level.

Hi, I am also having this issue, some of my ceiling lights are going unavailable very often.
Can this be cause by not selecting the correct device type by any chance?


Compile error when build velox with -mavx512f flag

Problem description
Here is my CMAKE_CXX_FLAGS opinion.
Setting CMAKE_CXX_FLAGS=-mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -mavx512f
FAILED: velox/common/base/tests/CMakeFiles/velox_base_test.dir/SimdUtilTest.cpp.o
ccache /usr/bin/c++  -DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_DYN_LINK -DBOOST_CONTEXT_DYN_LINK -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_REGEX_DYN_LINK -DBOOST_SYSTEM_DYN_LINK -DBOOST_THREAD_DYN_LINK -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -DVELOX_ENABLE_PARQUET -I../../. -I../../velox/external/xxhash -I../../third_party/xsimd/include -isystem ../../velox -isystem ../../velox/external -isystem ../../velox/external/duckdb -isystem ../../velox/external/duckdb/tpch/dbgen/include -isystem /usr/include/libdwarf -isystem /usr/include/libiberty -isystem ../../third_party/googletest/googletest/include -isystem ../../third_party/googletest/googletest -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -mavx512f -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-implicit-fallthrough          -Wno-empty-body          -Wno-class-memaccess          -Wno-comment          -Wno-int-in-bool-context          -Wno-redundant-move          -Wno-array-bounds          -Wno-maybe-uninitialized          -Wno-unused-result          -Wno-format-overflow          -Wno-strict-aliasing          -Wno-type-limits          -Wno-stringop-overflow -g -fPIE   -std=gnu++17 -MD -MT velox/common/base/tests/CMakeFiles/velox_base_test.dir/SimdUtilTest.cpp.o -MF velox/common/base/tests/CMakeFiles/velox_base_test.dir/SimdUtilTest.cpp.o.d -o velox/common/base/tests/CMakeFiles/velox_base_test.dir/SimdUtilTest.cpp.o -c ../../velox/common/base/tests/SimdUtilTest.cpp
In file included from ../.././velox/common/base/SimdUtil.h:419,
                 from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil-inl.h: In instantiation of ‘int32_t facebook::velox::simd::indicesOfSetBits(const uint64_t*, int32_t, int32_t, int32_t*, const A&) [with A = xsimd::avx512f; int32_t = int; uint64_t = long unsigned int]’:
../../velox/common/base/tests/SimdUtilTest.cpp:65:70:   required from here
../.././velox/common/base/SimdUtil-inl.h:222:39: error: static assertion failed
  222 |             static_assert(Batch::size == 4);
      |                                  ~~~~~^~~~
In file included from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil.h: In instantiation of ‘xsimd::batch<T, A> facebook::velox::simd::getHalf(xsimd::batch<From, A>, const A&) [with To = long unsigned int; bool kSecond = true; From = int; A = xsimd::avx512f]’:
../../velox/common/base/tests/SimdUtilTest.cpp:280:50:   required from here
../.././velox/common/base/SimdUtil.h:333:63: error: no matching function for call to ‘facebook::velox::simd::detail::GetHalf<long unsigned int, int, xsimd::avx512f>::apply<true>(xsimd::batch<int>&, const xsimd::avx512f&)’
  333 |   return detail::GetHalf<To, From, A>::template apply<kSecond>(data, arch);
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
In file included from ../.././velox/common/base/SimdUtil.h:419,
                 from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil-inl.h:907:36: note: candidate: ‘static xsimd::batch<long unsigned int, A> facebook::velox::simd::detail::GetHalf<long unsigned int, int, A>::apply(xsimd::batch<int, A>, const xsimd::avx2&) [with bool kSecond = true; A = xsimd::avx512f]’
  907 |   static xsimd::batch<uint64_t, A> apply(
      |                                    ^~~~~
../.././velox/common/base/SimdUtil-inl.h:909:7: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::avx2&’
  909 |       const xsimd::avx2&) {
      |       ^~~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:916:36: note: candidate: ‘static xsimd::batch<long unsigned int, A> facebook::velox::simd::detail::GetHalf<long unsigned int, int, A>::apply(xsimd::batch<int, A>, const xsimd::sse4_1&) [with bool kSecond = true; A = xsimd::avx512f]’
  916 |   static xsimd::batch<uint64_t, A> apply(
      |                                    ^~~~~
../.././velox/common/base/SimdUtil-inl.h:918:7: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::sse4_1&’
  918 |       const xsimd::sse4_1&) {
      |       ^~~~~~~~~~~~~~~~~~~~
In file included from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil.h: In instantiation of ‘uint32_t facebook::velox::simd::crc32U64(uint32_t, uint64_t, const A&) [with A = xsimd::avx512f; uint32_t = unsigned int; uint64_t = long unsigned int]’:
../../velox/common/base/tests/SimdUtilTest.cpp:317:41:   required from here
../.././velox/common/base/SimdUtil.h:344:43: error: no matching function for call to ‘facebook::velox::simd::detail::Crc32<long unsigned int, xsimd::avx512f>::apply(uint32_t&, uint64_t&, const xsimd::avx512f&)’
  344 |   return detail::Crc32<uint64_t, A>::apply(checksum, value, arch);
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
In file included from ../.././velox/common/base/SimdUtil.h:419,
                 from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil-inl.h:1020:3: note: candidate: ‘static uint32_t facebook::velox::simd::detail::Crc32<long unsigned int, A>::apply(uint32_t, uint64_t, const xsimd::sse4_2&) [with A = xsimd::avx512f; uint32_t = unsigned int; uint64_t = long unsigned int]’
 1020 |   apply(uint32_t checksum, uint64_t value, const xsimd::sse4_2&) {
      |   ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1020:44: note:   no known conversion for argument 3 from ‘const xsimd::avx512f’ to ‘const xsimd::sse4_2&’
 1020 |   apply(uint32_t checksum, uint64_t value, const xsimd::sse4_2&) {
      |                                            ^~~~~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1026:19: note: candidate: ‘static uint32_t facebook::velox::simd::detail::Crc32<long unsigned int, A>::apply(uint32_t, uint64_t, const xsimd::avx&) [with A = xsimd::avx512f; uint32_t = unsigned int; uint64_t = long unsigned int]’
 1026 |   static uint32_t apply(uint32_t checksum, uint64_t value, const xsimd::avx&) {
      |                   ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1026:60: note:   no known conversion for argument 3 from ‘const xsimd::avx512f’ to ‘const xsimd::avx&’
 1026 |   static uint32_t apply(uint32_t checksum, uint64_t value, const xsimd::avx&) {
      |                                                            ^~~~~~~~~~~~~~~~~
In file included from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil.h: In substitution of ‘template<class T, class A> using HalfBatch = typename facebook::velox::simd::detail::HalfBatchImpl<T, A>::Type [with T = int; A = xsimd::avx512f]’:
../.././velox/common/base/SimdUtil-inl.h:991:26:   required from ‘struct facebook::velox::simd::detail::Filter<int, xsimd::avx512f, 4>’
../.././velox/common/base/SimdUtil.h:309:37:   required from ‘xsimd::batch<U, A> facebook::velox::simd::filter(xsimd::batch<U, A>, BitMaskType, const A&) [with T = int; BitMaskType = int; A = xsimd::avx512f]’
../../velox/common/base/tests/SimdUtilTest.cpp:218:29:   required from ‘void {anonymous}::{anonymous}::testFilter(std::initializer_list<_Tp>) [with T = int]’
../../velox/common/base/tests/SimdUtilTest.cpp:251:73:   required from here
../.././velox/common/base/SimdUtil.h:85:7: error: invalid use of incomplete type ‘struct facebook::velox::simd::detail::HalfBatchImpl<int, xsimd::avx512f, void>’
   85 | using HalfBatch = typename detail::HalfBatchImpl<T, A>::Type;
      |       ^~~~~~~~~
../.././velox/common/base/SimdUtil.h:78:8: note: declaration of ‘struct facebook::velox::simd::detail::HalfBatchImpl<int, xsimd::avx512f, void>’
   78 | struct HalfBatchImpl;
      |        ^~~~~~~~~~~~~
../.././velox/common/base/SimdUtil.h: In substitution of ‘template<class T, class A> using HalfBatch = typename facebook::velox::simd::detail::HalfBatchImpl<T, A>::Type [with T = float; A = xsimd::avx512f]’:
../.././velox/common/base/SimdUtil-inl.h:991:26:   required from ‘struct facebook::velox::simd::detail::Filter<float, xsimd::avx512f, 4>’
../.././velox/common/base/SimdUtil.h:309:37:   required from ‘xsimd::batch<U, A> facebook::velox::simd::filter(xsimd::batch<U, A>, BitMaskType, const A&) [with T = float; BitMaskType = int; A = xsimd::avx512f]’
../../velox/common/base/tests/SimdUtilTest.cpp:218:29:   required from ‘void {anonymous}::{anonymous}::testFilter(std::initializer_list<_Tp>) [with T = float]’
../../velox/common/base/tests/SimdUtilTest.cpp:260:68:   required from here
../.././velox/common/base/SimdUtil.h:85:7: error: invalid use of incomplete type ‘struct facebook::velox::simd::detail::HalfBatchImpl<float, xsimd::avx512f, void>’
   85 | using HalfBatch = typename detail::HalfBatchImpl<T, A>::Type;
      |       ^~~~~~~~~
../.././velox/common/base/SimdUtil.h:78:8: note: declaration of ‘struct facebook::velox::simd::detail::HalfBatchImpl<float, xsimd::avx512f, void>’
   78 | struct HalfBatchImpl;
      |        ^~~~~~~~~~~~~
In file included from ../.././velox/common/base/SimdUtil.h:419,
                 from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../.././velox/common/base/SimdUtil-inl.h: In instantiation of ‘xsimd::batch<U, A> facebook::velox::simd::reinterpretBatch(xsimd::batch<T, A>, const A&) [with T = unsigned char; U = signed char; A = xsimd::avx512f]’:
../../velox/common/base/tests/SimdUtilTest.cpp:360:42:   required from ‘void {anonymous}::validateReinterpretBatch() [with T = signed char]’
../../velox/common/base/tests/SimdUtilTest.cpp:373:36:   required from here
../.././velox/common/base/SimdUtil-inl.h:1253:50: error: no matching function for call to ‘facebook::velox::simd::detail::ReinterpretBatch<unsigned char, signed char, xsimd::avx512f>::apply(xsimd::batch<signed char, xsimd::avx512f>&, const xsimd::avx512f&)’
 1253 |   return detail::ReinterpretBatch<T, U, A>::apply(data, arch);
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::sse2&) [with T = unsigned char; U = signed char; A = xsimd::avx512f]’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::sse2&’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                                                            ^~~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::avx&) [with T = unsigned char; U = signed char; A = xsimd::avx512f]’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::avx&’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                                                            ^~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h: In instantiation of ‘xsimd::batch<U, A> facebook::velox::simd::reinterpretBatch(xsimd::batch<T, A>, const A&) [with T = short unsigned int; U = short int; A = xsimd::avx512f]’:
../../velox/common/base/tests/SimdUtilTest.cpp:360:42:   required from ‘void {anonymous}::validateReinterpretBatch() [with T = short int]’
../../velox/common/base/tests/SimdUtilTest.cpp:374:37:   required from here
../.././velox/common/base/SimdUtil-inl.h:1253:50: error: no matching function for call to ‘facebook::velox::simd::detail::ReinterpretBatch<short unsigned int, short int, xsimd::avx512f>::apply(xsimd::batch<short int>&, const xsimd::avx512f&)’
 1253 |   return detail::ReinterpretBatch<T, U, A>::apply(data, arch);
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::sse2&) [with T = short unsigned int; U = short int; A = xsimd::avx512f]’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::sse2&’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                                                            ^~~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::avx&) [with T = short unsigned int; U = short int; A = xsimd::avx512f]’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::avx&’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                                                            ^~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h: In instantiation of ‘xsimd::batch<U, A> facebook::velox::simd::reinterpretBatch(xsimd::batch<T, A>, const A&) [with T = unsigned int; U = int; A = xsimd::avx512f]’:
../../velox/common/base/tests/SimdUtilTest.cpp:360:42:   required from ‘void {anonymous}::validateReinterpretBatch() [with T = int]’
../../velox/common/base/tests/SimdUtilTest.cpp:375:37:   required from here
../.././velox/common/base/SimdUtil-inl.h:1253:50: error: no matching function for call to ‘facebook::velox::simd::detail::ReinterpretBatch<unsigned int, int, xsimd::avx512f>::apply(xsimd::batch<int>&, const xsimd::avx512f&)’
 1253 |   return detail::ReinterpretBatch<T, U, A>::apply(data, arch);
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::sse2&) [with T = unsigned int; U = int; A = xsimd::avx512f]’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::sse2&’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                                                            ^~~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::avx&) [with T = unsigned int; U = int; A = xsimd::avx512f]’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::avx&’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                                                            ^~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h: In instantiation of ‘xsimd::batch<U, A> facebook::velox::simd::reinterpretBatch(xsimd::batch<T, A>, const A&) [with T = long unsigned int; U = long int; A = xsimd::avx512f]’:
../../velox/common/base/tests/SimdUtilTest.cpp:360:42:   required from ‘void {anonymous}::validateReinterpretBatch() [with T = long int]’
../../velox/common/base/tests/SimdUtilTest.cpp:376:37:   required from here
../.././velox/common/base/SimdUtil-inl.h:1253:50: error: no matching function for call to ‘facebook::velox::simd::detail::ReinterpretBatch<long unsigned int, long int, xsimd::avx512f>::apply(xsimd::batch<long int>&, const xsimd::avx512f&)’
 1253 |   return detail::ReinterpretBatch<T, U, A>::apply(data, arch);
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::sse2&) [with T = long unsigned int; U = long int; A = xsimd::avx512f]’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1074:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::sse2&’
 1074 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::sse2&) {
      |                                                            ^~~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:29: note: candidate: ‘static xsimd::batch<U, A> facebook::velox::simd::detail::ReinterpretBatch<T, U, A>::apply(xsimd::batch<T, A>, const xsimd::avx&) [with T = long unsigned int; U = long int; A = xsimd::avx512f]’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                             ^~~~~
../.././velox/common/base/SimdUtil-inl.h:1080:60: note:   no known conversion for argument 2 from ‘const xsimd::avx512f’ to ‘const xsimd::avx&’
 1080 |   static xsimd::batch<T, A> apply(xsimd::batch<U, A> data, const xsimd::avx&) {
      |                                                            ^~~~~~~~~~~~~~~~~
../.././velox/common/base/SimdUtil-inl.h: In instantiation of ‘xsimd::batch_bool<T, A> facebook::velox::simd::detail::fromBitMaskImpl(int) [with T = int; A = xsimd::avx512f]’:
../.././velox/common/base/SimdUtil-inl.h:137:61:   required from ‘static xsimd::batch_bool<T, A> facebook::velox::simd::detail::BitMask<T, A, 4>::fromBitMask(int, const A&) [with T = int; A = xsimd::avx512f]’
../.././velox/common/base/SimdUtil.h:289:44:   required from ‘xsimd::batch_bool<T_out, A> facebook::velox::simd::fromBitMask(BitMaskType, const A&) [with T = int; BitMaskType = int; A = xsimd::avx512f]’
../../velox/common/base/tests/SimdUtilTest.cpp:294:48:   required from here
../.././velox/common/base/SimdUtil-inl.h:48:21: error: static assertion failed
   48 |     static_assert(N <= 8);
      |                   ~~^~~~
In file included from ../../third_party/xsimd/include/xsimd/xsimd.hpp:61,
                 from ../.././velox/common/base/SimdUtil.h:24,
                 from ../../velox/common/base/tests/SimdUtilTest.cpp:17:
../../third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:122:22: error: inline function ‘xsimd::batch<int> xsimd::operator+(const xsimd::batch<int>&, const xsimd::batch<int>&)’ used but never defined [-Werror]
  122 |         friend batch operator+(batch const& self, batch const& other) noexcept
      |                      ^~~~~~~~
cc1plus: all warnings being treated as errors

System information
Velox System Info v0.0.2
Commit: 94644c62f34daf966d32dadb5a0e059530eeaff9
CMake Version: 3.16.3
System: Linux-5.4.0-144-generic
Arch: x86_64
C++ Compiler: /usr/bin/c++
C++ Compiler Version: 9.4.0
C Compiler: /usr/bin/cc
C Compiler Version: 9.4.0
CMake Prefix Path: /usr/local;/usr;/;/usr;/usr/local;/usr/X11R6;/usr/pkg;/opt
Conda Env

# packages in environment at /mnt/DP_disk1/zzh/miniconda3:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
_openmp_mutex             4.5                       1_gnu    defaults
alabaster                 0.7.13                   pypi_0    pypi
babel                     2.11.0                   pypi_0    pypi
brotlipy                  0.7.0           py39h27cfd23_1003    defaults
ca-certificates           2022.3.29            h06a4308_1    defaults
certifi                   2021.10.8        py39h06a4308_2    defaults
cffi                      1.15.0           py39hd667e15_1    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.1.3                    pypi_0    pypi
colorama                  0.4.4              pyhd3eb1b0_0    defaults
conda                     4.12.0           py39h06a4308_0    defaults
conda-content-trust       0.1.1              pyhd3eb1b0_0    defaults
conda-package-handling    1.8.1            py39h7f8727e_0    defaults
cryptography              36.0.0           py39h9ce1e76_0    defaults
docutils                  0.19                     pypi_0    pypi
duckdb                    0.6.2.dev623             pypi_0    pypi
idna                      3.3                pyhd3eb1b0_0    defaults
imagesize                 1.4.1                    pypi_0    pypi
importlib-metadata        6.0.0                    pypi_0    pypi
jdot                      0.5                      pypi_0    pypi
jinja2                    3.1.2                    pypi_0    pypi
ld_impl_linux-64          2.35.1               h7274673_9    defaults
libffi                    3.3                  he6710b0_2    defaults
libgcc-ng                 9.3.0               h5101ec6_17    defaults
libgomp                   9.3.0               h5101ec6_17    defaults
libstdcxx-ng              9.3.0               hd4cf53a_17    defaults
markupsafe                2.1.1                    pypi_0    pypi
ncurses                   6.3                  h7f8727e_2    defaults
numpy                     1.23.5                   pypi_0    pypi
openssl                   1.1.1n               h7f8727e_0    defaults
packaging                 23.0                     pypi_0    pypi
pip                       21.2.4           py39h06a4308_0    defaults
protobuf                  3.20.2                   pypi_0    pypi
pycosat                   0.6.3            py39h27cfd23_0    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pygments                  2.14.0                   pypi_0    pypi
pyopenssl                 22.0.0             pyhd3eb1b0_0    defaults
pysocks                   1.7.1            py39h06a4308_0    defaults
python                    3.9.12               h12debd9_0    defaults
pytz                      2022.7                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.1.2                h7f8727e_1    defaults
regex                     2022.10.31               pypi_0    pypi
requests                  2.27.1             pyhd3eb1b0_0    defaults
ruamel_yaml               0.15.100         py39h27cfd23_0    defaults
setuptools                61.2.0           py39h06a4308_0    defaults
six                       1.16.0             pyhd3eb1b0_1    defaults
snowballstemmer           2.2.0                    pypi_0    pypi
sphinx                    6.1.3                    pypi_0    pypi
sphinx-rtd-theme          0.5.1                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.3                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                    pypi_0    pypi
sphinxcontrib-htmlhelp    2.0.0                    pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.3                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.5                    pypi_0    pypi
sqlite                    3.38.2               hc218d9a_0    defaults
substrait-validator       0.0.8                    pypi_0    pypi
tk                        8.6.11               h1ccaba5_0    defaults
tqdm                      4.63.0             pyhd3eb1b0_0    defaults
tzdata                    2022a                hda174b7_0    defaults
urllib3                   1.26.8             pyhd3eb1b0_0    defaults
wheel                     0.37.1             pyhd3eb1b0_0    defaults
xz                        5.2.5                h7b6447c_0    defaults
yaml                      0.2.5                h7b6447c_0    defaults
zipp                      3.11.0                   pypi_0    pypi
zlib                      1.2.12               h7f8727e_1    defaults

CMake log
No response

Did velox make some changes that caused the compilation to fail?

CC: @Yuhta @kgpai @majetideepak

This is by design, some code is not adapted to AVX512.  You can add some individual compilation unit to be compiled with AVX512 enabled, but adding it globally requires some work.

@Yuhta @mbasmanova @majetideepak Are there plans for implementing full AVX512 support? I just can tell that the gain seems massive as compared with ARROW - instructions execute at least an order of magnitude faster so it would be kind of game-changer  - having in mind that engines like Presto won't utilize the Java 18: Vector API soon, as there is no special effort put in porting it for Java higher then version 11, Spark seems closer to this - theoretically supports JAVA 17.
This speed-up can be achieved in most data-centers running on Intel Xeons for example :-).

@Mionsz We did some experiments back a while and AVX512 was exhibiting downclocking behavior on our servers, making the overall gain very little.  That is why we deprioritize adopting of AVX512.  You can try add support for it if you have time, it should not be too hard but it's also not trivial.

We have not seen any great win from avx512 over avx2. Maybe 1.3x is the highest we have been notified about. Gather for example, last time we checked, did worse than avx2. But could have changed since.  Most but not all servers have this. AMD Milan does not.
So, where is there an order of magnitude? What about dropping clock frequency for avx512?
Thanks
Orri
From: Miłosz Linkiewicz @.>
Sent: Saturday, March 18, 2023 7:37 PM
To: facebookincubator/velox @.>
Cc: oerling @.>; Mention @.>
Subject: Re: [facebookincubator/velox] Compile error when build velox with -mavx512f flag (Issue #4267)
@Yuhta https://github.com/Yuhta  @mbasmanova https://github.com/mbasmanova  @majetideepak https://github.com/majetideepak  Are there plans for implementing full AVX512 support? I just can tell that the gain seems massive as compared with ARROW - instructions execute at least an order of magnitude faster so it would be kind of game-changer - having in mind that engines like Presto won't utilize the Java 18: Vector API soon, as there is no special effort put in porting it for Java higher then version 11, Spark seems closer to this - theoretically supports JAVA 17.
This speed-up can be achieved in most data-centers running on Intel Xeons for example :-).
—
Reply to this email directly, view it on GitHub https://github.com/facebookincubator/velox/issues/4267#issuecomment-1475077902 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AKPPPT4RSQE3UDELN2HGR5DW4ZWLDANCNFSM6AAAAAAVYVKP6A .
You are receiving this because you were mentioned.Message ID: @.***>


Prefixed property and comment at end of line issue

When there's a prefixed property, and a comment at the end of the same line, the following error is hit. I've also included a CSS sample, which caused this issue for me. It's pulled from normalize.css.
TypeError: Cannot read property 'indexOf' of undefined
    at path/to/project/node_modules/perfectionist/dist/index.js:112:37
    at Array.filter (native)
    at path/to/project/node_modules/perfectionist/dist/index.js:110:32
    at Array.forEach (native)
    at path/to/project/node_modules/perfectionist/dist/index.js:107:26
    at path/to/project/node_modules/perfectionist/dist/index.js:121:15
    at path/to/project/node_modules/gulp-postcss/node_modules/postcss/lib/container.js:102:26
    at Root.each (path/to/project/node_modules/gulp-postcss/node_modules/postcss/lib/container.js:89:22)
    at Root.eachInside (path/to/project/node_modules/gulp-postcss/node_modules/postcss/lib/container.js:101:21)
    at applyExpanded (path/to/project/node_modules/perfectionist/dist/index.js:84:9)

html {
  font-family: sans-serif; /* 1 */
  -ms-text-size-adjust: 100%; /* 2 */
  -webkit-text-size-adjust: 100%; /* 2 */
}

Thanks!

Thanks for reporting, will take a look soon.

Released as 1.0.2.

Thanks for the quick turnaround!


How to restrict dragging of widget inside a specific container or a prarent container & not allow dragging all over the phone screen ?

I want to restrict the dragging behaviour inside a container only but when we try to reorder here we can drag the widgets all over the screen.
Below is my widget tree

Column[Container(child : xyzWidget), Container( child : ReorderableGridView) ]

My requirment is if user wants to reorder the widgets of ReorderableGridView he should be able to drag only within ReorderableGridView's parent container, he should not be able to drag all over the phone screen.
Please suggest your answers
Thanks  in advance

The reason is, this library use Navigator's Overlay.
But I have no time to try to fix this recently. Sorry.

Please try version ~2.2.6-alpha.3~
Fixed ReorderableGridView. Not in ReorderableSliverGridView.
And you can see demo/demo_custom if you use ReorderableWrapperWidget

2.2.6-alpha.3 intro a new issue https://github.com/huhuang03/reorderable_grid_view/issues/54.
So don't use 2.2.6-alpha.3. And this issue is not resolved...

Fixed in 2.2.6-alpha.12.
Work in ReorderableGridView, not work in ReorderableSliverGridView, Please reopen a new issue if you want this in ReorderableSliverGridView


Fix line-directive print errors on different python versions

Most Python files have
from __future__ import print_function

but this one doesn't, even though it has
print(map_line_from_source_file(source_file, source_line, target_file))

@swift-ci please smoke test

This broke the doctests, that you get by running the tool with no arguments:
**********************************************************************
File "/Users/Shared/dabrahams/s/swift-master/utils/line-directive", line 231, in __main__.run
Failed example:
    print output.replace(target1.name,'TARGET1-NAME').replace(target2.name,'TARGET2-NAME') + 'EOF'
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest __main__.run[13]>", line 1
        print output.replace(target1.name,'TARGET1-NAME').replace(target2.name,'TARGET2-NAME') + 'EOF'
                   ^
    SyntaxError: invalid syntax
**********************************************************************
File "/Users/Shared/dabrahams/s/swift-master/utils/line-directive", line 247, in __main__.run
Failed example:
    print subprocess.check_output([__file__, 'foo.bar', '21', target1.name]),
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest __main__.run[14]>", line 1
        print subprocess.check_output([__file__, 'foo.bar', '21', target1.name]),
                       ^
    SyntaxError: invalid syntax
**********************************************************************
File "/Users/Shared/dabrahams/s/swift-master/utils/line-directive", line 249, in __main__.run
Failed example:
    print subprocess.check_output([__file__, 'foo.bar', '8', target2.name]),
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/doctest.py", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest __main__.run[15]>", line 1
        print subprocess.check_output([__file__, 'foo.bar', '8', target2.name]),
                       ^
    SyntaxError: invalid syntax
**********************************************************************
1 items had failures:
   3 of  16 in __main__.run
***Test Failed*** 3 failures.


sqlsrv_errors, error message returns some driver version

Hello,
Issue
When i try to get error message (for example: raiserror on trigger) i get message with driver version included:
$errors = sqlsrv_errors();
print_r($errors);
die();

the result is:
Array
(
    [0] => Array
        (
            [0] => 42000
            [SQLSTATE] => 42000
            [1] => 50000
            [code] => 50000
            [2] => [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]My Error Text
            [message] => [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]My Error Text
        )
)

Question

[Microsoft][ODBC Driver 13 for SQL Server][SQL Server] - Why this going on message?
Why i get the same information 0 AND SQLSTATE, 1 AND CODE...

System info
PHP version: 7.1.5
Linux version: Ubuntu 16.04.2 LTS
sqlsrv_client_info:
[DriverName] => libmsodbcsql-13.1.so.8.0
[DriverODBCVer] => 03.52
[DriverVer] => 13.01.0008
[ExtensionVer] => <IP_ADDRESS>
sqlsrv_server_info:
[SQLServerVersion] => 12.00.2000

@genyslt ,
[Microsoft][ODBC Driver 13 for SQL Server][SQL Server] means the error comes from the ODBC driver. (for instance, if you try to bind binary data but your encoding is utf-8, the ODBC will complain about conversion problems. The SQLSRV driver passes the binary data and the encoding to the ODBC driver but does not check whether the conversion is valid).
The message shown in [0] and [SQLSTATE] are the same. It is designed this way so the user can access the information using a numeric key or a associative key.

Why this ODBC complain about conversion problems written on message key? The message must be original message from SQL Server. Any complaints can go to other key?

@genyslt,
The SQLSRV driver is a wrapper around the Microsoft ODBC driver. You can see here for a list of possible errors from the Microsoft ODBC driver. The message doesn't have to be the original message from the SQL Server because the error could come from the SQL Server, the Microsoft ODBC Driver, or the PHP SQLSRV Driver. Please see here for the return values of sqlsrv_error().

An because of that message that contains driver version etc. you can't use RAISERROR or THROW inside your Stored Procedure to return an error back to APP.
Instead you need to use some SELECT or OUTPUT parameter to forward your message back to APP.
Did someone was thinking about that we can't use a proper error handling because of that?
My customer doesn't need to know ODBC version when I use:
;THROW 50000, @errormsg, 1;


Migrating from sodiumoxide to ring

What do you think about the idea of migrating this crate's crypto primitives to the ones from ring? I'm not sure if it supports everything that is needed yet. But it would be nice to be able to drop the libsodium dependency.

In general I prefer ring to sodiumoxide, however the Macaroons format presently uses Salsa20Poly1305, which ring does not support.


ci: use yq to extract version number from the TOML file

https://github.com/PRQL/prql/pull/3122#issuecomment-1653053214
There is no need to remove spaces or quotation marks because yq correctly parses the TOML and extracts the version number.

It does not seem working on Windows
  ./.github/set_version.sh
  shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
  env:
    CARGO_TERM_COLOR: always
    CLICOLOR_FORCE: 1
    RUSTFLAGS: -C debuginfo=0
./.github/set_version.sh: line 5: yq: command not found

I thought yq was available on GitHub Actions, but it doesn't seem to be included on Windows at this time. (actions/runner-images#7443)
https://github.com/actions/runner-images/blob/win22/20230724.1/images/win/Windows2022-Readme.md

FYI cargo metadata & jq is probably the "optimizer" (not necessarily good!) approach...

FYI cargo metadata & jq is probably the "optimizer" (not necessarily good!) approach...

Good idea! Thanks.


Can't select Transliterated text from Suggestion.

Environment:
Magento 2.1.4 EE
Craftyclicks_Clicktoaddress (ver: 1.1.1)
Steps to reproduce:

Set "Transliterate for non ASCII characters" to "Yes" under Store -> Configuration -> CRAFTY CLICKS -> Click to Address -> Advanced in the admin panel.
In the frontend go to checkout and try to search for an address (600028: India postal code in our case).
Click through multiple levels of suggestions

Expected Result:
When selecting the last suggestion, the address should be auto-filled in the checkout address form.
Actual Result:
The following Notice is thrown when trying to select the address suggestion:
"CraftyClicks Debug Error Message: [JS503] API address retrieve request resulted in a JS error."

Points to be noted:

When "Transliterate for non ASCII characters" is set to "No", all work fine.
First two levels of suggestion are selectable, only the last level throws the above error.
JS minification and merge are enabled in Magento.

Thanks for reporting this issue.
We identified this to be a problem with our underlying clicktoaddress-js library.
The latest pushed commit on master (v1.1.2) has been now fully tested and resolves this issue.
Do let us know if you still see any issues around transliteration.


Error thrown when uploading video from a url source, but video is still uploaded

Expected Behavior
Uploading a file and waiting for the task id and the task to finish shouldn't throw an error
Actual Behavior
Uploading a file and waiting for the task to finish throws this error, BUT the file upload succeeds...
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/Users/ashleyxue/work/www/backend/ads/helpers.py", line 133, in upload_test
    upload_task = upload_video(source=source)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ashleyxue/work/www/backend/ads/helpers.py", line 114, in upload_video
    task = tl_client.task.create(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ashleyxue/work/www/venv/lib/python3.11/site-packages/twelvelabs/resources/task.py", line 139, in create
    res = self._post(
          ^^^^^^^^^^^
  File "/Users/ashleyxue/work/www/venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 97, in post
    return self._request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ashleyxue/work/www/venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 38, in _request
    raise self._make_status_error(e.response)
twelvelabs.exceptions.InternalServerError: Error code: 500 - runtime error: invalid memory address or nil pointer dereference

Steps to Reproduce
Running this, with source being some video url source. I was using "https://video-iad3-1.xx.fbcdn.net/v/t42.1790-2/370851398_210657385060503_7699108706487054458_n.mp4?_nc_cat=108&ccb=1-7&_nc_sid=c53f8f&_nc_ohc=9RHMZG_n6l4Q7kNvgFGlAMH&_nc_ht=video-iad3-1.xx&oh=00_AYDv1bdSjwF8832d6cznwr-SWyOtG2csaQz73F5y4CUepQ&oe=664C6CA3" to test.
twelvelabs==0.1.23
task = tl_client.task.create(
      index_id=<INDEX_ID>,
      url=source,
  )

print(f"Task id={upload_task.id}")

def on_task_update(task: Task):
      print(f"  Status={upload_task.status}")

upload_task.wait_for_done(sleep_interval=5, callback=on_task_update)

if upload_task.status != "ready":
      raise RuntimeError(f"Indexing failed with status {upload_task.status}")
print(f"The unique identifer of your video is {upload_task.video_id}.")

Detailed Description
N/A
Possible Solution
I'm not really sure what's happening, but seems like something with the tasks endpoint? Unsure if I'm just missing something when initializing or calling the method.

I also tried using the endpoint directly, copy pasted: https://docs.twelvelabs.io/reference/create-video-indexing-task
And am getting this error:
'{"code":"video_resolution_too_high","message":"The resolution of the video is too high. Please use a video with resolution between 360p(480x360) and 2160p(3840x2160). Current resolution is 360p."}\n'

I get a different error when I call task.create. The URL that @ashleyxue529 used as a source has expired, but https://metadaddy-public.s3.us-west-004.backblazeb2.com/minnie01.mov has no expiry.
Source:
url = 'https://metadaddy-public.s3.us-west-004.backblazeb2.com/minnie01.mov'
task = tl_client.task.create(
    TWELVE_LABS_INDEX_ID,
    url=url
)

Error:
% python quick_test.py
Traceback (most recent call last):
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 31, in _request
    response.raise_for_status()
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/httpx/_models.py", line 758, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api.twelvelabs.io/v1.2/tasks'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ppatterson/src/b2-twelvelabs-example/quick_test.py", line 39, in <module>
    task = TWELVE_LABS_CLIENT.task.create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/resources/task.py", line 143, in create
    res = self._post(
          ^^^^^^^^^^^
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 107, in post
    return self._request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 35, in _request
    raise self._make_status_error(e.response)
twelvelabs.exceptions.BadRequestError: Error code: 400 - {'code': 'parameter_unknown', 'message': 'The following parameters are unknown: dummy. Remove these parameters and try again.'}

The problem is in this code in Task.create():
        else:
            # Request should be sent as multipart-form even file not exists
            files["dummy"] = ("", "")

If file is None, then files["dummy"] is set to ("", ""), which prompts the error from the service.
However, since url is set, there is no need to send files in the request, so I tried modifying Task.create() to remove the else clause with its assignment to files["dummy"]. I received a different error:
Traceback (most recent call last):
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 31, in _request
    response.raise_for_status()
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/httpx/_models.py", line 758, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://api.twelvelabs.io/v1.2/tasks'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ppatterson/src/b2-twelvelabs-example/quick_test.py", line 40, in <module>
    task = tl_client.task.create(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/resources/task.py", line 147, in create
    res = self._post(
          ^^^^^^^^^^^
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 107, in post
    return self._request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ppatterson/src/b2-twelvelabs-example/.venv/lib/python3.11/site-packages/twelvelabs/base_client.py", line 35, in _request
    raise self._make_status_error(e.response)
twelvelabs.exceptions.BadRequestError: Error code: 400 - {'code': 'parameter_unknown', 'message': 'The index_id parameter is unknown. Remove these parameters and try again.'}

So, it looks like there's a bug in the service - it seems to be assuming that there will always be file data in the request.
I was able to work around the bug by sending the URLs as if they were files:
        # Remove assignment to files["dummy"]
        ...
        if url is not None:
            files['video_url'] = io.BytesIO(url.encode())
        if transcription_url is not None:
            files['transcription_url'] = io.BytesIO(transcription_url.encode())

        try:
            res = self._post(
                "tasks", data=remove_none_values(data), files=files, **kwargs
            )
            ...

This fixes the immediate problem, but it seems like a hack. The service should be able to handle the case where there are no files in the request.


rc.6 - for tracking purposes

As if rc.5 wasn't enough work ;)
I was going to look at updating soon if it's not on already being worked on.
If using rc.6 with next tag here, this error will crash your app:

tns_modules/nativescript-angular/renderer.js:3:33: JS ERROR SyntaxError: Importing binding name 'core_private' is not found.

Closing this since the recent posted issue here: https://github.com/NativeScript/nativescript-angular/issues/439 reports same thing.


Cancel in-progress GPU actions

Summary
When updating a PR branch, a lot of concurrent CICD actions can be triggered. This creates a backlog and it takes time for the tests to actually get around to running on the most recent commit. Github makes it easy to cancel in-progress actions when new actions are trigged within a group. This will help avoid wasting GPU time running CICD on out of date commits.

Hardware Type:
[ ] run make test to ensure correctness
[ ] run make checkstyle to ensure code style
[ ] run make test-convergence to ensure convergence


Allow ThemeIcon in WebviewPanel.iconPath

Issue Type: Feature Request
You can only pass an Uri or an object containing Uris to WebviewPanel.iconPath, while iconPath on QuickInputButton and TreeItem also supports ThemeIcon. This would increase codicon reusability.
VS Code version: Code 1.42.0 (ae08d5460b5a45169385ff3fd44208f431992451, 2020-02-06T10:51:34.058Z)
OS version: Windows_NT x64 10.0.18362

@bpasero I believe we should change how editors get their icons to support this. My proposal is to add a IEditorInput.icon property that lets editor inputs set their icon directly instead of going through css rules based on the resource. This could also be used for our welcome page and a few other places in VS Code core that have to use css for this today: https://github.com/microsoft/vscode/blob/ceb8d830f88e516ff7074127189ef01497b2feb2/src/vs/workbench/contrib/welcome/page/browser/welcomePage.css#L223

Works for me.

@aeschli Adding you for help with #92122. Let's try looking into this in May

Any update? Looking forward to use ThemeIcons in webviews!

What is the current workaround?

Now @benibenj introduced support for editors to return a ThemeIcon:
https://github.com/microsoft/vscode/blob/b4870983b477caabcb6ef75b5bed4aa11592bfaf/src/vs/workbench/common/editor/editorInput.ts#L167-L173
So I think this can move along, unless I miss something.

What about setting a ThemeIcon to an editor? @bpasero
I might be missing something.

Yeah I muss the bigger picture of this issue, I was just to mention we made some progress in core for the reading side.


Introduce error type for io module

We can introduce two error type like
`For io module`
enum IoError {
    Appender(AppenderError)
}

// For `ParquetWriter`
struct AppenderError {}

@Xuanwo @liurenjie1024 What do you think?

For error handling, we need to the following things:

Error code that help users to decide how to take actions.
Error Message that describe what happend.
Error Context that provide more information for users.
Error Causes that help users to figure out what happened (with backtrace support).

An error enum is not a good idea to me. I will start a PR for this.


DOC: Clarify Python release documentation

Adds language to clarify the difference between ITK wheel archives which contain loadable Python packages and ITK build archives which cache build artifacts for use in subsequent module builds.
Based on discussion in https://github.com/InsightSoftwareConsortium/ITKPythonPackage/pull/216

PR Checklist

[x] Updated API documentation (or API not changed)

Refer to the ITK Software Guide for
further development details if necessary.

/azp run ITK.macOS


Improve code style (part 1)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.22%. Comparing base (2f073fc) to head (baec2bc).

Additional details and impacted files
@@             Coverage Diff             @@
##              main     #651      +/-   ##
===========================================
- Coverage   100.00%   99.22%   -0.78%
===========================================
  Files           27       27
  Lines         2077     2073       -4
  Branches       312      310       -2
===========================================
- Hits          2077     2057      -20
- Misses           0       16      +16

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


feat(import): Add ability to import treesitter groups

Subject to the same experimental status as #109 but should make it easier to start using the new features that introduced.

:100:


Ant Design preset

Apparently there is some work required to get Ant Design working with Storybook. If we could package that into a preset, it would help people use the tools together.
Here's what that would look like:

[ ] @storybook/preset-ant-design package under packages/preset-ant-design
[ ] Copy structure of an existing preset (e.g. preset-typescript)
[ ] Include README and sample storybook

Presets reference docs here: https://storybook.js.org/docs/presets/introduction/

@shilman I can help with this issue.

Hey @shilman, sorry for being late. I just tested on this on my project and it works awesome! I needed some extra setup on the options and it covered perfectly! Kudos to you and @beizhedenglong.
I added into documentation for this: https://github.com/storybookjs/storybook/pull/8254
Just FYI, the preset i used.
// My preset.js
const theme = require('../configs/anyDateTime.theme');

module.exports = [
  {
    name: '@storybook/preset-ant-design',
    options: {
      lessOptions: {
        modifyVars: theme,
        javascriptEnabled: true
      }
    }
  }
];

Closing this! Great job guys!!!


Quiz Added

Title and Issue number

Close #3035
I have added the customized Quiz. Can it be merged now?

Checklist:

[x] I have mentioned the issue number in my Pull Request.
[] I have commented my code, particularly in hard-to-understand areas
[] I have created a helpful and easy to understand README.md

@MohitGupta121 Can it be merged now?

Sorry for the mistake.I have now deleted those files.The PR contains only the quiz part now.Can it be merged now?


Add call to chuck norris API

added footer
added temporary class for joke
added chuck.js to run the fetch function