Issues accessing the SNI (server name) from the TLS inspector within External Authorization CheckRequest

Title: Access the SNI (server name) from the TLS inspector within External Authorization CheckRequest
Description:
We make use of External Authorization as both a network and HTTP filter. In both cases we set the include_tls_session value to true. For the documentation is states:

(bool) Specifies if the TLS session level details like SNI are sent to the external service.
When this field is true, Envoy will include the SNI name used for TLSClientHello, if available, in the tls_session.

When include_tls_session is set to true one would expect that service.auth.v3.AttributeContext.TLSSession is set. I have ensured that https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/listener_filters/tls_inspector is set and can confirm that in all cases %REQUESTED_SERVER_NAME% is populated in the logs. I am using the SNI dynamic forward proxy to route the traffic based on the SNI.
Is there possibly an issue with the SNI being sent to External Authorization when using the TLS Inspector? Is there another way I can access the SNI if this manner isn't working properly? Perhaps through some form of metadata that comes across in the CheckRequest.
Note, I have built the authorization endpoint with gRPC using https://github.com/envoyproxy/go-control-plane. i.e. handling the requests with Check(ctx context.Context, req *auth.CheckRequest) (*auth.CheckResponse, error).
Relevant Links:

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/ext_authz/v3/ext_authz.proto#envoy-v3-api-msg-extensions-filters-network-ext-authz-v3-extauthz
https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_authz/v3/ext_authz.proto#envoy-v3-api-msg-extensions-filters-http-ext-authz-v3-extauthz
https://github.com/envoyproxy/go-control-plane

cc @ggreenway @KBaichoo @esmet @tyxia

The documentation isn't clear about this, but include_tls_session only works when using a tls transport socket. This could probably be fixed to also work with TlsInspector; a PR to do that would be welcome.

@ggreenway Thanks for the clarification. Are you able to give me a few hints about where to look in the code about where the external auth looks at the transport socket when include_tls_session is true? I presume I could just read the server name from SNI (i.e. TLS inspector) and then are probably filters that already read it like the SNI dynamic forward proxy.

Start by looking here and here. And then look at how the sni dynamic forward proxy is looking it up, and in CheckRequestUtils::setTLSSession if the existing path is empty, check the value from tls inspector.

@ggreenway https://github.com/envoyproxy/envoy/pull/34100 is my attempt to handle this. I'm having troubles understanding what exact unit test (or tests) that I broke. I know the file, just not which tests.


Onboarding OTT

We need to ensure that OTT has access to all the platforms and invites, need to be a team member on the sender pipeline team
Checlist:
OTT needs to have the following

[ ] All meeting invites - Yemi
[x] Access to the sender pipeline slack channel
[ ] Access to Github - Jim
[ ] Access to zenhub - Yemi
[ ] Assign 1st story - Yemi
[x] Walkthrough of engineering process/practices - Jim and Carlos

Please update bullets to done, when assigned task is completed.

@jimduff-usds Can we close this now?

closing


Package config variables not available to npm-alternatives

A quick repro case:
"config": {
	"foo": "bar"
},
"scripts": {
	"vars": "echo $npm_package_config_foo",
	"fails": "run-s vars",
	"works": "npm run vars"
}

npm run ___ can run any of the three scripts successfully (as you'd expect).
pnpm can run vars directly or works, but fails on fails with:
 ERROR   ERROR  Unknown options: '<pkgname>:foo'
I suspect this is very related to #193, but the description there sounded like it was coming at this error from a different direction, so I didn't want to hijack Jakob's issue.
I haven't tested yarn, though #196 sounds like a similar fix could potentially address issues there as well? Especially since a script of yall -s vars runs successfully with pnpm.

Just felt in the same issue!
+1

+1

Also encountering this issue. A workaround that works for commitizen is to use a .czrc config file. This, however, has to be fixed since other packages that use the package.json config field might not provide stand-alone config files.

This is kinda important, and this repo hasn't had a release since 2018, anyone know if there's a popular fork yet?

Bump, is there a fix?

+1


[Bug]: Improve persist of everything-alpha

Prerequisites

[X] I have written a descriptive issue title.
[X] I have searched all issues/PRs to ensure it has not already been reported or fixed.
[X] I have verified that I am using the latest version of Scoop and corresponding bucket.

Package Name
everything-alpha
Expected/Current Behaviour
Irrelevant
Steps to Reproduce
Irrelevant

Possible Solution
In addition to the currently stored configuration, everything-alpha also provides an empty file called NO_ALPHA_INSTANCE (need to be created manually by the user) to disable the Everything 1.5a instance and a folder called plugins to store user-installed plug-ins.
Hopefully they can be added to the persist.

Scoop and Buckets Version
Irrelevant

Scoop Config
Irrelevant

PowerShell Version
Irrelevant

Additional Softwares
No response

In addition to the plugins folder and NO_ALPHA_INSTANCE file mentioned above, I also found that after adding Omit Filter, there will be an additional configuration file Omit Results.csv in the installation folder.


Binary data types not supported for Publish message contents

This is because the Message argument on Publish function is declared as string.

This is caused by the underlying lib which is using string as argument.
github.com/eclipse/paho.mqtt.golang
The clean way to fix this would be to fix it in there.

can you give me a js sample file sending binary in js so I can test I made a fix in https://github.com/pmalhaire/xk6-mqtt/tree/byte_array_support

Thanks. Just try encoding any string with CBOR. For example this one https://github.com/paroga/cbor-js
Basically I encode before sending and decode after receiving.

The thing needed for this is the way to convert js array to golang []byte and back.

note you can do the binary to hex and hex to binary in the js code instead of injecting it in golang.

Yeah. I actually do that. But the thing that was missing was to be able to send bytes in the mqtt message. As before my modification it was converting everything to string. Will try out the new version and let you know how it works. Thanks again!

The issues is that the js arraybuffer does not have a corresponding golang type (or at least I did not find how to do it) so I am kindof stucked on this for now.


Add file path loading and type casting

Added loading of files via $(path) syntax
Added casting of types via type:val syntax

Coverage decreased (-3.5%) to 81.395% when pulling 06408f56ad4a570477aa6c77349555c21a8ae3ac on X2-300 into 0c998568e8101ac9892bd216c86e5b142a02569c on master.

👍

Looks good to me 👍


fix: 安全组规则返回项目id信息

这个 PR 实现什么功能/修复什么问题:
安全组规则返回项目id信息
是否需要 backport 到之前的 release 分支:

release/3.2

/area region
/cc @zexi

Codecov Report

Merging #6666 into master will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #6666      +/-   ##
==========================================
- Coverage    7.14%    7.14%   -0.01%
==========================================
  Files         647      647
  Lines      109345   109351       +6
==========================================
- Hits         7818     7816       -2
- Misses     100929   100936       +7
- Partials      598      599       +1

Flag
Coverage Δ

#aFlag
7.14% <0.00%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/compute/models/dbinstance_accounts.go
2.07% <0.00%> (-0.01%)
:arrow_down:

pkg/compute/models/dbinstance_databases.go
3.39% <0.00%> (-0.02%)
:arrow_down:

pkg/compute/models/elasticcache_accounts.go
2.85% <0.00%> (-0.01%)
:arrow_down:

pkg/compute/models/elasticcache_acls.go
3.75% <0.00%> (-0.02%)
:arrow_down:

pkg/compute/models/elasticcache_backups.go
3.75% <0.00%> (-0.02%)
:arrow_down:

pkg/compute/models/secgrouprules.go
3.39% <0.00%> (-0.02%)
:arrow_down:

pkg/appsrv/response.go
66.17% <0.00%> (-2.95%)
:arrow_down:

pkg/cloudcommon/validators/validators_cert.go
39.77% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9307660...61ff154. Read the comment docs.


test: optimistic the duration time of unit tests

:tada: This PR is included in version 2.10.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[5.7] Show which packages are newly discovered

This PR makes artisan package:discover show which packages are newly discovered. I got the idea after seeing this tweet by Freek van der Herten. Being able to see which packages are newly discovered is nice, but i have to admit it isn't really useful.

This PR works by creating a bootstrap/cache/packages-previous.php file when running artisan package:discover. The command then uses that file next time it is run to figure out which packages are newly discovered.
I'm not really happy with how this PR is implemented, but creating a cached file of the previously discovered packages is the best solution i could think of. We can't use the real package manifest because it is always deleted when running any composer command (see ComposerScript::clearCompiled).
Another problem i ran into was that while Laravel is booting, PackageManifest::providers and PackageManifest::aliases are called, they create a new package manifest if one doesn't exist yet.
This effectively means that currently the PackageDiscoverCommand doesn't actually do anything. When a new package is installed, the existing package manifest is deleted, then while Laravel is booting it creates a new package manifest, then the PackageDiscoverCommand is called and creates the same package manifest again.

No plans to implement this. I don't really like creating a new file just for this feature, etc.


Pass message to RoOriginateError to improve error messages.

Passing this message enables better error messages accross ABI boundaries in cppwinrt and cswinrt code.
//cc @dmachaj

Even easier! @jlaanstra  see https://github.com/jlaanstra/wil/pull/1 which swaps the call to RoOriginateErrorW that takes the string directly.

I'd love to see some before and after debugger spew if you have it. I find that very helpful to make an "it should be better" change obviously better.

With the changes I pushed up, you now get this in the debugger:
Exception thrown at 0x00007FFF92A64B2C (KernelBase.dll) in witest.exe: WinRT originate error - 0x80004005 : 'Puppies not allowed'.

@jonwis This is great, merged it into this change.


fixed typo in matrix in linear_algebra.ipynb

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Janne Hofgren Patala seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


[cmd] Remove --tidy-config flag

CodeChecker analyze --tidy-config is a rarely used and redundant flag. ClangTidy options can be given through .clang-tidy file, --analyzer-config, --checker-config and --tidyargs flags.

I agree, though maybe it'd be more user friendly if we would depracate this flag first.

Alright, I restored the flag's functionality. It will be removed in the next release.


Replicas cannot really be 0

please see comment here
https://github.com/etsy/statsd/issues/468

it renders the statsd proxy "useless" as the proxy is written .. so that should probably have "replicas" set to 1 anyways.

Fixed in 3.1.1


Node stop rework

Fixes #520

This does not address #519 - that will come in a subsequent PR

A couple of questions and minor corrections but otherwise LGTM


Remove _cohorts subfolder from _data folder in theme repo

The _data folder should be kept in the theme, but the _cohorts subfolder removed, as it is not needed in the we-are-ols.org website.
By doing this:

The people info will be stored in theme repo where it will be available in every website using it.
Cohort information will not be stored in the theme, but in the openseeds repo.

No longer applies under the new single-website plan


bug: Storage stringification leads to redundant JSON encoding

E.g., app.json:
{
   "bridges": "[\"SOME_UUID\"]",
   "SOME_UUID": "{\"filter\": ...}"
}

And SOME_UUID.json:
{
    "root.operationalCredentials": {
        "trustedRootCertificates": [
            "{\"__object__\":\"Uint8Array\",\"__value__\":\"...\"}",
        ]
    },
    ...
}

This is not a bug, this is intended. The app knows how to handle it.

Sure, input and output handling matches but that doesn't make it correct. It just makes storage larger, slower and harder to read for no benefit.


Historic conversation

BP to class HistoricConversionQueryBuilder and removes the Override annotation.

HistoricConversionQueryBuilder stands all by itself, with no place in the RI. the actual ConversionQueryBuilder is located in the APi, so a "patch" or workaround in the RI makes no sense. If a Java 7 BP of the Lib-Incubator where it is was desired, please create this bp.


Xnero: Watch 0gadget.com

Xnero requests the watch of the watch_keyword 0gadget\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

0gadget\.com has been seen in 2 true positives, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Debugger is stepping in the code with action step over

1. Describe the bug
I have a strange behavior with the debugger. When I start the debugger and call a report without a breakpoint. The report is performed without the debugger starting. But if I add a breakpoint before the report and call the report again. The debugger is stopping at the breakpoint, that's what I expect. But when I use now the Step Over, the debugger is stopping at the triggers in the report. For example at the trigger OnAfterGetRecord. And is not stepping over the whole report. Same behavior is in pages.
2. To Reproduce
Steps to reproduce the behavior:

Add a breakpoint in page 42 "Sales Order" in the line DocumentAttachmentDetails.RunModal(); (Line 1267)
Start a session with a active debugger
Open a Sales Order and use the action Attachements
The debugger is now stopping at the breakpoint
Use now Step Over
The debugger is stopping in the page 1173 "Document Attachment Details" in function GetCaptionClass (Why is the debugger stopping here?)
When I use now the action Step out
The debugger is stopping on different functions like GetCaptionClass, OnNewRecord and OnAfterGetCurrRecord

3. Expected behavior
I expect, that the debugger is stopping at the breakpoint like in step 4. But wenn I use Step Over, the debugger should not stop anywhere in the page
4. Actual behavior
The debugger is stopping in the page 1173 "Document Attachment Details" and is not stepping over
5. Versions:

AL Language: v13.0.971907
Visual Studio Code:
Version: 1.88.1 (user setup)
Commit: e170252f762678dec6ca2cc69aba1570769a5d39
Date: 2024-04-10T17:41:02.734Z
Electron: 28.2.8
ElectronBuildId: 27744544
Chromium: 120.0.6099.291
Node.js: 18.18.2
V8: 12.0.267.19-electron.0
OS: Windows_NT x64 10.0.22621
Business Central: Version: W1 23.1 (Platform 23.0.13804.0 + Application 23.1.13431.13812)
List of Visual Studio Code extensions that you have installed: AL Language

In your launch.json do you have breakOnError or breakOnRecordWrite set? If so, was there an error or record write where the unexpected breaks happened?

Hi, I have "breakOnError":  "ExcludeTry" there is no error and no write
{
     "name":  "8.X DEV LOKAL",
     "server":  "http://PROD-BC23-1-dev",
     "serverInstance":  "BC",
     "dependencyPublishingOption":  "Ignore",
     "type":  "al",
     "environmentType":  "OnPrem",
     "breakOnError":  "ExcludeTry",
     "authentication":  "UserPassword",
     "request":  "launch"
 }

There are some examples where the debugger stops. The first one is on the breakpoint OnAction DocAttach

So this is by design for now, and has been for as long as I can tell. Because the breakpoint is on the last line in the method, when you step over it the method scope no longer exists and we stop on the next frame to be executed which just happens to be the GetCaptionClass method.
From what I understand, your ask is that it behaves more like a continue (F5) instead?
@kalberes - maybe you have more input on this scenario?

Yes it is since a long time. But not only if it is the last Step
When I add a breakpoint in the Sales Order page here

And Step over, the debugger is not moving to the MoveNegSalesLine.ShowDocument().
It is moving there

Can you make sure that you don't have any entries in your breakpoint list in VSC for this position:

Old breakpoints are not correctly shown in vsc but recognized by the debugger, if they are set in a "wrong" tab. (see #7611)

I have no other breakpoints in the list.

I think it is by design now and you cannot step over modal state. This has been like this like forever, since I am not aware of anyone changing modal UI (page, report) execution context.
If you put a breakpoint on the next line (ShowDocument) it should be hit though. If not that is a bug.

Yes with F5 it steps to the next line


Formatting fix

Fixes formatting for RTD, as noted in #1520. Line 118 (<summary> block) doesn't work yet...

By wrapping our ordered list items in inline-code formatted (backticks ``) our markdown is interpreted very differently, which breaks some existing functionality.
What is the reason for treating these ordered list items as inline-code?

The lists often break in RTD with extra spacing, so using the backtick formatting seems to work better for maintaining numbered lists without causing confusion in the future. Thanks for flagging the other issues, @jakobsandberg , those are also now resolved (removing the extra spacing, unrelated issue).


fix: heads

SUMMARY

Merge Alembic heads manually.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Thanks @betodealmeida.


AoT compile fails when using ambient TSDs

I'm submitting a ...  (check one with "x")
[X] bug report
[ ] feature request
[ ] support request

I'm not sure if this is an issue with the seed, an issue with the Angular AOT compiler, or an issue with TypeScript itself.
If I add a custom TSD (d.ts) to tools/manual_typings/project, I can do normal dev and prod builds OK. The build correctly handles the use of ambient definitions.
However, the AOT build will fail during compile.ahead.prod ("Cannot find name 'XXX'.").
The only way I could get it to work was to use more of an @types approach:

I created a folder containing my definition (in an index.d.ts) file under tools/manual_typings/project
I modified the src/client/tsconfig.json to add a typeRoot and type:

"typeRoots": [
  "../../node_modules/@types",
  "../../node_modules",
  "../../tools/manual_typings/project"
],
"types": [
  "node",
  "jasmine",
  "protractor",
  "systemjs",
  "XXX"
]

I can create a branch that shows the issue if that would help, but I suspect you can reproduce it pretty easily. But as I said, I'm not sure if this means the way the seed project handles ambient definitions needs to change, or if this is expected behavior with the current AoT compiler.

Seems like an issue with the seed. Thanks for reporting!

@brian428 after the build passes I'll merge and the issue should be fixed. Again, thanks for letting me know!

Thanks, Minko. A quick follow up: while this seems to fix the build, I'm still having to use the "types" approach to get IDEA to see the ambient types. I'm using TypeScript 2.1.5, so not sure if this is something that changed in TS 2.1.x regarding nested tsconfig files, or if it's some bug in IDEA/WebStorm 2017.1. I'm leaning toward the latter, since things seem to build fine from the command line.
Just thought I'd mention this here in case anyone else runs into the same problem.

Yeah, the manual typings are added by the build itself. In order to resolve your issue you have to add them manually. I'll add one additional line which drops duplicate files from tsconfig.json.

Hmm, interesting. When I tried to specify specific files in the files or include tsconfig arrays, it seemed to cause other problems (Angular decorators started being flagged as unknown). Almost like files or include killed the inclusion of types/typeRoots. I hated to do it, but I ended up having to use a TS  tag to include the TSD.
I don't want to turn this into an annoying time sink for you, but if you have any ideas off the top of your head, I'm open to hearing them. :-)

I'll put this into my todo list and think of a solution tomorrow or Wednesday.

Something like this as content of your src/client/app/tsconfig.json should work:
{
  "compilerOptions": {
    "target": "es5",
    "module": "commonjs",
    "declaration": false,
    "removeComments": true,
    "noLib": false,
    "lib": ["es2016", "dom"],
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "sourceMap": true,
    "pretty": true,
    "allowUnreachableCode": false,
    "allowUnusedLabels": false,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "noImplicitUseStrict": false,
    "noFallthroughCasesInSwitch": true,
    "typeRoots": [
      "../../node_modules/@types",
      "../../node_modules"
    ],
    "types": [
      "node",
      "jasmine",
      "protractor",
      "systemjs"
    ]
  },
  "files": [
    "./app/main.ts"
  ],
  "exclude": [
    "node_modules",
    "dist",
    "src"
  ],
  "compileOnSave": false
}

Just tried it, VSCode doesn't complain about the experimental decorators.


Implement vote functionality

Add reviewId prop to reviewListItem

add a blank route for review vote

Implement vote functionaliyt and styliing

 [5] implement vote functionality

 [1] fix 'voted for this' to 'voted for this review' in vote component when vote count is 2

AWESOME


param nrow in make_grid is misleading

import torch
import torchvision
label_img = torch.rand(100, 3, 32, 32)
x = torchvision.utils.make_grid(label_img, nrow=20, padding=10)
torchvision.transforms.ToPILImage()(x).save("example.png")

produces this image:

which has 5 rows and 20 columns. Although the docs say
nrow (int, optional) – Number of rows in grid. Final grid size is (B / nrow, nrow). Default is 8.

Which says the output will actually be (B/nrow, nrow). Perhaps this param should be called ncol, or we should change the output shape to be (nrow, B/nrow)?
I think, to keep backward compatibility changing the output shape to (nrow, B/nrow) would make more sense.
Happy to send a PR if you agree?

Maybe we could change the docs to read "number of images in a row"? This way we don't change the API, and it still makes sense from the parameter name?

Yeah - this is a lot easier and wont have "unexpected" changes for people who are used to using it in its current form. I'll send a PR for this.

Fixed via #250


Borged crew does not count Borg playtime

Description
#31629 added Borg time tracking for midround posibrains, but the brains of gibbed mobs still do not switch to the Borg job when put into an MMI

Pretty sure nothing tracks for time when you change positions/jobs, I mean it could be a pain too if your an antag and steal a commands ID and get free Captain time


Command: refactor to remove mdc global variable

Fixes #2015
Signed-off-by: Brandon Forster<EMAIL_ADDRESS>

Codecov Report

Merging #2079 into master will decrease coverage by 0.09%.
The diff coverage is 2.73%.

@@            Coverage Diff            @@
##           master    #2079     +/-   ##
=========================================
- Coverage   42.19%   42.09%   -0.1%
=========================================
  Files         184      184
  Lines       12027    12055     +28
=========================================
  Hits         5075     5075
- Misses       6655     6683     +28
  Partials      297      297

Impacted Files
Coverage Δ

internal/core/command/router.go
0% <0%> (ø)
:arrow_up:

internal/core/command/init.go
0% <0%> (ø)
:arrow_up:

internal/core/command/restDevice.go
0% <0%> (ø)
:arrow_up:

internal/core/command/device.go
31.3% <20%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0735833...39c1e88. Read the comment docs.


[SU-197] Remove broken state history from data tab

Several components on the data tab read information from state history (scroll position, current page, etc). However, no data they read this way is ever visible to a user.
Some children of WorkspaceData read from state history on their first render firstRender:
https://github.com/DataBiosphere/terra-ui/blob/86327f299540fe87d0570c7e9e2dcff597492f28/src/components/data/EntitiesContent.js#L369
firstRender is a state variable in WorkspaceData and passed down to children:
https://github.com/DataBiosphere/terra-ui/blob/86327f299540fe87d0570c7e9e2dcff597492f28/src/pages/workspaces/workspace/Data.js#L393-L401
On mount, WorkspaceData loads metadata:
https://github.com/DataBiosphere/terra-ui/blob/86327f299540fe87d0570c7e9e2dcff597492f28/src/pages/workspaces/workspace/Data.js#L528-L531
Which clears entityMetadata state:
https://github.com/DataBiosphere/terra-ui/blob/86327f299540fe87d0570c7e9e2dcff597492f28/src/pages/workspaces/workspace/Data.js#L434-L436
While entity metadata is loading, a spinner is the only child rendered. Other children are unmounted:
https://github.com/DataBiosphere/terra-ui/blob/86327f299540fe87d0570c7e9e2dcff597492f28/src/pages/workspaces/workspace/Data.js#L553-L555
Thus, by the time children such as EntitiesContent are mounted, firstRender will be false and they will not use anything from state history.
I could attempt to make this work correctly. However, the WorkspaceData component is messy enough that I'd rather clear out broken functionality and reduce complexity before attempting a refactor.
Also, currently, using state history in the data tab is complicated by having multiple pages (data tables, snapshots, reference data, files, etc) all share the same URL (<namespace>/<name>/data). Thus, they also share the same state history entry, which is what necessitated reading state history based on firstRender. This functionality will be much easier to restore in a possible future where all of those pages have their own URLs and their own entries in state history.
This also removes state history from UploadPreviewTable. There, any restored state is useless without the table data itself, which is ephemeral.

Rebased to resolve merge conflicts.


Paper uses averages instead of the minimum for security analysis

Perhaps the one key factor that differentiates security (and adversarial robustness) from other general forms of robustness is the worst-case mindset from which we evaluate. This paper uses the mean throughout to evaluate both attacks and defenses.
Using the mean over various attacks to compute the “security” of a defense completely misunderstands what it means to perform a security evaluation in the first place.
For example, the paper bolds the column for the NAT defense when evaluated on CIFAR-10 because it gives the highest “average security” against all attacks. However, this is fundamentally the incorrect evaluation to make: the only metric that matters in security is how well a defense withstands attacks targeting that defense. And in this setting, the alternate adversarial training approach of Madry et al. is strictly stronger.
For this reason, when the paper says that all the defenses are "more or less" effective, it's completely misrepresenting what is actually going on. In fact, almost all of the defenses studied offer 0% robustness to any actual attack. By misrepresenting this fact, the work of Madry et al. and Xu et al. which actually mostly satisfy their security claims aren't appropriately

We do agree that security is the worst-case guarantee, and no system/model is absolutely secure from this perspective. However, from a practical or statistical point of view, security is often a kind of relative or statistical security. For instance, Windows 10 is often considered to be more secure for users than Windows XP, although both of them offer 0% security guarantee to actual attacks in your opinion.
Therefore, from a statistical or practical security perspective, rather than whether one particular defense is secure or not in the worst case scenario, we try to capture the overall security differences of different types of defenses, such as "For complete defenses, most of them have the capability of defending against some adversarial attacks, but no defense is universal. Particularly, the defenses that retrain their models usually perform better than others without retraining".
We believe it is also important to let the readers know which model is statistically more secure, while we agree that no model is 100% secure against worst-case attacks. In addition, we would also like to point out that Madry et al.  is only more secure within the strict perturbation magnitude bound; because it is trained intensely within the norm-bounded ball, and this cannot be viewed as the “worst case scenario” in security. So it is not exactly accurate to claim “Madry et al.  is more robust than other models”, and again it is for sure not secure under powerful attackers who can add larger perturbation in the worst case scenario attack, but this doesn’t mean this model is not useful.

Definitely, there is no such thing as perfect security and some things can be more robust than others.
Fortunately, we have a way to measure this. Accuracy.
For example, the model of Madry et al. gives roughly 45% accuracy on CIFAR-10 with l_infinity distortion of 0.031. PixelDefend give 9%. I'm not making the argument "neither model gives 100% robustness and so they're both useless", but rather making the argument that we must compare worst-case robustness when dealing with security. That is what makes security different than other fields.


FIPS: Add EDDSA public key validation.

EVP_PKEY_public_check() can be used by ED25519 and ED448 in order to determine if the public key is a valid point on the curve.
The FIPS ACVP tests require public key validation tests. See https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/EDDSA-KeyVer-1.0/internalProjection.json
Note that this is NOT required to be called before EDDSA signature verification since it is done internally.

Checklist

[ ] documentation is added or updated
[x] tests are added or updated

I have not added siggen and sigver tests since there are already many tests in evp_test (see evppkey_ecx.txt)

This pull request is ready to merge

Merged to the master branch. Thank you for your contribution.


Automatically generate subtrees for Visual Studio

Take a look at CMakeGenerateTrees.txt, it is fully handmade. Maybe we could find a way to generate those subtrees in a more automatic way.

https://cmake.org/cmake/help/v3.0/command/source_group.html

This issue seems obsolete. Do you confirm ?

Well it's kind of a "we will  do it when someone bothers"...

As far as I see, CMakeGenerateTrees.txt doesn't exist anymore.
What was the purpose of this file ?
The solution today generated by cmake seems good to me.

The idea was to have the folder hierarchy mirrored in the file hierarchy of VS instead of having all files at the same level in each project.

I see, fair enough. Sill open though :)

This could be fixed in VS 2017 which I've heard supports CMake natively

Ok. Todo then.

We be fixed by WIP on branch fixMVSC.

This is currently fixed by PR #195.


Move to Microsoft.UI.Xaml 2.4.0

This brings support for "Compact" tab sizing, which compresses all inactive tabs to just the size of their icons plus the close button. Neat!
It also just keeps us generally up-to-date and good citizens.

I am so dismayed.

This is one of those things that held me off (at least Azure can build with it) from doing this PR myself.

Okay this happens even in (app=light, system=dark), (app=light, system=light), and (app=dark, system=light).
I'm trying to look through MUX changes to see if there's something that stands out. microsoft-ui-xaml#2377 looks related, but I can't easily bisect on our side to see if that's it...

It absolutely was microsoft-ui-xaml#2377.
If you insert
<StaticResource x:Key="AccentButtonForeground" ResourceKey="SystemControlBackgroundChromeWhiteBrush" />

into our App.xaml, that effectively reverts the part of https://github.com/microsoft/microsoft-ui-xaml/commit/1785888fe6408a9c57d7a44a9ed40b5e5b82cc17 that broke this. I'm not sure what the side effects of that are, but those are my findings so far.

Looking through our 1.1/1.x/backlog milestones. I'm guessing this would close:

#5360

I'll edit this comment if/as I find more.

So, in the ongoing saga of the Accent Button color, this was reverted upstream in https://github.com/microsoft/microsoft-ui-xaml/pull/2420. I'm not sure what their release cadence is like - we could probably ship this PR with the workaround above for now, and revert the change when we pull the next MUX. @DHowett-MSFT I'll leave that up to you.

We can probably safely wait a bit for this one to land. It's not pressing that we hit MUX 2.4 :)

Updating will probably solve more bugs than just the one mentioned above. Nobody has just reported yet. I think the workaround should be added and merge the PR. There's a higher chance of dropping bugs than gaining new ones for Terminal.

We are very early in the next release cycle, and we can afford to wait up to 7 days to make sure we're doing the right thing. There's no urgency here :smile: because it's not landing in time for v1.0.

We might want to wait one more MUX release, so we can get https://github.com/microsoft/microsoft-ui-xaml/pull/2201 too 😉

Ha, should be in the next preview I'd guess. Fix bugs and gain new features is always a win/win.

2.4.2 was released and contained a bug fix. We could move to that for now and update again when .3 is available for the Edge look.

IIRC we explicitly need the -prerelease versions of the MUX package to get the symbols hooked up right? I might be wrong on that, but I think there was a reason.

We need the prerelease versions because we need the DLL to get into our package instead of being inside a framework package dependency.

Stable ones don't provide a dll? Also, I've been experimenting with dependabot. If we did get inside a framework package dependency, it would automatically create a pull request to bump the version. We already have other nuget packages that require updates that are being read in by dependabot. Adding MUX to the list would make more sense to me. https://app.dependabot.com/ You should enable this repo for it and see what I mean.

Unfortunately, they don’t! They inject a PackageDependency into the application manifest, whereas prereleases include the DLL, assets and localization content directly in your package.

Strange they don't provide it as an option. Wonder if they have an issue/feature request for it already.

I’ve got a good email thread with the project lead over there about this :smile:

Seems counterproductive to force apps to rely on pre-release versions that are by their very nature, unstable and subject to change but if that's what that team wants to force you to do, ¯_(ツ)_/¯

We are in a unique position. The framework is intended to be consumed through a dependency package, which will be automatically managed by the store.
The reason we can’t use that is a long-standing bug (since Windows 8!) in elevation. It turns out that Terminal is the only package that both runs as admin and uses WinRT class activation! Those things together make our lives rather difficult.
I don’t blame the Xaml team for not catering to a use case 0.5% of applications will ever hit right now with their initial efforts to take the “UWP” ui toolkit out of the platform. Like, it was never designed for this, it’s no wonder it doesn’t work 100% right.
WinUI3, on the horizon, offers a better solution to so many of these issues.

Yes, I'm hoping we can drop the UWP bits all together and just throw all the XAML into WPF utilizing the UWP APIs being ported such as https://github.com/microsoft/microsoft-ui-xaml/issues/2005

Very extremely unlikely to use WPF. We are entirely native code :p

Entirely? 🧐

The Windows Terminal* is entirely native code. The WpfControl is obviously C#, and so is ColorTool, and maybe some random tests and utils, but the Terminal is native, and conhost is native.

Well you need the XAML to go somewhere and it seems to me that WPF will be more helpful than trying to go full UWP and should alleviate the elevation issue.

Switching from one framework to another while remaining in a Win32 application model application (make no mistake; WPF is Win32) will do absolutely nothing to change our station in regards to package activation, elevation or dependencies. WinUI3 is still going to be made up of WinRT classes that need to be instantiated one way or another!

@carlos-zamora @zadjii-msft this is ready to review.


[WIP] Use upstream units for mdadm

Motivation for this change
Fix RAID1 rebuild on each reboot.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

[WIP] because I still need to test it better. I don't have good ideas for automated testing for now.

@GrahamcOfBorg build mdadm

@edolstra I think so too but we already have systemd.generatorPackages. I'm all for merging them all, what do you think?

Tested on Intel IMSM array, before the patch a reboot was practically guaranteed to start a resync. After the patch and four reboots no more resyncs. I think this is ready.

This is ready with some cosmetic changes. I also removed generator-packages completely after thinking more, we should indeed just use systemd.packages everywhere.

@Mic92 All fixed, thanks!
@GrahamcOfBorg build systemd mdadm


Double escaping url's

Rails url helpers escape special characters when creating urls, so when jQCloud escapes the url it double escapes it.

Becomes...

  John's Bday

Which doesn't link correctly.
Suggested fix
Adding a boolean option encode_urls that would allow the user to decide

Just for other fellow developers that used this excellent library: the library already has an encodeURI option that can be set to false/true and it is true by default.


user server channel to fix telemetry lose data

@SummerSun,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Improve password recovery

Is your feature request related to a problem? Please describe.
If browser(s) are not installed in default location nothing is recovered
Describe the solution you'd like
Add a search option or automatic search ability
Additional context
To save space like I do, others may do the same and install non essential apps on secondary drive(s)
saving space on my main SSD/HDD So, browsers and other may not be in default location causing
password recovery to fail.

"c:\nirsoft_tools\nircmd.exe" exec hide "c:\nirsoft_tools\WebBrowserPassView.exe" /stab "c:\output\delicious_passwoooords_baby.csv"
you can do a lot with remote shell commands and nirsoft tools.

"c:\nirsoft_tools\WebBrowserPassView.exe" /stab "c:\output\delicious_passwoooords_baby.csv"
you can do a lot with remote shell commands, nirsoft tools and some creativity
https://www.nirsoft.net/utils/web_browser_password.html

That does not work for me. Am I doing something wrong? It just opens the program. "C:\Users\<username>\Downloads\Test\WebBrowserPassView.exe" /stab "C:\Users\<username>\Downloads\Test\Done.csv"

@BurntDog can i have your email ?

@BurntDog can i have your email ?

The reason?

That does not work for me. Am I doing something wrong? It just opens the program.

you're using wrong version of web_browser_password
it's explained somewhere on the website but it's easy to miss in that huge wall of text.
there are 2 version of browser password tool:

one has command line support, can be used silently in the background for some nefarious reasons. antivirus doesn't like that and marks it as a hacking tool.
second version is visible window gui only, can't be sneaky, so av doesn't mind.

get one with commands from here:
https://www.nirsoft.net/password_recovery_tools.html
that part where it says::
1.  Click this download link.
2.  Enter 'download' as the user name, and 'nirsoft123!' as the password.
3.  After downloading the package, extract the files from it using the following password: nirsoft123!

remember to add pass tool to antivirus exceptions first or av will hunt it down
and done. now that command will work,
pass tool will run without showing window and will output results to csv

That does not work for me. Am I doing something wrong? It just opens the program.

you're using wrong version of web_browser_password.exe
it's explained on thes website, but it's easy to miss that on that huge wall of text.
there are 2 version of browser password tool:

one has command line support and can be used silently in the background, possibly for nefarious reasons. antivirus doesn't like that and marks it as potentially risky hacking tool.
that's why there's a second version that can only be used with visible window gui. problem sold, no sneakery peakery, so av doesn't mind.

get version with commands from here:
https://www.nirsoft.net/password_recovery_tools.html
that part where it says::
1.  Click this download link.
2.  Enter 'download' as the user name, and 'nirsoft123!' as the password.
3.  After downloading the package, extract the files from it using the following password: nirsoft123!

remember to disable antivirus before using password tool, or add it exceptions, or av will hunt it down instantly.
and done. now that command will work:
"C:\Users\<username>\Downloads\Test\WebBrowserPassView.exe" /stab "C:\Users\<username>\Downloads\Test\Done.csv"
pass tool will run without showing anything on the screen and will output results to csv

Thanks for the help!

"c:\nirsoft_tools\WebBrowserPassView.exe" /stab "c:\output\delicious_passwoooords_baby.csv"
you can do a lot with remote shell commands, nirsoft tools and some creativity
https://www.nirsoft.net/utils/web_browser_password.html

They removed that feature sadly :(


Wocoommerce automatic emails sent in English

Hello,
We have installed your plugin in a multilanguage site in Spanish, English and German.
We have done several orders, with the site in Spanish (including Cart and Checkout pages in Spanish), but the Wocoommerce emails are sent in English.

Can you reproduce this issue on default Wordpress theme (eg Storefront)?
Can you reproduce this issue when all other plugins are disabled except WooCommerce, Polylang and Hyyan WooCommerce Polylang Integration?
What product versions and settings are you using when this issue occurs?

PHP: 5.6.35
WordPress: 4.9.4
WooCommerce: 3.3.4 (we have also downgraded to 3.2.6, as jonmoore said us is not tested for 3.3.x)
Polylang: 2.3.4
Hyyan WooCommerce Polylang Integration: 1.0.4 (we have marked every option)
Browser: -

Steps to Reproduce
It's just the automatic emails sent by Woocommerce after an order is done.
What I Expected
As I am making the order in Spanish, the emails should be sent in Spanish, but we receive it in English (eve the primary language in Polylang is Spanish). However, If I change afterwards the order status in the backend, those emails are sent in Spanish; but the automatic ones (like the “new order” email for customer and admin), always in English.
What Happened Instead
Automatic Woocommerce emails sent always in English.
Thanks in advance.

UPDATE: We have just realised it only happens with TPV payment; with bank transfer emails are sent in Spanish.
We have tried two different Redsys modules, and also with two different bank account gateways.
In all cases, the TPV payment returns Woocommerce emails in English...
Could it be a bug in your module?

what is TPV payment ?
there is no standard payment gateway called TPV, have you added another plugin, which is not compatible with the language switching?
Note, when raising a ticket you are supposed to paste in WooCommerce\Status\Get System Report which would include this and other information.

well, taking into account that:

the problem only occurs with this gateway Pasarela de pago para Redsýs
the gateway Pasarela de pago para Redsýs is "No probado con la versión activa de WooCommerce"
the gateway Pasarela de pago para Redsýs is a paid product and should have support available
this component is a free product only supported by volunteers in their spare time, who have no access to the code for Pasarela de pago para Redsýs

.. really you should raise a call with Pasarela de pago para Redsýs

Hello back Jon,
We have been doing some new tests, and we can confirm now that Redsýs (POS) is not the problem.
Using PayPal payment (just with the Woocommerce plugin) it happens the same: automatic sent emails after payment, admin-new-order.php & customer-completed-order.php, are both sent in English.
We even have even try adding inside those templates this code: echo get_locale(), and we get: [es_ES], but all strings in English (although translations are present, since you get Spanish mails if you change order status manually).
So, problem must be in the "trigger" of that automatic emails when the payment is done...
Any clue about that issue?
Thanks in advance.

Thanks for investigating,
for info I haven't had time to look at this - actually I run Woo 3.1.2 though I hope to test upgrade to 3.3.4 soon.
I do remember there are two ways of triggering email in woocommerce, one is immediate and the other is with asynchronous with cron, there isn't an option for it in the user interface but you can turn the cron off by returning false on the filter woocommerce_defer_transactional_emails and then the email will be send from the UI in the correct language.
It may be that extra code is needed to force the language switch (reload the translation files) when in cron context - I thought something was already done for this, I'll check (but probably not this week), in the meantime please try the workaround with woocommerce_defer_transactional_emails

for the record, I retested that this is working for me with woo 3.1.2
The site main language is English, secondary language is Spanish
I make an order in Spanish I get the customer email in Spanish.
I don't use woocommerce_defer_transactional_emails filter.
There is a setting to enable the email translation in Settings, WooPoly, Features - but I suppose you already have this.
And the language switching in the Email.php specifically gets order language and forces reload of the text domain for translation.

for me, it's Fixed in #329
as per related discussion on #319
Note that as of writing this isn't merged yet, just pick up the replacement ajax file from the commit on #329  :
https://github.com/marian-kadanka/woo-poly-integration/blob/888df716c43c0191c32e2b8bfc11d7ea0a9ed055/src/Hyyan/WPI/Ajax.php
props @marian-kadanka

Fixed in #329

Hola me pasa algo muy similar, tengo un sitio en Frances e Ingles pero Woocommerce solo funciona en ingles al enviar el email al cliente.  Existe alguna forma para los email esten en el idioma del usuario?

make sure you are using the latest from https://github.com/hyyan/woo-poly-integration/releases
I don't think it was released to wordpress.org


kubectl unable to terminate pod -- runc exit with error code 1

Description
I'm creating the pod in kubernets v.1.30.x. All is working nice with containerd. Further, once we go to delete pod is not going to remove instead keep in Terminating status. While error logs found follow errors.

I expect container should be stopped and terminated.
runc -v
runc version 1.1.12-0ubuntu3
spec: 1.0.2-dev
go: go1.22.2
libseccomp: 2.5.5
containerd -v
containerd github.com/containerd/containerd 1.7.12
kubectl delete po/ng

sudo journalctl -k | grep -F audit | grep -F containerd | grep -Fv ALLOWED
Aug 03 16:07:23 h32.lab kernel: audit: type=1400 audit(1722681443.231:255): apparmor="DENIED" operation="signal" class="signal" profile="cri-containerd.apparmor.d" pid=6528 comm="runc" requested_mask="receive" denied_mask="receive" signal=kill peer="runc"
Aug 03 16:08:07 h32.lab kernel: audit: type=1400 audit(1722681487.213:256): apparmor="DENIED" operation="signal" class="signal" profile="cri-containerd.apparmor.d" pid=6546 comm="runc" requested_mask="receive" denied_mask="receive" signal=kill peer="runc"
Aug 03 16:08:07 h32.lab kernel: audit: type=1400 audit(1722681487.238:257): apparmor="DENIED" operation="signal" class="signal" profile="cri-containerd.apparmor.d" pid=6552 comm="runc" requested_mask="receive" denied_mask="receive" signal=kill peer="runc"
Solutions: Pod/container only can terminate by killing process ID.
https://github.com/containerd/containerd/issues/6080
crictl stop 13746f6907f7b
E0803 15:48:46.295164    5941 remote_runtime.go:366] "StopContainer from runtime service failed" err=<
rpc error: code = Unknown desc = failed to kill container "13746f6907f7bc702294005c199e1903dc02fe30ad848950c567dd66dea19bfd": unknown error after kill: runc did not terminate su
ccessfully: exit status 1: unable to signal init: permission denied
: unknown

containerID="13746f6907f7b"
FATA[0000] stopping the container "13746f6907f7b": rpc error: code = Unknown desc = failed to kill container "13746f6907f7bc702294005c199e1903dc02fe30ad848950c567dd66dea19bfd": unknown
error after kill: runc did not terminate successfully: exit status 1: unable to signal init: permission denied
: unknown

root@h32:~# crictl stats
CONTAINER           NAME                      CPU %               MEM                 DISK                INODES
13746f6907f7b       ng                        0.00                3.199MB             94.21kB             19
4a7e2c8daa87f       etcd                      3.07                35.96MB             36.86kB             11
66392610b59e3       kube-apiserver            7.87                195.3MB             53.25kB             15
8c4f8a579c5be       kube-controller-manager   2.16                45.2MB              73.73kB             21
8ee1605e85154       coredns                   0.19                13.39MB             36.86kB             11
bdf84141f5c32       kube-scheduler            0.38                16.81MB             12.29kB             7
c9f2adbfecf0a       kube-proxy                0.11                11.96MB             65.54kB             24
cc13c2c660029       coredns                   0.12                13.17MB             36.86kB             11
root@h32:~# crictl inspect 13746f6907f7b | jq '.info.pid'
4871
root@h32:~# ps auxf | grep 4871
root        6566  0.0  0.0   9144  2176 pts/2    S+   16:08   0:00                          _ grep --color=auto 4871                                                                    root        4871  0.0  0.1  11404  7552 ?        Ss   15:31   0:00  _ nginx: master process nginx -g daemon off;
root@h32:~# kill -9 4871
root@h32:~# ps auxf | grep 4871                                                                                root        6654  0.0  0.0   9144  2176 pts/2    S+   16:08   0:00                          _ grep --color=auto 4871
root@h32:~# crictl inspect 13746f6907f7b | jq '.info.pid'                                                                                E0803 16:08:48.635190    6657 remote_runtime.go:432] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container "
13746f6907f7b": not found" containerID="13746f6907f7b"                                                                                FATA[0000] getting the status of the container "13746f6907f7b": rpc error: code = NotFound desc = an error occurred when try to find container "13746f6907f7b": not found
Steps to reproduce the issue

Describe the results you received and expected
I'm creating the pod in kubernets v.1.30.x. All is working nice with containerd. Further, once we go to delete pod is not going to remove instead keep in Terminating status. While error logs found follow errors.

I expect container should be stopped and terminated.
What version of containerd are you using?
1.7.12
Any other relevant information
runc version 1.1.12-0ubuntu3
spec: 1.0.2-dev
go: go1.22.2
libseccomp: 2.5.5
containerd github.com/containerd/containerd 1.7.12
uname -a
Linux h32.lab 6.8.0-39-generic #39-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul  5 21:49:14 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
....
"imagePullProgressTimeout": "5m0s",
"drainExecSyncIOTimeout": "0s",
"containerdRootDir": "/var/lib/containerd",
"containerdEndpoint": "/run/containerd/containerd.sock",
"rootDir": "/var/lib/containerd/io.containerd.grpc.v1.cri",
"stateDir": "/run/containerd/io.containerd.grpc.v1.cri"
},
"golang": "go1.22.2",
"lastCNILoadStatus": "OK",
"lastCNILoadStatus.default": "OK"
}
Show configuration if it is related to CRI plugin.
disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2
[cgroup]
path = ""
[debug]
address = ""
format = ""
gid = 0
level = ""
uid = 0
[grpc]
address = "/run/containerd/containerd.sock"
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
tcp_address = ""
tcp_tls_ca = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
[metrics]
address = ""
grpc_histogram = false
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
deletion_threshold = 0
mutation_threshold = 100
pause_threshold = 0.02
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
cdi_spec_dirs = ["/etc/cdi", "/var/run/cdi"]
device_ownership_from_security_context = false
disable_apparmor = false
disable_cgroup = false
disable_hugetlb_controller = true
disable_proc_mount = false
disable_tcp_service = true
drain_exec_sync_io_timeout = "0s"
enable_cdi = false
enable_selinux = false
enable_tls_streaming = false
enable_unprivileged_icmp = false
enable_unprivileged_ports = false
ignore_image_defined_volumes = false
image_pull_progress_timeout = "5m0s"
max_concurrent_downloads = 3
max_container_log_line_size = 16384
netns_mounts_under_state_dir = false
restrict_oom_score_adj = false
sandbox_image = "registry.k8s.io/pause:3.8"
selinux_category_range = 1024
stats_collect_period = 10
stream_idle_timeout = "4h0m0s"
stream_server_address = "<IP_ADDRESS>"
stream_server_port = "0"
systemd_cgroup = false
tolerate_missing_hugetlb_controller = true
unset_seccomp_profile = ""
[plugins."io.containerd.grpc.v1.cri".cni]
  bin_dir = "/opt/cni/bin"
  conf_dir = "/etc/cni/net.d"
  conf_template = ""
  ip_pref = ""
  max_conf_num = 1
  setup_serially = false

[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "runc"
disable_snapshot_annotations = true
discard_unpacked_layers = false
ignore_blockio_not_enabled_errors = false
ignore_rdt_not_enabled_errors = false
no_pivot = false
snapshotter = "overlayfs"
  [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
    base_runtime_spec = ""
    cni_conf_dir = ""
    cni_max_conf_num = 0
    container_annotations = []
    pod_annotations = []
    privileged_without_host_devices = false
    privileged_without_host_devices_all_devices_allowed = false
    runtime_engine = ""
    runtime_path = ""
    runtime_root = ""
    runtime_type = ""
    sandbox_mode = ""
    snapshotter = ""

    [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options]

  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
      base_runtime_spec = ""
      cni_conf_dir = ""
      cni_max_conf_num = 0
      container_annotations = []
      pod_annotations = []
      privileged_without_host_devices = false
      privileged_without_host_devices_all_devices_allowed = false
      runtime_engine = ""
      runtime_path = ""
      runtime_root = ""
      runtime_type = "io.containerd.runc.v2"
      sandbox_mode = "podsandbox"

snapshotter = ""
      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
        BinaryName = ""
        CriuImagePath = ""
        CriuPath = ""
        CriuWorkPath = ""
        IoGid = 0
        IoUid = 0
        NoNewKeyring = false
        NoPivotRoot = false
        Root = ""
        ShimCgroup = ""
        #SystemdCgroup = false
        SystemdCgroup = true

  [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
    base_runtime_spec = ""
    cni_conf_dir = ""
    cni_max_conf_num = 0
    container_annotations = []
    pod_annotations = []
    privileged_without_host_devices = false
    privileged_without_host_devices_all_devices_allowed = false
    runtime_engine = ""
    runtime_path = ""
    runtime_root = ""
    runtime_type = ""
    sandbox_mode = ""
    snapshotter = ""

    [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options]

[plugins."io.containerd.grpc.v1.cri".image_decryption]
  key_model = "node"

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = ""

  [plugins."io.containerd.grpc.v1.cri".registry.auths]

[plugins."io.containerd.grpc.v1.cri".registry.configs]
  [plugins."io.containerd.grpc.v1.cri".registry.headers]

  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]

[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
  tls_cert_file = ""
  tls_key_file = ""

[plugins."io.containerd.internal.v1.opt"]
path = "/opt/containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.internal.v1.tracing"]
sampling_ratio = 1.0
service_name = "containerd"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.nri.v1.nri"]
disable = true
disable_connections = false
plugin_config_path = "/etc/nri/conf.d"
plugin_path = "/opt/nri/plugins"
plugin_registration_timeout = "5s"
plugin_request_timeout = "2s"
socket_path = "/var/run/nri/nri.sock"
[plugins."io.containerd.runtime.v1.linux"]
no_shim = false
runtime = "runc"
runtime_root = ""
shim = "containerd-shim"
shim_debug = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
sched_core = false
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.service.v1.tasks-service"]
blockio_config_file = ""
rdt_config_file = ""
[plugins."io.containerd.snapshotter.v1.aufs"]
root_path = ""
[plugins."io.containerd.snapshotter.v1.blockfile"]
fs_type = ""
mount_options = []
root_path = ""
scratch_file = ""
[plugins."io.containerd.snapshotter.v1.btrfs"]
root_path = ""
[plugins."io.containerd.snapshotter.v1.devmapper"]
async_remove = false
base_image_size = ""
discard_blocks = false
fs_options = ""
fs_type = ""
pool_name = ""
root_path = ""
[plugins."io.containerd.snapshotter.v1.native"]
root_path = ""
[plugins."io.containerd.snapshotter.v1.overlayfs"]
mount_options = []
root_path = ""
sync_remove = false
upperdir_label = false
[plugins."io.containerd.snapshotter.v1.zfs"]
root_path = ""
[plugins."io.containerd.tracing.processor.v1.otlp"]
endpoint = ""
insecure = false
protocol = ""
[plugins."io.containerd.transfer.v1.local"]
config_path = ""
max_concurrent_downloads = 3
max_concurrent_uploaded_layers = 3
[[plugins."io.containerd.transfer.v1.local".unpack_config]]
  differ = ""
  platform = "linux/amd64"
  snapshotter = "overlayfs"

[proxy_plugins]
[stream_processors]
[stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
path = "ctd-decoder"
returns = "application/vnd.oci.image.layer.v1.tar"
[stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
path = "ctd-decoder"
returns = "application/vnd.oci.image.layer.v1.tar+gzip"
[timeouts]
"io.containerd.timeout.bolt.open" = "0s"
"io.containerd.timeout.metrics.shimstats" = "2s"
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[ttrpc]
address = ""
gid = 0
uid = 0

Are you using a custom build of runc? I see that runc version used is 1.1.12, but compiled with go1.22.2.
Ref: https://github.com/opencontainers/runc/pull/4292, runc had an issue when using go version less than 1.22.4 from the 1.22 series.

It is not a custom installation as i used apt-get package manager. I'm using "ubuntu 24.04" as base OS.

dpkg -l | grep runc
ii  runc                                          1.1.12-0ubuntu3                          amd64        Open Container Project - runtime
runc -v
runc version 1.1.12-0ubuntu3
spec: 1.0.2-dev
go: go1.22.2
libseccomp: 2.5.5

You may want to open an issue on Launchpad for Ubuntu's runc package.

Thanks @Rico00121.  This does look like a duplicate which has been fixed in v1.7.16 / v1.6.32:

https://github.com/containerd/containerd/pull/10129
https://github.com/containerd/containerd/pull/10130


TVM for ROCm 2.x is currently not working

Environment: Ubuntu 18.04 + ROCm 2.2 + TVM (built from current master with ROCM = ON)
I ensure the target TVM library successfully detect and link with ROCM, and the tuning procedure runs successfully, however, while executing tvm.build(s, arg_bufs, 'rocm', name='matmul'), it failed with the following error:
WARNING:autotvm:Too many errors happen in the tuning. Now is in debug mode
Finish loading 500 records
DEBUG:autotvm:Finish loading 500 records
Cannot find config for target=rocm, workload=('tvm_matmul_tune_op', 4, 256, 256). A fallback configuration is used, which may bring great performance regression.
WARNING:autotvm:Cannot find config for target=rocm, workload=('tvm_matmul_tune_op', 4, 256, 256). A fallback configuration is used, which may bring great performance regression.

Best config:
,None,None
[14:47:54] /host/docker/matmul_tvm/tvm/src/pass/vectorize_loop.cc:362: Detect vector condition in Vectorized Loop, scalarizing...
[14:47:54] /host/docker/matmul_tvm/tvm/src/pass/vectorize_loop.cc:362: Detect vector condition in Vectorized Loop, scalarizing...
Traceback (most recent call last):
  File "matmul_autotvm.py", line 260, in <module>
    search_matmul_config(4, 256, 256, 500) # m, k, n, num_trials
  File "matmul_autotvm.py", line 165, in search_matmul_config
    func = tvm.build(s, arg_bufs, 'rocm', name='matmul')
  File "/host/docker/matmul_tvm/tvm/python/tvm/build_module.py", line 617, in build
    fhost, mdev = _build_for_device(flist, tar, target_host)
  File "/host/docker/matmul_tvm/tvm/python/tvm/build_module.py", line 484, in _build_for_device
    mdev = codegen.build_module(fdevice, str(target)) if fdevice else None
  File "/host/docker/matmul_tvm/tvm/python/tvm/codegen.py", line 36, in build_module
    return _Build(lowered_func, target)
  File "/host/docker/matmul_tvm/tvm/python/tvm/_ffi/_ctypes/function.py", line 206, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (2) /host/docker/matmul_tvm/tvm/build_rocm/libtvm.so(TVMFuncCall+0x61) [0x7f9598de3f01]
  [bt] (1) /host/docker/matmul_tvm/tvm/build_rocm/libtvm.so(+0x14b2e9) [0x7f95986992e9]
  [bt] (0) /host/docker/matmul_tvm/tvm/build_rocm/libtvm.so(+0x231aaa) [0x7f959877faaa]
  File "/host/docker/matmul_tvm/tvm/src/codegen/codegen.cc", line 46
TVMError: Check failed: bf != nullptr: Target rocm is not enabled

All target TVM libraries link to ROCm successfully:
/host/docker/matmul_tvm/tvm/build# ldd libtvm.so
        linux-vdso.so.1 (0x00007fff995ed000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc8f0e65000)
        libhip_hcc.so => /opt/rocm/lib/libhip_hcc.so (0x00007fc8f0560000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc8f01d7000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc8efe39000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc8efc21000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc8ef830000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc8f1d7f000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc8ef611000)
        libhc_am.so => /opt/rocm/lib/libhc_am.so (0x00007fc8ef3b1000)
        libmcwamp.so => /opt/rocm/lib/libmcwamp.so (0x00007fc8ef197000)
        libhsa-runtime64.so.1 => /opt/rocm/hsa/lib/libhsa-runtime64.so.1 (0x00007fc8eeee0000)
        libhsakmt.so.1 => /opt/rocm/lib/libhsakmt.so.1 (0x00007fc8eecbc000)
        libelf.so.1 => /usr/lib/x86_64-linux-gnu/libelf.so.1 (0x00007fc8eeaa2000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc8ee89a000)
        libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007fc8ee68f000)
        libpci.so.3 => /lib/x86_64-linux-gnu/libpci.so.3 (0x00007fc8ee482000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc8ee265000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fc8ee04a000)
        libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fc8ede2c000)

I printed all the registered providers:
For TVM built with CUDA on, TVM runtime detects following providers:
[codegen.build_aocl] [codegen.build_opengl] [codegen.build_cuda] [codegen.build_metal] [codegen.build_opencl] [codegen.build_aocl_sw_emu] [codegen.build_sdaccel] [codegen.build_c] [codegen.build_stackvm]
and no errors for the whole tuning procedure.
For TVM built with OpenCL on, TVM runtime detects following providers:
[codegen.build_aocl] [codegen.build_opengl] [codegen.build_metal] [codegen.build_opencl] [codegen.build_aocl_sw_emu] [codegen.build_sdaccel] [codegen.build_c] [codegen.build_stackvm]
and no errors for the whole tuning procedure.
For TVM built with ROCm on, TVM runtime detects following providers:
[codegen.build_aocl] [codegen.build_opengl] [codegen.build_metal] [codegen.build_opencl] [codegen.build_aocl_sw_emu] [codegen.build_sdaccel] [codegen.build_c] [codegen.build_stackvm]
So it's clear that codegen.build_rocm is not detected.

Have you enabled LLVM? The rocm target won't be enabled if USE_LLVM is off (even if USE_ROCM is on)

@masahi After enabling LLVM, the runtime turned from undetected exception into segment fault crash:
ul_tvm/tune_rocm/tvm/build/libtvm.so(+0x1addf2) [0x7f0ca5986df2]\n  [bt] (4) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x37ef43) [0x7f0ca5b57f43]\n  [bt] (3) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x375624) [0x7f0ca5b4e624]\n  [bt] (2) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x356601) [0x7f0ca5b2f601]\n  [bt] (1) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x355983) [0x7f0ca5b2e983]\n  [bt] (0) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x146ee3) [0x7f0ca591fee3]\n  File "/host/docker/matmul_tvm/tune_rocm/tvm/src/pass/inject_double_buffer.cc", line 217\nTVMError: Check failed: loop_nest_.size() != 0U (0 vs. 0) : Double buffer scope must be inside a loop',),), error_no=2, all_cost=0.020066261291503906, timestamp=1555834931.7148871)     [('tile_k', [1, 16, 256]), ('tile_y', [2, 128, 8, 2]), ('tile_x', [4, 1, 256, 4]), ('auto_unroll_max_step', 512)],,None,9493746
No: 9   GFLOPS: 0.00/0.00       result: MeasureResult(costs=(TimeoutError(),), error_no=6, all_cost=10, timestamp=1555834931.71503)     [('tile_k', [16, 32, 8]), ('tile_y', [2, 1, 8, 256]), ('tile_x', [16, 2, 16, 8]), ('auto_unroll_max_step', 1500)],,None,30057978
No: 10  GFLOPS: 0.00/0.00       result: MeasureResult(costs=(TimeoutError(),), error_no=6, all_cost=10, timestamp=1555834937.216011)    [('tile_k', [2, 2048, 1]), ('tile_y', [64, 8, 4, 2]), ('tile_x', [32, 4, 4, 8]), ('auto_unroll_max_step', 1500)],,None,29449613
Finish loading 558 records
Cannot find config for target=rocm, workload=('tvm_matmul_tune_op', 4096, 4096, 4096). A fallback configuration is used, which may bring great performance regression.

Best config:
,None,None
[08:22:17] /host/docker/matmul_tvm/tune_rocm/tvm/src/pass/vectorize_loop.cc:362: Detect vector condition in Vectorized Loop, scalarizing...
[08:22:17] /host/docker/matmul_tvm/tune_rocm/tvm/src/pass/vectorize_loop.cc:362: Detect vector condition in Vectorized Loop, scalarizing...
'gfx906' is not a recognized processor for this target (ignoring processor)
'gfx906' is not a recognized processor for this target (ignoring processor)
free(): invalid pointer
Aborted (core dumped)

Is your card really gfx906? Then I think you need a fairly recent LLVM.

@masahi Yes, all other rocm apps run well. I am using LLVM 6.0, can you suggest a specific version that you have tested for ROCm 2.x? Seems like I tried some other LLVM version but not working well for TVM.

I have only used LLVM 6.0 with my gfx803 card. If other rocm apps are working, you can try the same LLVM version that comes with your rocm installation.

@masahi I upgraded to llvm-8.0 this time, and TVM throws another kinds of exception:
[12:52:35] /host/docker/matmul_tvm/tune_rocm/tvm/src/pass/vectorize_loop.cc:362: Detect vector condition in Vectorized Loop, scalarizing...
[12:52:35] /host/docker/matmul_tvm/tune_rocm/tvm/src/pass/vectorize_loop.cc:362: Detect vector condition in Vectorized Loop, scalarizing...
[codegen.build_aocl] [codegen.build_opengl] [codegen.build_metal] [codegen.build_opencl] [codegen.build_aocl_sw_emu] [codegen.build_sdaccel] [codegen.build_c] [codegen.build_stackvm] [codegen.build_rocm] [codegen.build_nvptx] [codegen.build_llvm] ..
Traceback (most recent call last):
  File "matmul_autotvm.py", line 259, in <module>
    search_matmul_config(64, 4096, 4096, 100) # m, k, n, num_trials
  File "matmul_autotvm.py", line 165, in search_matmul_config
    func = tvm.build(s, arg_bufs, 'rocm', name='matmul')
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/build_module.py", line 617, in build
    fhost, mdev = _build_for_device(flist, tar, target_host)
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/build_module.py", line 484, in _build_for_device
    mdev = codegen.build_module(fdevice, str(target)) if fdevice else None
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/codegen.py", line 36, in build_module
    return _Build(lowered_func, target)
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/_ffi/_ctypes/function.py", line 206, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (5) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(TVMFuncCall+0x61) [0x7f4af81a0d51]
  [bt] (4) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x159819) [0x7f4af7a0e819]
  [bt] (3) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x23f655) [0x7f4af7af4655]
  [bt] (2) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x27aee6) [0x7f4af7b2fee6]
  [bt] (1) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x894d94) [0x7f4af8149d94]
  [bt] (0) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x8e839b) [0x7f4af819d39b]
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/_ffi/_ctypes/function.py", line 71, in cfun
    rv = local_pyfunc(*pyargs)
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/contrib/rocm.py", line 67, in callback_rocm_link
    rocm_link(tmp_obj, tmp_cobj)
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/contrib/rocm.py", line 39, in rocm_link
    stderr=subprocess.STDOUT)
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ld.lld': 'ld.lld'

@masahi Seems like TVM is not working well with ROCm 2.2

hmm I don't know what's happening. It's been a while since I used rocm backend. Does a very simple example like vector add work?

@masahi Can you provide such an sample? Maybe the code I am using not correct as well.

https://github.com/dmlc/tvm/blob/master/tutorials/tensor_expr_get_started.py
You can replace cuda with rocm.

@masahi Also failed with the same error:
vecadd# python3 example.py
<class 'tvm.tensor.Tensor'>
Traceback (most recent call last):
  File "example.py", line 155, in <module>
    fadd(a, b, c)
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/_ffi/function.py", line 144, in __call__
    return f(*args)
  File "/host/docker/matmul_tvm/tune_rocm/tvm/python/tvm/_ffi/_ctypes/function.py", line 206, in __call__
    raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (3) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(TVMFuncCall+0x61) [0x7fb25ce7ed51]
  [bt] (2) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x95b917) [0x7fb25ceee917]
  [bt] (1) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x95b64f) [0x7fb25ceee64f]
  [bt] (0) /host/docker/matmul_tvm/tune_rocm/tvm/build/libtvm.so(+0x1470a3) [0x7fb25c6da0a3]
  File "/host/docker/matmul_tvm/tune_rocm/tvm/src/runtime/rocm/rocm_module.cc", line 98
  File "/host/docker/matmul_tvm/tune_rocm/tvm/src/runtime/module_util.cc", line 73
ROCMError: Check failed: ret == 0 (-1 vs. 0) : hipModuleGetFunction myadd_kernel0 failed with error: hipErrorNotFound

Ok thanks. I will take a look

hi, @masahi @ghostplant
I have same problem, and I found the solution that works for me. So, I would sharing my experience on this. The hipErrorNotFound error was got when your environment is ROCm2.3 (not compatible with LLVM6 or 7) or use LLVM8+.
The reason is that LLVM uses -mattr=+code-object-v3 in default for HSACO generation, but the ROCm hipModuleLoad function which only accepts a HSACO with code object v2.
It means you need to pass -mattr=-code-object-v3 as the option explicitly. Note the value given to -mattr,  the plus sign + be replaced to minus sign -.

@masahi My error is caused on LLVM 8+. It is interesting why TVM for ROCm has to use LLVM as hcc from ROCm is already based on LLVM clang.

thanks @fundamat for the info, this is very helpful.
@ghostplant From rocm we only use their HIP runtime. For codegen all we need is LLVM's AMDGPU backend. The hcc compiler is for compling HIP code (cuda like language supported by AMD). We don't generate HIP but LLVM IR directly, so we don't need hcc.

@masahi Thanks, hope to get fixed soon for ROCm 2.3

hi @fundamat can you point me to the documentation so that I can understand your finding?

@masahi
You can get more information at here: https://llvm.org/docs/AMDGPUUsage.html#code-object-metadata
And I just simply add the -mattr=-code-object-v3 in BuildAMDGPU at codegen_amdgpu.cc:182
config << "-mtriple=amdgcn-amd-amdhsa-hcc -mcpu=gfx"
       << DetectROCMComputeVersion(target) << " -mattr=-code-object-v3 "
       << target.substr(4, target.length() - 4);

Change target also works, no need to change codegen.
On Wed, Apr 24, 2019 at 19:21 fundamat<EMAIL_ADDRESS>wrote:

@masahi https://github.com/masahi
You can get more information at here:
https://llvm.org/docs/AMDGPUUsage.html#code-object-metadata
And I just simply add the -mattr=-code-object-v3 in BuildAMDGPU at
codegen_amdgpu.cc:182
config << "-mtriple=amdgcn-amd-amdhsa-hcc -mcpu=gfx"
<< DetectROCMComputeVersion(target) << " -mattr=-code-object-v3 "
<< target.substr(4, target.length() - 4);
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/tvm/issues/3058#issuecomment-486495030, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAJTLXUHKWTG4U2I2AMFVCLPSEIS7ANCNFSM4HHJ2GPA
.

--
Bing Xu

@fundamat After applying your patch, I tried the tuning again, and hipErrorNotFound are solved, but following "Segment Fault" again as I explained earlier.
I am using LLVM-8 (current lastest stable version), and ROCm-2.3. As LLVM-9 is not stable so I didn't use it.
Can you share your specific library version for LLVM and ROCm?

@ghostplant
I have no idea on "Segment Fault" problem, but I'd be happy to share my library version used:
LLVM-9 (manually build)
ROCm 2.3

@fundamat I built the lastest LLVM-9 on ubuntu 18.04 but the makefile fail on half way based on gcc-7.
@msakai Is there a Dockerfile that contains TVM for ROCm environment which is reproducible?

No sorry, our ROCm support is limited (not many people are interested) and I'm not familiar with docker. You can add one if you like.

@masahi If I can build TVM for ROCm without problems, I can write one. However, the environment for TVM + ROCm is not working, and the purpose I ask for a Dockerfile for ROCm is just to get an environment that uses TVM + ROCm without failures.

I've tested ROCm 2.3 + LLVM 6.0 on my gfx803 card. vector add test and VGG inference work. But for some workload tvm crashes with
### HCC STATUS_CHECK Error: HSA_STATUS_ERROR_INVALID_ISA (0x100f) at file:mcwamp_hsa.cpp line:1195

In particular, resnet workload doesn't work. I'll try upgrading LLVM.
@fundamat What is your ld.lld version? Is tutorial/relay_quick_start.py working for you?

@ghostplant Does segfault still occur on vector add? Please try the simplest test first.

I've tested ROCm 2.3 + LLVM 6.0 on my gfx803 card. vector add test and VGG inference work. But for some workload tvm crashes with
### HCC STATUS_CHECK Error: HSA_STATUS_ERROR_INVALID_ISA (0x100f) at file:mcwamp_hsa.cpp line:1195

In particular, resnet workload doesn't work. I'll try upgrading LLVM.
@fundamat What is your ld.lld version? Is tutorial/relay_quick_start.py working for you?

# [RUN TEST]
docker run -it --privileged --network=host tvm-rocm python3 tensor_expr_get_started.py

And this is my output:
/root/docker$ docker run -it --privileged --network=host tvm-rocm python3 tensor_expr_get_started.py
<class 'tvm.tensor.Tensor'>
'gfx906' is not a recognized processor for this target (ignoring processor)
'gfx906' is not a recognized processor for this target (ignoring processor)
free(): invalid pointer

It seems you need a newer LLVM.

@masahi OK, I updated to 7, 8, 9 respectively (updated in my scripts), and gfx906 warning no longer exists, but they all outputs Segmentation fault (core dumped).

@masahi I rebuilt llvm-9 from source and segfault is solved, but installing llvm-9 from binary will segfault them. After that, I tried tuning applications, and a very small group of applications can run successfully, while the majority will output various exceptions, while all cases run well using cuda / opencl.

@masahi I have another question. Seems like source code generation after tuning is not working for ROCm platform, as TVM codegen using other platform like CUDA, OpenCL can output respective kernel source code, but TVM codegen for ROCm platform just outputs nothing..

you can use get_source("llvm") or get_source("asm").

@masahi OK, but how can I compile them after getting the llvm/asm-level source code?

@masahi For asm, I think I can follow https://gpuopen.com/amdgcn-assembly/, but how about codes from llvm format?

The output of get_source is not meant to be combined by other means. What do you want to do?

@masahi I want to tune an op with specific config, get the best-working kernel source code, and freeze it as a hipModule.

given that this question has evolved to more general discussions, I would recommend starting a conversation on https://discuss.tvm.ai/ :)

@masahi Do you know why CUDA/OpenCL platform can get C source, but ROCm platform can't?

Because CUDA and OpenCL backends generate kernel source as C strings, while ROCm backends generates LLVM IR

@antinucleon Actually, most of my tuning results in this failure but not happen in CUDA backend:
``sh
error: local memory limit exceeded (73728) in default_function_kernel0
Do you know the solution to this error?

This is not clean, but you can modify this block for rocm target.
 if 'cuda' in self.task.target.keys or 'opencl' in self.task.target.keys:
            remote = request_remote(self.key, self.host, self.port)
            ctx = remote.context(str(self.task.target), 0)
            max_dims = ctx.max_thread_dimensions
            kwargs['check_gpu'] = {
                'max_shared_memory_per_block': ctx.max_shared_memory_per_block,
                'max_threads_per_block': ctx.max_threads_per_block,
                'max_thread_x': max_dims[0],
                'max_thread_y': max_dims[1],
                'max_thread_z': max_dims[2],
            }

For rocm, max_shared_memory_per_block should be 48KB, and max threads per block should be 256. Don't forget to add "if 'rocm in self.task.target.keys".

@masahi Nice, it works.

@masahi Do you know whether this warning matters for tuning?
DEBUG:autotvm:No: 1000  GFLOPS: 0.00/0.00       result: MeasureResult(costs=(InstantiationError('Traceback (most recent call last):\n  [bt] (1) /opt/tvm/build/libtvm.so(TVMFuncCall+0x65) [0x7fc0db08f915]\n  [bt] (0) /opt/tvm/build/libtvm
.so(+0x8f2e8b) [0x7fc0db08be8b]\n  File "/opt/tvm/python/tvm/_ffi/_ctypes/function.py", line 71, in cfun\n    rv = local_pyfunc(*pyargs)\n  File "/opt/tvm/python/tvm/autotvm/measure/measure_methods.py", line 595, in verify_pass\n    rais
e InstantiationError("Skipped because of invalid gpu kernel")\ntvm.autotvm.task.space.InstantiationError: Skipped because of invalid gpu kernel',),), error_no=1, all_cost=0.02058720588684082, timestamp=1557040253.856357)    [('tile_k', [
16, 2, 64]), ('tile_y', [1, 2, 32, 32]), ('tile_x', [1, 1024, 1, 2]), ('auto_unroll_max_step', 1500)],,None,12857110
Too many errors happen in the tuning. Now is in debug mode
WARNING:autotvm:Too many errors happen in the tuning. Now is in debug mode

If you specify constraints on shared mem size, number of threads, etc, verify_pass would reject kernel configurations that do not satisfied that constraints (invalid configurations are still considered because of random nature of autotvm). I think the warning you got is the result of rejection.
As long as autotvm visits configuration that satisfy your constraints, autotvm should be able to make progress.
I also recommend making 'auto_unroll_max_step' smaller. 1500 is too big and causes a pressure on register usage. 125-256 should be good.

@masahi Thanks, I updated this parameter, though I didn't find the tuning result get better.

@tqchen I cannot sign up Tvm discuss successfully, my email server is qq.com and I cannot receive the activation email, also no new emails from junk category.


Upgrade cluster to 4.10.* - add delay after roles creation

CS first attempts to validate the new roles right after the creation fails.
Add 5 seconds delay with a spinner to reduce the number of retries on conflict in CS.
It's better to wait in the CLI, instead of blocking both CLI and CS.

Related: SDA-5753

@tzvatot @pvasant PTAL

lgtm, best if @pvasant  will also TAL

/lgtm


Accessing host docker daemon from container

Is there a well defined way to have host docker daemon accessible within a container? My current approach is to mount /var/run/docker.sock inside the container when I run it, then use the http api on the socket. Is this a safe way?

@eknkc This is only 'safe' if you're not allowing untrusted code and people to do this. You probably want to allow this only for something you're dong on your development box, not on a build server in production.
I believe this question has been answered. I'm going to close the issue now. Please feel free to comment if you have any more questions.


feat: validate email on login

Summary
Add basic email validation before attempting to send the
magic link through Notify.
Also fix a bug with the prod Docker build/push workflow.
Related

#298

Looks good! I like how you added email validation - we should have thought of this earlier!

This is me worrying about our Notify API key 😅


Sqlalchemy field names can break flask admin

This is more of a note for future searching by me or others, though getting it into the docs would be nice.
Flask admin dynamically constructs wtforms from your sqlalchemy models. This means that your sqlalchemy fields are properties on the wtform. Sounds great! However, sqlalchemy lets you name fields anything you want, such as process. As it happens, process is a preexisting property in wtforms. Specifically, it's a function used for constructing the form.
If you have a sqlalchemy model that has a process field, it will silently override the function in wtforms. Then, during page creation, you get the following obtuse stack trace (note the process function call at the end):
Traceback (most recent call last):
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 130, in handle
    self.handle_request(listener, req, client, addr)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 171, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask_debugtoolbar/__init__.py", line 124, in dispatch_request
    return view_func(**req.view_args)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask_admin/base.py", line 68, in inner
    return self._run_view(f, *args, **kwargs)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask_admin/base.py", line 359, in _run_view
    return fn(self, *args, **kwargs)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask_admin/model/base.py", line 1668, in create_view
    form = self.create_form()
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask_admin/model/base.py", line 1063, in create_form
    return self._create_form_class(get_form_data(), obj=obj)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/wtforms/form.py", line 212, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/flask_admin/form/__init__.py", line 13, in __init__
    super(BaseForm, self).__init__(formdata=formdata, obj=obj, prefix=prefix, **kwargs)
  File "/home/srathbun/development/google-ical-integration/env/local/lib/python2.7/site-packages/wtforms/form.py", line 278, in __init__
    self.process(formdata, obj, data=data, **kwargs)
TypeError: __call__() takes exactly 1 argument (4 given)

Be aware of this when constructing your sqlalchemy models, and if you run into another weird issue, try changing your field names. If the problem goes away, you have another conflict somewhere.

Sounds like we need some specific warnings/errors raised when conflicting field names are encountered, or some kind of workaround to transcode them to a useable name transparently?

got same error. And "process" is very important word in domain in which I'm working now

It is limitation of wtforms library. Yes, it has certain methods in the Form class that can be overridden and break it.
Alternative solution would be optional automatic field prefixing, but this might complicate application development, as developer would have to remember that field is called "name" in a model and "field_name" in a form (very important for all sort of on_model_change hooks).

It is sad to hear that. But it can not be helped. Thank you

Closing this now. But please re-open if it's still a problem.

Does anyone have a workaround for this? Aside from renaming the field in the database?


no issue really

hi mate,
where can we discuss your project? if we have questions for example: very long cache / and matching wrong comics from vine etc?
appreciate a link of something to any boards when i can ask questions? thanks

The project is not dead, however it is on hiatus. I’ve unfortunately had to put it aside due to family matters over the last 8 months. I am hoping to pick it back up again at some point soon.
As a consumer of FOSS software, I know this can be frustrating, but please understand, as the sole maintainer it can be very difficult to continue traction when it is a side-project.
To answer your original question: GitHub is currently where we communicate about Tenma. If you have a specific question, you are welcome to search through the issues, or create one yourself.
I’ll close this comment since it answers your original question, but feel free to open another issue if you have any other questions!


[option] Switching case doesn't apply to the current character

Hello, I'm a bit troubled by some default behavior of tt9: switching case changes the case of the last entered character. I think some people (including me) expect it to change the case of the next character. Would it be possible to add an option that activates that behavior?

Hey!
From what I understand, you would like to have the equivalent of CAPS LOCK on a computer keyboard. This is certainly possible, but changing the text case would have to happen via another key. I can't simply throw away the current functionality, because the #-key does not simply change the text case, but it changes the input mode (123, ABC, abc, Predictive). And I am sure, you would want that, too.
In short, it is possible to add a new CAPS LOCK hotkey, but changing the text case is possible even now, so I consider this issue of low priority.

I seem to recall that Clam-TT9 was kinda clever about this at one point - pressing mode-change while a suggestion was active would (as TT9 currently does) cycle thru the capitalization options, but would THEN drop back to standard lowercase once the suggestion was accepted.
This MIGHT prove a worthwhile mechanism to move since then it could just be a logic inversion and a toggle in the options. i.e. 'Capitalize previous/Capitalize Next on mode-key'

I don't think I was clear about what I meant.
Currently, if I type 2 # it will type a then change its case to A. I think there should be an option to change that behavior to selecting the letter a then changing the mode to capital letters.
More generally I think that other key presses than the last key press should have the same effect as Ok followed by that key. Eg left and right arrows should accept the current letter then go back or forward. I'm only talking about modes AB and ab here, not the actual T9.

I seem to recall that Clam-TT9 was kinda clever about this at one point - pressing mode-change while a suggestion was active would (as TT9 currently does) cycle thru the capitalization options, but would THEN drop back to standard lowercase once the suggestion was accepted.

But this still works. In Predictive mode, the text case changes automatically, even taking into account personal name fields (where every word is supposed to start with a capital) or email/password fields, where text case must not be enforced.

Currently, if I type 2 # it will type a then change its case to A. I think there should be an option to change that behavior to selecting the letter a then changing the mode to capital letters.

The option exists even now. Just do as you described, press # to change the mode and select "AB" or "ab", depending on your need.

Eg left and right arrows should accept the current letter then go back or forward. I'm only talking about modes AB and ab here

Agreed.

Currently, if I type 2 # it will type a then change its case to A. I think there should be an option to change that behavior to selecting the letter a then changing the mode to capital letters.

The option exists even now. Just do as you described, press # to change the mode and select "AB" or "ab", depending on your need.

I think we are talking about two different things. Currently, to type Ad I need to press: 2, OK, #, 3. I think this does not feel logical and would like to press 2, #, 3 and not have to type on OK.

Currently, to type Ad I need to press: 2, OK, #, 3

Actually, there is no need to press OK after every key. Let's say you want to type "Adam", then you would press: 2, 3#, 2 , 6, OK to get the word. Or you could even press 0 at the end to type a space and start a new word. See, it really takes the minimum amount of key presses, it's just that you press # after the letter, not before it.
Now, I could technically add a hotkey for changing the text case only, but it feels redundant. It will not speed typing and it just copies existing functionality...