Should CS still be deasserted when flush fails?

https://github.com/rust-embedded/embedded-hal/blob/84dfc74b59279eb433fde0c2b928570c4da59136/embedded-hal-bus/src/spi/shared.rs#L29
If that flush fails, shouldn't the error fall through to the de-assertion at the end of the function instead of returning immediately?

Isn't the result saved in op_result, which is bubbled up later, after the deassertion? The comment hints that this is exactly the intention.
So CS will be deasserted also in case of an error.

Apologies, I can't read :sweat_smile:, I missed the fact that that bit is in a closure, so the ? does the right thing.


Advisory search: sort by average_severity does not work

Some Context
Given the Advisories UI table (image below), I'd like to sort Advisories by Severity.

The Advisory UI Table is populated by GET /api/v1/advisory. The JSON is similar to:

{
    "items": [
        {
            "identifier": "CVE-2023-20862",
            "hashes": [
                "sha256:e6a2bc1084c77809af965dc4facb0c28aca67210f054eb8de4778190b8c6347e"
            ],
            "issuer": {
                "id": 2,
                "name": "Red Hat Product Security",
                "cpe_key": null,
                "website": null
            },
            "published": "2023-04-19T00:00:00Z",
            "modified": "2023-11-14T22:09:12Z",
            "title": "spring-security: Empty SecurityContext Is Not Properly Saved Upon Logout",
            "average_severity": "medium",... this is the field I want to SortBy
            "average_score": 6.3,
            "vulnerabilities": [
                {
                    "identifier": "CVE-2023-20862",
                    "non_normative": {
                        "non_normative": true,
                        "identifier": "CVE-2023-20862",
                        "title": "spring-security: Empty SecurityContext Is Not Properly Saved Upon Logout",
                        "discovered": "2023-07-31T00:00:00Z",
                        "released": "2023-04-19T00:00:00Z"
                    },
                    "severity": "medium",
                    "score": 6.3
                }
            ]
        },
    ],
    "total": 2595
}

Here is the problem
When I try to search Advisories and order them by average_severity I get an error. The request I am executing is:
GET /api/v1/advisory?sort=average_severity:asc&limit=10&offset=0

And this is the exact error I get from the backend 400 Bad Request:
{
    "error": "Query error",
    "message": "query syntax error: Invalid field name for sort: 'average_severity'"
}

This might  be just me not knowing how to properly create the query for sorting, or it might be a real bug.
Could you help me with this please?

On a side note. The query for sorting by published works so my queries are grammatically correct
GET /api/v1/advisory?sort=published:asc&limit=10&offset=0

The average_severity string is a synthetic string generated based upon the average_score just as a helper.
Can the UI treat sort-by-severity as exactly sort-by-score?
Else, I presume it'll sort lexographically on the strings "high" "medium" "low" which... isn't very sensible.

This one will be tricky. Because the results are post-processed after being fetched from the db. Unless it's possible to move that processing logic into the db (as a view or some stored proc), I'm not sure how make the pagination work correctly if sorting on one of those calculated fields.

After package?q= I'll look into if we can't SQL calculate that column (or at least the score, if not the severity word) so perhaps it'd be sortable using your stuff @jcrossley3

Cool. In general, we probably need to be conscious of the fact that folks might try to sort on any field they see in a JSON response. If those fields are calculated, expectations may not be met.

I think this harkens back to the Sikula discussion a bit.  Since we DTO, things in JSON may not be in the DB, or at least not in the JSON-shaped format.
In this case, I suspect I can sort out some SQL aggregate/sum/average to give us a named column, even if synthetic, but we may hit a wall at some point.

Can the UI treat sort-by-severity as exactly sort-by-score?

Technically possible. But I believe just like me, any other client will try to pursue my same journey: "sort by average_severity"
Generally speaking i think this specific thing should be addressed in the server side and not on the client side.
Filtering
@bobmcwhirter @jcrossley3
I just tried to also add filtering to the table "filter by average_severity" which is a very valid use case for a client: E.g. "give  me all advisories with high average_severity". And I have hit the same issue: "query syntax error: Invalid field name for filter: 'average_severity'".
I thought about opening another issue as the title of this issue refers only to SORTING, but I think whatever is the solution it might apply to both Sorting and Filtering.

Just so we're clear: our filtering/sorting/pagination stuff ONLY works on database fields.
Any required filtering/sorting/pagination of DTO's will need to be done by hand within the API request handlers.

@bobmcwhirter I confirm SortBy average_score works. But FilterBy average_severity does not work. The title of this issue is exclusively for SORTING so I am happy to close this issue and open a new one for Filtering by average_severity


Bower install moment-recur#1.0.5 provides old code

In the current (latest) version the moment.tz call in dateOnly is like this (which is correct)
    return moment.tz(this.format('YYYY-MM-DD'), 'UTC');

However, when you bower install you get an old version
    return moment.tz(this.format("YYYY/MM/DD"), "UTC");

which will throw a deprecation warning

Bumped the bower config version - this should work correctly now.


Custom logos read from files

If the logo specified with --logo isn't recognized, the program will check if it's a file that can be read, and if it is, the contents of the file will be used. Otherwise it'll use the unknown logo.

I'll fix all of this when I get some time later. Sorry if some things don't
make much sense, I'm pretty much inexperienced in both C development and
doing pull request.
The logo name being amogus was something I forgot to remove.
strlen gave me wrong results when trying to load logos that included UTF-8
characters, so I added that function to support them.
I didn't know about getline, as I said I barely have experience with C.
I'll take a look at it.
El mié., 26 may. 2021 20:48, Linus Dierheimer @.***>
escribió:

@LinusDierheimer commented on this pull request.
In README.md
https://github.com/LinusDierheimer/fastfetch/pull/57#discussion_r640023556
:

+This fork adds custom logos read from files:
+

This changes are fine for your fork but shouldn't be in this repository.
In src/common/logo.c
https://github.com/LinusDierheimer/fastfetch/pull/57#discussion_r640026982
:

const char* color = doColor ? "\033[91m" : "";

strcpy(logo->color, "\033[91m");

I don't think we should set the same color for every logo loaded from
file. Use the --color option to set it.
In src/common/logo.c
https://github.com/LinusDierheimer/fastfetch/pull/57#discussion_r640027637
:

@@ -249,6 +269,37 @@ static void loadGarudaLogo(FFlogo* logo, bool doColor)
sprintf(logo->chars[19], FASTFETCH_TEXT_MODIFIER_BOLT"%s                                              "FASTFETCH_TEXT_MODIFIER_RESET, color);
}

+static void loadLogoFromFile(FFlogo* logo, bool doColor, char* pathToFile)
+{

strcpy(logo->name, "amogus");

No need to do this, as we never load the logo by name. Maybe just set a
null byte at index 0 to be save.
In src/common/logo.c
https://github.com/LinusDierheimer/fastfetch/pull/57#discussion_r640028607
:

while(!feof(fp)) {

   fgets(buff, sizeof(buff), fp);

   normalizedBuff = strtok(buff, "\n");

   sprintf(logo->chars[logoHeight++],   FASTFETCH_TEXT_MODIFIER_BOLT"%s%s"FASTFETCH_TEXT_MODIFIER_RESET, color, normalizedBuff);

   len = utf8_strlen(normalizedBuff);

   logoWidth = len > logoWidth

       ? len

       : logoWidth;

}

Wouldn't it be simpler to use getline?
In src/common/logo.c
https://github.com/LinusDierheimer/fastfetch/pull/57#discussion_r640031317
:

else if(access(logo, R_OK) == 0)

   loadLogoFromFile(&config->logo, config->colorLogo, logo);

I think we should also introduce an extra option for this. Maybe
--logo-file? This way the user has better control about what logo is
loaded. Anyway we can leave this a a fallback for --logo.
In src/common/logo.c
https://github.com/LinusDierheimer/fastfetch/pull/57#discussion_r640034882
:

const char* color = doColor ? "\033[91m" : "";

strcpy(logo->color, "\033[91m");

FILE* fp = fopen(pathToFile, "r");
char buff[255];
char* normalizedBuff;
int len;

int logoWidth = 0, logoHeight = 0;
while(!feof(fp)) {

   fgets(buff, sizeof(buff), fp);

   normalizedBuff = strtok(buff, "\n");

   sprintf(logo->chars[logoHeight++],   FASTFETCH_TEXT_MODIFIER_BOLT"%s%s"FASTFETCH_TEXT_MODIFIER_RESET, color, normalizedBuff);

   len = utf8_strlen(normalizedBuff);

Im not into this topic, why isn't a simple strlen enough? The point of
Ascii logos is that they can be encoded using ascii so i'm not sure if we
need to support unicode.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/LinusDierheimer/fastfetch/pull/57#pullrequestreview-669411588,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJEFZTQQPTW6VZ5YECNXVRLTPU7BXANCNFSM45SQWNAA
.

Thanks for the contribution!

It doesn't load the OS logo anymore on my machine, but i will fix that by my self. You did great work providing this code to build upon.

Yes, in my machine both work as it should


Integration Test - Release - 2022/01/14 06:46:19

Auto create for integration test. IssueCommandFlow

#pri1

#pri2

Just type '#pri3'

#pri3

#assign PRMergerTest1

#assign NonAllowedUser

The following users don't have valid GitHub IDs, or are not collaborators on this repo: nonalloweduser

#reassign PRMergerTest2

#reassign NonAllowedUser

The following users don't have valid GitHub IDs, or are not collaborators on this repo: nonalloweduser

#please-close

GitHub user PRMergerTest2 has been removed from the MicrosoftDocs organization, so they were automatically removed as an assignee.

#unassign PRMergerTest1

#unassign NonAllowedUser


⚠ Spryr has degraded performance

In 36c2f51, Spryr (https://spryr.com) experienced degraded performance:

HTTP code: 200
Response time: 4397 ms

Resolved: Spryr performance has improved in 2616597 after 7 minutes.


fix: handle fault for function stack not found in stack resources

Description of changes

handle the case where StackResources is empty or undefined and print a better message than 'Cannot read property 'PhysicalResourceId' of undefined'

Checklist

[x] PR description included
[x] yarn test passes

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #12084 (7865de8) into dev (984c0f8) will decrease coverage by 3.31%.
The diff coverage is 53.78%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##              dev   #12084      +/-   ##
==========================================
- Coverage   49.45%   46.14%   -3.31%
==========================================
  Files         692      820     +128
  Lines       33424    37578    +4154
  Branches     6840     7642     +802
==========================================
+ Hits        16529    17341     +812
- Misses      15370    18561    +3191
- Partials     1525     1676     +151

Impacted Files
Coverage Δ

...sync-simulator/src/data-loader/opensearch/index.ts
45.45% <0.00%> (ø)

...s/amplify-appsync-simulator/src/type-definition.ts
100.00% <ø> (ø)

...sync-simulator/src/velocity/util/dynamodb-utils.ts
6.55% <0.00%> (ø)

...y-appsync-simulator/src/velocity/util/map-utils.ts
10.00% <ø> (ø)

...tor/src/velocity/util/transform/dynamodb-filter.ts
4.93% <0.00%> (ø)

...ync-simulator/src/velocity/util/transform/index.ts
21.05% <0.00%> (ø)

...appsync-simulator/src/velocity/value-mapper/map.ts
86.88% <ø> (ø)

...ackages/amplify-category-auth/src/commands/auth.ts
0.00% <0.00%> (ø)

...es/amplify-category-auth/src/commands/auth/help.ts
0.00% <0.00%> (ø)

.../amplify-category-auth/src/commands/auth/import.ts
0.00% <0.00%> (ø)

... and 536 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Iframe should be ignored by rule "react/self-closing-comp"

Is there an existing issue for this?

[X] I have searched the existing issues and my issue is unique
[X] My issue appears in the command-line and not only in the text editor

Description Overview
Self closing <iframe> tag is not recommended and sometimes may cause unexpected behaviour/bugs in different browsers
I want to use react/self-closing-comp all over the project and every time I face iframe element I need to put comment to disable eslint rule
{/* eslint-disable-next-line react/self-closing-comp */}
<iframe
...
></iframe>

There are lots of iframe elements and I don't want to overload the project with such comments.
Expected Behavior
My proposal is to extend the configuration of html field of react/self-closing-comp rule to support tags that must be ignored. For instance:
"react/self-closing-comp": ["error", {
  "html": {
    ignore: ["iframe"]
  }
}]

This configuration has to ignore <iframe> tag when applying the rule
eslint-plugin-react version
v7.30.1
eslint version
v8.17.0
node version
v16.15.0

jsx isn't html, and a self-closing iframe tag in jsx can't possibly cause any problems in browsers, so this is a non-concern.


Developments

What does this PR do?
This pull request was created to check houndCI
Description of Task to be completed?
refactor of code and ensure no houndCI flag is raised
How should this be manually tested?
npm test
Any background context you want to provide?
No background context
Screenshots (if appropriate)
Not Applicable
Questions:
Not Applicable.

closed because hounci failed to acknowledge es6 syntax


Add signed mail support

This should be hopefully enough to support getting the attachments of signed mails. At least I was able to download the attached invoices that a company sends me. Regular attachments also still worked.
Another user also raised this issue some time ago https://github.com/munir131/attachment-downloader/issues/21

@lpirkwieser Apologies for the late response. Thanks for contribution


Rasa v3 to rasa x. When rasa 3.x can support Rasa X ?

What problem are you trying to solve?
When rasa 3.x can support Rasa X ?
Please give me release date. Because i'm trying to build digital assistant. So if i build it on Rasa 2.x it will expire soon. So i need to go on Rasa 3.x. I think its not only my opinion also so many developers wants to know new Rasa X release date.
What's your suggested solution?
No response
Examples (if relevant)
No response
Is anything blocking this from being implemented? (if relevant)
No response
Definition of Done
No response

Thanks for submitting this feature request 🚀 @alopez will get back to you about it soon! ✨

@sara-tagger he is on leave until January 6, 2022. So no one knows about new Rasa X release date?

@ganbaaelmer We don't have release date for you now, but we are actively working on this

Please head to the Compatibility Matrix page


plnkr.co is down

Currently I get the following error message if I go to www.plnkr.co:
"NO APPLICATION WAS FOUND FOR plnkr.co".
None of the plunker URLs work at this time (http://plnkr.co/edit/xxxxx?p=preview).
Can you please check it.
Thanks.

Like many, much larger sites, Plunker was affected by the DDoS attack on Dyn (see: http://dyn.com/blog/dyn-statement-on-10212016-ddos-attack/). Now that Dyn is back to business, Plunker's service is restored.


fix: Fix cloudflare_record snippet

tf-cloudflare_record snippet was a chef role
I did generate the snippets file but it created many differences due to some ^M (CRLF) characters in random places.

@dan-hughes re: was Chef role
err that's a bit of a shame from my side. thanks for the PR. Merging it now and will release to Marketplace. thanks


Avoid using "with" function. It is not recommended. Angular cant build …

…apps including "with" functions anymore.
Easy Fix to make it work and buildable.

@anowak Please merge this PR

@Anexo Oh, but this is not the source file. You just fixed the dist version. I think with appears during build process. I don't see it in the source code.

@Anexo Lit's rewrite the lib on TypeScript! ;-)


react package updated

🛠 Fixes issue(s)
Closes #163

👨💻 Change(s) proposed
Updated react packages and nanoid required a major update so updated that also.

✔ Checklist (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers

📷 Screenshots

Thanks for the PR, will have a look at it soon.

Looks good, thanks :muscle:


[Synapse] Revert linter work in purely auto generated packages

This PR reverts the linter error fixes done in #13650 and #13651 for Synapse packages as the code here is purely auto generated. Any manual fixes done here are futile when the code gets re-generated

cc @deyaaeldeen


[INFINITY-2205] Uninstall after reconciliation.

Create task kill steps for each task, processing kills as part of the plan.
This guarantees reconciliation is complete before the uninstall scheduler does anything.

@nickbp feedback addressed.


houseclub android;while registering,after writing name and username when ı click register button it says error loading data.I can see some rooms and even get in the rooms and listen but i can't search.and friends can't find me.

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Smartphone (please complete the following information):

Device: [e.g. Google Pixel]
OS: [e.g. Android 11]
Version [e.g. 1.0.7]

Additional context
Add any other context about the problem here.

You will have to logout and login again

I think this has something to do with how the program handle the "username taken" situation, I tried it with 2 of my friends I changed their username at registration and it worked just fine, but be careful you can change your username Just once.

I think this has something to do with how the program handle the "username taken" situation, I tried it with 2 of my friends I changed their username at registration and it worked just fine, but be careful you can change your username Just once.

Thanks ,it solvedy problem


Configure Dependabot to update Cargo and GitHub Action dependencies

Description of change
This change means we'll pick up new dependency updates on a regular basis for both crate dependencies in Mountpoint as well as our CI.
Relevant issues:

Closes #268

Tested in fork: https://github.com/dannycjones/mountpoint-s3/pulls?q=is%3Apr+is%3Aclosed+author%3Aapp%2Fdependabot
Does this change impact existing behavior?
No change to Mountpoint itself.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

Need to rebase, closing this and reopening #331


Able to pass arguments to function which do not match typing.

TypeScript Version:  2.1.6
Code

interface Data {
  something: string;
  somethingElse: number;
}

function test(arg: Partial<Data>) { }
function test2(arg: { [key: string]: any }) { }
function test3(arg: { [key: string]: string }) { }

test({ something: "true" }); // OK
test(() => false); // OK
test(false); // OK
test(true); // OK
test([]); // OK

test2({ something: "true" }); // OK
test2(() => false); // OK
test2(false); // Argument of type false not assignable to...
test2(true); // Argument of type true not assignable to...
test2([]); // OK

test3({ something: "true" }); // OK
test3(() => false); // Argument of type () => boolean not assignable to...
test3(false); // Argument of type false not assignable to...
test3(true); // Argument of type true not assignable to...
test3([]); // Argument of type undefined[] not assignable to...

Expected behavior:
Compiler should throw an error when assigning parameters which do not match the interface or parameter type in a function call.
Actual behavior:
test3 has expected behaviour. test2 and test1 have unexpected behaviour on most types of invalid arguments.

The playground (http://bit.ly/2lfXje9) seems to agree with your assessment.  Perhaps your tsconfig is off?

Yeah, correct. What I am saying is that in the first and 2nd tests, there are false positives, compared to test3, which is correctly reporting the errors.

I think the way open types are handled, [] or () => false is assignable to {[key:string]: any}.  The Partial<Data> being assignable to true|false does seem a little weird though, but that may be me not understanding exactly how Partial works.


chore(falco-chart): update falcoctl version since it's outdated

What type of PR is this?
/kind feature
/kind chart-release
Any specific area of the project related to this PR?
/area falco-chart
What this PR does / why we need it:
The 'default' version of falcoctl in the chart's values.yaml is outdated. This is confusing and may lead to unexpected falcoctl behaviour.
Which issue(s) this PR fixes:
-
Checklist

[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] CHANGELOG.md updated

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: CarpathianUA
Once this PR has been reviewed and has the lgtm label, please assign cpanato for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @CarpathianUA, thanks for the PR!
Next week we are going to release a new chart for falco 0.36 which will ship also a newer version of falcoctl.

Hi @CarpathianUA, thanks for the PR!
Next week we are going to release a new chart for falco 0.36 which will ship also a newer version of falcoctl.


Querytimeout

Added setQueryTimeout method

Hi, this lib is written in coffee-script your pull request modifies the .js file.
lib folder is destination for the coffee-script and regenerated on each build.
All changes should be made to src/*.coffee files.
See here for building.
Your implementation is correct but as i said it should be made in src/statement.coffee file.
I leave this pleasure to you ;). Please make the changes and send a new pull request.
Thanks.


Support heterogeneous nodes

We should support clusters with different runtimes (is this actually a think ?). The current approach only supports homogeneous nodes (all nodes have the same container runtime).

I think we don't need this and actually I don't know of any clusters that are setup in this way. If for some reasons we need this feature we an open this issue again.


Rewrite everything and make it secure

Replace /dev/urandom related functionality and implement libsodium for better security (memory zeroing, scudo)

no :3


emit style on #restful_form

Hi a small one.
The style field is not emitted on element_restful_form
Add to line: 39 please:
	{style, WithName#restful_form.style},

Thanks! Fixed.


Add Larod

The acap-rs workspace doesn't currently provide wrappers for the larod library. When it is ready, I hope this merge request will provide that.
Immediate goals:

Provide safe Rust wrappers around the Larod functions, and abstract into rust structs and impls where appropriate.
Provide tests to verify functionality.

Since larod uses peripheral hardware, I plan to build the tests and execute them on an Axis P1468-LE.
Checklist before requesting a review

[ ] I have performed a self-review of my own code
[ ] I have verified that the code builds perfectly fine on my local system
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have verified that my code follows the style already available in the repository
[ ] I have made corresponding changes to the documentation

Just adding a comment that I am still relatively new to Rust, so suggestions to improve my code is very welcome. I will also be learning about functionality of the larod library during the process of creating this wrapper. So, guidance on interfacing with larod is also very welcome.

Looking at the larodError documentation a bit more closely, I realize what I have so far won't work, and maybe larod cannot be wrapped?
https://axiscommunications.github.io/acap-documentation/docs/api/src/api/larod/html/structlarodError.html
Notably larodError, which is passed to nearly every function, must be uninitialized. It might be impossible for rust to produce such a pointer https://doc.rust-lang.org/std/mem/union.MaybeUninit.html.

Since you say the code is not ready I will focus on answering your questions and assist you when you ask for it for now. Consequently I haven't looked much at the code but I address your comments below.

Just adding a comment that I am still relatively new to Rust

Diving in at the deep end with unsafe. A word of warning; In my opinion unsafe combines the worst of C(++) and Rust, and writing safe wrappers is especially frustrating since many of the safety aspects that we need to guarantee are not documented - sometimes not even considered -  by the library authors. To make matters worse, we often cannot even test empirically if our code is correct because UB does not mean that it will segfault or do anything weird right now, it means it could do it now or at any point in the future. So, know know what you are getting yourself into, that it is ok to quit and don't let this experience discourage you from using Rust because there's another side to the language that is carefree and fun :sun_with_face: :rainbow:
What is your programming background? I will try to adjust my communication accordingly.

I will also be learning about functionality of the larod library during the process of creating this wrapper.

That makes two of us.

Notably larodError, which is passed to nearly every function, must be uninitialized.

I have found that sometimes the documentation should be taken with a grain of salt (I have found errors) and other times it is important to read it carefully. In any case the examples can be a good supporting resource. in vdo_larod.c ^1 we find this:
larodError* error = NULL;
// ...
if (!larodConnect(&conn, &error)) {

We clearly don't pass an uninitialized variable but a pointer to null. I think this is what the documentation means when it says "An uninitialized handle to an error".
Compare this to send_event.c ^2 which is makes heavy use of glib and, presumably, follows glib error conventions:
GError* error = NULL;
// ...
if (!ax_event_handler_declare(event_handler,
                                  // ...
                                  &error)) {

Thanks for the feedback. My background is varied engineering (R&D, so jack of all trades master of none sort of thing). I've written production C++ for embedded microcontrollers, C++ for AI using OpenCV and libtorch, lots of Python, and a few web backend type services using Rust. Again, I'm no expert, but I've done a lot of dabbling in various things.
I'm ok diving into the deep end. Just more to learn hopefully :).
One piece from the larod docs that I worry a bit about is this quote, "error can also be NULL if one does not want any error information." I worry that if I initialize the pointer to Null, we'll just never get any error information back.

I'm comparing to axevent again because I recently worked on those bindings.
I think this is what passing a null pointer would look like: https://github.com/AxisCommunications/acap-native-sdk-examples/blob/9b00b2fdf23672f8910421653706572201c2ed8b/axevent/send_event/app/send_event.c#L167
Compare that to passing a (not-null) pointer to a null pointer: https://github.com/AxisCommunications/acap-native-sdk-examples/blob/9b00b2fdf23672f8910421653706572201c2ed8b/axevent/send_event/app/send_event.c#L176

@apljungquist I just set up the P1468-LE camera and set up a remote testing workflow. I've added some documentation and configuration files in this commit.
It might be useful for you in your endeavors. I have confirmed it works for the larod library so far on the P1468-LE running OS 11.11.73. It might be worth considering as a cherry pick, or I could open a separate merge request for it.

Neat. I  didn’t know about runner = ["/workspaces/acap-rs/remote-test.sh"] so I implemented this in cargo-asap-sdk. I’m traveling over the weekend but when I’m back I’ll try out your solution and think about how I would like to leverage it.
I currently think of tests in two categories:

Library test. This is what you have written and implemented support. We typically upload them to /tmp/ and run them as root. Starting with AXIS OS 12.0 this will be difficult or impossible to do
Application tests. These can be packaged as an EAP and installed on the device as any app. They run as the same user as the app would, which is required for APIs like message broker. This likely will continue to work or, in the worst case, the test output would have to be retrieved from the system logs.

Where you aware of cargo-acap-rs test? Asking because if not then I have a marketing problem.

Were you aware of cargo-acap-sdk test? Asking because if not then I have a marketing problem.
I might have been, but have been caught up in thinking through the API I'd like to achieve that I may have forgot about it. I have definitely used cargo-acap-sdk build to build an eap.

Using that may ultimately be a better approach than using the runner as discussed above.

Happy to see you are still working on this 🙂 What's are you aiming for (the whole API vs some subset, a minimal safe abstraction vs highly ergonomic abstraction, integration with VDO, etc) and how far do you think you have come?
Is there anything I can do to help?
I'm sorry I still haven't made time to study the APIs and your implementation.

Yep, I'm still working on it when I find time. I was initially planning on doing the whole API, but I'm starting to realize some elements of the larod API don't make a ton of sense. I'm also thinking covering the whole API is a bit more work than I initially realized. So, I'm leaning more towards a minimal but functional library for now that strikes a balance between being a relatively thin wrapper and providing an idiomatic Rust API.
I am also starting to work on the VDO library so I can get frames from the camera. That might be a bit more difficult looking at the imageprovider.c file in the vdo-larod example. Setting up background threads and interfacing the GLib will take a bit of thinking through. I'm aiming for an API that would work something like this:
fn main() -> Result<()> {
    let stream = Stream::builder()
        .channel()
        .format()
        .width()
        .height()
        .build();
    stream.start();
    while let Ok(frame) = stream.next_frame() {
        ...
    }
    Ok(())
}

Example liblarod Confusion
The larodCreateTensors function exists and simply takes a size_t number of tensors to create. I believe the library allocates some vector-like data structure on the heap and hands back a pointer to that data structure. The docs say to call larodDestroyTensors which takes a pointer to the larodConnection.
These two functions have me a bit confused. the larodCreateTensors does not seem to require a larodConnection, so it would seem the lifetime is like 'static. However, since the data structure is a C-allocated thing, we have to hand it back to C to deallocate and we don't seem to have a good function for doing that. Other functions, such as larodCreateModelInputs take a pointer to the larodModel, for which a larodConnection is needed in the first place. So the lifetime of these tensors is inherently tied to the lifetime of the larodConnection. Fine. Also, when larodDisconnect is called, any allocated tensors can be deallocated since they can easily be associated with that initial larodConnection.
I did a quick test (I believe, I can't find it now) where I used larodCreateTensors,  larodConnect, and larodDestroyTensors(larodConnection* conn, ... ) and got no errors. That seems to invite the possibility for any client to deallocate any other client's tensors as long as they have a pointer to the container of tensors, which doesn't seem particularly safe. Maybe there is a use case for that, but I thought I would leave out a Rust version of larodCreateTensors for now.
Code Conventions
One thing I was struggling with was the combination of supported preprocessor configurations. We know a-priori what backends support what kind of operations. So, I would really like to create a Rust Preprocessor API which encodes these supported combinations in the type system. We could then lean on the Rust type system to make it mostly impossible to code an incompatible combination of parameters. I couldn't find a nice way of doing it with Enums. The typestate pattern might be able to do it but looked like it was going to quite a bit of library code to do.
For now, I'm following the C library pretty closely with a builder pattern and doing the appropriate checks in a .build() method at runtime and returning an Err if an invalid combination is requested.


Debian Packaging using OBS

I've been trying to set up the Open Build System to build some debian packages. This would vastly simplify the installation process (and adds the possibility to uninstall jasper, too). Unfortunately, I'm too incompetent for debian packaging.
If anyone wants to lend a helping hand, here's the project:
https://build.opensuse.org/project/monitor/home:HolzhausGE
Something must be wrong with my debian.rules, because even though sphinxbase builds and installs, pocketsphinx refuses to detect sphinxbase.

Hi.  Are you still interested in making Debian packages?  I'm a Debian developer, and could help out, and preferably help you make this package into an official Debian package available from the Debian package archive.  I had a look at the source, and suspect a good start is to create a setup.py file to install the required files to their correct locations and port the code to python 3 (aka #724).  As Python 2 is no longer supported upstream, every package in Debian need to work with Python 3 or get thrown out.


Fix out of date docs

As discussed on Gitter, rustful::server::Listening has some out of date docs. It states that fn close does not work, but the linked issue is now closed. Fixing this should be fun and ez.
Looking more closely, this is actually a struct in Hyper re-exported here, ill add this issue to Hyper, and not here....

This is fixed in master in Hyper, but is not released. When they cut a new release, and we update, it will be fixed.

Oh, you know what? I just realized that Listening this is reexported from Hyper, so no wonder that I had no recollection of that note. I'm definitely getting too tired. Too many meetings... I'll close this straight away, since it's on their side and the upcoming async API will also probably not reexport it at all.
Still, thank you for taking your time to report this., and sorry for not realizing it earlier.


Setting notifications permissions to yes on launching does not work anymore (xcode 9, iOS 11)

Description
Suddenly setting the notifications permissions to yes on launching does not work anymore.
It worked before, but now I have a system popup asking the user to allow notifications. That popup should not happen.
I think this popup is back again since I upgraded to xcode 9.0. Which also means the simulator is upgraded to iOS 11.0
During the test startup Detox reports:
detox info 7: Trying to set permissions...
detox info 7: Permissions are set
However, it seems they do not.
Steps to Reproduce
before(async () => {
  await detox.init(config, { launchApp: false });
  await device.launchApp({ permissions: { notifications: 'YES' } });
});

Node, Device, Xcode and macOS Versions

Node: 8.4.0
Device: simulator iPhone 7 iOS 11.0
Xcode: 9.0
macOS: 10.12.6
applesimutils: HEAD-1c62171
detox: 5.8.1

Same here... I'm trying to get this working since yesterday....

The problem seems running iOS 11 in the simulator. Because I do not have the problem anymore if I switch back to iOS 10.3(.1)
I installed iOS 10.3.1 (yes, .1) in xcode 9 and then selected that version in the package.json like below. For some reason there only 10.3 is accepted.
After this the system notifications popup does not appear anymore.
  "detox": {
    "configurations": {
      "ios.sim.debug": {
        "binaryPath": "ios/build/Build/Products/Debug-iphonesimulator/HighFive.app",
        "build": "xcodebuild -project ios/HighFive.xcodeproj -scheme HighFive -configuration Debug -sdk iphonesimulator -derivedDataPath ios/build",
        "type": "ios.simulator",
        "name": "iPhone 7, iOS 10.3"
      }
    }
  }

This will be fixed soon in AppleSimUtils.

@LeoNatan It's still not working even after installing Detox and AppleSimUtils latest versions 😞

@ahmedlhanafy I’ll look into it.

@ahmedlhanafy Unable to reproduce your problem. What does applesimutils --version return?

@LeoNatan My applesimutils version is 0.5.11 .
Can this problem arise because I don't ask for push notifications permission when the app starts? In my case, I ask for permissions once the user logs in

0.5.11 supports Xcode 9.
But I don’t understand—if you are not asking for permission, how do you know it’s not working? If you ask for permission and no system alert comes up, that means it is working, otherwise not working.

Sorry for the confusion, what I meant is, I ask for the permission once the user logs in. So when simulating login with Detox, it gets stuck on the first screen (after login) that shows the permission popup. I don't ask for the permission right away when the app starts, I delay asking for it until the user logs in.

cc @LeoNatan


Allow admin to update vacation request

Description
Allow admin to update vacation request
Related Issues

https://github.com/codescalers/CSHR/issues/397

Checklist

[ ] Tests included
[ ] Build pass
[ ] Documentation
[ ] Code format and docstrings
[ ] Screenshots/Video attached (needed for UI changes)

please merge development

Could be only one button to update

The dates have not changed

Could be only one button to update

How to reproduce this?

The dates have not changed

Already fixed here

The dates have not changed

Already fixed here

I think it shouldn't be added here


Curious if it's possible to use this library together with my get_it_mixin

Hi, just stumbled upon this new package.
My get_it_mixin is a simpler hooks like package to observe objects inside get_it.
If I understand it correctly you still need stateful widgets to make it work.
I think it could indeed work.
@esDotDev what do you think?

I have been using this lately and it works great :D Mostly I use it for AnimationControllers so it doesn't conflict with anything I use GetItMixin for.

Awesome, I was almost sure that it does. It's really sad that the get_it family doesn't get that much interest as the other solutions

Ya, I don't really get it (get it!?) :p . All we can do is keep fighting the good fight :D
Here's code snippet on how they work together fine:
class _DrinkPageState extends State<DrinkPage> with GetItStateMixin, ReactiveHostMixin {
  late final anim = ReactiveAnimationController(this)
    ..ctrl.duration = Times.medium
    ..ctrl.forward();

  @override
  Widget build(BuildContext context) {
    final drinks = watchX((DrinksManager m) => m.all);
    ...
  }


useLassoSelection

This advanced hook would allow to select points in scatter series.

this would useDrawPath
by default it should be filled. I would suggest contour line in black 2 points thick and filled with black translucent: 0.8
accepts a callback that received for each scatter series (and only the scatter series) all the points that are inside the lasso selection
we need a way to highlight the selected points that are selected

Could first be done in a story, no need to create any hook.
The following dependency could be used: https://www.npmjs.com/package/point-in-polygon
We can take the PCA data of ecstasy but in a new story.

i see that useDrawPath is made using polyline and we should make a new version with polygone (add a prop if we want it polyline /polygone ) to be like the example (the first point is attached to the last one )


Documentation for useCSSLayers and CLI package

The website docs are missing information about the CLI and the full set of supported configuration options.

Can I help with this?


Create dropbox app and token for metacell

From this link https://www.dropbox.com/developers/apps/create :

create an app
set permission to app folder and set this to the folder will contain all the datasets
name your app, not really any preference in this sense, "neuroscan-promoterdb" would work but if you prefer anything else it's fine for us, just let us know what the name will be.
click the button "create" in the bottom-right corner

Once clicked you will be redirected to another page, similar to the screenshot below here

From this view, scroll down a little bit and:

from "App secret" click show
from access token expiration dropdown select "no expiration", this will be temporary just for development, then we can define a strategy to have short-lived tokens in the future.
from "generate access token" click the button generate
forward to us via email the api key, api secret and the generated token or take a screenshot that show all and attach that in an email.

Thanks!

We need to allow the App to read and write files, this can be set in the permissions folder as shown below:

@ddelpiano it seems that the token is expired or invalid.
also it's missing the file_requests.read permission
can you ask for a new, non expiring, token


Transaction Message does not Change

The message for the transaction never changes. Even when the user already approved the transaction, the message displayed is that the user needs to approve it (see the last part of the gif below). We could update the message when the user has already approved the transaction to say it's now just waiting on the tx to be confirmed in the blockchain.

see https://github.com/celo-org/use-contractkit/pull/205 for GIF


validate_assignment with side effect validators

Bug
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
             pydantic version: 1.7.2
            pydantic compiled: True
                 install path: /home/arthur/Documents/odmantic/.venv/lib/python3.8/site-packages/pydantic
               python version: 3.8.1 (default, Jan  5 2020, 23:12:00)  [GCC 7.3.0]
                     platform: Linux-4.17.14-041714-generic-x86_64-with-glibc2.27
     optional deps. installed: ['typing-extensions']

from pydantic import root_validator, BaseModel

class Rectangle(BaseModel):
    width: float
    height: float
    area: float = None

    class Config:
        validate_assignment = True

    @root_validator()
    def set_ts_now(cls, v):
        v["area"] = v["width"] * v["height"]
        return v

r = Rectangle(width=1, height=1)
assert r.area == 1
r.height = 5
assert r.area == 5 # AssertionError, r.area has not been modified

I don't know if it should be allowed to use validators this way or if it's a real bug.
I'd be happy to try to implement a fix :smile:

Hi @art049
Yes you're right. Since #1971 has been solved, this is the expected behaviour but it seems we update only the current field and not all. I can make a quick fix if you want should take couple of minutes

Thanks @PrettyWood !


cargo check --tests not working

cargo check --help tells

--tests                      Check all tests

I expect that at least it cheks syntax of tests.
I created simple project cargo new foo,
and edit src/lib.rs like this:
#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        baguaga
    }
}

I run cargo check --tests (cargo 0.22.0-nightly (88aa6423a 2017-07-22)) and
it reports nothing:
$ cargo check --tests
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
$ cargo test
   Compiling foo v0.1.0 (file:///tmp/foo)
error[E0425]: cannot find value `baguaga` in this scope
 --> src/lib.rs:5:9
  |
5 |         baguaga
  |         ^^^^^^^ not found in this scope

error: aborting due to previous error

error: Could not compile `foo`.

To learn more, run the command again with --verbose.

I expected the same error report from cargo check --tests as from cargo test
foo directory in attachment.
foo.zip

This is fixed by #4592.  cargo check --tests now really does all tests.


Remove DatabaseSnapshot

It is no longer being used.

This change is

OK I think I know why the presubmit tests keeps failing:
https://source.cloud.google.com/results/invocations/cc2db8f8-955e-4b55-bcb1-8f7c672d5664/log
It's failing at:
+ ./integration/run_compatibility_tests.sh -p domain-registry-dev -s sql

Based on the description [here], its running the deployed test suite (which does not contain the fix in #2107) against the schema in this PR. We need to deploy a new version first.
What's more interesting is how the bug that #2107 fixed slipped in in the first place. It should have been blocked by presubmit tests. It looks like something on kokoro changed silently which made this test start to fail.


Bug: hide wxWindow interface properties that forms can't use

Description:

Forms like wxFrame that use the wxWindow xml interface have derived_ properties which are ignored. The UI should hide these properties since they have no effect.

First off, this is a feature request, not a bug. Secondly, there are no more derived_ properties. There are subclass properties, but those are valid.
Closing unless more specific details provided.


not woking

Dev Home version
0.1201.442.0
Windows build number
10.0.22631.3447
Other software
OS Build Version: 10.0.22631.3447.amd64fre.ni_release.220506-1250
.NET Version: .NET 8.0.1
Steps to reproduce the bug
I was trying to download things
Expected result
the app to download
Actual result
it would not let me search anything
Included System Information
CPU: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
Physical Memory: 7.85GB (1.26GB free)
Processor Architecture: x64
Included Extensions Information
Extensions:
Microsoft.Windows.DevHome_0.1201.442.0_x64__8wekyb3d8bbwe
Microsoft.Windows.DevHome_0.1201.442.0_x64__8wekyb3d8bbwe
Microsoft.Windows.DevHomeGitHubExtension_0.1200.442.0_x64__8wekyb3d8bbwe
Widget Service:
MicrosoftWindows.Client.WebExperience_424.1301.450.0_x64__cw5n1h2txyewy

/logs
/needinfo


[Translatable] paginating when using TranslationWalker

Hi,
i'm not sure where to post this issue, but i'll start here.
I'm having trouble paginating filtered results when using the TranslationWalker.
For pagination i use the KnpPaginationBundle, which uses Doctrine's own CountOutputWalker to calculate the total amount of entities in the pagination.
In order to search in translations I've enabled the TranslationWalker.
To test this I've set up a very simple project with 1 entity: Product, that has a translated name.

Is there any way to fix this?

These are the packages I'm using:
"require": {
        "php": ">=5.5.9",
        "symfony/symfony": "~2.8.0",
        "doctrine/orm": "^2.5",
        "doctrine/doctrine-bundle": "^1.6",
        "doctrine/doctrine-cache-bundle": "^1.2",
        "symfony/swiftmailer-bundle": "^2.3",
        "symfony/monolog-bundle": "^2.8",
        "sensio/distribution-bundle": "^5.0",
        "sensio/framework-extra-bundle": "^3.0.2",
        "incenteev/composer-parameter-handler": "^2.0",
        "knplabs/knp-menu-bundle": "~2.0.0",
        "knplabs/knp-menu": "~2.0.1",
        "knplabs/knp-components": "~1.2.5",
        "knplabs/knp-paginator-bundle": "~2.4.2",
        "stof/doctrine-extensions-bundle": "~1.1.0"
    },

the problem is that both, knp paginator and translation walker, use sql walker. there can be only one of them and paginator overrides it. If you are sure that you will paginate only orm entities, try a simple solution like one I use it may not look as elegant as knp, but it is small and simple to change. Though it works with query builder, just download source as your project bundle and edit to add translation query hints. this pager has almost no source code.. won't be hard to maintain compared to others.

Thanks for the reply.
I briefly considered writing my own pagination, but I skipped that (I see you didn't, I'll keep that bundle in mind for future reference) and I've worked around it by creating my own count-query with a clone of the already filtered querybuilder and passing the result as query-hint to the paginator (https://github.com/KnpLabs/KnpPaginatorBundle/blob/master/Resources/doc/manual_counting.md).

@l3pp4rd I've created the best of both worlds by maintaining the original KNP-interface and functionality but internally using the DataDogPaginatorBundle: https://github.com/SamsonIT/SimplerKNPPaginatorBundle


Consider adding http zstd compression

Today we support gzip so Vespa delivers a gzip compressed response when the client uses "Accept-Encoding": "gzip", we could also support better and faster compression if we could understand "Accept-Encoding: zstd" and deliver a zstd compressed response. Also relevant for our feeding apis.
I believe this would be a great as it reduced egress cost for multi-cloud usages. curl has support for decompressing zstd compressed content as well https://daniel.haxx.se/blog/2020/08/19/curl-7-72-0-more-compression/

Jetty has currently no builtin support for zstd encoded request/response content. We'll have to implement it ourself. On approach is to create a new Jetty handler + input/output interceptors for (de)compression of zstd content. Jetty's GzipHandler does that.

I think for our internal feed api's it would make a lot of sense to support it, for query/read traffic where we don't do client implementations we could point to libraries supporting it for reducing network transfer cost.

Created feature request on Jetty's issue tracker: https://github.com/eclipse/jetty.project/issues/6600.


DxvkMemoryAllocator: Memory allocation failed

Software information
The Witcher 3, all settings maxed out, full HD, Nvidia Hairworks all characters + AA4
System information

GPU: NVIDIA 1050 Ti 4GB
Driver: 396.54.9 vulkan beta
Wine version: 3.19 master with Proton patches
DXVK version: v0.90-46-g963bd66

Log files

d3d11.log: cannot reproduce with debug enabled
dxgi.log: cannot reproduce with debug enabled

After loading a saved game, the game freezes just milliseconds after starting to fade in the screen. Since the game still fades in, everything is dark but it looks like everything is rendered correctly - no models or textures missing, NV hairs are also working. This happens only sometimes.
Looking at the logs I see
terminate called after throwing an instance of 'Dxvk::DxvkError'

Turning on full debug logging of dxvk eliminates the issue, it's not longer reproducible.
The frozen game can be successfully and instantly killed with SIGKILL.

Please apply the following patch to DXVK to get more descriptive error output:
dxvk-error.patch.txt
I've never seen this problem or anything like it, and I test Witcher 3 a lot. With the current set of information I won't be able to do anything, though.

@SveSop I'm currently working with bleeding edge builds here... Proton rebased to 3.19 including some code to optimize the process scheduler priorities to reduce priority inversion effects, and bleeding edge dxvk from git built als winelib. This boosts SOTTR performance from 19 to 33 fps for me here (even 35 fps with latest wine-3.19). And it reduces stutter and fps dips in TW3 and PoE. Also, intermittent freezes in SOTTR are fixed. I'm also working on some avrt patches so that native xaudio can properly gain realtime priority (currently, only built-in xaudio does that, and only with staging patchset). I'm going to soon push these updates to my repository but I'm currently not satisfied with it, and I want to test quality a little more. Also, wine had some commits lately breaking compatibility with esync and d3d related patches from Proton which I need to iron out (I think I fixed most by now).
I don't think that the wine version has anything to do with it, or if it has, it's something that'll show up here as soon as Proton would be officially based off a newer wine version.

Ah yeah, sorry. This one should work:
dxvk-error.patch.txt
Again, SOTTR works fine on my end.

SOTTR also radically dropped performance for me during one of my last rebases, from 30 fps to 10 fps (with vsync+triple buffer). But I don't know if this is due to code changes in wine-master or in DXVK. I'm currently trying to figure out if my wine-master rebase went wrong. There are currently many conflicting changes going on and I'm reintegrating patches from their updated sources now. I already reverted my own code changes as a first step but that didn't help. So there seems nothing wrong with those. Ah well... sigh

Can you just test things with a clean wine-tkg setup (if you're on arch) or something similar to rule out issues with your wine build?

@doitsujin Okay, something strange is going on. Out of desperation, I zapped the shader cache from $STEAMAPPS/shadercache/$GAMEID and the crash in SOTTR is gone, plus it's back to normal performance (the perceived performance even looks smoother now). The first benchmark run was clearly full of stutters as expected. Subsequent runs are fine now. Also, graphic distortions in SOTTR are gone (like Lara missing her clothes or hair).
Does this make sense to you? I wonder if TW3 benefits from a cache clear, too. Let me try...
PS: Don't try to reproduce Lara missing clothes and expecting some fun, the developers seem to have thought of this. :-)

That's weird and should probably not happen, but yeah, might be worth tryng for TW3 as well.

Is the cache depending on the DXVK version somehow? And are there safeguards against broken shader caches?

Okay, I already found that there's a safeguard using sha1 sums of each state cache entry, and a version header. So how did it break for me?

Not sure. Did you manage to confirm whether it was DXVK's state cache or the Nvidia driver cache that was causing issues?

I nuked both and only then discovered that this wasn't the best idea to find which one actually caused the problem. :-(

Okay, I got TW3 to crash again, this time logging worked (that logging patch should be in mainline, shouldn't it?):
0029:err:clipboard:convert_selection Timed out waiting for SelectionNotify event
0029:err:clipboard:convert_selection Timed out waiting for SelectionNotify event
DxvkMemoryAllocator: Memory allocation failed
terminate called after throwing an instance of 'dxvk::DxvkError'
004d:fixme:seh:dwarf_get_ptr unsupported encoding 9b
004d:fixme:seh:dwarf_get_ptr unsupported encoding c4
004d:fixme:seh:dwarf_get_ptr unsupported encoding 7d
004d:fixme:seh:dwarf_get_ptr unsupported encoding 9b
004d:fixme:seh:dwarf_get_ptr unsupported encoding c4
004d:fixme:seh:dwarf_get_ptr unsupported encoding 7d
004d:fixme:seh:dwarf_get_ptr unsupported encoding 9b
004d:fixme:seh:dwarf_get_ptr unsupported encoding 4a
004d:fixme:seh:dwarf_get_ptr unsupported encoding a9
004d:fixme:seh:dwarf_get_ptr unsupported encoding 9b
004d:fixme:seh:dwarf_get_ptr unsupported encoding 4a
004d:fixme:seh:dwarf_get_ptr unsupported encoding a9
004d:fixme:seh:dwarf_get_ptr unsupported encoding 9b
004d:fixme:seh:dwarf_get_ptr unsupported encoding 4a
004d:fixme:seh:dwarf_get_ptr unsupported encoding a9
004d:fixme:seh:dwarf_get_ptr unsupported encoding 9b
004d:fixme:seh:dwarf_get_ptr unsupported encoding 4a
004d:fixme:seh:dwarf_get_ptr unsupported encoding a9
004d:err:seh:call_stack_handlers invalid frame 3986f519 (0x39672000-0x39870000)
004d:err:seh:NtRaiseException Exception frame is not in stack limits => unable to dispatch exception.

Looking at the code, it seems like I should somehow manage to reproduce this error even wtih DXVK logging turned on...

DxvkMemoryAllocator: Memory allocation failed indicates that you're running out of memory (not necessarily VRAM).

Okay, I renamed the issue title to reflect the original problem. I think the "cache corruption" in SOTTR is really a different issue and should be reported separately by me if it occurs again.
It's strange that this can happen even very early after starting the game, read: When I just loaded a saved game the first time after starting The Witcher 3. I'll report back with new findings.
Actually, my system was loaded with some development applications which like to take a good amount of RAM while this issue occurred the last time. But it still had plenty of RAM left, around 8 GB. After all, TW3 is usually not THAT memory hungry (being an older game).

Here's an update:
err:   DxvkMemoryAllocator: Memory allocation failed
  Size:      134217728
  Alignment: 256
  Mem flags: 0x7
  Mem types: 0x681
DxvkMemoryAllocator: Memory allocation failed
terminate called after throwing an instance of 'dxvk::DxvkError'

# free -m
              total        used        free      shared  buff/cache   available
Mem:          15931        8063        1415         184        6452        7092
Swap:         67583        1304       66279

# nvidia-smi after the crash
Fri Nov  9 21:03:35 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.54.09              Driver Version: 396.54.09                 |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 105...  Off  | 00000000:01:00.0  On |                  N/A |
| 54%   43C    P5    N/A /  75W |   1697MiB /  4006MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1040      G   /usr/libexec/Xorg                           1120MiB |
|    0      2537      G   /usr/bin/kwin_x11                             47MiB |
|    0      2547      G   /usr/bin/krunner                               1MiB |
|    0      2549      G   /usr/bin/plasmashell                         241MiB |
|    0      4762      G   ...quest-channel-token=6856907186180940681   256MiB |
|    0      6145      G   ...ra/.local/share/Steam/ubuntu12_32/steam    20MiB |
|    0      6153      G   ./steamwebhelper                               1MiB |
|    0      6172      G   ./steamwebhelper                               4MiB |
+-----------------------------------------------------------------------------+

I played this game for extended hours (sometimes 12 in a row, yes I'm an addict of this game) with previous versions of DXVK. So I wonder why I see this now.

Does it still work on older versions?
I don't see why allocating a 128MB buffer in system memory would suddenly fail when it previously didn't, especially since the memory allocator hasn't been touched in a long time.

After trying some games, I see that multiple games are affected... Skyrim SE freezes on loading screens or in the middle of the game, looking at the logs I also see DXVK complain that very moment about memory.
It looks like Chrome hogs a lot of GPU memory, Xorg was holding almost 3 GB of GPU memory. Restarting Chrome fixes that, and stopping Chrome gets rid of the issues in Skyrim SE. I don't think it has anything to do with the DXVK version but coincidence is that other processes occupy GPU memory. Shouldn't such memory swap out to system memory? Maybe something changed in the NVIDIA driver?

System memory that needs to be made visible to the GPU cannot be swapped out as far as I'm aware, and the Nvidia driver might have further limitations (probably for a good reason). fwiw I've seen similar issues on amdgpu under low-memory conditions, although not directly related to DXVK.
In any case, if you consider this issue resolved by closing third-party applications, please close the issue.

There's definitely a bug somewhere leaking memory... If I play long enough, VRAM eventually fills up to 3.7 of 4 GB and then games either freeze, crash or behave strangely (like flickering or missing textures/models). Something changed, I'm just not sure what. I'm using the same NV driver version since some time now, so it's not too likely that the graphics driver changed something. I followed DXVK master closely. Maybe something new in DXVK triggers such a bug?

You can monitor DXVK's VRAM consumption with DXVK_HUD=memory. I haven't seen any behaviour that would indicate a leak.

I've also notice (DXVK_HUD=memory) that on some games like Dark Souls 3 my old lady GTX 960 with 2 GB vram is unplayable. Memory consumption is about 1.8-2.0 GB (just like in windows), but just try to sit at the bonfire, death or teleport to another area and oh boy it spikes to ~2.6-3.4 GB. And now I'm playing Dark Souls 3 with bullet time (slow motion). Fallout 4, Skyrim SE, ReCore DE also eats way above my vram limit.
But games like Dark Souls Remastered, Deus Ex HR, Divinity OS 2 DE, GTA V, Hard Reset Redux, Shadow Warrior 2, Witcher 3 works perfect.

I have same issue with 4GB GPU mem and SOTTR. But I can work around it using medium textures. It's looks like game do not free memory immediately but using some garbage collector/reuse mechanism (or just leaks and it's not so noticeable on windows). On windows some GPU defragmenter is working and also keeps hottest memory resident on GPU while in dxvk if texture allocated on the host memory than it keeps on host forever.

I changed the title because it is not game specific. It's visible in different games. I'm playing SOTTR even with low textures and it happens after 3-4 hours (sometimes earlier). In SOTTR another effect probably resulting from this is the second benchmark always runs slower than the first, going into a game level, then returning to the benchmark, it's even slower now. So there's overhead accumulating somewhere.
SOTTR: Results in graphical glitches first, then freezes, finally crashes to desktop after some time of thrashing the harddisk
TW3: Just freezes, often just after the initial saved game load, on rare occasions it crashes to desktop
SkyrimSE: Just freezes, either during a loading screen (endless loading screen) or midst in the game, sound continues to play
Anyone of you knows what changed lately in your systems? This effect was much less visible some time ago (or even didn't exist, hard to say).

You could try, but that only has an effect when you actually run out of VRAM. But you won't ever run out of VRAM in Witcher 3, for example.

@doitsujin Okay, after testing for a while, available VRAM makes a huge difference for performance. All games crash if I do not enable overcommitting. Crashing is a bad experience, so we probably need something more intelligent here. Meanwhile, I patched my version of DXVK to enable overcommitting by default which gets rid of all the crashes I experienced lately. I rather reach a savepoint with bad performance than have a crash which forces me to repeat parts of the game. There's always an opt-in to gracefully quit the game and clean up VRAM somehow.
Games are affected in different ways, I've tested two so far:

In SOTTR, under low VRAM, performance degrades vastly, by a factor of at least 2, sometimes 3 oder 4. This has a very big impact.
In TW3 the frame dips become much more visible: With almost all VRAM available, the frame dips are hardly noticeable, I managed to run TW3 with only ~800MB VRAM available and frame dips became very noticeable. But using the mod "HD Reworked Project" with its optimized configuration helped a lot: While it adds very high quality textures to the game (which should even worsen the situation), it also includes a configuration with optimized texture streaming parameters and that helps a lot: TW3 feels much smoother.
I didn't test the third so much yet but SkyrimSE doesn't seem to have noticeable performance problems with low VRAM despite I'm using HD and immersion mods which make it use 2-3 GB of VRAM.

So first: Maybe this issue should be tagged "performance".
Second: Is there anything intelligent that DXVK could do about memory management? In retrospect to the issue reporting degrading performance with high quality textures in SOTTR, is it possible for DXVK to somehow prioritize what goes to VRAM and what goes to system RAM? Is it possible that DXVK could discard allocations from VRAM, or swap them between system memory and VRAM based on usage patterns?
I wonder how Windows manages this... It either has ways to manage and swap VRAM with system RAM, or the games behave different there and can actually manage this on their own. Then the question is, why can't they do it when running under wine/DXVK?

I must be doing something wrong i guess.
Replaced Proton 3.16 dxvk files with 1724d51 and played about 2 hours. Never went past 2.8GB allocated mem.
GTX970 4GB w/396.54.9 driver.
1080p all on "Ultra" in TW3. Perhaps you use 4K?

When DXVK fails to allocate a resource in VRAM, it allocates it in RAM which of course comes with a pretty big performance penalty. That shouldn't crash though. If anything it should crash only if you allow overcommiting.
Windows does the same but it is a bit smarter and also moves existing resources out of VRAM to make room for more important ones.

@K0bin I think it's quite the other way: Overcommitting allows memory to be allocated from system RAM even if the game didn't ask for it, otherwise it fails which crashes the game (because DXVK refuses to continue).
I wonder if it is possible to make DXVK similar smart and let it move resources out of the way. But I guess this has to be fixed higher up the stack layer, i.e. the driver itself or Xorg must be willing to give up resources when requirements are coming in...

@K0bin I think it's quite the other way: Overcommitting allows memory to be allocated from system RAM even if the game didn't ask for it, otherwise it fails which crashes the game (because DXVK refuses to continue).

No, its not the other way, see #527.

I wonder if it is possible to make DXVK similar smart and let it move resources out of the way. But I guess this has to be fixed lower down the graphic stack layers, i.e. the driver itself or Xorg must be willing to give up resources when requirements are coming in...

It's possible but it's very hard and a lot of work. This has to be done inside DXVK though, Vulkan explicitly leaves memory management to the application. That's probably not going to happen any time soon, so your best option is to just lower your graphics settings.

@K0bin Interesting... DXVK definitely terminates here when the error occurs. Revisiting the allocation code, it shouldn't do that. It should just return DxvkDeviceMemory() instead of result. So I conclude something is going wrong just a little bit later? Like accessing zero pointers?

It only throws the error when neither the VRAM nor the System RAM allocations succeed. Once that happens, it's too late to continue in any meaningful way anyway.

I think its not a good test for me with GTX970 to use over 3.5GB vram for comparing performance, cos of the 970 memory configuration thingy. https://hexus.net/tech/news/graphics/79925-nvidia-explains-geforce-gtx-970s-memory-problems/
So this "bug" is mostly crashes due to OTHER apps using up gpu memory, and dxvk not pushing this (useless) memory usage out of the way to up performance? :)

@doitsujin Yes this is what the code says but I'm sure there's still sysmem available, or the system could just swap stuff out to disk to make some small allocation of 128M available. Could this be a driver bug? After all you're not allocating through standard C/C++ functions but through vulkan functions.

i don't know what the issue is, but it seems that something eats unusual amounts of memory on your end. Have you tried running those games on a simple WM (like fluxbox) without any applications running in the background?

Stupid question from my end: Is the problem here that there is a memory leak eating more and more vram until the game crashes in eg. TW3?
If so, i dont really see the same problem on my end, as those 2.8GB allocated vram shown happened in the first 2-3 minutes of playing TW3 yesterday, and did not increase over the course of 2 hours of me playing, loading/saving games several times without change. In use (commited? dont remember the wording) hovered around 2.2GB - 2.5GB mostly.
I did not try to crash the game on purpose by overloading vram in some other manner tho.
Just trying to troubleshoot on a different system than yours to weed out any possible non-dxvk issues.

Did a wee bit of testing back and forth, and can't really say i am able to make something eat so much vram.. Opening 10 chrome windows did not chunk out a huge deal of vram either tbh, but for all i know you could be running 200+ windows while editing a 4K movie in the background :)
What i DID notice however (no difference between Proton 3.16 w/dxvk 0.90 vs. building my own dxvk from git) was the "Memory Allocated" from the DXVK hud only increased and never went down even if i loaded a saved game with less "Memory used".
That might be intended, and tbh SHOULD not be an issue as long as it is <4GB i guess?
Eg.
nVidia-SMI: Witcher3: 1488
nVidia-SMI: (Total): 1880
DXVK Memory Allocated: 2030
DXVK Memory Used:      1837

Loading save games from different spots + running around and so on would up the "Memory Allocated" upwards, even tho "memory used" goes up/down as needed. Not really sure what the discrepency between nVidia-SMI (who i would deem to be "accurate" in usage directly from the driver) and "Memory used"?
nVidia-SMI "Total" memory was 1880, and somewhat more in line with "DXVK Memory" i guess, but nVidia one includes Xorg, gnome-shell and stuff like that, so i would not think DXVK would be able to "read" that?
I did not test hours upon hours of gameplay, but from the 2 hours i played yesterday mentioned above, i had 2.8GB "Memory allocated", so i guess it MIGHT be something that just grows and grows until it gets a problem? Is the mem allocation something that SOMETIMES gets cleared out? (Or rather SHOULD).

DXVK uses a chunk allocator, thus it usually doesn't cleanup because some bit of information will always be left in a chunk. Chunks are allocated probably in 64 MB blocks, within each chunk you'll have a free list of blocks from which DXVK will allocate into the biggest block available (except a free block matches exactly in size), if the allocation request type matches the chunk type. It's similar to how btrfs manages its device space. If a chunk becomes completely free, it could be de-allocated, but that really doesn't make much sense because probably you would request a new chunk of memory just moments later. If no free block can be found, a new chunk will be allocated from the device. Thus, it's normal that the memory usage only increases until it peaks at some value. A chunking allocator is pretty much the best thing you can do if you need to handle different and incompatible types of allocations. You just need to properly tune the chunk size so you can fit all types of allocations without too much overhead and without too much wasted space. The "allocated" counter is probably what's been allocated as chunks, the "used" counter is what's actually used across all chunks. The difference is wasted space which wasn't used or couldn't be used due to incompatible memory type flags.
As far as I understood, chunks are allocated from the driver or the vulkan layer which in turn decides if it allocates from the device or from system memory (depending on the flags given). Within each chunk, memory is managed by DXVK itself by keeping lists of free blocks (pairs of offset/size).
What happens in my case seems to be: DXVK asks vulkan for a new chunk of device-local memory, vulkan says "no", DXVK tries again without the "device-local" flag, thus it allows to use non-local memory which is slower because it is accessed over the PCI bus. But the vulkan says "no" again. But there's plenty of system RAM available to allocate such a chunk. I can only guess why that is. Maybe vulkan cannot find system memory that would be mappable by the GPU. Not all of your physical address space may be available to the GPU because of chipset limitations, or because other devices already mapped that, i.e. another GPU, or I don't know what.
Overcommitting "solves" this because it lets vulkan pretend that unused chunk memory isn't going to be used any time soon. Thus, such memory is still available to other allocations. Your Linux kernel does a similar thing: Allocated memory only becomes mapped to real memory if something writes to the memory blocks. Otherwise it stays idle. It accounts for the allocated RAM but not the used RAM. It's the "virt" counter you'd see in top: virt is allocated space. But things start crashing if one application now actually wants to use its allocated but yet unused memory: The GPU won't find any space to put that request, it fails, crash. Linux solves this by swapping to disk. The GPU could request the driver to swap to sysmem. But as I understood, vulkan leaves that completely to the application. So DXVK would be in charge of doing so. But DXVK doesn't implement this. It's complicated. It should be avoided as long as you can.
So in turn that means: Overcommitting does not crash for me, thus a lot of VRAM is only allocated but not used. So Chrome (or Xorg) seems to allocate a lot of VRAM just because it can but it never uses it.
To the experts: Does this make sense?
I'm running with two monitors, left one is a full-HD TV (which I actually use for gaming from the couch, with a wireless controller), and the right one is a 4k PC monitor. I do no video editing but some browser tabs may host paused or finished youtube videos (which tend to be streamed in 4k quality). I also have multiple gmail tabs open. At least back in 2014 there was a bug in Chrome where it would slowly eat away your VRAM if you have gmail opened over longer periods of time. But that was fixed since then.
So overall I'm probably running a virtual framebuffer of (1920+3840)x2160 pixels at 32 bit color depth (I think it doesn't use 24 bit buffer representation, but color space is 24 bit). With triple buffering, that's about 142 MB of screen buffer. Probably there's some padding and alignment but nothing to worry about...

Thanks for a thorough explanation :)
I use 2x1080p monitors, but rarely have i ever seen vram used past 2GB in the cruddy old games i play... save for TW3 (probably old aswell), and have after a while of playing up toward 2.8GB allocated mem.
Now.. i dont do 12+ hour gaming sessions without logging off, nor do i have many many chrome tabs open while i game.  This COULD ofc be worse if i play for a lot longer, as i said (and to your explanation) the game COULD be allocating chunks until vram is all spent? Dunno.
How long does it take you if you do a clean boot and just load  up steam and start TW3 until you get errors? Cos troubleshooting stuff that is in the realm of "Oh.. yeah, you need to do a 12 hour playingsession before that happens" is kinda.. uhm.. Well :)
As i said, chrome seemed hard pressed to really use much vram for me, so i am looking for something different perhaps.. some example code that can be started over until vram is spent perhaps? Found some references to GLSLHacker (GeeXLab) and some 4GB vram test thingy, but was not able to find that anymore. Opening a 4K video on youtube seems to be using a whopping 70MB of vram for me, so i dunno...

@kakra

The GPU could request the driver to swap to sysmem. But as I understood, vulkan leaves that completely to the application. So DXVK would be in charge of doing so.

Actually no, it isn't. Once a memory chunk is allocated, Vulkan apps don't really have to bother with it, residency is magaged by the driver. Even for device-local memory types, there is no guarantee that memory allocated from them is actually located in VRAM, it can be paged out if necessary.

@doitsujin So we are back to "that doesn't seem to happen here". It only strengthens the theory this is a driver / graphic stack issue here... Maybe related to configuration or hardware memory layout...

Okay, I managed to let SOTTR allocate more than 4GB of memory now without a crash, also TW3 allocated around 3GB without a crash now - with Chrome and some other windows opened. I have a theory of what was going wrong in my system but I need to test this a little more. The "slow down over time" issue in SOTTR also seems to be gone but since my system does a lot of background activity currently, I'd like to defer the performance testing a little more. Currently, stuttering is a lot more apparent now and I'm not sure if it comes from background activity or switched settings. But the games seem to cope well now with Xorg/Chrome allocating a lot of memory, overall memory footprint of those seems a little bit lower now. I'll report back.

Apparently, during testing various kernel configurations I managed to crash my filesystem the hard way. I probably lost some important changes to the wine code, one of which is hard to recreate. Replacement drives are ordered because I want to keep around the broken file system for trying recovery. This throws me back about 1-2 weeks, so I'm going to pause working on this for a few days.
But to recap what I found out so far: Vulkan (or NVIDIA) seems to interact with THP very badly (at least in combination with wine). It wasn't able to allocate more RAM because there was just no mappable memory block left to allocate. This is probably a memory fragmentation issue. Usually, the kernel would defer huge page creation then. But I also noticed that my kernel didn't properly enable IOMMU (which seems to be important for NVIDIA). I was still testing that part when the crash occured.
Since THP can be a pretty nice, performance enhancing feature, I wanted to work out a proper configuration and document that. First tests showed that it makes a difference in performance. Overall fps was mostly identical but I did notice audio-dropouts every now and then which I didn't notice before.
So I probably take the chance to rebase my work to wine 3.21 then. I was just finished with preparing and cleaning up the 3.20 release when everything went down the virtual drain. :-(
I think I'm back up running by next weekend. Thank you, Murphy, that I discovered my daily backup wasn't working that very same day.
Note to myself: Don't mix zswap with some workloads. Push often even if still WIP.
Conclusion: If someone is seeing this issue, too, it may be due to THP being enabled and not being fully and/or correctly configured. Could you check? grep ^ /sys/kernel/mm/transparent_hugepage/*

Nvidia have separate nvidia-uvm driver part for Unified Memory Access support. Not sure if it's CUDA only, but may be all CPU visible allocations goes through it and may be it have some tunable options.
Anyway, dxvk do reuse allocated blocks + games do allocation/frees in runtime, so fragmentation issue can't be resolved only by driver/system, GPU mem defragmenter is needed. Otherwise there ~2x GPU mem is needed in worst case scenario (i.e. game can only use 2GB for 4GB card).

During my tests I enabled HMM which is probably needed for full uvm support. But the uvm module is not used (used count is 0), and also it would probably be auto loaded if something accesses the API. Games don't it seems. CUDA probably should but I'm not sure how to test that.
The chunk allocator of DXVK seems to do a pretty good job, I've not seen a waste of 2x, usually just a couple of 100 MB. If the factor raises to 2 or more, that's usually only of short duration because a game discards and reloads stuff. The strategy to fill the largest free blocks in a chunk first seems to work very well: It leaves space for bigger allocations, or if a big allocation cannot fit, it will allocate a new chunk and this allocation will already take a good part of it, so overall slack is reduced.
But in my case I think it's not a problem of the GPU memory but the Linux kernel wasn't able to find a contiguous block of system memory. Usually, a THP defrag should happen then which introduces latency. But due to latency, I turned on deferred defrag: Thus, an allocation would be broken down into 4k pieces and defragged later. I think what happens here: Such a broken allocation cannot be mapped for the GPU then and the allocation request will be rejected, resulting in the out of memory error. If you turn off defrag, you'd easily run out of memory for applications early, even when top still states enough memory available. THP uses 2M page size here. It reduces pressure on the TLB which can result in up to 10% performance difference.
Maybe the memory stats should show what the largest block from the free list currently is. I'm not sure if the Linux kernel could tell you something similar. But displaying this can be difficult as there are many different chunk types.

@kakra
Couple of questions cos im a total n00b.

Why would you need IOMMU for wine? (IOMMU being pci-passthrough for use with KVM/QEMU)
Is not nvidia-uvm for CUDA?
Would you need to compile the nVidia kernel module with hmm=1 to get that support, given that you ofc have a kernel with:

CONFIG_ARCH_HAS_HMM=y
CONFIG_HMM=y
CONFIG_HMM_MIRROR=y

Sorry if my quesions are stupid... just trying to see the connection with DXVK and these options :)

IOMMU can be used for proper DMA remapping according to NVIDIA driver readme. And when the crash happened, I was just starting with HMM. I didn't check yet if Gentoo automatically picks up the HMM setting. The crash came first. But modinfo stated HMM is available as far as I remember. About UVM/CUDA: You are probably right.
I think IOMMU and DMA remapping could work around the fragmentation issue but I'm not sure. Without hard IOMMU the driver uses software bounce buffers which can be slower and disables some other features.

IOMMU is a virtualization feature that afaik requires your bios and hardware to support "Intel vt-d" (Also needs to be enabled in the motherboard uefi bios).
IOMMU is most commonly used for mapping dma/memory for your gpu and other stuff to be used in a VM (pci-passthrough). Or, possibly gain the usage of 4GB gpu memory in a 32-bit os?? I dont understand what would be needing IOMMU in wine or dxvk tbh...
HMM on the other hand might be interesting... if the nVidia driver actually support it?

All pre-PCI Express GPUs and non-Native PCI Express GPUs (often known as bridged GPUs) are limited to 32 bits of physical address space, which corresponds to 4 GB of memory. On a system with greater than 4 GB of memory, allocating usable DMA buffers can be a problem. Native PCI Express GPUs are capable of addressing greater than 32 bits of physical address space and do not experience the same problems.

https://us.download.nvidia.com/XFree86/Linux-x86_64/331.79/README/dma_issues.html
Seem to be talking about PRE-PciE cards... So from that i would understand its a benefit if you have a PCI nvidia gpu (with only 32-bit address space support). Probably not useful at all for a GTX970 PCI-E card.. no?

There usually is no "IOMMU setting" in the bios. If i need to enable IOMMU in the kernel, i have to enable the bios setting called "Intel Virtualization - vt-d". If that is "disabled" (default), i cant (or get an error) use IOMMU. There is 2 parts in this, 1 is the CPU, and other is the motherboard.
From what i gather from the nVidia documents, it is not something you should need other than in special cases. I guess if you have a mining rig with 20 8GB 1080 cards or something, this is needed to overcome some memory boundaries? Dunno.
https://download.nvidia.com/XFree86/Linux-x86_64/396.54/README/README.txt
Chatpter 35 talks about

For example, it is common for a system with 512 GB of RAM installed to have
physical addresses up to ~513 GB. In this scenario, a GPU with an addressing
capability of 512 GB would force the driver to fall back to the 4 GB DMA zone
for this GPU.

And i get:
cat /proc/driver/nvidia/gpus/0000\:01\:00.0/information
Model: 		 GeForce GTX 970
IRQ:   		 150
GPU UUID: 	 GPU-6d548f8e-645a-6933-c892-f86a2f2fe8aa
Video BIOS: 	 <IP_ADDRESS>.f1
Bus Type: 	 PCIe
DMA Size: 	 40 bits
DMA Mask: 	 0xffffffffff
Bus Location: 	 0000:01:00.0
Device Minor: 	 0

1 Terabyte (40 bits)

I dont have >1TB memory, so i would think im in the "safe-zone".
PS. To enable IOMMU by default and not have to use intel_iommu=on, you need to compile the kernel with CONFIG_INTEL_IOMMU_DEFAULT_ON=y
IMO: Not something you "need" unless you have >1TB memory (possibly), or a large number of GPU's.. or unless you use KVM/QEMU to run a VM with pci-passthrough. (And if you DO, you usually would want to pass the nVidia GPU through to the vm, and need to disable that totally with no driver)

@SveSop I think IOMMU is not dependent on VT-d but vice-versa if you need features like PCI-passthru.
You are citing the manual. It only tells you that the driver falls back to the 4 GB DMA zone if you have more than 512 GB of memory because the GPUs cannot address more than that.
IOMMU allows to bypass the DMA zone because address translation and process isolation is done in the north bridge.
My idea was that the driver can utilize IOMMU to overcome fragmentation issues introduced by THP: A contiguous mapping of GPU memory would no longer need to align with the system memory the same way... IOMMU could just remap memory access when THP deferred page merging allocates memory in 4k pages first.
It's not that I want to overcome memory addressing issues - I don't have those: my system has 16 GB of RAM. And I don't want to remap memory because I want to do VM passthru. I just wanted to try to break the constraints between memory mapping within the GPU and memory mapping in sysmem.
BTW: My BIOS says "VT-d not available" because my CPU does not support that feature. But it should still have IOMMU support. I get no error when I put intel_iommu=on into the kernel cmdline, it says DMAR uses IOMMU now. Surprisingly it's not enough to set CONFIG_INTEL_IOMMU_DEFAULT_ON=y - it simply doesn't generate the same message in dmesg for DMAR.
If you think you don't need it: I didn't ask you to enable it on your side. I'm still researching if I can make use of it, and will document if I found something useful. Current findings are: The DXVK error occurs when I use THP=always. With THP=madvise the error no longer shows up but I get audio-dropouts - not sure why or what happens yet. With THP=never some workloads in my system are slower, probably also with THP=madvise. This is because there are many more TLB cache misses then. I was working with THP=always for a long time now successfully (since they introduced deferred defrag to overcome latency spikes). Only lately, DXVK started to show errors. But that's not a DXVK issue probably. Something else changed. But since switching THP modes definitely changes things, I'm probably on the right track.
I'm only guessing what the problem may be: DXVK asks for 64 MB of memory. GPU allocates contiguous memory, the driver asks the kernel to find 64 MB of contiguous memory in sysmem. It cannot (or doesn't want to) do that because deferred defrag is turned on, it allocates 64 MB as 4k pages instead of 2M pages scattered around physical memory. The GPU cannot map that because it's not contiguous yet, allocation fails, DXVK makes boom. But without deferred defrag I get very noticeable latency spikes throughout the whole system, without even looking at graphics performance in DXVK. In theory, with IOMMU it no longer matters what the physical layout of GPU mapped memory is in sysmem. In practice, I'm not sure if IOMMU could handle such big allocations. From reading the manuals, it's usually meant to be used with much smaller memory blocks.
Old school DMA is limited to 4 GB address space without address translation support as far as I figured out yet. So DMA buffers could only be placed in the first 4 GB of memory. IOMMU can bypass this.  DMA also doesn't support process isolation wrt to memory access by device, IOMMU does. That's why it's an integral useful feature for virtualization but it is not limited to be used for that.  DMA allows PCI(e) devices to address system memory. IOMMU allows addresses to be transparently translated to other physical locations within the north bridge.

@SveSop, I wrote:

IOMMU can be used for proper DMA remapping according to NVIDIA driver readme. And when the crash happened, I was just starting with HMM. I didn't check yet if Gentoo automatically picks up the HMM setting. The crash came first. But modinfo stated HMM is available as far as I remember. About UVM/CUDA: You are probably right.

BTW: I enabled HMM and verified that the driver actually compiled with HMM enabled. But my system is unstable with this (freezing more or less randomly), I stopped following this idea and reverted the driver change.

It is a lot i dont understand about this, so i guess its kinda silly for me to comment on that.
I do not really think you would gain much benefits from IOMMU, but could ofc be a faulty assumption. There is also a BIOS option (on my MB) that reads: Above 4G decoding that has some notion about "remapping PCIe cards above 4GB limit". That could be relevant.
This setting might also vary among manufacturers, or possibly not be a option at all, for all i know. Mentioning this as it is for my motherboard a separate "option" that has something to do with remapping this > 4GB instead of the "normal" <4GB address. If Asus does this "right" or not, i have no idea... I do have it enabled tho.
When it comes to HMM, it does look interesting, so some experimenting on this would be warranted. What option did you set, or how did you compile the nVidia binary driver with to get HMM support?

@SveSop
In Gentoo it's easy, create /etc/portage/package.env with this line:
x11-drivers/nvidia-drivers hmm

Then create /etc/portage/env/hmm with this line:
NVIDIA_BUILD_SUPPORTS_HMM=1

Now reinstall the driver, add a modprobe.conf entry to add option nvidia-uvm hmm=1, reboot.
If your system becomes unstable, just revert the changed line in /etc/portage/package.env and rebuilt.
I cannot say how to do this with other distributions. If you use the original nvidia installer, it's probably a matter of running with an exported variable:
$ sudo NVIDIA_BUILD_SUPPORTS_HMM=1 /path/to/installer/NVIDIA-Linux-...run

The header files in the installer explicitly state that this needs kernel patches upstreamed before becoming default, thus it is currently only opt-in by setting this variable.

Seems as HMM is part of nvidia-uvm. nvidia-uvm is the "cuda" module is it not?
lsmod | grep -i nvidia
nvinvidia_drm             45056  4
nvidia_modeset       1093632  10 nvidia_drm
nvidia              14090240  425 nvidia_modeset
drm_kms_helper        208896  1 nvidia_drm
drm                   532480  7 drm_kms_helper,nvidia_drm
ipmi_msghandler       110592  2 ipmi_devintf,nvidiadia

So, no nvidia-uvm module loaded. I do not use cuda.
I could try to add NVIDIA_BUILD_SUPPORTS_HMM=1 when building the kernel module through dkms, but i dont really see the point if it adds HMM support for nvidia-uvm module, as i dont use cuda.
Starting a cuda test through wine did load the module tho
[  895.239558] nvidia-uvm: Loaded the UVM driver in 8 mode, major device number 511
nvidia_uvm            917504  0
But not sure about the relevance of that and dxvk?

No, UVM is not CUDA. CUDA is for accessing the compute units of the GPU, UVM is the unified memory management which is not exclusively reserved for use by CUDA. But I don't think that anything else currently uses that except CUDA. So there's no relevance to DXVK.
UVM enables applications (and also CUDA) to simplify memory access, application would no longer need to manually copy data back and forth. It could theoretically be used generally for any GPU acceleration, not just CUDA but also 3D. As far as I can tell, neither Vulkan nor the driver itself use that currently, only the CUDA API probably does.
But enabling HMM seems to have some effect otherwise my system wouldn't become unstable with it. But at this stage, performance tests are impossible for me because of that.
But please stop thinking about CUDA and virtualization. UVM, HMM, IOMMU, DMAR etc are just building blocks of those, it's not the implementation of CUDA or VT-d. You aren't forced to build a house out of those bricks, you could also build a tower or a bridge. I think CUDA is used by NVAPI which in turn is used by some games. But apparently the wine implementation of that is imcomplete (tho it forwards calls to CUDA as far as I understand the code).

I'm closing this with the following results: Disabling transparent huge pages in the kernel fixes the problem at the cost of quite noticeable performance loss. But there's a compromise: Using transparent huge pages in "madvise" mode keeps around memory optimizations for running software that supports it, so TLB misses have less impact on DXVK performance. But the advantage is only a slight one to nothing - depending on your system configuration.
I'm instead experimenting with a patch to wine to support large page mode for applications. "Middle-earth: Shadow of War" is supposed to support it and I can successfully enable it in the game now - but tracing wine shows that the game still does not make the needed system calls.
If anyone wants to help with that, here's my wine-proton project:
https://github.com/kakra/wine-proton
Another issue discovered during the course of this report: Web browsers tend to allocate a lot VRAM directly and through the xserver. This has a direct impact on performance of the game. I doubt there's anything DXVK could do about it - shuffling idle VRAM of foreign processes between sysmem and VRAM is probably not concern of DXVK. Such VRAM shortage has direct consequences under transparent hugepages: The game may crash because of memory allocation failures despite having plenty of memory available as initially outlined in this report. It's probably some hidden memory fragmentation problem. There's probably nothing to do about it unless the Vulkan/graphics driver would be aware of transparent huge pages. The fix currently is to not force transparent huge pages but use "madvise" mode instead, I recommend to combine it with deferred defragmentation.

I have the same issue, any news ?


Return 0 fee if buy equals sell token

We have been seeing some logs of the form:

WARN orderbook::fee: Computed negative fee after applying fee discount: -10000000000000000, capping at 0

which according to my analysis stem from the fact the frontend is sometimes querying us with sellToken == buyToken fee queries (cf. slack message). In those case the gas cost is 0 so our fixed fee discount makes it negative.
This PR adds an early exit for this code path. Note, that as soon as the quote endpoint is used this type of query would return a BadRequest error.
Test Plan
Unit tests still pass. Querying http://localhost:8080/api/v1/fee?sellToken=0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2&buyToken=0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2&amount=6837606881482500&kind=sell locally yields 0 fee estimate without a warning.

Codecov Report

Merging #1432 (6ad364d) into main (829f96c) will increase coverage by 0.00%.
The diff coverage is 83.33%.

@@           Coverage Diff           @@
##             main    #1432   +/-   ##
=======================================
  Coverage   65.96%   65.96%
=======================================
  Files         155      155
  Lines       30488    30492    +4
=======================================
+ Hits        20110    20113    +3
- Misses      10378    10379    +1


HNC: allow kubectl hnc tree to show all namespaces if no argument is given

(Filed as a result of #93)
Currently kubectl hnc tree requires an argument, which is the root namespace of the tree to display. We should also allow the user to omit the root argument and print every namespace (and their subnamespaces) instead.
/good-first-issue

hello @adrianludwin I would love to pick this up

/assign @shivi28


Execute without create restore point

Thank you for this wonderful collection of scripts.
Please make it possible to execute them without creating a restore point

I'm pretty sure you can already do that. On the wizard page with the two radio buttons, you can select not to create a restore point.

I'm closing this. Let me know if you have any other questions.