Kong fails to start with permissions error on kong.yaml

Bug report
Describe the bug
When starting the local development environment with supabase start, the Kong container fails to start:

stack traceback:
[C]: in function 'error'
/usr/local/share/lua/5.1/kong/init.lua:490: in function 'init'
init_by_lua:3: in main chunk
nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:490: error parsing declarative config file /var/lib/kong/kong.yml:
/var/lib/kong/kong.yml: Operation not permitted

When checking the supabase/.temp/kong.yml file, it appears to have the correct permissions:

$ ls -hal supabase/.temp/
total 8
drwxr-xr-x  3 user  staff    96B Feb 15 14:22 .
drwxr-xr-x  7 user  staff   224B Feb 15 14:27 ..
-rw-r--r--  1 user  staff   2.8K Feb 15 14:22 kong.yml

I even made the whole supabase directory as permissive as possible (chomd -R 777 supabase) and tried again without success:

$ ls -hal supabase/.temp/
total 8
drwxrwxrwx  3 user  staff    96B Feb 15 14:47 .
drwxrwxrwx  7 user  staff   224B Feb 15 14:47 ..
-rwxrwxrwx  1 user  staff   2.8K Feb 15 14:47 kong.yml

To Reproduce
Steps to reproduce the behavior, please provide code snippets or a repository:

Start supabase
$ supabase start
Try to access the API and see that it does not respond properly:
$ curl http://localhost:54321
Observe the kong container logs and see the traceback

Expected behavior
I expect the development environment to start and the APIs to be available
System information

OS: macOS 12.1
Version of supabase-cli: 0.17.3
Version of Docker: 20.10.12, build e91ed57

Additional context
I also tried to steps on this similar issue without success

I figured it out. Turns out it was the macOS permissions system rather than the file system's permissions. I had to grant Docker "Full Disk Access" in System Preferences > Security.
More info about this here: https://stackoverflow.com/questions/58482352/operation-not-permitted-from-docker-container-logged-as-root


Integration of logspout with dind

closes #80

Oh, this is really nice! But I'm not a huge fan of adding a mandatory extra dependency... I'd love to know what other people think? Should we merge it by default, or make a separate image maybe ... ?

Well, the feature sped up our development, so I think others would also benefit. But I understand the concerns.

There is now an official docker:dind image upstream!
I can suggest two options:

find a way to integrate your logspout work with it somehow;
maintain your own fork of "my" dind repo.

If you choose to do the latter, I'll be happy to reference your repo in the README of this one to indicate that it's more up-to-date!

Thank you. That sounds reasonable. I will think how to proceed.


link in thrimbletrimmer to download the current cut

This will make grabbing bits to build a supercut or other edit easier.
No new functionality needed serverside, just call restreamer with the cutting args.

This already exists. Next to the "submit" button in Thrimbletrimmer is a "Download" button, which does exactly that.


XMI  MIDI controllers

[x] delete drivers/miles namespace
[x] delete test drivers/miles namespace
[ ] xmi midi controllers, need to do a new "OplDriver" sort of OplXmiDriver to start with as there are also sounds/timbres/banks to deal with.

xmi midi controllers, need to do a new "OplDriver" sort of OplXmiDriver to start with as there are also sounds/timbres/banks to deal with.


Use .ts file extensions instead of .js

There appears to be some typos where imports are referring to .js files whereas the @filename annotations indicate they are .ts files. This corrects that mismatch.

@orta I'm clearly a novice here, but could you explain further? Given these lines:
https://github.com/microsoft/TypeScript-Website/blob/4a670b334df7be35c480f640009ce698a7bab02b/packages/documentation/copy/en/handbook-v2/Modules.md?plain=1#L68-L77
Do the filenames get resolved after tsc has processed the .ts files to generate the .js ones? Is that why even though index.ts references hello.js, this works because tsc would have compiled it to that new target to import? I'm confused as to how tsc would import from hello.js during parsing since it would be working with hello.ts.

You're welcome to read up on the different module resolution strategies TypeScript has built in via the reference docs on the you edited  and here - but the code samples on the side default to node-style resolution where you give the .js because javascript that is the only lanaguge natively supported in nodejs. A type would crash the engine..
To give people code samples which wouldn't throw in that environment isn't something we'd want to do.


Reworked to improve flexibility

I was browsing GitHub when I found this awesome blog template from @ashish-makes, and I wanted to use it for my purpose.
After aditiong a few things a make it mine, I thought this project could become even greater with more flexibility.
So I reworked a few things on it:

Added categories models to make easier their management and  thier assignations to posts
Added project models, to allow an easier management instead of writing the html code
About class to make the section text easier to edit
Skill model, to show custom skills/values on the about page
reworked search form to use created categories
commented contact page url assignation (only because I don't need it)
added and icon to the nav bar
added script to load a virtual env (named .env), update teh database and start the dev server (default address/port)

The blog is still basically the same, but the admin pannel now offers more possibilities.
Once again, thanks @ashish-makes for your increbible work!

Hi @PetchouDev,
the categories you added are very buggy and also you leaked your API key.
Except from that this is a nice PR.

What do you mean by very buggy ?
I now the logic I used to make sure the user set a category is quite messy…
And I think it make the category menu overflow weirdly on some resolutions
But it actually seemed to work well
About the api key, I don’t know even remember using one… where was it ? I just forwarded the port via SSH so I was able to connect via localhost (which doesn’t require the api key).
Anyway I will look at this deeper in the weekend, thank for your answer!

Hi @PetchouDev,
I am not 100% sure anymore since it is now some weeks ago that I modified your code, but I think the categories broke the search completely for instance.
About the overflow I think that this is not your fault, since this always happens, even using the original repo.
The API key I meant is the tiny.cloud one


Docstring conventions and some testing style.

Added some notes on docstring comments and on the structure of test cases.

+1

+1, all good.


Installer: Improve NATS configuration

Action Item
With the latest update of the NATS charts to 0.13.2 we can configure our NATS installation.
Acceptance Criteria

[ ] automountServiceAccountToken should be set to false either in the ServiceAccount or in the PodSpec.
[x] Set Resource limits for CPU and RAM.
[x] runAsNonRoot should be set to true or runAsUser should be set to a value > 10000 either in the container SecurityContext or in the PodSecurityContext.
[x] allowPrivilegeEscalation should be set to 'false'.
[x] readOnlyRootFilesystem should be set to 'true'.
[x] Update the chart version (check https://github.com/nats-io/k8s/blob/main/helm/charts/nats/README.md and the changelog for info about breaking changes)

DoD

[x] Acceptance criteria are met
[x] Manual test to verify everything works fine
[x] Integration tests are passing

value automountServiceAccountToken is not exposed in PodSpec of nats, so ti cannot be configured. Filed an issue about the details: https://github.com/nats-io/k8s/issues/527


[CELEBORN-48][BUG] Channel inactive may cause new client use old stream id to fetch data

What changes were proposed in this pull request?
When changing client, the origin client's corresponding streamId may be removed by channel inactive
Replica should request a new StreamHandle for the new client again. Newly returned numChunks should be the same.
Why are the changes needed?

What are the items that need reviewer attention?
Related issues.
Related pull requests.
How was this patch tested?
/cc @related-reviewer
/assign @main-reviewer

duplicate  with https://github.com/apache/incubator-celeborn/pull/1000, close this one


add all mac, linux arches

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.


Dead keys + space bar not working through ssh or in wsl 2

Windows Terminal version
1.20.10303.0
Windows build number
10.0.22631.0
Other Software
Bash, either in WSL 2 or through SSH on debian-based OS's (tested on Debian, Ubuntu and RasberryPi OS Bookworm)
Steps to reproduce

Use a keyboard layout that uses dead keys (e.g. US International)
Press a "dead key" (e.g. ')
Press the space bar

Expected Behavior
The corresponding character to appear (e.g. an apostrophe)
Actual Behavior
Only a space character appears.

I think this may be related to #16641

@Meryl Yeah, sorry about this. It's the same underlying cause.

Stephen f Balogh
On Tue, Feb 6, 2024 at 6:04 PM Dustin L. Howett @.***>
wrote:

Closed #16642 as completed via #16645.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

urueurrhhrjr


Add Common Objects in 3D (CO3D)

Adding a Dataset

Name: Common Objects in 3D (CO3D)
Description: See blog post here
Paper: link to paper
Data: link to data
Motivation: excerpt from above blog post:

As the first data set of its kind, CO3D will aptly enable reconstruction of real-life 3D objects. Indeed, CO3D already provides training data to enable our NeRFormer to tackle the new-view synthesis (NVS) task. Here, photorealistic NVS is a major step on the path to fully immersive AR/VR effects, where objects can be virtually transported across different environments, which will allow connecting users by sharing or recollecting their experiences.
Besides practical applications in AR/VR, we hope that the data set will become a standard testbed for the recent proliferation of methods (including NeRFormer, Implicit Differentiable Renderer, NeRF, and others) that reconstruct 3D scenes by means of an implicit shape model.

Instructions to add a new dataset can be found here.

？


Not working with Lumen 5.4

Trying to make it work with Lumen 5.4, first error was that share() is not define, which I have modified with singleton(). Than I get the error that optimize is not defined. Looks like the command php artisan optimize is called and Lumen does not have this command.

@BogdanHossu should be compatible now, tested with latest Lumen version 5.4.6

When following the installation step by step, I encounter this error
Class 'Codedge\Countries\CountriesServiceProvider' not found
Do you think I am doing something wrong? The provider is registered into my bootstrap/app.php.

Hmm.. sounds strange. Please run composer update and composer dump-autoload. After that it should work.

Unfortunately I have tried this a lot of times but with no success.

I'll updated the package - there was sth. wrong with proper loading, as you mentioned. Please run composer update again and fetch the latest 1.6.1 version

Thank you @codedge, I can confirm that now it worked for me.


Leaderboard selfuser fix

Hide attack button on current user tab at leaderboard page
Current user tab as in green colour


[Bug]: Invalid Add type cast code action

Description
Consider the following scenario where the Add type cast code action is suggested,
https://user-images.githubusercontent.com/61020198/196413664-84ef1187-b77a-4294-8930-3759929ed355.mov
Improve the code action to apply the type cast to the entire expression.
Steps to Reproduce
type RecordType record {
    json|error fieldName;
};

function toString(RecordType r) {
    json|error s1 = r.fieldName;
    string s2 = s1 is json? s1.toJson() : "";
}

Affected Version(s)
No response
OS, DB, other environment details and versions
No response
Related area
-> Editor
Related issue(s) (optional)
No response
Suggested label(s) (optional)
No response
Suggested assignee(s) (optional)
No response

Now the type cast is being added to the middle expression (s1.toJson()). Therefore closing this issue.


Enforce Yarn, remove package-lock.json and add Jest testing to pre-commit hooks

Closes #227
Remove -
package-lock.json
Add -

.npmrc file
.yarnrc file
preinstall script for Yarn enforcement
Jest testing on pre-commit hooks

Just out of curiosity, why are the lock files being removed? I understand the point of not having both, but we should at least have one of them, right?

@thetimbanks
Agreed !! we should have at least 1 (preferably yarn.lock) but not mandatory to have it on projects, we can prevent using NPM on adding/updating packages. It will also add a prerequisite of everyone knowing/using yarn towards contributing to the project.
It can be achieved by preinstall script-
"scripts": {
    "preinstall": "node -e \"if(process.env.npm_execpath.indexOf('yarn') === -1) throw new Error('You must use Yarn to install, not NPM')\""
}

@rparet @vitorpamplona  - Let me know your thoughts after discussing it with the core dev team and I can update PR accordingly.

I think we should continue to use just yarn.lock so we can keep everyone on the same versions of the packages.

@thetimbanks updated PR

@pathakparth can you resolve the conflicts

@harshvitra, @tremblerz  it would be great if we can get this merged else it will keep running into conflicts.

Even if you keep yarn.lock when someone new goes and does a yarn install yarn will automatically update some of the dependencies underneath.

@pitops, addressing your comment:

Even if you keep yarn.lock when someone new does a yarn install yarn will automatically update some of the dependencies underneath.

from yarn install docs:

If yarn.lock is absent, or is not enough to satisfy all the dependencies listed in package.json (for example, if you manually add a dependency to package.json), Yarn looks for the newest versions available that satisfy the constraints in package.json. The results are written to yarn.lock.

So, updated yarn.lock on yarn install should not be a concern as long as everybody installs packages using yarn add


feat: add family and version to ledger object ids

Signed-off-by: Daniel Bluhm<EMAIL_ADDRESS>

Good catches @swcurran; put this PR together a bit too absent-mindedly during the call :sweat_smile: I'll take a look


Not working with node 6.7.0

Hey thanks for the beautiful work you do. I hope the tutorial will be out soon.
Is it normal with node 6.7.0 that when i run npm install, webpack is launched and I have ERROR in Cannot find module 'babel-core' . I can't launch the project.
Thanks

Thanks for the kind words :)
We need to update the dependencies, but if you npm install babel-core that should do the trick. We're currently switching the app over to create-react-app (not quite done yet but should be soon), so this shouldn't be an issue at all in the future!

This is solved with #24 -- git pull and npm install to get it running on an existing codebase!


Crash Report - Phantomjs 1.9.1 on Ubuntu 64 bit

I've attached the stack trace from the crash generated as per the Crash Reporting Wiki page.
I am seeing a Phantom crash under serious load. I'm making extensive use of Phantom webpages and the web server module. Any ideas on what may be going on?
If there is any more information that would be useful to debug please let me know.
Stack trace:
Operating system: Linux
                  0.0.0 Linux 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64
CPU: amd64
     family 6 model 45 stepping 7
     4 CPUs

Crash reason:  SIGSEGV
Crash address: 0x91

Thread 0 (crashed)
 0  phantomjs + 0x1711007
    rbx = 0x00007fff9c2e1fb0   r12 = 0x0000000006dcddf0
    r13 = 0x00007f23c00662f8   r14 = 0x0000000002ab18e0
    r15 = 0x0000000006dcde40   rip = 0x0000000001b11007
    rsp = 0x00007fff9c2e1f40   rbp = 0x0000000000000091
    Found by: given as instruction pointer in context

Thread 1
 0  libc-2.15.so + 0xbf83d
    rbx = 0x00007f2455daecf0   r12 = 0x0000000002ab39c0
    r13 = 0x00007f239f274680   r14 = 0x0000000002abf1e0
    r15 = 0x0000000002abf1b0   rip = 0x00007f24568d483d
    rsp = 0x00007f2455daec40   rbp = 0x00000000ffffffff
    Found by: given as instruction pointer in context
 1  libc-2.15.so + 0xbf6db
    rip = 0x00007f24568d46dc   rsp = 0x00007f2455daec50
    Found by: stack scanning
 2  phantomjs + 0x193fbaf
    rip = 0x0000000001d3fbb0   rsp = 0x00007f2455daec58
    Found by: stack scanning
 3  phantomjs + 0x2e2c
    rip = 0x0000000000402e2d   rsp = 0x00007f2455daec68
    Found by: stack scanning
 4  libc-2.15.so + 0x109f7
    rip = 0x00007f24568259f8   rsp = 0x00007f2455daec70
    Found by: stack scanning
 5  phantomjs + 0x24c7
    rip = 0x00000000004024c8   rsp = 0x00007f2455daec78
    Found by: stack scanning
 6  ld-2.15.so + 0xa522
    rip = 0x00007f2457eeb523   rsp = 0x00007f2455daecc8
    Found by: stack scanning
 7  libc-2.15.so + 0xbf848
    rip = 0x00007f24568d4849   rsp = 0x00007f2455daecd8
    Found by: stack scanning
 8  libpthread-2.15.so + 0xfcaf
    rip = 0x00007f24573f5cb0   rsp = 0x00007f2455daece8
    Found by: stack scanning
 9  libpthread-2.15.so + 0xb103
    rip = 0x00007f24573f1104   rsp = 0x00007f2455daed90
    Found by: stack scanning
10  libc-2.15.so + 0xe54f
    rip = 0x00007f2456823550   rsp = 0x00007f2455daed98
    Found by: stack scanning
11  ld-2.15.so + 0x15234
    rip = 0x00007f2457ef6235   rsp = 0x00007f2455daedd0
    Found by: stack scanning
12  libpthread-2.15.so + 0xe9f9
    rip = 0x00007f24573f49fa   rsp = 0x00007f2455daede0
    Found by: stack scanning
13  libpthread-2.15.so + 0xbe17
    rip = 0x00007f24573f1e18   rsp = 0x00007f2455daedf0
    Found by: stack scanning
14  phantomjs + 0xcfcbd
    rip = 0x00000000004cfcbe   rsp = 0x00007f2455daee20
    Found by: stack scanning
15  libc-2.15.so + 0x977f
    rip = 0x00007f245681e780   rsp = 0x00007f2455daee28
    Found by: stack scanning
16  libpthread-2.15.so + 0x1187f
    rip = 0x00007f24573f7880   rsp = 0x00007f2455daee50
    Found by: stack scanning
17  libpthread-2.15.so + 0x1187f
    rip = 0x00007f24573f7880   rsp = 0x00007f2455daee80
    Found by: stack scanning
18  phantomjs + 0xcff28
    rip = 0x00000000004cff29   rsp = 0x00007f2455daeea0
    Found by: stack scanning
19  libpthread-2.15.so + 0x7e99
    rip = 0x00007f24573ede9a   rsp = 0x00007f2455daeeb0
    Found by: stack scanning
20  libpthread-2.15.so + 0x1187f
    rip = 0x00007f24573f7880   rsp = 0x00007f2455daef58
    Found by: stack scanning
21  libc-2.15.so + 0xf3cbc
    rip = 0x00007f2456908cbd   rsp = 0x00007f2455daefc0
    Found by: stack scanning

Thread 2
 0  libc-2.15.so + 0xed023
    rbx = 0x0000000000000000   r12 = 0x00007f24554add80
    r13 = 0x0000000000000008   r14 = 0x00007f2450000a98
    r15 = 0x00007f2450000d30   rip = 0x00007f2456902023
    rsp = 0x00007f24554adb50   rbp = 0x00007f24500008e0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x18ea916
    rip = 0x0000000001cea917   rsp = 0x00007f24554adb60
    Found by: stack scanning
 2  ld-2.15.so + 0xa29d
    rip = 0x00007f2457eeb29e   rsp = 0x00007f24554adba0
    Found by: stack scanning
 3  phantomjs + 0x18ebbd6
    rip = 0x0000000001cebbd7   rsp = 0x00007f24554adbf0
    Found by: stack scanning
 4  phantomjs + 0x18db0a6
    rip = 0x0000000001cdb0a7   rsp = 0x00007f24554adc10
    Found by: stack scanning
 5  phantomjs + 0x17f3332
    rip = 0x0000000001bf3333   rsp = 0x00007f24554adc20
    Found by: stack scanning
 6  linux-gate.so + 0x958
    rip = 0x00007fff9c39b959   rsp = 0x00007f24554adc30
    Found by: stack scanning
 7  librt-2.15.so + 0x415c
    rip = 0x00007f245760715d   rsp = 0x00007f24554adc60
    Found by: stack scanning
 8  phantomjs + 0x18324f3
    rip = 0x0000000001c324f4   rsp = 0x00007f24554adc70
    Found by: stack scanning
 9  phantomjs + 0x18323f8
    rip = 0x0000000001c323f9   rsp = 0x00007f24554adcb0
    Found by: stack scanning
10  phantomjs + 0x18eb595
    rip = 0x0000000001ceb596   rsp = 0x00007f24554adcc0
    Found by: stack scanning
11  phantomjs + 0x18ec681
    rip = 0x0000000001cec682   rsp = 0x00007f24554add30
    Found by: stack scanning
12  phantomjs + 0x18c1aa4
    rip = 0x0000000001cc1aa5   rsp = 0x00007f24554add80
    Found by: stack scanning
13  phantomjs + 0x18c2298
    rip = 0x0000000001cc2299   rsp = 0x00007f24554add90
    Found by: stack scanning
14  phantomjs + 0x18c1f07
    rip = 0x0000000001cc1f08   rsp = 0x00007f24554addb0
    Found by: stack scanning
15  phantomjs + 0x17d1765
    rip = 0x0000000001bd1766   rsp = 0x00007f24554ade10
    Found by: stack scanning
16  phantomjs + 0x17d4631
    rip = 0x0000000001bd4632   rsp = 0x00007f24554ade60
    Found by: stack scanning
17  libpthread-2.15.so + 0x7e99
    rip = 0x00007f24573ede9a   rsp = 0x00007f24554adeb0
    Found by: stack scanning

Thread 3
 0  libc-2.15.so + 0xed023
    rbx = 0x0000000000000200   r12 = 0x0000000003f1e300
    r13 = 0x0000000003f1e3e0   r14 = 0x0000000003f1e970
    r15 = 0x00007f240f0bada0   rip = 0x00007f2456902023
    rsp = 0x00007f240f0bad80   rbp = 0x0000000000000000
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d345
    rip = 0x000000000044d346   rsp = 0x00007f240f0bad90
    Found by: stack scanning
 2  phantomjs + 0xc0007e
    rip = 0x000000000100007f   rsp = 0x00007f240f0bae50
    Found by: stack scanning
 3  libpthread-2.15.so + 0x7e99
    rip = 0x00007f24573ede9a   rsp = 0x00007f240f0baeb0
    Found by: stack scanning

Thread 4
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f24080008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f240e8b79f0   rbp = 0x00007f24080008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f240e8b7a20
    Found by: stack scanning

Thread 5
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23f40008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f240e0b69f0   rbp = 0x00007f23f40008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f240e0b6a20
    Found by: stack scanning

Thread 6
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23f00008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f240d8b59f0   rbp = 0x00007f23f00008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f240d8b5a20
    Found by: stack scanning

Thread 7
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23e80008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f240d0b49f0   rbp = 0x00007f23e80008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f240d0b4a20
    Found by: stack scanning

Thread 8
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x00007f23fc005ed0   r12 = 0x004189374bc6a7ef
    r13 = 0xffffffffffffffff   r14 = 0x00007f23fc005ef8
    r15 = 0x00007f23fc006010   rip = 0x00007f24573f1d84
    rsp = 0x00007f240c8affd0   rbp = 0x00007f23fc006010
    Found by: given as instruction pointer in context
 1  phantomjs + 0x17d5364
    rip = 0x0000000001bd5365   rsp = 0x00007f240c8b0000
    Found by: stack scanning
 2  phantomjs + 0x17d049c
    rip = 0x0000000001bd049d   rsp = 0x00007f240c8b0080
    Found by: stack scanning
 3  phantomjs + 0x2387b
    rip = 0x000000000042387c   rsp = 0x00007f240c8b00b0
    Found by: stack scanning
 4  phantomjs + 0x195844f
    rip = 0x0000000001d58450   rsp = 0x00007f240c8b0148
    Found by: stack scanning

Thread 9
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23e40008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f2407ffc9f0   rbp = 0x00007f23e40008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f2407ffca20
    Found by: stack scanning

Thread 10
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23e00008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f24077fb9f0   rbp = 0x00007f23e00008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f24077fba20
    Found by: stack scanning

Thread 11
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23f80008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f2406ffa9f0   rbp = 0x00007f23f80008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f2406ffaa20
    Found by: stack scanning

Thread 12
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f23ec0008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f24067f99f0   rbp = 0x00007f23ec0008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f24067f9a20
    Found by: stack scanning

Thread 13
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x0000000003f1e940   r12 = 0x00007f24000008c8
    r13 = 0x0000000003f1e300   r14 = 0x0000000000000000
    r15 = 0x0000000003f1e3e0   rip = 0x00007f24573f1d84
    rsp = 0x00007f2405ff89f0   rbp = 0x00007f24000008c0
    Found by: given as instruction pointer in context
 1  phantomjs + 0x4d8e7
    rip = 0x000000000044d8e8   rsp = 0x00007f2405ff8a20
    Found by: stack scanning

Thread 14
 0  libpthread-2.15.so + 0xbd84
    rbx = 0x000000000541bbf0   r12 = 0x004189374bc6a7ef
    r13 = 0xffffffffffffffff   r14 = 0x000000000541bc18
    r15 = 0x000000000476f030   rip = 0x00007f24573f1d84
    rsp = 0x00007f23d2ffcd10   rbp = 0x00007f23b28e6f10
    Found by: given as instruction pointer in context
 1  phantomjs + 0x17d5364
    rip = 0x0000000001bd5365   rsp = 0x00007f23d2ffcd40
    Found by: stack scanning
 2  linux-gate.so + 0xa1a
    rip = 0x00007fff9c39ba1b   rsp = 0x00007f23d2ffcd80
    Found by: stack scanning
 3  phantomjs + 0xd9379
    rip = 0x00000000004d937a   rsp = 0x00007f23d2ffcdc0
    Found by: stack scanning
 4  phantomjs + 0xd76a08
    rip = 0x0000000001176a09   rsp = 0x00007f23d2ffcde0
    Found by: stack scanning
 5  phantomjs + 0xd76c32
    rip = 0x0000000001176c33   rsp = 0x00007f23d2ffce20
    Found by: stack scanning
 6  phantomjs + 0xd8b8a
    rip = 0x00000000004d8b8b   rsp = 0x00007f23d2ffce50
    Found by: stack scanning
 7  phantomjs + 0x17d4631
    rip = 0x0000000001bd4632   rsp = 0x00007f23d2ffce60
    Found by: stack scanning
 8  libpthread-2.15.so + 0x6e0a
    rip = 0x00007f24573ece0b   rsp = 0x00007f23d2ffce70
    Found by: stack scanning
 9  libc-2.15.so + 0x3b4817
    rip = 0x00007f2456bc9818   rsp = 0x00007f23d2ffce78
    Found by: stack scanning
10  libpthread-2.15.so + 0x7e99
    rip = 0x00007f24573ede9a   rsp = 0x00007f23d2ffceb0
    Found by: stack scanning

Thread 15
 0  libc-2.15.so + 0xed023
    rbx = 0x00007f23d37fdd30   r12 = 0x0000000000011448
    r13 = 0x0000000000000024   r14 = 0x00007f23c4104888
    r15 = 0x00007f23c4104b20   rip = 0x00007f2456902023
    rsp = 0x00007f23d37fdb50   rbp = 0x00000000000b1ce5
    Found by: given as instruction pointer in context
 1  phantomjs + 0x18ea817
    rip = 0x0000000001cea818   rsp = 0x00007f23d37fdb60
    Found by: stack scanning
 2  phantomjs + 0x18ebbd6
    rip = 0x0000000001cebbd7   rsp = 0x00007f23d37fdbf0
    Found by: stack scanning
 3  phantomjs + 0x18db0a6
    rip = 0x0000000001cdb0a7   rsp = 0x00007f23d37fdc10
    Found by: stack scanning
 4  phantomjs + 0x17f3363
    rip = 0x0000000001bf3364   rsp = 0x00007f23d37fdc20
    Found by: stack scanning
 5  linux-gate.so + 0x958
    rip = 0x00007fff9c39b959   rsp = 0x00007f23d37fdc30
    Found by: stack scanning
 6  librt-2.15.so + 0x415c
    rip = 0x00007f245760715d   rsp = 0x00007f23d37fdc60
    Found by: stack scanning
 7  phantomjs + 0x18324f3
    rip = 0x0000000001c324f4   rsp = 0x00007f23d37fdc70
    Found by: stack scanning
 8  phantomjs + 0x18ec2dd
    rip = 0x0000000001cec2de   rsp = 0x00007f23d37fdca0
    Found by: stack scanning
 9  phantomjs + 0x18323f8
    rip = 0x0000000001c323f9   rsp = 0x00007f23d37fdcb0
    Found by: stack scanning
10  phantomjs + 0x18eb595
    rip = 0x0000000001ceb596   rsp = 0x00007f23d37fdcc0
    Found by: stack scanning
11  phantomjs + 0x18ec681
    rip = 0x0000000001cec682   rsp = 0x00007f23d37fdd30
    Found by: stack scanning
12  phantomjs + 0x18c1aa4
    rip = 0x0000000001cc1aa5   rsp = 0x00007f23d37fdd80
    Found by: stack scanning
13  phantomjs + 0x18c2298
    rip = 0x0000000001cc2299   rsp = 0x00007f23d37fdd90
    Found by: stack scanning
14  phantomjs + 0x18c1f07
    rip = 0x0000000001cc1f08   rsp = 0x00007f23d37fddb0
    Found by: stack scanning
15  phantomjs + 0x17d1765
    rip = 0x0000000001bd1766   rsp = 0x00007f23d37fde10
    Found by: stack scanning
16  phantomjs + 0x17d4631
    rip = 0x0000000001bd4632   rsp = 0x00007f23d37fde60
    Found by: stack scanning
17  libpthread-2.15.so + 0x7c82
    rip = 0x00007f24573edc83   rsp = 0x00007f23d37fde70
    Found by: stack scanning
18  libc-2.15.so + 0x3b4817
    rip = 0x00007f2456bc9818   rsp = 0x00007f23d37fde78
    Found by: stack scanning
19  libc-2.15.so + 0x3b4827
    rip = 0x00007f2456bc9828   rsp = 0x00007f23d37fde80
    Found by: stack scanning
20  libc-2.15.so + 0x3b482f
    rip = 0x00007f2456bc9830   rsp = 0x00007f23d37fde88
    Found by: stack scanning
21  libpthread-2.15.so + 0x7e99
    rip = 0x00007f24573ede9a   rsp = 0x00007f23d37fdeb0
    Found by: stack scanning

Loaded modules:
0x00400000 - 0x026cbfff  phantomjs  ???  (main)
0x7f23de463000 - 0x7f23de4f8fff  DejaVuSans-Oblique.ttf  ???
0x7f23de819000 - 0x7f23de8aefff  DejaVuSans-BoldOblique.ttf  ???
0x7f2404205000 - 0x7f2404252fff  DejaVuSerifCondensed-Bold.ttf  ???
0x7f2404253000 - 0x7f240429ffff  DejaVuSansMono-Bold.ttf  ???
0x7f24042a0000 - 0x7f240433afff  DejaVuSansCondensed-Bold.ttf  ???
0x7f2404349000 - 0x7f2404399fff  DejaVuSerifCondensed.ttf  ???
0x7f240439a000 - 0x7f24043ebfff  DejaVuSansMono.ttf  ???
0x7f24043ec000 - 0x7f2404489fff  DejaVuSansCondensed.ttf  ???
0x7f240c0af000 - 0x7f240c0b5fff  gconv-modules.cache  ???
0x7f2455db0000 - 0x7f245600afff  libssl.so.1.0.0  ???
0x7f245600c000 - 0x7f24563cffff  libcrypto.so.1.0.0  ???
0x7f24563d4000 - 0x7f24565fdfff  libexpat.so.1.5.2  ???
0x7f24565fe000 - 0x7f2456814fff  libz.so.<IP_ADDRESS>  ???
0x7f2456815000 - 0x7f2456bcefff  libc-2.15.so  ???
0x7f2456bd4000 - 0x7f2456de9fff  libgcc_s.so.1  ???
0x7f2456dea000 - 0x7f24570e5fff  libm-2.15.so  ???
0x7f24570e6000 - 0x7f24573d0fff  libstdc++.so.6.0.16  ???
0x7f24573e6000 - 0x7f24575fefff  libpthread-2.15.so  ???
0x7f2457603000 - 0x7f245780afff  librt-2.15.so  ???
0x7f245780b000 - 0x7f2457a0efff  libdl-2.15.so  ???
0x7f2457a0f000 - 0x7f2457c44fff  libfontconfig.so.1.4.4  ???
0x7f2457c45000 - 0x7f2457ee0fff  libfreetype.so.6.8.0  ???
0x7f2457ee1000 - 0x7f2457f02fff  ld-2.15.so  ???
0x7f2457f5f000 - 0x7f2457f67fff  6d41288fd70b0be22e8c3a91e032eec0-le64.cache-3  ???
0x7f2457f68000 - 0x7f24580f0fff  locale-archive  ???
0x7fff9c39b000 - 0x7fff9c39bfff  linux-gate.so  ???

Please upgrade to PhantomJS 2.0. 1.9.x is no longer supported, and known to have many crasher bugs that are fixed in 2.0.


implement Targaryen cards from Someone Always Tells chapter pack

french spoilers:

https://i62.servimg.com/u/f62/11/77/80/59/f2bf4910.jpg
https://i.redditmedia.com/N2neyXUtS5sJ8oU0x2sdGqWstM5jjeG209e3G7N2xVw.jpg?w=510&s=63221ef369256348854b4e15db3d5711
english card namers aren't available yet, so this should not be merged as is


[API design] Get request details

Clean code 👍

@haithamalnaeb
So in situations like this, when the issue has been closed and I need to edit something, should I  create a new pull request ?! or there is another way!

@KaramNa , yes you have to make a new PR,
I noticed the security issue after I closed it,
That's why it is better to have more than one approve, like in big Repo's


Feature/wbd endpoint

[x] Upload wb data to s3 in two csv (population and gdp)
[x] Create migration and model
[x] Import data to the database
[x] Create controller to serve the data

Creates the following new endpoint:
WB (World Bank) Extra Country Data

GET /api/v1/wb_extra/:iso will return a response with the following

extra params:
startYear: min limit to the year. If it's not included will get the closest min year
endYear: min limit to the year. If it's not included will get the closest max year
format:
[
    {
        "population": "30739250",
        "GDP": "13834300571",
        "year": "1961"
    },
    {
        "population": "31023366",
        "GDP": "16138545209",
        "year": "1962"
    }, ...
]

Where:

population: is the total population for the year and the iso code of the country;
GDP: is the gross domestic product for the year and the iso code of the country;
year: year

Could you please satisfy @houndci-bot 's recommendations (when sensible? Thanks!


Text direction RTL

Please add support for Arabic text (RTL)
Right now the text is added on the left side of the text edit area

As a workaround, I've added CSS style to force it to be ltr direction: ltr !important; to the main wrapper.
But this is not resolving the problem as I need it to be rtl.

I know. We'll put this in our todo and try it when we can afford it.


Trying to load docker container on O3R VPU and I get "no space left on device"

Hi ,  I am trying to deploy custom code to the O3R VPU (starting off with just the basics for now but the overall goal is to deploy some lightweight pytorch models for inference on the camera streams)
I followed the steps of this tutorial
I downloaded the docker image provided by ifm with docker pull ghcr.io/ifm/ifm3d:latest-l4t@sha256:71a34e14f279a76e25b840b6eb0297a393b09f3ad311b0f51667ef00560bd928
I used docker save to make it a tar file and transferred it to the vpu via scp
When i run docker load  it proceeds loading layers so far until I get

Error processing tar file(exit status 1): write /home/ifm/venv/lib/python3.9/site-packages/numpy/polynomial/tests/test_printing.py: no space left on device

I inspected the docker image and its size is 1.02GB
df - h returns the following when run of the vpu via ssh:

o3r-vpu-c0:~$ df -h
Filesystem       Size  Used Avail Use% Mounted on
none             1.6G     0  1.6G   0% /dev
/dev/mmcblk0p1   6.1G  4.2G  1.6G  74% /
tmpfs            1.9G  4.0K  1.9G   1% /dev/shm
tmpfs            1.9G  9.8M  1.9G   1% /run
tmpfs            1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs            1.9G  4.0K  1.9G   1% /tmp
tmpfs            1.9G   40K  1.9G   1% /var/volatile
/dev/mmcblk0p33  1.2G  9.0M  1.1G   1% /opt/ifm/data
overlay          1.2G  9.0M  1.1G   1% /etc/systemd/network
tmpfs            383M     0  383M   0% /run/user/989

Is there any step I am missing to get a container with ifm3d libraries running on the vpu?

Hi @niallomahony93
There seem to be two problems here:

The Docker save to tar and scp copy to the VPU and an additional docker load on the VPU takes "double" the space of the container to unpack it. This is a known bug. Please have a look at this description on how to avoid file size limitations on the VPU

docker save <image> | ssh -C<EMAIL_ADDRESS>docker load

The VPU space seems to be quite “full” already. If you don't need older Docker containers and Images anymore, please consider a docker system prune. This will remove dangling images, not used containers, etc.
If the space is not release directly (after a docker system / container prune), some min or a reboot is sometimes required.

If you want to deploy code using CUDA a specialized firmware is required, which comes with the CUDA libs and dependencies.
Please get in touch via email at<EMAIL_ADDRESS>for a download link.

Thank you, I was able to deploy the container with the instructions provided and will be in contact re: firmware for CUDA deployment


Path change native event Fix #823

this PR includes two more PRs that I based my work on: getPath and the original path change (Which I ended up not using)

Fixes #823 (since github didn't autolink it from title)


Should someone else maintain this?

Yes. That will happen in the future. Can't give you an exact timeline.

@Qmando please don't let this die without support.
Usecase for ElastAlert is pretty unique and it stands well in the ELK ecosystem as an open source alternative to licensed Elastic alerting features. Moreover, the functionality is way way richer than what Elastic guys have under the hood.
You did a great job with building maintaining it, now please hand it over to someone who deserves.
Cheers!

Hi I wonder whether there are some updates? This is a very helpful and useful project!

Hi I wonder whether there are some updates? This is a very helpful and useful project!

People can follow on this discussion here to pitch in their opinions about an official fork and the process of it: jertel#77


Unauthorized tagging operation

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v20.12.2
Amplify CLI Version
12.11.1
What operating system are you using?
windows
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
nope
Describe the bug
I attempted to add a custom resource that enables a backup plan with a KMS key (based on https://aws.amazon.com/blogs/storage/automate-backups-for-aws-amplify-graphql-backends-with-aws-backup/).
I've updated the CDK and changed some names and the policy owner in the example code, which I do not believe is the issue (see attached CDK code if needed).
The error I receive does not seem relevant and links to an access policy that I do not process when I execute amplify push.
🛑 CFN Deployment failed for custom resources.
Name: amplifyclientdevelopkeyA93F5F9B (AWS::KMS::Key), Event Type: create, Reason: Resource handler returned message: "Encountered a permissions error performing a tagging operation, please add required tag permissions. See https://repost.aws/knowledge-center/cloudformation-tagging-permission-error for how to resolve. Resource handler returned message: "Unauthorized tagging operation"" (RequestToken: e3913e9e-a5bf-ef1a-befa-81f271edd41e, HandlerErrorCode: UnauthorizedTaggingOperation), IsCustomResource: true

Is this an Amplify error, a cdk-script error or do I need to fellow the steps in the link?
cdk-stack.ts
import * as AmplifyHelpers from '@aws-amplify/cli-extensibility-helper';
import { Construct } from 'constructs';
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as kms from 'aws-cdk-lib/aws-kms';
import * as backup from 'aws-cdk-lib/aws-backup';
export class cdkStack extends cdk.Stack {
  constructor(
    scope: Construct,
    id: string,
    props?: cdk.StackProps,
    amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps,
  ) {
    super(scope, id, props);
    /* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
    new cdk.CfnParameter(this, 'env', {
      type: 'String',
      description: 'Current Amplify CLI env name',
    });

    // This line is changed
    const keyAdmin = new iam.AccountRootPrincipal();
    // This line is changed
    const backupAdmin = new iam.Role(this, 'BackupAdminRole', {
      assumedBy: new iam.AccountRootPrincipal(),
    });
    backupAdmin.addManagedPolicy(
      iam.ManagedPolicy.fromAwsManagedPolicyName('AWSBackupFullAccess'),
    );

    const key = new kms.Key(
      this,
      `amplify-client-${AmplifyHelpers.getProjectInfo().envName}-key`,
      {
        removalPolicy: cdk.RemovalPolicy.RETAIN,
        alias: `alias/amplify-client-${
          AmplifyHelpers.getProjectInfo().envName
        }-key`,
        description:
          'KMS key for encrypting the objects in your AWS Backup Vault',
        enableKeyRotation: false,
        admins: [backupAdmin, keyAdmin],
        policy: new iam.PolicyDocument({
          statements: [
            new iam.PolicyStatement({
              actions: [
                'kms:Create*',
                'kms:Describe*',
                'kms:Enable*',
                'kms:List*',
                'kms:Put*',
                'kms:Update*',
                'kms:Revoke*',
                'kms:Disable*',
                'kms:Get*',
                'kms:Delete*',
                'kms:ImportKeyMaterial',
                'kms:TagResource',
                'kms:UntagResource',
                'kms:ScheduleKeyDeletion',
                'kms:CancelKeyDeletion',
              ],
              principals: [keyAdmin],
              resources: ['*'],
            }),
            new iam.PolicyStatement({
              actions: [
                'kms:Encrypt',
                'kms:Decrypt',
                'kms:ReEncrypt*',
                'kms:GenerateDataKey*',
                'kms:DescribeKey',
              ],
              principals: [backupAdmin],
              resources: ['*'],
            }),
          ],
        }),
      },
    );

    const plan = new backup.BackupPlan(this, 'amplify-client-plan', {
      backupPlanName: `amplify-client-plan-${cdk.Fn.ref('env')}`,
      backupPlanRules: [
        backup.BackupPlanRule.daily(),
        backup.BackupPlanRule.weekly(),
        backup.BackupPlanRule.monthly1Year(),
      ],
      backupVault: new backup.BackupVault(this, 'Vault', {
        backupVaultName: `amplify-client-vault${
          AmplifyHelpers.getProjectInfo().envName
        }`,
        encryptionKey: key,
        accessPolicy: new iam.PolicyDocument({
          statements: [
            new iam.PolicyStatement({
              effect: iam.Effect.DENY,
              principals: [new iam.AnyPrincipal()],
              actions: ['backup:DeleteRecoveryPoint'],
              resources: ['*'],
              conditions: {
                StringNotLike: {
                  'aws:PrincipalArn': [backupAdmin.roleArn],
                },
              },
            }),
          ],
        }),
      }),
    });

    plan.addSelection(`ClientSelectionByTag`, {
      resources: [
        backup.BackupResource.fromTag('user:Stack', cdk.Fn.ref('env')),
      ],
    });
  }
}

Expected behavior
Should not complain about tagging permissions?
Reproduction steps

amplify add custom
replace cdk-stack.ts with the attached code above
amplify push

Project Identifier
Project Identifier: 0e0c35de10e80e85c555ebedce4575e5
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

I just verified my IAM profile and I'm member of admin group, and the policy of AdministratorAccess and does indeed include TagResource in KMS.  Could it be that amplify use another profile?
I've successfulle creared other custom resource before but not KMS and and with latest amplify CLI.

Hey @bergmorten, we can check the profile the project is using in the .config/local-aws-info.json file. The .config folder may be hidden in Vscode, you can disable this in the .vscode/settings.json.
On testing with the provided example, i did run into the error with a user that utilized AdministratorAccess-Amplify. On adding a inline permission with tagging permissions and other resource permissions such as backup,kms I was able to do a successful push.

Thanks I'll try this :-)

@bergmorten do reach out to us if you are experiencing issues with this.

@ykethan An delayed update ( took a few day off last week).
First I tried to add inline policy for my IAM account:
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "VisualEditor0",
			"Effect": "Allow",
			"Action": [
				"kms:TagResource",
				"kms:UntagResource"
			],
			"Resource": "*"
		}
	]
}

This did not change anything. Then I tried to deploy via amplify console / continuous deploys trigger. This also resulted in the same error. Amplify console use the amplifyconsole-backend-role which has full AdministratorAccess.

I need continuous deployment to work since I've multiple stacks/tenants which triggers deployment. I do not understand why AdministrorAccess is not sufficient?

@bergmorten retested this, but did not observe this behavior on my push.
user permissions

created a custom resource with the code provided and pushed

Very odd, I see that you have kms:* , while I only set kms:TagResource and kms:UnTagResource. Must I do re-login for the policy to be effective?
Still I think it very odd that administrator access which has Action:* is not sufficient? Why is inline policy better than AWS managed roles?

Hey @bergmorten, with AWS Admin access should have access to creating the resources. In my testing i used a user with Amplify Admin access which is a scoped policy for Amplify resources. I utilized the inline permission to provide additional permission which in your case should not be needed as it is AWS Admin access.
As a quick check could you ensure the access key matches the .config/local-aws-info.json profile credentials locally present at ~/.aws/credentials for example.

I got the aws-amplify-console to push the custom CDK, the error I had previously with console was not a tag resource error 👍  Since I now have pushed the CDK, it think it will be more difficult to debug the CLI access issue.
My content of amplify/.config/local-aws-info.json is:
{
  "develop": {
    "configLevel": "amplifyAdmin"
  }
}

Since it is missing profileName and useProfile, I assume it use 'default'?
In ~/.aws/credentials I've two accounts, default and  amplify-account, both with an aws_access key & secret (no SSO).
The account amplify-account point to a IAM user which has only one policy attached (administratorAccess).

However the default points to a IAM user with multiple policies, including both administrator and administratorAccess-Amplify.

The default administrator has type job function, could this be an issue?
or, is it a issue that default  has multiple policies where some allow other not? I thought this was ok...

Hey @bergmorten, could you try running amplify configure project -> AWS Profile setting -> yes -> Select the authentication method you want to use:  AWS profile and select the profile. This update the project to use the profile. Could you let us know if this mitigates the push issue. Do ensure you run a pull first to sync the changes made on the console build.

@ykethan I do confirm that this did work. I did checkout of an another environment, which did not yet had custom backup pushed. Then updated my amplify -> aws profile setting,  and after that a amplify pull & push worked :-)
Thanks for your help.
I think you should consider to make backup and a restore guide a part of the amplify tools and guides. It's a bit odd that a such important task is not part of amplify?

Hey @bergmorten, we recently introduced Amplify Gen 2, which provides advanced customizations and integrations using CDK under the hood.
Do check out our documentation at Amplify documentation and let us know if this would assist with your use case.  Examples on deletion protection and backup: https://docs.amplify.aws/react/build-a-backend/add-aws-services/deletion-backup-resources/

Hi @bergmorten @ykethan , chiming in as I was having this issue as well trying to create a KMS Key through Amplify custom resource with CDK, been tracking this issue, and finally what worked for me was to update the .config/local-aws-info.json so that it uses the default profile instead of the Amplify-Admin one, not sure why it would fail with the amplify one as it has the TagResource and UntagResource permission on CloudFormation actions, only thin I see is that it has a string like pattern check on the resources property, could this be affecting the create process in this case?:

Closing the issue due to inactivity. Do reach out to us if you are still experiencing this issue


Consider adding a simple effect checker for safe vs unsafe

Rust has an excellent way to explicitly demarcate parts of API that are not memory safe: unsafe blocks and unsafe functions. The combination of two lets one to be sure that the code you're writing is memory safe as long as you don't use unsafe parts. And when you do it's going to be explicitly visible which parts are not safe.
I've prototyped a simple version of very similar checking in topic/unsafe branch. It's built using a combination of macro annotations (@unsafe), blackbox macros (unsafe { ... }) and @compileTimeOnly.
Even though implemenation looks reasonably decent to me the main pain point comes from the fact that one can't abstract over safety of the code (i.e. no effect polymorphism.) It might be possible to encode it with some extra macros but I don't see any way to make it robust. The area needs some extra research before we commit to this design.

This idea is a better fit for Scala Native project https://github.com/scala-native/scala-native/issues/44


User StructureGraph for DistinctPathFinder

I'm trying to add the PR that was previously intended for pymatgen.
#https://github.com/materialsproject/pymatgen/pull/1435
My understanding of the current implementation of DistinctPathFinder is that it only looks for distinct paths and does not keep track of all paths.
My usage scenario requires the full list of hops (I'm planning on analyzing full migrations pathways using either charge density of IDPP).  This will require some redesign of the class.
If there are no objections, I would like to rewrite the DistinctPathFinder with the StructureGraph to keep track of all the symmetry equivalent hops.  I think I can keep the output as a list of MigrationPaths so it won't affect the rest of the codebase very much.
However, doing this means that we will have to compare any two hops to check if they are equivalent.  The only way I can see for doing this is by adding two dummy atoms at the endpoints for each hop and performing structure matching.
So I will have to replace how the set of symmetrically inequivalent hops are generated.
This is going to take some effort and I just wanted to make sure the devs agree with my approach before I start coding this up.  And make sure doing this will not disrupt other ongoing efforts.
Thanks in advance.

Additional comment:
Using the end-point comparing approach should also make enumerating the vacancy migration much more straight forward.
Assuming the input is a structure with all the sites occupied.
For the interstitial we will just structurematch an empty structure with two dummy atoms at two of the nodes in the structuregraph.
For the vacancy we still add two dummy atoms at the nodes of the structuregraph which are atomic sites.  Only difference is we add the working ion on all the other sites for the structurematcher fitting.

Just to clarify - what do you mean by the PathFinder does not keep track of all paths?  You mean even symmetrically equivalent pathways? That can be generated simply by applying the symmetry operations of the spacegroup to all distinct paths.
The code already checks whether paths are equivalent. Just to clarify that the start and end points are INSUFFICIENT to denote whether a path is distinct. You need the endpoints and the midpoint of the pathway. I suggest you looking at the MigrationPath.__eq__ method to understand how distinct paths are determined. I think this will work in your case as well.
Also, the current code works with vacancy migration. To make it work with an additional atom migration, all that needs to be done is to insert dummy atoms into the distinct sites and postprocess it.
I suggest you throw the code a few example structures with vacancy migration and see what it gives you first before making mods.

Thanks for the input!  I really helped clear things up.

You mean even symmetrically equivalent pathways? That can be generated simply by applying the symmetry operations of the spacegroup to all distinct paths.
That was the additional part that I needed.  I will try to generate the full network of paths thisway and compare with the structureGraph results to convince myself.

The code already checks whether paths are equivalent. BTW, just the start and end points alone are INSUFFICIENT
OK, agreed.  I think I have a better idea of what needs to be done now.
I will reach out if I have anymore questions

The code already checks whether paths are equivalent. BTW, just the start and end points alone are INSUFFICIENT

Hi, I know this is closed, but i don't understand how to apply the symmetry operations of the group in practical scripting terms. I have the exact same issue as the original poster

The code already does this automatically for you. So you do not need to apply the operations yourself. If you want to know how the operations are applied, you can look at the source code.


In Core2 there no M5.Power functions

In Core2 there no M5.Power functions.
I know that here is AXP192 chip.
But there no getBatteryLevel() function

You can use the function GetBatVoltage() to read his voltage, the standard voltage is 3.7V.

#58 https://github.com/m5stack/M5Core2/pull/58


Codecov settings

Move settings from .github/workflows/test.yml to codecov.yml
Change the path specification method from direct specification to regular expression pattern

Codecov Report
Patch coverage has no change and project coverage change: -34.83 :warning:

Comparison is base (c40feb7) 96.12% compared to head (82e8e0e) 61.30%.

:exclamation: Current head 82e8e0e differs from pull request most recent head d20ca92. Consider uploading reports for the commit d20ca92 to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #105       +/-   ##
===========================================
- Coverage   96.12%   61.30%   -34.83%
===========================================
  Files           6       22       +16
  Lines         129      783      +654
  Branches        7      134      +127
===========================================
+ Hits          124      480      +356
- Misses          2      253      +251
- Partials        3       50       +47

Flag
Coverage Δ

unittests
61.30% <ø> (-34.83%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 22 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

As a result, the coverage value will decrease, but that value is the true value.


_1password-gui: 8.0.27 -> 8.0.28

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for _1password-gui is: "Multi-platform password manager"
meta.homepage for _1password-gui is: "https://1password.com/"
meta.changelog for _1password-gui is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 8.0.28 with grep in /nix/store/qnp5f4lzkjlrgkhlr09p79cr241xh2lq-1password-8.0.28

directory tree listing: https://gist.github.com/c4d0286392650d479e521fbd4643352e

du listing: https://gist.github.com/3409f110942eb8aac74b82ea6e47512f

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
0 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
0 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
_1password-gui

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/qnp5f4lzkjlrgkhlr09p79cr241xh2lq-1password-8.0.28 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A _1password-gui https://github.com/r-ryantm/nixpkgs/archive/a12f4b172f35d9e6d8b180de6bd38a7a9c147557.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/qnp5f4lzkjlrgkhlr09p79cr241xh2lq-1password-8.0.28
ls -la /nix/store/qnp5f4lzkjlrgkhlr09p79cr241xh2lq-1password-8.0.28/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

_1password-gui

Maintainer pings
cc @danieldk @timstott for testing.

I downloaded the binary fro r-ryantm's cachix and launched it. When you hit the 'Visit 1Password.com to Sign In' link, it doesn't do anything and I see a segfault in the stderr.

Odd, I re-authenticated and it works fine here. However, the segfault in your case seems to be in gio (or xdg-open), not 1Password itself.

I'm not sure how the 1password versions work at all -- it seems like a big jump from 0.9.9-3 to 8.0.28? FWIW, my system is:

The version number is kinda strange. Maybe they want to sync it with the macOS and Windows versions. On the other hand, 0.9.28 just came out?
https://releases.1password.com/linux/


add LimitNOFILE to service files

在trojan-go的systemd模版中添加LimitNOFILE解决使用一段时间发生断连的情况
参考：
https://github.com/p4gefau1t/trojan-go/pull/313

Some vps not support that kind of setting i think.
Isn't better to set certain limit number.
LimitNOFILE=51200

I have changed it on my vps. But disconnection still happened. it may need more test

@CiXY you may try this

LimitNOFILE=51200
LimitNPROC=infinity

For public servers add more zero xD 512000. I set this one with traffic vnstat in vps 150-400Mbps.


Fix escaping HTML <

Oops, my fault. BTW any chance we could change to MarkDown (like README.md) for generating docs.html? I can't promise but I might be able to convert docs.html to markdown for you.

The website has a new home: https://github.com/vlang-io/website
I'll use Markdown in the future, yes.


Missing resources

I found some mistakes on Step 14 of the codelab:

min_height is missing in values/dimens.xml
@color/colorPrimary should actually be @color/purple_500 for newer versions of Android Studio (I am using version 4.1.2)

min_height is added as part of the "Add NewWordActivity" step

I found the min_height but not the colorPrimary.


[Performance bug] Incorrect register dependency tracking

Got a slight issue with the following Spectre-BTB gadget. It appears that there is a dependency where it shouldn’t.
.global gadget
gadget:
  # Windowing gadget
  addi sp, sp,-8
  li t0, 1
  # Slow down the commits using dependencies, also slow down the branch execution to avoid pipeline flush too early
  div t0, jump_dst, t0
  # Triggering speculation
  jalr x0, t0

.global gadget_middle
gadget_middle:
  # Access secret
  lb t0, 0(secret_ptr)
  # Select one bit of secret
  srl t0, t0, bit_select # shift bit
  andi t0, t0, 1 # select bit
  slli t0, t0, 10 # 6 = LOG2_BYTES_PER_LINE # set to size of cache line or 0
  or t0, t0, shared_mem_ptr # compute address in shared mem depending on bit
  # address is therefore either shared_mem_ptr or shared_mem_ptr + 64
  # Access shared memory
  lb t0, 0(t0)

.global gadget_end
gadget_end:
  addi sp, sp, 8
  ret

gives with konata.

The prediction is correct but the lb after the jump does not execute (because of t0 false dependency ?).
The correct speculative execution is flushed and reexecuted.
If I replace t0 with t1 in the windowing gadget
li t1, 1
div t1, jump_dst, t1
# Triggering speculation
jalr x0, t1

everything is working correctly.

I am not familiar enough with the core for a proper debug. But if you have some ideas where to look for, I can give it a try.

Hi,
Yes, that's curious.
Can your software code + binaries + assembly code so i can reproduce ? Woud need to check what is reaching the LSU / dispatch
thanks ^^

You can find all that here : L4-SpectreBTB.zip
This is derived from your SideChannels folder so you may copy/paste only parts of the code and it should work.
(I am preparing a practical on microarchitecture security with these)

Thanks for the zip ^^
i can reproduce.
I checked and all of this seems normal, for non-obvious reasons XD
So, but your sit belt :
What is happening on your first screenshot is the following :

PC=0x80000578 is interpreted by the post decode predictor as a JALR which should use the RAS to pop where to go.
https://github.com/SpinalHDL/NaxRiscv/blob/ec3ee4dc0d7e6351f2df8c2ca7ad136c224d6321/src/main/scala/naxriscv/prediction/DecoderPredictionPlugin.scala#L162
From the RISC-V spec :
-"A JAL instruction should push the return address onto a return-address stack (RAS) only when rd=x1/x5"

x5 => t0
This will override the prediction made by the BTB, by the one from the RAS, throwing the 0x8000057C aways and "wrongly" correcting the PC to 0x80000294
So, avoid using t0 for jumps ^^

Wow, nice catch indeed. I forgot that t0 was allowed to impact the RAS. Thanks !

I forgot that t0 was allowed to impact the RAS

Same XD


🛂 Add Data Platform Apps and Tools OIDC provider

Data Platform's Open Metadata instance running on EKS in Data Platform's Apps and Tools account needs to assume roles in our data accounts (data-platform-development) to collect metadata from Glue and S3
We will use IRSA to achieve this in the cluster
An explicit deny prevented me from doing this in the environments repository
Signed-off-by: Jacob Woffenden<EMAIL_ADDRESS>

This is no longer required.
I've removed the OIDC provider from data-platform-development account.


Gas Overlays going invisible

Description
I've had this happen rarely while playing, but sometimes my gas overlays disappear completely. This renders me unable to see Plasma, CO2, Miasma, Tritium, as well as fires. Restarting and relogging do not fix the issue.
Reproduction
I am not sure how to reproduce this, as it happens rarely, like maybe 2 games out of 100 games.
Screenshots

As you can see, the plasma miner appears empty, despite other players being able to see the purple overlay.
Additional context
I apologize on not being able to know the reasoning why this happens, but I do notice it when I start many fires in the mixer tank with a plasma/o2 mix. I'm just reporting so there is a record if others have the same issue.

Related to https://github.com/space-wizards/space-station-14/issues/9294

It seems to be the same bug, so closing as a duplicate.
The bug may or may not get fixed by #9619.


Add github action for code formatting

gotta exclude utils/install_poetry.py
as it was adopted mostly as is from https://install.python-poetry.org/


[BUG]: Azure Functions Deploy task failing repeatedly

New issue checklist

[X] I searched for existing GitHub issues
[X] I read pipeline troubleshooting guide
[X] I checked how to collect logs

Task name
Azure Functions Deploy
Task version
2.238.1
Issue Description
Our CD pipeline that deploys our services and function apps is failing in one environment only. The environment happens to be in Azure CN cloud. However, other deployments to the Azure CN cloud worked fine, only one environment seems to fail repeatedly. The error message reads:
##[error]Error: Failed to get resource ID for resource type 'Microsoft.Web/Sites' and resource name ''. Error: Could not fetch access token for Azure. Status code: endpoints_resolution_error, status message: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.partner.microsoftonline.cn//v2.0/.well-known/openid-configuration
When I call the well known endpoint in my browser it works just fine.
Additionally, all my other deployments work fine, we are not configuring this environment any differently other than choosing the right service principal. The Service Principal itself was checked and is valid.
Environment type (Please select at least one enviroment where you face this issue)

[ ] Self-Hosted
[X] Microsoft Hosted
[ ] VMSS Pool
[ ] Container

Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Ubuntu latest
Relevant log output
2024-04-23T06:15:22.5394220Z ##[section]Starting: Deploy Catalog Cache Function App
2024-04-23T06:15:22.5402680Z ==============================================================================
2024-04-23T06:15:22.5402913Z Task         : Azure Functions Deploy
2024-04-23T06:15:22.5403083Z Description  : Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications
2024-04-23T06:15:22.5403334Z Version      : 2.238.1
2024-04-23T06:15:22.5403491Z Author       : Microsoft Corporation
2024-04-23T06:15:22.5403656Z Help         : https://aka.ms/azurefunctiontroubleshooting
2024-04-23T06:15:22.5403827Z ==============================================================================
2024-04-23T06:15:25.0280162Z Got service connection details for Azure App Service:'biz-common-ecn-sand-catalogcache-func'
2024-04-23T06:15:35.2136506Z ##[error]Error: Failed to get resource ID for resource type 'Microsoft.Web/Sites' and resource name '<my function app>'. Error: Could not fetch access token for Azure. Status code: endpoints_resolution_error, status message: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.partner.microsoftonline.cn/<tenantid>/v2.0/.well-known/openid-configuration
2024-04-23T06:15:35.2205561Z ##[section]Finishing: Deploy Catalog Cache Function App

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION]

Repro steps
No response

Is there any documentation about what permissions are needed by the Service Connection? At the moment my service connection has a Contributor role at the resource group level, the same resource group under which the function app resides

Also having this issue, but rather than Azure Functions, we're trying to deploy an app service to Azure China. Specifically it's the 'Azure App Service deploy' task v4 that we're using, although I also tried v3 and had the same problem.
The service connection is fine -- I have several other pipelines running Azure Powershell tasks under the same service connection, and they run without issue. But our two Azure App Service deploy tasks fail completely:
2024-04-23T13:25:40.3283519Z ##[section]Starting: Deploy: set app service to deployed image
2024-04-23T13:25:40.3293014Z ==============================================================================
2024-04-23T13:25:40.3293173Z Task         : Azure App Service deploy
2024-04-23T13:25:40.3293283Z Description  : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2024-04-23T13:25:40.3293471Z Version      : 4.238.1
2024-04-23T13:25:40.3293562Z Author       : Microsoft Corporation
2024-04-23T13:25:40.3293656Z Help         : https://aka.ms/azureappservicetroubleshooting
2024-04-23T13:25:40.3293786Z ==============================================================================
2024-04-23T13:25:44.0584652Z Got service connection details for Azure App Service:'api-3t--app-service--dev'
2024-04-23T13:25:53.2883895Z ##[error]Error: Failed to get resource ID for resource type 'Microsoft.Web/Sites' and resource name '***'. Error: Could not fetch access token for Azure. Status code: endpoints_resolution_error, status message: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.partner.microsoftonline.cn/***/v2.0/.well-known/openid-configuration
2024-04-23T13:25:53.2919412Z ##[section]Finishing: Deploy: set app service to deployed image

The only workaround I could find for this was to:

Set up and run a self-hosted agent (we normally use Azure hosted agents)
Run a Squid forward proxy server in a docker container and set the environment variable VSTS_HTTP_PROXY to point at it

This suggests to me that it might be related to this issue in azure/msal-node.
This is really frustrating and has taken me almost all day to fix.

@abagonhishead Thanks for the information on the workaround, I'll take that back to our delivery infrastructure team and see if they can help me with that.
But yes, still hoping for a fix here :(

It looks like  more and more people face this issue as seen at the link below. Still waiting for a response here.
https://developercommunity.visualstudio.com/t/Deploying-to-Azure-China:-Could-not-fetc/10652428?viewtype=all

@abagonhishead I think I've found another workaround. I switched the agent from a linux agent to a windows agent. So it looks like this issue only affects linux agents. Still, only a workaround.

I'm also having the same issue recently. I managed to deploy application by switching from AzureRmWebAppDeployment@4 tasks for (Web App & Functions) to AzureFunctionApp@1 & AzureWebApp@1.
However, it's failing now on AzureAppServiceManage@0 when starting the App Service.
It used to work perfectly, I suspect a regression in these tasks with latest releases.

@FinVamp1 is there any update on this issue? It has been several weeks so just checking in

any updates? the same issue here


Add FreeBSD x86 and x86_64 native library cross-compilation via Docker

This uses empterdose/freebsd-cross-build:9.3 to compile for FreeBSD x86 and x86_64. This has been tested via Komga by some of our users and reported to work well.
Note that dockcross doesn't support any FreeBSD build.
I thought about creating a derived Docker image that would have bash, java and perl installed, but given the frequency of rebuilds this is probably overkill.
Supersedes #494

Great! Merged


Workflow fixtures documentation clarification

In the fixtures documentation it describes several built-in fixtures, some of which I've been using are data_path and temp_path.
In the detailed usage it shows that data_path is being imported directly from virtool_workflow.storage.paths. When I try to use it as it is described the import gets marked as an unused in my IDE as well as Codacy, and the fixture still seems to be accessible when the import is removed.
Does the detailed usage represent an older implementation of these fixtures?

In the fixtures documentation it describes several built-in fixtures, some of which I've been using are data_path and temp_path.
In the detailed usage it shows that data_path is being imported directly from virtool_workflow.storage.paths. When I try to use it as it is described the import gets marked as an unused in my IDE as well as Codacy, and the fixture still seems to be accessible when the import is removed.
Does the detailed usage represent an older implementation of these fixtures?

A fixture will be available for use as long as there is a reference to it's class stored in the enclosing, or global, scope. In practice this means that as long as the module containing the fixture has been imported the fixture can be used.
The IDE is showing the unused import error as it considers the data_path parameter to be a local variable
of the function and does not consider it a usage of the data_path fixture from module scope.
When python imports a module a reference to that module is stored in sys.modules. Upon another import of that
same module, sys.modules will be checked first and the module's code will not be executed a second time. Additionaly using the from ... import ... syntax also imports the entire module and stores a ModuleType object into sys.modules.
What this means for us is that as long as the module containing a specific fixture (such as data_path) has been imported at any point in the execution of the program the fixture will be available, regardless of whether or not it was imported within the workflow module.
The runtime for virtool_workflow will implicitly import all of the documented fixtures prior to executing the workflow defined in the workflow.py file. It will also import the fixtures.py file prior to running. This means that when using workflow run to execute your workflow, the data_path fixture and others will already be available. However, when running outside of the runtime context those modules will not be implicitly imported, which is why I included the import statements in the documentation.
Also the import mechanism was previously a bit broken and did not consistently import the required modules. The implicit imports are now working as of the 0.3.0 release. There has also been a runtime pytest fixture included in
virtool_workflow_runtime.test_utils which will load all fixtures which would be present in the runtime environment.
A section describing how to use the runtime pytest fixture is now given in the fixture docs.
I think at this point that the explicit imports in the docs have become misleading. I will remove them from the examples. I'll also add a section on how to avoid the unused import warning when pulling in additional fixtures.


R200 does not work on NUC6i5SYK

I can start the demo app, but the demo app can run for about 30 seconds and then crash the PC.
i have to re-start it after each crash.
Both sample cpp-tutorial-1-depth and c-tutorial-1-depth runs correctly.
Seems that if the sample do not require additional window, it works correctly.
sample cpp-config-ui return error: intel_do_flush_locked failed: input/output error.
I have tested it on win 10 and it is works fine.
| Camera Model | R200 |
| Firmware Version | <IP_ADDRESS>  |
| Operating System & Version |  Ubuntu 14.04.4 |
| Kernel Version (Linux Only)  | <IP_ADDRESS>400-generic |
| Build System | Makefile |

Hi @meitiever. This isn't related to an issue with software, but some kind of opengl driver incompatibility. As noted, this only affects example apps that open an opengl window. Let us know if you fix it, but otherwise we can't provide any support.


Dropping Packets Error Code 6?

Hello Jesse, I am trying to use Divert.Net to create a simple packet latency simulator, could you explain why the packets are being randomly dropped?
    Thread receiving;
    Thread sending;
    Diversion diversion;

    public struct packet
    {
        public Address address;
        public byte[] buffer;
        public uint receiveLength;
        public uint sendLength;
        public DateTime dt;
    }

    packet rp = new packet();
    packet sp = new packet();
    Queue pq = new Queue();

    public int pc = 0; //packet count

    public void run()
    {
        rp.address = new Address();
        rp.buffer = new byte[65535];
        rp.receiveLength = 0;
        rp.sendLength = 0;
        rp.dt = DateTime.Now;

        while (true)
        {
            if (!diversion.Receive(rp.buffer, rp.address, ref rp.receiveLength))
            {
                 continue;
            }
            else
            {
                rp.dt = DateTime.Now.AddTicks(1000000); //adds timestamp + 100ms
                pq.Enqueue(rp); //adds packet to queue
                pc++;
           }
       }
    }

    public void send()
    {
        while (true)
        {
            if (pc > 0)
            {
                sp = (packet)pq.Peek(); //gets first packet in the queue
                if (sp.dt <= DateTime.Now) //checks if packet arrival time is <= than current time
                {
                    //diversion.SendAsync(sp.buffer, sp.receiveLength, sp.address, ref sp.sendLength);
                    if (!diversion.Send(sp.buffer, sp.receiveLength, sp.address, ref sp.sendLength))
                    {
                        try
                        {
                            Invoke(new Action(() => { richTextBox1.AppendText(System.Runtime.InteropServices.Marshal.GetLastWin32Error().ToString() + "\n"); }));
                        }
                        catch { }
                        pq.Dequeue();
                        pc--;
                    }
                    else
                    {
                        pq.Dequeue();
                        pc--;
                    }
                }
            }
        }
    }

    private void button1_Click(object sender, EventArgs e)
    {
        diversion = Diversion.Open("outbound", DivertLayer.Network, 100, 0);

        receiving = new Thread(run);
        receiving.Start();

        sending = new Thread(send);
        sending.Start();
    }

    private void button2_Click(object sender, EventArgs e)
    {
        sending.Abort();
        receiving.Abort();
        if(pq.Count > 0)
        {
            pq.Clear();
        }
        diversion.Close();
    }`

This is using Windows Forms and I have compiled Divert.Net to x64.
Side note: I am a beginner with this, but if you could advise it would be appreciated.

@TodM0 Where are you getting error code 6? If you look here, and you're properly fetching the error code, then error 6 means ERROR_INVALID_HANDLE which means that you're performing an operation against an invalid handle.
It's been a while, but there should be something like Valid or IsValid property on the diversion object to ensure that you've got a valid handle. Ensure that you're running as administrator, and possibly try using the PInvoke version at https://github.com/TechnikEmpire/DivertPInvoke. This way you can follow the official documentation exactly and greatly eliminate the odds that a bug is due to my wrapper.

@TechnikEmpire Thanks for the quick reply. I am getting the error after diversion.send, but if I remove the timestamp checks, so that there is no delay in sending the packets everything works fine. I did try the PInvoke version prior to Divert.Net, and both produce the same random dropped packets when trying to delay them from sending instantly.

If I had to guess, I'd say that you're overflowing the queue in WinDivert
by waiting too long. There is one and you can configure it to be larger
(search the docs) and that might help, but what I would suggest is
switching to a threaded implementation using both the asynchronous send and
receive functions. This way at least you'll be parallelizing the work so
your stacking up of packets on the internal queue isn't linear and
therefore exponential.
That's of course all assuming that my guess is accurate as to the cause.
On Wed, Jan 25, 2017 at 1:22 AM TodM<EMAIL_ADDRESS>wrote:

@TechnikEmpire https://github.com/TechnikEmpire Thanks for the quick
reply. I am getting the error after diversion.send, but if I remove the
timestamp checks, so that there is no delay in sending the packets
everything works fine. I did try the PInvoke version prior to Divert.Net,
and both produce the same random dropped packets when trying to delay them
from sending instantly.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/TechnikEmpire/Divert.Net/issues/12#issuecomment-275029233,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKtty6SV51fiXrzDaU6-rYiV51VGqzrMks5rVupCgaJpZM4LtGY-
.

--
Jesse Nicholson

@TechnikEmpire I have tried increasing the WinDivert queue length to the maximum with no luck. Should the WinDivert queue length really be an issue when WinDivert automatically drops packets after a set queue time anyway? If the packets are sent by my own C# queue I fail to understand why they wouldn't be being sent all of the time, even though the packets are sent the majority of the time with the added 100ms delay as required. I have also ensured my own queue is not becoming full.
I will delve into the asynchronous methods, but I did not expect it to be this difficult to simply add delay to packets. Looking at my code provided can you see any fault or provide any pointers on how to change it to be asynchronous? Thanks for the support.

I'll do an actual test myself just give me a few. I was taking a stab in the dark but you have valid objections.

@TechnikEmpire Thanks. Using ping google.com -t is how I am seeing the lost packets.

@TodM0 My best guess now is that you're not calling the CalculateChecksums functions. It's a method of the Diversion class in this library. Below is threaded example using the PInvoke library that functions exactly as you intend with zero packet loss.
using System;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Net.NetworkInformation;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace TestDivert
{
    internal unsafe class Program
    {
        public static volatile bool running = true;

        private static void Main(string[] args)
        {
            Console.CancelKeyPress += delegate
            {
                running = false;
            };

            var divertHandle = WinDivertMethods.WinDivertOpen("outbound", WINDIVERT_LAYER.WINDIVERT_LAYER_NETWORK);

            if(divertHandle != IntPtr.Zero)
            {
                Parallel.ForEach(Enumerable.Range(0, Environment.ProcessorCount), x => RunDiversion(divertHandle));
            }

            WinDivertMethods.WinDivertClose(divertHandle);
        }

        private static void RunDiversion(IntPtr handle)
        {
            Console.WriteLine("Starting diversion thread.");

            byte[] packet = new byte[65535];

            while(running)
            {
                WINDIVERT_IPHDR* ipv4Header = null;
                WINDIVERT_IPV6HDR* ipv6Header = null;
                WINDIVERT_ICMPHDR* icmpv4Header = null;
                WINDIVERT_ICMPV6HDR* icmpv6Header = null;
                WINDIVERT_TCPHDR* tcpHdr = null;
                WINDIVERT_UDPHDR* udpHdr = null;

                uint readLength = 0;
                WINDIVERT_ADDRESS addr = new WINDIVERT_ADDRESS();
                if(!WinDivertMethods.WinDivertRecv(handle, packet, (uint)packet.Length, ref addr, ref readLength))
                {
                    continue;
                }

                ////Console.WriteLine("Read {0} bytes into packet, packet direction is {1}.", readLength, addr.Direction.ToString());
                ////Console.WriteLine(BitConverter.ToString(packet, 0, (int)readLength));

                fixed (byte* inBuf = packet)
                {
                    byte* payload = null;

                    uint payloadLen = 0;

                    WinDivertMethods.WinDivertHelperParsePacket(inBuf, readLength, &ipv4Header, &ipv6Header, &icmpv4Header, &icmpv6Header, &tcpHdr, &udpHdr, &payload, &payloadLen);

                    if(ipv4Header != null && tcpHdr != null && payload != null)
                    {
                        // Do IPV4 TCP specific stuff here where a payload is present.

                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;
                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got TCP IPV6 packet from port {0} to port {1}.", tcpHdr->SrcPort, tcpHdr->DstPort);
                    }

                    if(tcpHdr != null && ipv4Header != null)
                    {
                        // Do IPV4 TCP specific stuff here where no payload is present.

                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;
                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got TCP packet from port {0} to port {1}.", tcpHdr->SrcPort, tcpHdr->DstPort);
                    }
                    else if(tcpHdr != null && ipv6Header != null)
                    {
                        // Do IPV6 TCP specific stuff here.

                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;

                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got TCP IPV6 packet from port {0} to port {1}.", tcpHdr->SrcPort, tcpHdr->DstPort);
                    }

                    if(ipv4Header != null && udpHdr != null)
                    {
                        // Do IPV4 UDP specific stuff here.

                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;

                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got UDP packet from port {0} to port {1}.", udpHdr->SrcPort, udpHdr->DstPort);
                    }
                }

                WinDivertMethods.WinDivertHelperCalcChecksums(packet, readLength, 0);

                // Halt the sending of this packet for roughly 100 msec.
                Thread.Sleep(100);

                WinDivertMethods.WinDivertSendEx(handle, packet, readLength, 0, ref addr, IntPtr.Zero, IntPtr.Zero);
            }
        }
    }
}

@TechnikEmpire The example code is great, and reduces the loss a lot, but I am still seeing lost packets unfortunately when left to run for longer.

Is this expected behaviour?

@TodM0 I'm running with 500 msec delays with zero loss right now. I don't think long running pings are a very reliable test for real network issues. If it's anything, I'd argue that it's not the diversion library. I'm on a flaky network myself (LTE connection) and I'm seeing zero loss.

@TodM0 If it's any reassurance, I'm using WinDivert in a commercial-grade content filter that does inline HTTP/S deep content inspection and classification, as well as blocking based on that analysis. It's very stable.

@TechnikEmpire I am just using the regular WinDivert, the pings are also randomly climbing, I know ping tests aren't the best way, but this shouldn't be happening. Does this give you any further clues as to what it could be?

If not I will assume it's a fault somewhere on my machine and will post back when I do some further research and tests. What I do know is applications such as Clumsy which use WinDivert work flawlessly on my machine to increase ping.

@TodM0 I'm not sure, this code is a direct pinvoke into WinDivert so it's not anything my code is doing. I'd look at handle use and priority on Clumsy and compare. Also if you're running clumsy maybe there's two instances of the divert driver running now. You can try 'sc stop WinDivert1.2" and same for WinDiver1.1 or whatever version clumsy has to ensure it's not a driver conflict. Lastly you could remove any firewall software and do Netsh winsock reset and reboot then re-test.

@TechnikEmpire I am a little confused how using Thread.Sleep() doesn't cause the queue to become blocked up. Wouldn't newly arriving packets during the sleep time become stalled too, possibly mounting up the delay? For example: Packet 2 arrives -> Packet 1 Arrives -> 100ms -> Send Packet. Packet 2 may now be waiting in excess of 150ms before being sent, or does WinDivert take care of this somehow?
I have also noticed increasing the packet latency this way is reducing my bandwidth is this normal and can it be avoided?

@TodM0 Regarding the buffer not overflowing, I'd say because such protocols are not going to just be blindly pumping out a nonstop stream of packets. They're sending, then waiting for a response.
With regards to the params, are you setting them correctly, or are you redefining header defs?
https://reqrypt.org/windivert-doc-v1.2.html#divert_set_param
Here's an example in code of me setting those params (c++):
https://github.com/TechnikEmpire/HttpFilteringEngine/blob/master/src/te/httpengine/mitm/diversion/impl/win/WinDiverter.cpp#L122
I have no problems with the WinDivert queue personally so I can't relate or comment on those. If you're starting to have WinDivert specific issues that you truly think are bugs, then please post on basil's repo (official WinDivert repo).

@TechnikEmpire It seems that packets are having to wait too long using Thread.Sleep() when performing bandwidth intensive events causing the ping to spike and occasionally lose packets.
I have tried modifying your example code to copy the packets into a new queue with timestamps to compensate for newly arriving packets during the active delay.
My problem now is that packets are refusing to send with error code 87 (invalid parameter), but the filter is correct it. This error is thrown upon attempting to send a packet after a set delay from the new queue.
Example:
using System;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Net.NetworkInformation;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Collections.Generic;

namespace DivertFix
{
    internal unsafe class Program
    {
        public static volatile bool running = true;
        public static Stopwatch sw = new Stopwatch();

        public struct packet
        {
            public byte[] data;
            public uint readL;
            public WinDivert.WINDIVERT_ADDRESS addr;
            public long timestamp;
        }

        public static Queue<packet> packet_queue = new Queue<packet>();

        private static void Main(string[] args)
        {
            Console.CancelKeyPress += delegate
            {
                running = false;
                sw.Stop();
            };

            var divertHandle = WinDivert.WinDivertMethods.WinDivertOpen("outbound", WinDivert.WINDIVERT_LAYER.WINDIVERT_LAYER_NETWORK, 1000, 0);

            if (divertHandle != IntPtr.Zero)
            {
                Parallel.ForEach(Enumerable.Range(0, Environment.ProcessorCount), x => RunDiversion(divertHandle));
            }

            WinDivert.WinDivertMethods.WinDivertClose(divertHandle);
        }

        private static void RunDiversion(IntPtr handle)
        {
            Console.WriteLine("Starting diversion thread.");

            byte[] packet = new byte[65535];
            sw.Start();

            while (running)
            {

                WinDivert.WINDIVERT_IPHDR* ipv4Header = null;
                WinDivert.WINDIVERT_IPV6HDR* ipv6Header = null;
                WinDivert.WINDIVERT_ICMPHDR* icmpv4Header = null;
                WinDivert.WINDIVERT_ICMPV6HDR* icmpv6Header = null;
                WinDivert.WINDIVERT_TCPHDR* tcpHdr = null;
                WinDivert.WINDIVERT_UDPHDR* udpHdr = null;

                uint readLength = 0;
                WinDivert.WINDIVERT_ADDRESS addr = new WinDivert.WINDIVERT_ADDRESS();
                if (!WinDivert.WinDivertMethods.WinDivertRecv(handle, packet, (uint)packet.Length, ref addr, ref readLength))
                {
                    continue;
                }

                ////Console.WriteLine("Read {0} bytes into packet, packet direction is {1}.", readLength, addr.Direction.ToString());
                ////Console.WriteLine(BitConverter.ToString(packet, 0, (int)readLength));
                fixed (byte* inBuf = packet)
                {
                    byte* payload = null;

                    uint payloadLen = 0;

                    WinDivert.WinDivertMethods.WinDivertHelperParsePacket(inBuf, readLength, &ipv4Header, &ipv6Header, &icmpv4Header, &icmpv6Header, &tcpHdr, &udpHdr, &payload, &payloadLen);

                    if (ipv4Header != null && tcpHdr != null && payload != null)
                    {
                        // Do IPV4 TCP specific stuff here where a payload is present.
                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;
                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got TCP IPV6 packet from port {0} to port {1}.", tcpHdr->SrcPort, tcpHdr->DstPort);
                    }

                    if (tcpHdr != null && ipv4Header != null)
                    {
                        // Do IPV4 TCP specific stuff here where no payload is present.

                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;
                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got TCP packet from port {0} to port {1}.", tcpHdr->SrcPort, tcpHdr->DstPort);
                    }
                    else if (tcpHdr != null && ipv6Header != null)
                    {
                        // Do IPV6 TCP specific stuff here.
                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;
                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got TCP IPV6 packet from port {0} to port {1}.", tcpHdr->SrcPort, tcpHdr->DstPort);
                    }

                    if (ipv4Header != null && udpHdr != null)
                    {
                        // Do IPV4 UDP specific stuff here.
                        //var srcIp = ipv4Header->SrcAddr;
                        //var dstIp = ipv4Header->DstAddr;
                        //Console.WriteLine("From {0} to {1}", srcIp.ToString(), dstIp.ToString());
                        //Console.WriteLine("Got UDP packet from port {0} to port {1}.", udpHdr->SrcPort, udpHdr->DstPort);
                    }
                }

                WinDivert.WinDivertMethods.WinDivertHelperCalcChecksums(packet, readLength, 0);

                //Copy packet to the new queue
                packet newpacket = new packet();
                newpacket.data = packet;
                newpacket.readL = readLength;
                newpacket.addr = addr;
                newpacket.timestamp = sw.ElapsedMilliseconds + 100;
                packet_queue.Enqueue(newpacket);

                //Free(packet); Pop the WinDivert packet from buffer, not sure how or if required?

                //Get the first packet in the new queue
                packet getpacket = packet_queue.Peek();

                //Check if the current time is greater than packet arrival time
                if (sw.ElapsedMilliseconds >= getpacket.timestamp) //removing this if statement allows unlagged packets to send without issue
                {
                    WinDivert.WinDivertMethods.WinDivertSendEx(handle, getpacket.data, getpacket.readL, 0, ref getpacket.addr, IntPtr.Zero, IntPtr.Zero);
                    packet_queue.Dequeue(); //remove first packet from the queue
                    Console.WriteLine(System.Runtime.InteropServices.Marshal.GetLastWin32Error()); //Error 87 - possibly due to addr ref?
                }
            }
        }
    }
}

Could it be that packets need dereferencing from the original queue before being put into the new queue? If you could take a quick look at the code provided, and removing the IF statement as commented you will see the code working, but without delay.
Thanks again for your time.

@TodM0 while I wish I could be of further assistance, this thread has turned into general programming and networking questions. Please direct such questions to stackoverflow.com. I have over a dozen respositories with plenty of unsolved tickets in each of them and while I try to be accommodating, I don't have the availability to address this any further. I'd like to keep my bug tickets reserved for issues with the code. Thanks and all the best.

@TechnikEmpire No problem and thanks for all the help.
I have moved this issue over to stackoverflow if you ever get chance to take a further look at it here.