Unable to access Lambda layer for Python with x86_64 architecture

Describe the bug
I am trying to follow the instructions for using the Lambda Support layer with Python, applying it to the sam-hello-world project described in the SAM tutorial. The layer does not appear to be accessible for x86_64 architecture.
Steps to reproduce
Follow the SAM tutorial. I changed python3.9 to python3.10.
Observe that the application deploys successfully.
Next, add in the Lambda Layer:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app

  Sample SAM Template for sam-app

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3
    MemorySize: 128

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.10
      Architectures:
        - x86_64
      Layers:
        - !Sub 'arn:aws:lambda:${AWS::Region}:901920570463:layer:aws-otel-python-x86_64-ver-1-20-0:3'
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

Run sam build and then sam deploy and you should see an error with the following text:
[…] is not authorized to perform: lambda:GetLayerVersion on resource: arn:aws:lambda:us-east-2:901920570463:layer:aws-otel-python-x86_64-ver-1-20-0:3 because no resource-based policy allows the lambda:GetLayerVersion action

If instead I change the architectures from x86_64 to arm64, run sam build again, then sam deploy will succeed. So this seems to be specific to the x86_64 architecture.
What did you expect to see?
I expected sam deploy to complete successfully with output like below:
CloudFormation events from stack operations (refresh every 5.0 seconds)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus                                       ResourceType                                         LogicalResourceId                                    ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           sam-hello-world                                      User Initiated
CREATE_IN_PROGRESS                                   AWS::IAM::Role                                       HelloWorldFunctionRole                               -
CREATE_IN_PROGRESS                                   AWS::IAM::Role                                       HelloWorldFunctionRole                               Resource creation Initiated
CREATE_COMPLETE                                      AWS::IAM::Role                                       HelloWorldFunctionRole                               -
CREATE_IN_PROGRESS                                   AWS::Lambda::Function                                HelloWorldFunction                                   -
CREATE_IN_PROGRESS                                   AWS::Lambda::Function                                HelloWorldFunction                                   Resource creation Initiated
CREATE_COMPLETE                                      AWS::Lambda::Function                                HelloWorldFunction                                   -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::RestApi                             ServerlessRestApi                                    -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::RestApi                             ServerlessRestApi                                    Resource creation Initiated
CREATE_COMPLETE                                      AWS::ApiGateway::RestApi                             ServerlessRestApi                                    -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment47fc2d5f9d                -
CREATE_IN_PROGRESS                                   AWS::Lambda::Permission                              HelloWorldFunctionHelloWorldPermissionProd           -
CREATE_IN_PROGRESS                                   AWS::Lambda::Permission                              HelloWorldFunctionHelloWorldPermissionProd           Resource creation Initiated
CREATE_COMPLETE                                      AWS::Lambda::Permission                              HelloWorldFunctionHelloWorldPermissionProd           -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment47fc2d5f9d                Resource creation Initiated
CREATE_COMPLETE                                      AWS::ApiGateway::Deployment                          ServerlessRestApiDeployment47fc2d5f9d                -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::Stage                               ServerlessRestApiProdStage                           -
CREATE_IN_PROGRESS                                   AWS::ApiGateway::Stage                               ServerlessRestApiProdStage                           Resource creation Initiated
CREATE_COMPLETE                                      AWS::ApiGateway::Stage                               ServerlessRestApiProdStage                           -
CREATE_COMPLETE                                      AWS::CloudFormation::Stack                           sam-hello-world                                      -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

What did you see instead?
CloudFormation events from stack operations (refresh every 5.0 seconds)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ResourceStatus                                       ResourceType                                         LogicalResourceId                                    ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CREATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           sam-hello-world                                      User Initiated
CREATE_IN_PROGRESS                                   AWS::IAM::Role                                       HelloWorldFunctionRole                               -
CREATE_IN_PROGRESS                                   AWS::IAM::Role                                       HelloWorldFunctionRole                               Resource creation Initiated
CREATE_COMPLETE                                      AWS::IAM::Role                                       HelloWorldFunctionRole                               -
CREATE_IN_PROGRESS                                   AWS::Lambda::Function                                HelloWorldFunction                                   -
CREATE_IN_PROGRESS                                   AWS::Lambda::Function                                HelloWorldFunction                                   Resource creation Initiated
CREATE_FAILED                                        AWS::Lambda::Function                                HelloWorldFunction                                   Resource handler returned message: "User:
                                                                                arn:aws:sts::109411978368:assumed-role/AWSReserved
                                                                                SSO_AdministratorAccess_14ea24479451150a/clasher
                                                                                is not authorized to perform:
                                                                                lambda:GetLayerVersion on resource:
                                                                                arn:aws:lambda:us-east-2:901920570463:layer:aws-
                                                                                otel-python-x86_64-ver-1-20-0:3 because no
                                                                                resource-based policy allows the
                                                                                lambda:GetLayerVersion action (Service: Lambda,
                                                                                Status Code: 403, Request ID:
                                                                                43f87ca0-7208-40c9-b06d-55043f3026ad)"
                                                                                (RequestToken:
                                                                                83c0cd8b-bb8d-8b27-2c2f-b470d56994be,
                                                                                HandlerErrorCode: AccessDenied)
ROLLBACK_IN_PROGRESS                                 AWS::CloudFormation::Stack                           sam-hello-world                                      The following resource(s) failed to create:
                                                                                [HelloWorldFunction]. Rollback requested by user.
DELETE_IN_PROGRESS                                   AWS::Lambda::Function                                HelloWorldFunction                                   -
DELETE_COMPLETE                                      AWS::Lambda::Function                                HelloWorldFunction                                   -
DELETE_IN_PROGRESS                                   AWS::IAM::Role                                       HelloWorldFunctionRole                               -
DELETE_COMPLETE                                      AWS::IAM::Role                                       HelloWorldFunctionRole                               -
ROLLBACK_COMPLETE                                    AWS::CloudFormation::Stack                           sam-hello-world                                      -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Error: Failed to create/update the stack: sam-hello-world, Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "ROLLBACK_COMPLETE" at least once

What version of collector/language SDK version did you use?
Version: arn:aws:lambda:us-east-2:901920570463:layer:aws-otel-python-x86_64-ver-1-20-0:3
What language layer did you use?
Config: Python

Hello, can you please use the layer - arn:aws:lambda:us-east-2:901920570463:layer:aws-otel-python-amd64-ver-1-20-0:3. The error you see is likely for using x86_64 instead of amd64.

@vasireddy99 Thanks for your reply. I have already described this above:

If instead I change the architectures from x86_64 to arm64, run sam build again, then sam deploy will succeed. So this seems to be specific to the x86_64 architecture.

I don't think it is an error to use x86_64 architecture for Python. I think the error is that this project has forgotten to provide the x86_64 version of the Lambda Layer, in addition to the arm64 version of the layer. Please let me know if you disagree.

@vasireddy99 Thanks for your reply. I have already described this above:

If instead I change the architectures from x86_64 to arm64, run sam build again, then sam deploy will succeed. So this seems to be specific to the x86_64 architecture.

I don't think it is an error to use x86_64 architecture for Python. I think the error is that this project has forgotten to provide the x86_64 version of the Lambda Layer, in addition to the arm64 version of the layer. Please let me know if you disagree.

arn:aws:lambda:us-east-2:901920570463:layer:aws-otel-python-x86_64-ver-1-20-0:3 is not the correct layer,
arn:aws:lambda:us-east-2:901920570463:layer:aws-otel-python-amd64-ver-1-20-0:3 is the correct layer

I misread your comment. You said amd64. My apologies.
To the best of my understanding, amd64 is not a standard architecture name for AWS Lambda, whereas x86_64 is. Here's another reference, from the SAM documentation.
I think it would be kind to users to provide an x86_64 layer since that matches the Lambda nomenclature.

I was following these instructions and, from the AWS docs, I was aware of x86_64 as an architecture value. It was not clear when the docs said

I agree with the point, modified the verbiage, that hopes to clear confusion.


Reflecting boundary conditions

Partially fixes https://github.com/parthenon-hpc-lab/athenapk/issues/56.
Only X3-boundaries are implemented. The boundary functions must be manually enrolled by the problem generator.

For tests, are you thinking of a simulation that would run and then we would check the solution, or just a unit test to check that the ghost cells are filled correctly for a given example of interior cells?


fix fix BE compile failure on Mac

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Fault tolerance

Added safe navigators to secrets response from client fetch
Moved error set state to within component
Added a catch block to creating signing keys

This will resolve https://github.com/sanity-io/sanity-plugin-mux-input/issues/23


Change terminal notification to unique thread ids

Terminal notifications should now display the number of friends that have messaged the user, not the total number of messages. Fixes #129

Thanks for the PR @ethan-ou! I've merged it 👍. Really appreciate your work here

No problems!


DBZ-866 Updating docs of 'source' block for MySQL connector

@jpechane Small fall-out of reviewing https://github.com/debezium/debezium/pull/607.

@gunnarmorling Applied, thanks!


Setup django

Setup a basic django backend

Assign this to me

No this is a course at a university we want to do it alone, thank you.


docs(icon guidelines): add requirement text

Pull request checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been reviewed and added / updated if needed (for bug fixes / features)
[x] Build (yarn build) was run locally and any changes were pushed
[ ] Unit tests (yarn test) were run locally and passed
[ ] Visual Regression Tests (yarn visual-regression) were run locally and passed
[ ] Linting (npm lint) was run locally and passed

Pull request type

Please check the type of change your PR introduces:

[ ] Bug fix
[ ] Feature
[ ] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[x] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

GitHub Issue Number: N/A
What is the new behavior?

fix typos
add SVG requirement (no title attribute) text

Does this introduce a breaking change?

[ ] Yes
[x] No

Testing

Other information

Texts were checked by UX writing.


Add notification when user's stylesheet has styles broken by shadow DOM

See https://github.com/atom/atom/issues/4962

This is probably not an issue anymore


UI suggestion - dropdown search on keypress

It would be nice to have this behaviour:
I have a dropdown with 2 items: 'abc', 'xyz'. When I have dropdown toggled ($(.ui.dropdown).dropdown('is visible') = true), if I press 'a' key on keyboard, 'abc' gets hovered (if I next press 'enter', 'abc' will become dropdown's value). Correspondingly, if I press 'x', 'xyz' gets hovered.

It does this in 2.0 already

:+1: :+1: :+1:


Introduce API endpoints for SEV attestation

What this PR does / why we need it:
The PR introduces new API endpoints for SEV attestation. It is based on the initial design proposal. It allows to perform the necessary pre-launch attestation steps:

retrieving platform certificates for verification sev/fetchcertchain: ~the API call is routed to the node where the VM is scheduled and it is processed by virt-handler. The certificates are fetched from libvirt by node-labeller in the init-container context (thus ATM the support of certificates rotation is limited);~  UPD: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked;
setting session parameters sev/setupsession: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked;
querying launch measurement sev/querylaunchmeasurement: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked. The measurement is returned to the caller;
injecting launch secret sev/injectsecret: the call is routed to the virt-launcher pod where the corresponding libvirt API is invoked.

The attestation process can be run either manually by calling the corresponding endpoints (e.g. using virtctl) or in an automated way by leveraging an attestation server and a controller that will monitor the VMI state and perform the required actions. The E2E functional test in tests/launchsecurity/sev.go demonstrates the general flow.
Attestation can be requested via VMI spec yaml:
spec:
  startStrategy: "Paused"
  domain:
    firmware:
      bootloader:
        efi:
          secureBoot: false
    launchSecurity:
      sev:
        attestation: {}

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:

The implementation of the new virtctl commands mapped to the SEV endpoints is not in the scope of this PR and will be added later in a follow-up (same applies to the attestation controller: not in the scope ATM). UPD: https://github.com/kubevirt/kubevirt/pull/7595
The PR requires libvirt >= 8.0.0: https://github.com/kubevirt/kubevirt/pull/7277
There is a proposal to generalize the attestation flow in libvirt: https://listman.redhat.com/archives/libvir-list/2022-February/msg00857.html. Any thoughts on that are welcome.

Release note:

Experimantal support of SEV attestation via the new API endpoints

/retest

/retest

/retest-required

I left some minor notes but overall I'm still digesting this PR :) Thanks a lot

Thanks for the review, @iholder-redhat . I rebased the changes and addressed the comments.

Thanks for the review, @iholder-redhat.

No problem! :) I've reviewed again and left some comments. Thanks a lot for your effort!

I rebased the changes and addressed the comments. Some things in this PR could be done differently. So I am open to discuss other approaches if any doubts ;)

To which other approaches do you refer? :smiley: Is it revolving around node-labeller implementation? Maybe it's good to open your concerns to discussions :)

To which other approaches do you refer? smiley Is it revolving around node-labeller implementation? Maybe it's good to open your concerns to discussions :)

Yeah, sorry if my pev comment was not very specific. Correct, one of the doubdts I had was about using the node-labeller: https://github.com/kubevirt/kubevirt/pull/7197#discussion_r867469420

Overall looks good to me!
I just have a couple of questions/remarks:

Have you thought to unpause the VM with the secret injection? I totally understand if you want to keep separate the commands but it could avoid an additional interaction
Does the session string require special formatting? I have only see that you verify if it is empty or not.

Overall looks good to me!

Thanks for the review!

I just have a couple of questions/remarks:

Have you thought to unpause the VM with the secret injection? I totally understand if you want to keep separate the commands but it could avoid an additional interaction

Yes, I thought about it. In my opinion for an API it is better to follow the 'single responsibility' principle as it gives more flexibility and simplifies the implementation. The additional interaction can always be hidden from the end-user at a different level. E.g. in virtctl sev inject-secret <VMI> --unpause.

Does the session string require special formatting? I have only see that you verify if it is empty or not.

The session itself is a binary blob. The string is just base64 representation. I was not sure how to properly validate it.

Updated the PR. The certificates are now fetched in virt-launcher context. Node labeller is not used anymore. The modification is added as a separate commit.

@vasiliy-ul Thanks very much for you hard work and patience :) Great stuff!
/lgtm
/reset-required

sorry, typo.. :)
/retest-required

/lgtm

I didn't wanted to steal you assign :)

Rebased the PR. Resolved conflicts with the generated files. No functional changes.

/retest

Minor comments after another look but it looks great /lgtm

Thanks. All addressed.

Thanks a lot @vasiliy-ul! Great work! A few minor comments below

Thanks for the review. Comments addressed.

/retest

Thanks a lot @vasiliy-ul! Great job!
/lgtm

/retest

/test pull-kubevirt-e2e-k8s-1.23-sig-storage-nonroot

Rebased and fixed the conflicts. No functional changes.

one more rebase...

/lgtm

Hi @vasiliy-ul , great to see this effort of integrating the SEV into Kubevirt. I feel obligated to ask if there is a slight possibility to get this tested somehow. I think our CI is a great advantage of Kubevirt. Even periodic or release jobs on an internal node or external cluster would be sufficient IMHO.

Hi @xpivarc, this needs SEV-capable hardware for testing. AFAIK in KubeVirt CI there are no AMD machines, right? If so, then I am afraid testing becomes problematic.

Hi @xpivarc, this needs SEV-capable hardware for testing. AFAIK in KubeVirt CI there are no AMD machines, right? If so, then I am afraid e2e testing becomes problematic. What would you suggest?

You are correct. I was hoping we could add the needed hardware to our CI(similarly to how we have added SRIOV capable hardware to our cluster or how we got the ARM cluster). My point is that most approvers/maintainers don't have this hardware available and therefore it is very hard to verify and ensure this feature will be working. Do you think this would be feasible?

Valid point. This is why the feature is now stated as experimantal. Do you mean connecting the hardware from our test lab to KubeVirt CI? Well, the hardware I use for testing is available only internally from within our infra. Unfortunately, it is not accessible from the outside.

@vasiliy-ul do you think we might find a way to mock the libvirt golang calls? It isn't the best option for testing it but at least we validate the new KubeVirt API calls and flow

@alicefr, I don't have a solution on top of my hand right now, but I think yes, it's possible. Though, with mocked libvirt it won't be e2e testing anymore. I am personally not sure if it's worth the effort since the 'mocked' flow is supposed to be covered by the unit-tests.

@vasiliy-ul yes you are right :/ emulation of course also doesn't work. How do you run the tests, on kind? AFAIK, we cannot use nested, so this is an additional problem for kubevirtci

Yeah, there is no SEV emulation. Right, it also does not work nested :( I just run the tests on a physical AMD machine. Not using kind though, but other k8s distros (k3s and rke2 mostly).

Valid point. This is why the feature is now stated as experimantal. Do you mean connecting the hardware from our test lab to KubeVirt CI? Well, the hardware I use for testing is available only internally from within our infra. Unfortunately, it is not accessible from the outside.

Maybe we can come up with a way of donating such a machine. One way would be sponsoring a bare-metal machine in IBM cloud. @fabiand FYI.
Maybe some the bare-metal machines are already AMD, or maybe we just have to rent a right one and we can use it with kind.
Let's see if we  can come up with a plan. I would prefer to have at least a rough discussion/idea before talking the PR in.

There seem to exist AMD offerings: https://cloud.ibm.com/gen1/infrastructure/provision/bm?packageId=2802&itemId=15824

@cwilkers @fabiand is there a way to donate resources or budget through CNCF for a project to have transparent donation reporting?

@cwilkers @fabiand is there a way to donate resources or budget through CNCF for a project to have transparent donation reporting?

I'm not sure, @aburdenthehand @jberkus  would you know?

The CNCF has a Community Infrastructure Lab that we can apply for. It's hosted by Equinix  and they have two types of AMD machines available (C3 medium and M3 large).
Would this be suitable?

We also need to consider that the host kernel requires the SEV support, so we need to check in which distro version is available

/remove-lifecycle stale

/remove-lifecycle stale

/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.23-sig-compute

Hey, just a gentle ping here. This PR is again ready for review. The new SEV test job is green:
https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/7197/pull-kubevirt-e2e-k8s-1.26-sev/1626499438073614336
What's changed:

the code has been rebased on the latest main
adapted the e2e tests to not use hardcoded hardware-specific SEV parameters
introduced SEV label to filter the tests properly
fixed issues with unit-tests on aarch64

The new code has been added in separate commits on top of the existing ones (to simplify the review process). If/When have some time, please take a look. Thanks!

/test pull-kubevirt-e2e-k8s-1.26-sev
/test pull-kubevirt-e2e-k8s-1.26-sig-storage
/test pull-kubevirt-e2e-arm64

/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.26-sev

Hey, @xpivarc, @rmohr, @alicefr, @iholder101, a gentle ping here :)
Firstly, thank you all for the previous reviews and comments. Now the topic about testing has been addressed. We currently have pull-kubevirt-e2e-k8s-1.26-sev job to test PRs on AMD hardware with SEV. There is also a periodic periodic-kubevirt-e2e-k8s-1.26-sev. This PR is currently green: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/7197/pull-kubevirt-e2e-k8s-1.26-sev/1639203141775790080. How can we move this forward? Could you have one more look here? Let's discuss if there are other concerns.

hi @vasiliy-ul, I'll give another pass to the PRs. Will be the CI job mandatory or optional?

hi @vasiliy-ul, I'll give another pass to the PR. Will be the CI job mandatory or optional?

Hi @alicefr, the presubmit job is not set as mandatory as of now. We have only one AMD node, which is not yet configured for parallel runs. Potentially we can set up kind there, but we also need to consider that there is a hardware limitation on how many SEV VMs can be run at the same time (and ensure that this does not end up in a bottleneck for the CI). Smth to work on in a follow-up I think.

hi @vasiliy-ul, I'll give another pass to the PR. Will be the CI job mandatory or optional?

Hi @alicefr, the presubmit job is not set as mandatory as of now. We have only one AMD node, which is not yet configured for parallel runs. Potentially we can set up kind there, but we also need to consider that there is a hardware limitation on how many SEV VMs can be run at the same time (and ensure that this does not end up in a bottleneck for the CI). Smth to work on in a follow-up I think.

Yeah, that's a good point. IMO, we should also only run the SEV tests on that machine and avoid that other test VMs are scheduled there. We'll probably need more fine-grained scheduling for the tests.

we should also only run the SEV tests on that machine and avoid that other test VMs are scheduled there

I think it is already done like that. Hm... Maybe @brianmcarey can confirm?

we should also only run the SEV tests on that machine and avoid that other test VMs are scheduled there

I think it is already done like that. Hm... Maybe @brianmcarey can confirm?

Yes currently only the SEV tests lanes are run against the SEV node - as optional lanes as you said.

Rebased the changes.
/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-code-lint

/test pull-kubevirt-code-lint

The lint lane is currently broken - there is a fix PR in the queue though -https://github.com/kubevirt/kubevirt/pull/9515

Hey, @xpivarc, @rmohr, @alicefr, @iholder101, a gentle ping here :)
Firstly, thank you all for the previous reviews and comments. Now the topic about testing has been addressed. We currently have pull-kubevirt-e2e-k8s-1.26-sev job to test PRs on AMD hardware with SEV. There is also a periodic periodic-kubevirt-e2e-k8s-1.26-sev. This PR is currently green: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/7197/pull-kubevirt-e2e-k8s-1.26-sev/1639203141775790080. How can we move this forward? Could you have one more look here? Let's discuss if there are other concerns.

I moved this to my next week's schedule. Glad to see we moved with the testing.

/test pull-kubevirt-e2e-k8s-1.26-sev

A gentle ping here. This PR would still benefit from some reviews :)
/test pull-kubevirt-e2e-k8s-1.26-sev

/lgtm
@vasiliy-ul there is a unit test failing in the ci

  ** pull-kubevirt-unit-test ** — Job succeeded.

@alicefr, from what I can see, the unit tests job is green. Or do I miss smth? There is a failed storage e2e test, but I dont think its related to this PR.

mmh I see the pull-kubevirt-goveralls failed

ah... I did not look at that job as it's not mandatory. Let's re-run them both. Previously there were no such failure...
/test pull-kubevirt-goveralls
/test pull-kubevirt-unit-test

The failed test uses 'real' timeouts/sleeps
https://github.com/kubevirt/kubevirt/blob/08f1b5c57d0cef851f4ba8122d493d10a9b2caa7/pkg/virt-handler/retry_manager_test.go#L59-L63
Depending on the CI load, this may fail. Does not seem related to this PR.

/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.26-sig-compute

Hi @alicefr, @iholder101. Previously, you reviewed the code here. Could you please re-lgtm/approve the PR (as you see fit)? I now rebased it and fixed all the conflicts (mostly with the generated code). There are no functional changes. The only thing is that I've moved the addition of sevctl tool to the test image into a separate PR (which is already merged https://github.com/kubevirt/kubevirt/pull/9892). Thanks!
/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.26-sev

/test pull-kubevirt-e2e-k8s-1.26-sig-storage

Hi @alicefr, @iholder101. Previously, you reviewed the code here. Could you please re-lgtm/approve the PR (as you see fit)? I now rebased it and fixed all the conflicts (mostly with the generated code). There are no functional changes. The only thing is that I've moved the addition of sevctl tool to the test image into a separate PR (which is already merged #9892). Thanks!

@alicefr, @iholder101, a gentle ping :) Would appreciate it if you could re-lgtm or approve the PR. Thanks!

/approve
/hold
Let's give some time to @iholder101 if he wants to have another look, but to me, it looks good
@vasiliy-ul great work and thanks for the perseverance!

Thanks a lot @alicefr for the review and valuable input!

Thank you for the review, @iholder101!
/test pull-kubevirt-unit-test

Looks like this PR broke https://github.com/openshift/openshift-restclient-python/blob/master/openshift/dynamic/discovery.py#L133, which expects VMInstancesSEVFetchCertChain/VMInstancesSEVQueryLaunchMeasurement/VMInstancesSEVSetupSession/VMInstancesSEVInjectLaunchSecret to be of the format '/'
cc: @myakove


Not working with youtube-dl command

Hello
I am trying to execute
youtube-dl --ignore-errors --yes-playlist  --extract-audio --audio-format mp3 --audio-quality 0 http://www.youtube.com/playlist?someurl command in oh-my-zsh
but it says
zsh: no matches found: http://www.youtube.com/playlist?someurl
BUT when i switch back to /bin/bash it works flawlessly

This is something to do with how zsh handles the input to itself. In the zsh shell you use the command as:
youtube-dl --ignore-errors --yes-playlist --extract-audio --audio-format mp3 --audio-quality 0 "http://www.youtube.com/playlist?someurl"
Use double quotes for the url

@GreenJoey So should i file an issue to zsh itself, or it's a feature not a bug ?

@ArtemBernatskyy alternatively you can simply mask the "?" with a "".

@ArtemBenatskyy There is no bug here, you have failed to understand how zsh pattern matching works. Please close.

@ArtemBernatskyy There is no bug here, you have failed to understand how zsh pattern matching works. Please close.

@ohjames but this wrong matching pattern causes user displeasure, doesn't it ?

@ArtemBernatskyy it's not wrong pattern matching at all, though you might be displeased by it.
Anyhow, as @ohjames already pointed out, there is no bug and the least a bug in oh-my-zsh.

Yes, mistakes cause you grief, so learn from them. ? like * is a global operator and must be escaped. Globbing operators are not blogs. Now close this.

ok, closing this for now, but i thought that open source were made for better products not for explaining why some bugs could be features )

@ArtemBernatsky You should realise that when your criticism comes from a lack of understanding that your concerns are invalid, you should see this as a chance to educate yourself. By levying out criticism without taking the time to understand you just come across as ignorant.
echo b* echoes all of the files beginning with a b. This is a really useful globbing pattern. When you do echo *.b?? it matches all files with an extension beginning with a b. This is useful for many.
So you can't have things both ways, both have access to globbing operators and not need to escape those globbing operators.

Omg @ohjames , i am just trying to tell that in ordinary terminal that command works, in zsh it doesn't, is it bug or feature this decision i will leave to the creators

can we add a line to the .zshrc to make it play nice with youtube-dl ?

alias youtube-dl noglob youtube-dl

awesome thanks

If anyone ends up here w/a similar problem with github issues because of the # command:
eg:
$ git co maintain_aspect_ratio_#30
> zsh: no matches found: maintain_aspect_ratio_#30

update ~/.zshrc with:
alias git="noglob git"


[SSCP][llvm-to-spirv] Do not use cmake INSTALL_DIR to install llvm-spirv translator

Hopefully fixes #1169

Can confirm that it fixes #1169.

My bad, should have tested one step further.
Now it requires root access during build, which is better than during configuration stage, but still not ideal:
$ cmake -S ~/AdaptiveCpp/ -B build
[...]
-- Configuring done (8.8s)
-- Generating done (0.0s)
-- Build files have been written to: /tmp/build
$ cmake --build build  -- -j12
[...]
[ 95%] Built target LLVMSPIRVLib
[100%] Linking CXX executable llvm-spirv
[100%] Built target llvm-spirv
Install the project...
-- Install configuration: ""
CMake Error at cmake_install.cmake:54 (file):
  file cannot create directory:
  /usr/local/lib/hipSYCL/ext/llvm-spirv/include/LLVMSPIRVLib.  Maybe need
  administrative privileges.

It also rebuilds the libSPIRV every time the build command is rung, even if the code and the cache are completely unchanged.

It also rebuilds the libSPIRV every time the build command is rung, even if the code and the cache are completely unchanged (this behavior is not specific to this PR).

This is because we need to patch llvm-spirv translator and cmake has no mechanism to figure out whether the patch is already applied, so it has to rebuild. It's massively annoying. Unfortunately it seems things get bleak when you want to include an external git dependency that needs patching with cmake.

Now it requires root access during build, which is better than during configuration stage, but still not ideal:

Right. I'm not sure how to fix that, short of just manually copying llvm-spirv to the destination directory during install (which might be brittle if llvm-spirv uses rpath, or changes their libraries etc).

Yaaay, packaging C++ dependencies is so fun. Well, I guess it can't be helped then. This MR already improves things: at least we don't need to run CMake as root, and we don't misuse INSTALL_DIR in ExternalProject_Add, so perhaps that's enough.
One solution would be to allow using an external LLVMSpirvTranslator installation. This would also play nicer with package managers, which might wish to handle dependencies themselves. Probably not terribly complicated to do, but a pain to test.

How would this work, given that we need a patched llvm-spirv translator?

Ah, right, the indexing of address spaces needs to be harmonized with the rest.

I'm not sure how to fix that, short of just manually copying llvm-spirv to the destination directory during install (which might be brittle if llvm-spirv uses rpath, or changes their libraries etc).

I guess the "uses rpath" issues might be solved by manually setting INSTALL_RPATH, and "changes their libraries", IMO, is less likely than some internal reorganization that breaks the patching process. Still perhaps too much effort for a relatively niche use-case.

I guess the "uses rpath" issues might be solved by manually setting INSTALL_RPATH, and "changes their libraries", IMO, is less likely than some internal reorganization that breaks the patching process. Still perhaps too much effort for a relatively niche use-case.

I do wonder whether there is a way to tell cmake to run the install target of external projects within our install target, not within the build target. Since we don't need llvm-spirv during the build itself, that would solve our problem..

Ok, I think I have found (and pushed to this branch) a solution that addresses both problems:

make && sudo make install pattern works as expected
llvm-spirv is not built for every single invocation of make.

The way it works feels a bit hacky, and I'd appreciate if someone with more cmake experience could have a look at it and tell whether this is an acceptable approach or totally UB:

Use FetchContent instead of ExternalProject to fetch the sources
Only if content is not yet populated apply the patch and run the cmake configuration. This happens with two execute_process calls (is there a better solution?).
Then use a custom build target to run cmake --build --config Release for LLVM-SPIRV translator if the project is built
Use a custom install() invocation to invoke another execute_process() to get get cmake --install to run when the project is installed.

From what I can tell:

Patch is applied correctly
llvm-spirv is rebuilt whenever the cmake configuration changes, but not for every single make
make && sudo make install can be used to install to privileged locations.

I guess the best course of action might be to merge and see if any issues pop up.


Error: Minified React error

Something went wrong in Ignition!
An error occurred in Ignition's UI. Please open an issue on the Ignition GitHub repo and make sure to include any errors or warnings in the developer console.

Hi @azerioid, can you please provide some more information (instead of opening issues with no context at all). Thanks!


Add: implement_strstr

Codecov Report

Merging #14 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #14   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          10     11    +1
  Lines         122    124    +2
=====================================
+ Hits          122    124    +2

Impacted Files
Coverage Δ

solution/implement-strstr/implement_strstr.go
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 472b9c2...57dec75. Read the comment docs.


[CosmosDB] Bump service api version

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.CosmosDB

replaced by #30573


Change the sidenav breakpoints to use Medium instead

Bug/issue #, if applicable: 86500277
Summary
Changes the sidebar breakpoints to use Medium as the point at which we go into a mobile view.
Dependencies
NA
Testing
Steps:

Point to a doccarchive with the index generated
View the page on desktop
Assert the sidebar is resizable and fully visible
Start resizing down to a tablet view, around 1000px it should follow the nav and move into a collapsed state.
Assert you can now toggle it open/closed
Keep it open and resize back into a larger screen.
Assert it automatically closes it without an animation

Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.

[x] Added tests
[x] Ran npm test, and it succeeded
[x] Updated documentation if necessary

@hqhhuang that should be fixed in - https://github.com/apple/swift-docc-render/pull/58

@swift-ci test


Added this-month-rocks

This is a website that I've made to inspire folks to do something fun this month. Made during the covid-19 pandemic for folks who are at home and may appreciate some inspiration. The website is fully collaborative, folks can add ideas via github.

Also can you rebase with master so the new linting runs

This should be it :)

Thanks @koaning!


With names

Update list of names

Hi @thefiend , I really apologize for this. I tried using your repo not knowing exactly how fork works. This PR should have been on my own repo and not on this one.


Fix #6495 : Added support to list-groups in Collapse class

Fix #6495

moved to https://github.com/yiisoft/yii2-bootstrap/pull/30


Changed lightGray, lightOrange, orange, and darkOrange colors.

After reviewing with the design team we thought it would be best to change/update a few of the colors in the design system. The first being lightGray — we wanted something lighter with more of a blue tint. The entire orange palette has also been updated slightly to allow for more legibility.

Codecov Report

Merging #169 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #169   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          31     31
  Lines         357    357
=====================================
  Hits          357    357

Impacted Files
Coverage Δ

src/theme.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0dc4de8...509f3dc. Read the comment docs.


Failing CI due to breaking change in pytest

In recent builds from https://github.com/alan-turing-institute/dodo/pull/79 and https://github.com/alan-turing-institute/dodo/pull/78 a failure has occurred on Travis which prints:
Traceback (most recent call last):
  File "/home/travis/miniconda/envs/p36/bin/pytest", line 10, in <module>
    sys.exit(main())
  File "/home/travis/miniconda/envs/p36/lib/python3.6/site-packages/_pytest/config/__init__.py", line 61, in main
    config = _prepareconfig(args, plugins)
  File "/home/travis/miniconda/envs/p36/lib/python3.6/site-packages/_pytest/config/__init__.py", line 182, in _prepareconfig
    config = get_config()
  File "/home/travis/miniconda/envs/p36/lib/python3.6/site-packages/_pytest/config/__init__.py", line 156, in get_config
    pluginmanager.import_plugin(spec)
  File "/home/travis/miniconda/envs/p36/lib/python3.6/site-packages/_pytest/config/__init__.py", line 530, in import_plugin
    __import__(importspec)
  File "/home/travis/miniconda/envs/p36/lib/python3.6/site-packages/_pytest/tmpdir.py", line 25, in <module>
    class TempPathFactory(object):
  File "/home/travis/miniconda/envs/p36/lib/python3.6/site-packages/_pytest/tmpdir.py", line 35, in TempPathFactory
    lambda p: Path(os.path.abspath(six.text_type(p)))
TypeError: attrib() got an unexpected keyword argument 'convert'

It appears that pytest seems to have the package attrs as a dependency.
REF: https://stackoverflow.com/questions/58189683/typeerror-attrib-got-an-unexpected-keyword-argument-convert
NOTE This is the same error that has been outlined in https://github.com/alan-turing-institute/dodo/issues/80

Looks like this is fixed now, and we've upgraded pytest to 5.x 😄


Add support for promhttp

InstrumentHandlerResponseSize func(obs prometheus.ObserverVec, next http.Handler) http.Handler from the promhttp package takes in "http.Handler" and returns a "http.Handler" for instrumenting the response size. However, martini only has martini.Handler which makes it impossible to pass into the promhttp instrumentation function, I'm wondering if it's possible to use martini with prometheus promhttp?

Manage to make the Go programme compile, but when it's done like this:
m := martini.New()
m.Use(promhttp.InstrumentHandlerInFlight(metrics.RequestGauge, m))

The whole programme just gets stuck

I was recently looking at adding opencensus instrumentation to orchestrator which uses Martini too. It suggests doing it passing in a custom http.Handler too here
Though I'm looking at writing my own middleware for it instead.


Impossible to use dynamodb stream trigger with exported stream ARN and configValidationMode="error"

Description
Hello !
I am using lift 1.17.0 and serverless 3.17.0.
I used the single table construct to deploy a dynamodb table and I am now trying to trigger a lambda function with the dynamodb stream of the table.
The event key for my function :
events: [
    {
      stream: {
        type: 'dynamodb',
        arn: '${construct:myTable.tableStreamArn}',
      },
    },
  ],
};

In my serverless configuration, I have configValidationMode: 'error',
I have the following error when deploying :
Error: Configuration error at 'functions.dispatchEvents.events.0.stream': unsupported string format
It works well when I use configValidationMode: 'warn', the lambda and the trigger are well deployed and working
How to Reproduce
service: my-app
configValidationMode: error
provider:
    name: aws

plugins:
    - serverless-lift

functions:
    myFunc:
        handler: './handler.main'
        events:
            - stream:
                type: dynamodb
                arn: ${construct:myTable.tableStreamArn}

constructs:
    myTable:
        type: database/dynamodb-single-table

Additional Information
No response

Thanks for reporting this issue @LouisPinsard :)
Digging around a little bit with your exemple serverless.yml, I've found out that at the time of Serverless Framework validation, the service definition input - or at least solely the myFunc.events[0].stream - passed into ajv validator has the following value { type: 'dynamodb', arn: '${Token[TOKEN.194]}' }
This indeed does not validate against the framework JSON Schema definition as it's neither a CloudFormation intrinsic function, neither a string starting with arn:. The ${Token...} string is a CDK placeholder, that will be resolved at later stage.
One of those stage is the variable resolver called on ${construct:...} occurrences in the service file definition. However, the current observation would led to believe variable resolution actually happens after service file validation, which I find really strange. It is also strange that similar issues were not submitted on Lift earlier to emphasize this problem, I'm guessing the arn property from stream event type has a much stronger typing than other places where ${construct:...} variable resolver is usually used.
This might be worth asking around on https://github.com/serverless/serverless side to check if we can have their input on this issue. Would you care opening a linked issue on their side referencing this one ?
In the meantime, I implemented a small PR showcasing this issue in Lift unit tests suite.

Serverless says it's Lift :shrug: Sounds like we're in a catch-22.

Resurrecting this one, is there anything I can do as an investigation to help support moving this on?
With serverless 4, the default is going to be set to "error" for config issues so lift will fail immediately with the documented approach.


Storing learning related parameters for checkpoints

For any pre-trained network, if you don't know the latest learning rate used then it is pain to fine-tune it with a good initial learning rate. therefore, I believe it is good to keep learning parameters with the checkpoint models for the continuation of training later.

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


Tainted Kernel

Describe the bug
Following a rebase from Fedora Kinoite 40 to Bazzite (KDE). Info Center claims the Linux Kernel is tainted.

What did you expect to happen?
Expected the Linux Kernel to remain untainted during the rebase.
Output of rpm-ostree status
State: idle
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bazzite:stable
                   Digest: sha256:d9780944c8e32563427bdf239fbcda168aafc9f9b2b538f2e02f5326485b36bc
                  Version: 40.20240605.0 (2024-06-05T16:37:35Z)

  ostree-unverified-registry:ghcr.io/ublue-os/bazzite:stable
                   Digest: sha256:d9780944c8e32563427bdf239fbcda168aafc9f9b2b538f2e02f5326485b36bc
                  Version: 40.20240605.0 (2024-06-05T16:37:35Z)

Hardware
Operating System: Bazzite 40
KDE Plasma Version: 6.0.5
KDE Frameworks Version: 6.2.0
Qt Version: 6.7.1
Kernel Version: 6.8.11-301.fsync.fc40.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 24 × AMD Ryzen 9 3900X 12-Core Processor
Memory: 62.7 GiB of RAM
Graphics Processor: AMD Radeon RX 6900 XT
Manufacturer: ASUS
Asus Crosshair Hero VIII Wifi
Installed Main Drive: Samsung SSD 970 EVO Plus 2TB NVME (no SMART errors)
BIOS: 4805
Secure Boot: On (despite what Info Center claims)
Extra information or context
No response

Expected, aftermarket kernel with akmods.

I can't find anything on the GitHub page or the COPR entry for the fsync kernel that this is expected. Could someone point me to some documentation that this is to be expected. Not saying I don't believe you I just like to understand why things work and behave the way they do.


(ZH) Link to PodOverhead configuration page fix.

Issue available at https://kubernetes.io/zh/docs/concepts/containers/runtime-class/
and introduced by original English documentation (see #17543)

/lgtm
/approve


set LSST_COMPILER for newinstall.sh

This is nessicary when using a compiler string that does not match the
default value hard coded in newinstall.sh.

LGTM


chore: remove kintsugi aura migration

Signed-off-by: Gregory Hill<EMAIL_ADDRESS>

Approved but there are merge conflicts


Using \wsl$ causes "init" CPU load

Your Windows build number:  Windows 10 Version 1909 (OS Build 18363.476)

What you're doing and what's happening:
Accessing WSL via \\wsl$ causes the init process to consume CPU visibly in Task Manager/top. In case of very frequent accesses (i.e. CLion) this will cause the process to consume a lot of CPU. The access are also quite slow it seems.

What's wrong / what should be happening instead:
I would not expect it to consume so much CPU.

There's also an open issue with CLion I created - https://youtrack.jetbrains.com/issue/CPP-17597
The issue occurs on two completely different machines and was already present in 1903. I am using WSL 1 with Kali.

Can you provide any Windows Event logs or strace logs in WSL? The instructions are provided here.

\\wsl$\ works by running a file server in the init process.

I see. But it seems that it is rather CPU heavy and slow - at least in the way CLion uses it. I will try to get some more information and data on this soon.

I see the pretty same slowness if I instruct my IDE to communicate with WSL over ssh. And it's again the init process taking up all CPU time.


Cannot create async background tasks

I'm not quite sure why this isn't working but I have code similar to this
@api.route("/rpc/work/{task_token}")
async def do_work(req, resp, *, task_token):
    """…"""
    @api.background.task
    async def process_task(task_token):
        await async_slow_function(task_token)
        something_else()

    process_task(task_token)
    resp.text = f"{task_token}"

In this case it seems like the functions in process_task never get executed or if they do they error silently, I even tried to print stuff inside at the top of async_slow_function but no output.
What I want to achieve is to launch a slow async worker in the background to do some heavy lifting while telling the requester that the task have been started. Background-task at the surface looks precisely like what I'm looking for but I'm rather new to asyncio so I might be missing something obvious. I tried searching issues here on github but didn't find anything mentioning async background tasks.

My current workaround looks like this
async def process_task(task_token):
    await async_slow_function(task_token)
    something_else()

@api.route("/rpc/work/{task_token}")
async def do_work(req, resp, *, task_token):
    """…"""

    loop = asyncio.get_event_loop()
    loop.create_task(process_task(task_token))

    resp.text = f"{task_token}"

It doesn't look as good/explicit as with the background task decorator but it seems to get the job done for now.

You need to await process_task(args)


neovim: unit tests for the lua functions

it would be nice to have unit tests for testing the lua functions like the ranges, the selection, etc (cursorless\cursorless.nvim\lua\cursorless)

done in https://github.com/saidelike/cursorless/pull/7


UXPROD-4304 update _index.md

Added and updated Inventory documentation regarding deleting instance records and staff suppression.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


cpu-stress Chaos Experiment

What This PR Does
Adds the cpu-stress chaos experiment

The experiment leverages the stress-ng tool for injecting stress chaos
The experiment accepts the number of stressors and CPU load as parameters for injecting chaos

Which Issue This PR Fixes
fixes #5
Checklist:

[x] Fixes #
[ ] PR messages has document related information
[ ] Labelled this PR & related issue with breaking-changes tag
[ ] PR messages has breaking changes related information
[ ] Labelled this PR & related issue with requires-upgrade tag
[ ] PR messages has upgrade related information
[ ] Commit has unit tests
[ ] Commit has integration tests
[x] E2E run Required for the changes

Build is failing, can you fix the issue with the ci-lint.

Build is failing, can you fix the issue with the ci-lint.

Updated.


Baba is Update

fixed level IDs and swapped names, idk how to do the thing that makes it work

No longer using ids


segfault from *bam with many contigs

Hi,
I get a segfault using freebayes on *bams that contain a large number of contigs (> 250k contigs).  I am interested in getting calls on only the ones that are of a certain size, but I can't seem to get it to run without crashing.  Any help would be greatly appreciated.  I included a small.fa and tmp.bam that produces this issue.
Thanks!
issue.zip

I'll check this out.
On Thu, Sep 15, 2016, 22:50 abishara<EMAIL_ADDRESS>wrote:

Hi,
I get a segfault using freebayes on *bams that contain a large number of
contigs (> 250k contigs). I am interested in getting calls on only the ones
that are of a certain size, but I can't seem to get it to run without
crashing. Any help would be greatly appreciated. I included a small.fa and
tmp.bam that produces this issue.
Thanks!
issue.zip https://github.com/ekg/freebayes/files/475548/issue.zip
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ekg/freebayes/issues/321, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAI4EVS03hTdezxK94A8wQ6KHHy3mg61ks5qqa-WgaJpZM4J-Vrf
.

I can confirm this. 200k contigs still works fine, 250k leads to segfault.
I guess one workaround for the moment is to concatenate those contigs into a pseudo scaffold and redo the mapping. Another one could be to reduce the number of "@SQ" BAM header lines to only those contigs that got reads mapped to.
I would appreciate the removal of this limitation, since some of my post-mapping-pre-variant-estimation filters are easier to implement with separate contigs as compared to a pseudo scaffold.
claudius

I understand you cannot run one job for every e.g. 1000 contigs?
On Sat, Nov 5, 2016, 09:22 Claudius Kerth<EMAIL_ADDRESS>wrote:

I can confirm this. 200k contigs still works fine, 250k leads to segfault.
I guess one workaround for the moment is to concatenate those contigs into
a pseudo scaffold and redo the mapping. Another one could be to reduce the
number of "@SQ https://github.com/SQ" BAM header lines to only those
contigs that got reads mapped to.
I would appreciate the removal of this limitation, since some of my
post-mapping-pre-variant-estimation filters are easier to implement with
separate contigs as compared to a pseudo scaffold.
claudius
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ekg/freebayes/issues/321#issuecomment-258598123, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAI4EVZsjdaZ6fDFFYDGx-z0TF3Fx6YGks5q7DzWgaJpZM4J-Vrf
.

Hi Eric,
sorry, but I can't figure out what you mean. The read mapping has to be be done against the whole reference, of course, resulting in BAM files with headers that contain all contigs of the reference. Limiting freebayes to a target of only 1000 contigs through a BED file leads to the same segfault:

freebayes -d -f Big_Data_ref.fa -t small.bed -T 0.01 -F 0 -C 0 -G 5 --min-coverage 3 --max-coverage 25 -V --use-mapping-quality par_34-9.sorted.bam
loading fasta reference Big_Data_ref.fa
Opening BAM format alignment input file: par_34-9.sorted.bam ...
done
Number of ref seqs: 583312
Making BedReader object for target file: small.bed ...
done
Number of target regions: 1000
no sample list file given, reading sample names from bam file
found 1 samples in BAM file
Segmentation fault (core dumped)

This is with version:  v1.0.2-58-g054b257. The BAM that I used for this test contains alignment records for reads mapping to 96,603 contigs in total, i. e. well below 250k. To me it looks like it's the number of ref seqs in the BAM header that is causing freebayes to segfault.
claudius

Do you get the same segfault with v1.1?
On Sun, Nov 6, 2016, 22:35 Claudius Kerth<EMAIL_ADDRESS>wrote:

Hi Eric,
sorry, but I can't figure out what you mean. The read mapping has to be be
done against the whole reference, of course, resulting in BAM files with
headers that contain all contigs of the reference. Limiting freebayes to
a target of only 1000 contigs through a BED file leads to the same segfault:
freebayes -d -f Big_Data_ref.fa -t small.bed -T 0.01 -F 0 -C 0 -G 5
--min-coverage 3 --max-coverage 25 -V --use-mapping-quality
par_34-9.sorted.bam
loading fasta reference Big_Data_ref.fa
Opening BAM format alignment input file: par_34-9.sorted.bam ...
done
Number of ref seqs: 583312
Making BedReader object for target file: small.bed ...
done
Number of target regions: 1000
no sample list file given, reading sample names from bam file
found 1 samples in BAM file
Segmentation fault (core dumped)
This is with version: v1.0.2-58-g054b257. The BAM that I used for this
test contains alignment records for reads mapping to 96,603 contigs in
total, i. e. well below 250k. To me it looks like it's the number of ref
seqs in the BAM header that is causing freebayes to segfault.
claudius
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ekg/freebayes/issues/321#issuecomment-258712613, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAI4EYf5A5se_yRFSs3hWs4rDotM4gqFks5q7kgvgaJpZM4J-Vrf
.

Hi Eric,
the segfault happens with freebayes from commit 39e5e4bcb (3 Nov) as well as.
claudius

Can you isolate the contig where the failure occurs? Are there any funny
features of this contig? For instance, depth being very high. Did you set
use-best-n-alleles?
On Thu, Jul 19, 2018, 06:23 Harish<EMAIL_ADDRESS>wrote:

Hi!
Did any of you guys (@ekg https://github.com/ekg , @claudiuskerth
https://github.com/claudiuskerth , @abishara
https://github.com/abishara ) solve the issue or is there a fix for the
same in any recent version? I'm calling variants from hexaploid wheat which
has about 735945 scaffolds and I'm facing the same issue.
Sorry for the necropost.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ekg/freebayes/issues/321#issuecomment-406158446, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAI4EdHKARwnLmsy8x0q6ZLLu26SIZdaks5uIBfOgaJpZM4J-Vrf
.


fix: Store build

What's the purpose of this pull request?

How it works?

How to test it?

Starters Deploy Preview

References

Closing this PR because we have fixed an issue, and the changes should be addressed in another PR. ✅


[QUESTION] How to respond to WebSocket Ping message?

When a WebSocket Ping message (data frame) is sent to a FastAPI WebSocket server, how can I receive it and return a Pong message?
Sending Ping using aiohttp as below, it doesn't seem that FastAPI WebSocket connection receives anything.
Sending Ping
async with aiohttp.ClientSession() as sess:
    async with sess.ws_connect(WS_ENDPOINT) as ws:
        await ws.ping()

Receiving Ping
@app.websocket("/ws")
async def websocket_endpoint(ws: WebSocket):
    await ws.accept()
    data = await ws.receive()

imho the simplest is not to use FastAPI decorator but subclassing Starlette WebSocketEndpoint and sending your pong in the on_receive method

Ping events are defined in the WebSocket spec: https://tools.ietf.org/html/rfc6455#section-5.5.2 and are handled automatically by Uvicorn, not by FastAPI.

@tiangolo
Thank you for your response.
I found the code that handles a Ping message in Uvicorn.
https://github.com/encode/uvicorn/blob/master/uvicorn/protocols/websockets/wsproto_impl.py#L186-L187


Login API

Great work here. I am wondering if you'd be interested in incorporating the login API for authentication using a username and password? I'd be willing to help out if interested.

Hey @topherreynoso! Thanks so much for the contribution and sorry for the late response. I'll be reviewing the code and hope to get back to you soon.


Less hardcoded Scala versions in tests and some bumps

Moved some test Scala version definitions up (into build.sc when possible)
Removed some hardcoded scala versions
Avoid compilation with Scala 2.12.4 compile, which is known to have issues with newer zinc (fixes some already present warnings)

I need to leave a main comment to leave a comment in code. Sorry.

No problem, but I don't see this restriction with my browser.
Any idea what's wrong with the NodeJS tests on Java 11?

Looks like it's combination of Scala 2.12.5 and Scala.js 0.6.3{2,3} which doesn't work.

@lolgab This PR is a preparation for Zinc version bump.


Sass error (undefined variable "$item-md-body-text-font-size") after update

From @Screeze on May 13, 2017 12:7
Ionic version:  (check one with "x")
[x] 3.x
I'm submitting a ...  (check one with "x")
[x ] bug report
After updating ionic, I get following error when building:
[15:02:00]  sass: ...ng/xx/xx/node_modules/ionic-angular/components/label/label.md.scss, line: 24
            Undefined variable: "$item-md-body-text-font-size".

      L23:  [text-wrap] .label-md {
      L24:    font-size: $item-md-body-text-font-size;
      L25:    line-height: $item-md-body-text-line-height;

Error: Failed to render sass to css

I have found the variable to be defined in xx\node_modules\ionic-angular\components\item\item.md.scss but somehow it seems this file is not included in the build.
I first posted this in the forum (https://forum.ionicframework.com/t/sass-error-after-update/90087) and seems like I am not the only one with that problem.
edit:
This only happens when building for Android, not when using ionic serve!
edit2:
This does not happen when "--prod" flag is NOT added to the build. This seems only to occur with "--prod" enabled.
Ionic info: (run ionic info from a terminal/cmd prompt and paste output below):
>ionic info

global packages:

    @ionic/cli-utils : 1.0.0
    Cordova CLI      : 7.0.1
    Ionic CLI        : 3.0.0

local packages:

    @ionic/app-scripts              : 1.3.7
    @ionic/cli-plugin-cordova       : 1.0.0
    @ionic/cli-plugin-ionic-angular : 1.0.0
    Ionic Framework                 : ionic-angular 3.2.0

System:

    Node       : v7.9.0
    OS         : Windows 10
    Xcode      : not installed
    ios-deploy : not installed
    ios-sim    : not installed

Copied from original issue: driftyco/ionic#11648

From @AmitMY on May 13, 2017 23:58
Please try v3.2.1, as it fixes some AoT issues that exist in v3.2.0

Thanks for opening an issue with us, we will look into this.

From @Screeze on May 14, 2017 1:45
I just tried, but still the same result, now using v3.2.1

global packages:
@ionic/cli-utils : 1.0.0
Cordova CLI      : 7.0.1
Ionic CLI        : 3.0.0

local packages:
@ionic/app-scripts              : 1.3.7
@ionic/cli-plugin-cordova       : 1.0.0
@ionic/cli-plugin-ionic-angular : 1.0.0
Ionic Framework                 : ionic-angular 3.2.1

System:
Node       : v7.9.0
OS         : macOS Sierra
Xcode      : Xcode 8.3.2 Build version 8E2002
ios-deploy : 1.9.1
ios-sim    : 5.0.13

I got the same error updating to 3.2.1 only run the --prod in android

thank you @rogeliotrejo13 your reply was very useful to me.

With Ionic CLI version 3.4.0, also got this error.
I solved this error with @rogeliotrejo13 's reply temporary.
Is this is a bug in ionic build script with --prod option?


Could not pass event BlockBreakEvent to Quests v3.2.4-b261

(Required) What is your Quests version / build number (do not say "latest")?
Quests 3.2.4-b261
(Required) Please describe the problem and how to replicate it. Any console errors?
Hey Guys,
this error keeps appearing in my console. Can you tell me what this is about?
https://pastebin.com/mcmafrd8

please use hastebin or pastebin
Thanks :)

@zippoking I have updated your original post to use a Pastebin link instead.
One of you Break Block objectives is causing the errors. It'd be helpful if you could post your quests.yml file (on Pastebin).

Hi,
thanks for the quick reply and sorry for not using pastebin.
Here is the story to the error:
I just installed the plugin, tried the default quests and then created my own quest ingame.
Just a very simple one. It was about breaking sugarcane. Unfortunately sugarcane didn´t work (i think i used ID 338, which is not the sugarcane block, so my mistake).
However i edited the quest ingame, changed the name and the goal to break 4 wood blocks.
This worked for me as a very simple test. A second player joined the server and then the error kept appearing in my console.
Here is the requested quests.yml file: https://pastebin.com/VWWFeeYA
Hope you guys can help me =)
Thhanks in advance.
Best regards
Zippokino

So it seems that the issue is fixed for me.
I didn´t get the error messages again.

@zippoking Sorry for the late reply. Glad it seems to be resolved. I don't see anything inherently wrong with your quests.yml, but if this comes up again, just comment here and I'll take another look.


refactor(server): introduce authentication to server class

This enables authentication for the new SDAM layer with a relatively minor changeset!
NODE-1620

@daprahamian there are a few tests in pool_tests.js, but generally yes I'm depending on native to test this. It's my hope that in the future (especially once Topology just supports a connection string) that we move most of those tests from native into core, ensuring that we are covering the right code in the right project.