XIONE-7775: Handle containers that don't respond to SIGKILL during cleanup

Description
In some scenarios on Xi1, processes can end up in a state where they don't respond to SIGKILL (assumed uninterruptible sleep).
In this case, Dobby may get stuck in a crash loop when attempting to cleanup the container.
The following logic is introduced:

At Dobby start, check for leftover containers

If container is in Running state, attempt to kill with SIGKILL

If SIGKILL success - delete container
If SIGKILL fail - mark container as "stuck" and do not attempt to delete

Add the container to our internal list of running containers in the Unknown state to prevent the container from being restarted

If container in stopped state, attempt to delete

If delete fails, then mark container as "stuck"

If we have any containers marked as "stuck", regularly check if they are still stuck. Kill/delete and remove from our list if/when they eventually die.

Test Procedure
Simulate container in uninterruptible sleep as per reproduction steps here: https://github.com/containers/crun/issues/868
Stop and restart dobby daemon. Dobby should handle the issue as per logic outlined above.
Test on STB and ensure no regressions
Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Other (doesn't fit into the above categories - e.g. documentation updates)

Requires Bitbake Recipe changes?

[ ] The base Bitbake recipe (meta-rdk-ext/recipes-containers/dobby/dobby.bb) must be modified to support the changes in this PR (beyond updating SRC_REV)

Holding off merge until https://github.com/containers/crun/pull/869 is merged
With crun PR applied, Dobby will not hang during container destroy


Classifiers UI?

Determine if we need to support browsing by classifiers, and if so we need to implement that UI. We also need a page that lists all of the available classifiers.

I'm going to add a page that list the classifiers.

And I'm struggling with Pyramid. Going to run through some tutorials and read some more code. I'll let this go for now, but get it back when I've gotten through a Pyramid tutorial.

Closing this as the new UI deals with it.


Proper Example of customised UI in demo project

I was looking at the demo project and trying to find customised UI examples, but there is no proper example present, can you guys prepare it and add it? it would be really helpful to inexperienced users

Available now: https://github.com/chat-sdk/chat-sdk-android#customization


binding.eval() handles def expressions differently from CRuby

I've noticed that this particular wierd corner case behaves differently between MRuby (3.2.0) and CRuby (3.2.1 and 3.0..6).  In this example, we capture the binding from an instance_exec call on the main object:
MAIN = self
module S
  @@repl_session = MAIN.instance_exec{ binding() }
  def self.session_eval(text)
    return @@repl_session.eval(text)
  end
end

S.session_eval 'def self.bar() puts "bar!"; end'
S.session_eval 'def foo() puts "foo!"; end'

bar()
foo()   # fails on mruby

On CRuby, both foo and bar are defined.  However, on MRuby, only bar (defined with self.bar) is visible in the toplevel object.
Oddly enough, if I define @@repl_session in the toplevel:
...
REPL_SESSION = MAIN.instance_exec{ binding() }
module S
  def self.session_eval(text)
    return REPL_SESSION.eval(text)
  end
end
...

everything works as expected.  My suspicion is that the current binding isn't getting self correctly, but I could be wrong.

The issue has been fixed by #6091.
Thank you for the report.

Thanks for the fix.


Add support for network interface based member binding

Docker swarm initializes service name as host name depends
on health check being configured, during hazelcast binding
docker might not have necessarily initialized service name as host
name and hence hazelcat Member binding fails.
To safegaurd hazelcast initialization during application startup
bind hazelcast based on network interface name,
Following is sample code snippet used for testing:
MemberAddressProviderConfig memberAddressProviderConfig =
new MemberAddressProviderConfig();
memberAddressProviderConfig.setEnabled(true);
memberAddressProviderConfig
.setImplementation(new DockerNetworkInterfaceBasedMemberAddressProvider(
memberAddressProviderProperties));

Please add documentation for how to use this option in the README, thank you!

@SachinB2 can you do the requested changes? is this PR still needed?

no response from PR submitter


ESET Protect CEF ingested logs parser

Is your feature request related to a problem? Please describe.
Not a problem but potential improvement.
The current ESET Protect parser only checks for log that have been ingested via syslog. Is there a plan to create a CEF parser that would check in the CommonSecurityLog table?
Describe the solution you'd like
Creation of a new parser for CEF ingested logs by ESET Protect. Same applies for the 2 available Analytics Rules.
Describe alternatives you've considered
Existing flow setup is still a viable option.
Additional context
Related CEF formatting of logs forwarded by ESET Protect:
https://help.eset.com/protect_admin/10.0/en-US/events_exported_to_cef_format.html

Hi @Rapsoulis we are working on your request, will update you shortly, thanks!

Hi @Rapsoulis we have reached out to the concerned team with your feature request, will update you once we hear back from them, thanks!

Hi @Rapsoulis we are waiting to hear back from the concerned team, will update you once we hear back from them, thanks!

Hi @Rapsoulis we have updated the solution owners about this issue, will update you once we hear back from them, thanks!

Hi @v-vdixit looking forward for further updates. Thank you for looking into this topic.

Hi @Rapsoulis we are waiting for update from solution owners on this, thanks!

Hi @Rapsoulis we are still waiting for update from solution owners, thanks!

Hi @Rapsoulis we are waiting for update from solution owners on this, thanks!

Hi @Rapsoulis, waiting for update from concerned team.

Hi @Rapsoulis, waiting for update from concerned team.

Hi @Rapsoulis, we have informed the concerned team, we will get back to you once there is an update.

Hi @Rapsoulis, Thanks for your suggestion. We've informed the respective team for this enhancement also you can raise a ticket with support team - https://techcenter.eset.nl/en/ for further updates.


Tackle-92 Assessment Confidence

Issue : https://github.com/konveyor/tackle-pathfinder/issues/69
Features covered

New endpoint "confidence" that will answer with a List of assessmentIds + confidence based on the risks for the answers of the assessment
Only considers assessments Complete

Unit , Integration and End2End Tests cases covered

Unit Test ( AssessmentSvcTest.java )

given_ApplicationsAssessed_when_AdoptionCandidate_then_ResuletIsTheExpected

Integration Test ( AssessmentsResourceTest.java )

given_ApplicationsAssessed_When_Confidence_Then_ResultIsTheExpected

End2End Test on minikube with native image ( check_api.sh )

Tests 15

Pre steps Minikube:

Install minikube : https://minikube.sigs.k8s.io/docs/start/
Start minikube : minikube start --kubernetes-version=v1.20.2 --cpus 4 --memory 12000
Enable ingress addons : minikube addons enable ingress
Create tackle namespace

kubectl create namespace tackle

Deploy tackle ( to have the keycloak instance running ) on your K8s cluster

$ kubectl apply -f  https://raw.githubusercontent.com/konveyor/tackle/main/kubernetes/kubernetes-tackle.yaml -n tackle

Edit the Ingress object to allow direct request to Pathfinder API

$ kubectl edit ingress tackle -n tackle

Add this below paths:
     - backend:
          service:
            name: tackle-pathfinder
            port:
              number: 8080
        path: /pathfinder
        pathType: ImplementationSpecific

Build and push

$ ./mvnw -U -B package -Dquarkus.container-image.push=true -Dquarkus.container-image.group={your quay user} -Dquarkus.container-image.registry=quay.io -Dquarkus.container-image.username={your quay user} -Dquarkus.container-image.password={your quay pwd} -Pnative

Change the image in the deployment tackle-pathfinder to be the one you have pushed
Rollout to update the image

$ kubectl rollout restart deployment tackle-pathfinder -n tackle

Test case with Minikube : Do the full API test
$ .github/scripts/check_api.sh

Expect

+++++ API CHECK SUCCESSFUL ++++++

Pre steps non containerised local test:

run keycloak

podman run -it --name keycloak --rm \
            -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin -e KEYCLOAK_IMPORT=/tmp/keycloak/quarkus-realm.json \
            -e DB_VENDOR=h2 -p 8180:8080 -p 8543:8443 -v ./src/main/resources/keycloak:/tmp/keycloak:Z \
            jboss/keycloak:12.0.2

run postgresql

podman run -it \
            --name postgres-pathfinder -e POSTGRES_USER=pathfinder \
            -e POSTGRES_PASSWORD=pathfinder -e POSTGRES_DB=pathfinder_db \
            -p 5433:5432 postgres:10.6

run application

./mvnw quarkus:dev

Test case with local non containerised : Do the full API test
$ .github/scripts/check_api.sh localhost:8085 localhost:8180

Expect

+++++ API CHECK SUCCESSFUL ++++++

@jonathanvila The response of the endpoint POST /confidence should also contain the applicationId rather than just the fields: assessmentId and confidence.
In order to create the graph https://docs.google.com/presentation/d/1RFDsQLfyuFd4vMqjlq4Q1tRQi8V-PC3jvMr7jLnTnJE/edit#slide=id.ga9b8077a6e_0_172 it is required to know which confidence was assigned to a certain application.
Let me give you this example:
Let's say we have 2 applications:
[
  {id: 1, name: "app1"},
  {id: 2, name: app2}
]

And then we send a request POST /confidence with the body:
[
  {applicationId: 1},
  {applicationId: 2},
]

The response of the previous request is:
[
  {"assessmentId": 99, confidence: 3},
  {"assessmentId": 100, confidence: 3},
]

Now the question is how can I know which assessment belongs to which application?
The final result is that I have 2 objects:
[
  {id: 1, name: "app1"},
  {id: 2, name: app2}
]

and
[
  {"assessmentId": 99, confidence: 3},
  {"assessmentId": 100, confidence: 3},
]

with no way to create a match between the previous 2 objects.
Could you please add the applicationId to the response of the endpoint?

:+1:


localns arguments are not longer accepted.

Initial Checks

[X] I confirm that I'm using Pydantic V2 installed directly from the main branch, or equivalent

Description
I used localns to create nested models dynamically using pydanticv1.
Define all models, ….update_forward_refs({"ModelNameA":ModelA","ModelNameA":ModelA",}) for all of them in the proper order and it works.
How to without localns using pydanticv2?
Example Code
No response
Python, Pydantic & OS Version
pydantic version: 2.0a1
                 install path: ~/venv/openapi3/lib/python3.10/site-packages/pydantic
               python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
                     platform: Linux-5.19.0-35-generic-x86_64-with-glibc2.35
     optional deps. installed: ['email-validator', 'typing-extensions']

Yeah, I realized we had eliminated this and was concerned someone might want the ability to specify the localns directly. I think we can re-add this into model_rebuild, I'll take a stab at it.

@commonism the "fix" for this from #5385 is included now in the just-released pydantic==2.0a2. Please let us know if you have any further issues with this.


Revert "[release-1.11] Try random iptables command to fix jwt test (#35399)"

This reverts commit dc50711e3fb687c816215ea43751c9327336ae02.
Closes #35438.

@jwendell: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio_release-1.11
dd89c435c0e224e0a3df55c5a45fb50f236e0e25
link
true
/test release-notes_istio_release-1.11

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test release-notes_istio_release-1.11


Issue running the cython 0.28.1 tests on aarch64

I don't know if this affects any versions prior to 0.28.1, because I haven't tested it.
But I'm having issues building and running the tests of cython 0.28.1 (python3) on aarch64 (specifically on an Raspberry Pi running NixOS).
I have reported this in the distribution bugtracker here: https://github.com/NixOS/nixpkgs/issues/41347
But it's probably something upstream.
I have managed to disable the execution of tests which results in a cython that "works for what I use it for" but probably not entirely since the tests fails :smile:
Here's a complete build log of me building cython and running the tests on my Raspberry Pi: https://ptpb.pw/jkjC

The failing test relies on "long double complex" as a type. Maybe the
architecture doesn't provide that?

I'm not familiar with the different architectures instructions sets or which features are available. I understand that ARM uses an entirely different instruction set than x86, and that different features may be available etc.
I'm not sure how to look this up or how that looking it up would help.
But can I then assume that you don't have any intention of finding some workarounds for architectures that doesn't support such datatypes?
Can I also assume that it's always been like this?
Is it just for running the tests that this datatype is needed or is it also in the actual software?

But can I then assume that you don't have any intention of finding some workarounds for architectures that doesn't support such datatypes?

We usually adapt tests when we notice that they are not cross-platform. And generating cross-platform code is obviously a project goal.
In this case, it's not immediately clear if the test is simply inappropriate for the platform, if it's a C compiler bug, or if there is an actual Cython bug hidden behind the test failure. Only further investigation can answer this.
But for unusual platforms that the core developers do not have access to (and/or do not have an interest in themselves), we rely on users of those platforms to do the debugging for us and/or tell us what we need to improve in what way. If no user of this platform has an interest in solving this problem, then it might simply not be worth solving. Just saying.

Can I also assume that it's always been like this?

Probably since the test was added.

Is it just for running the tests that this datatype is needed or is it also in the actual software?

It's only needed when used by user code, such as this test.

Cool, thanks for those clarifications. I'll try to adopt the nix build to skip the tests that fails on aarch64 when building on that architecture to get those packages working :)

If we could understand the root cause, and skip this in Cython itself, that
would be ideal.
On Mon, Jun 4, 2018 at 1:39 PM, Elis Hirwing<EMAIL_ADDRESS>wrote:

Cool, thanks for those clarifications. I'll try to adopt the nix build to
skip the tests that fails on aarch64 when building on that architecture to
get those packages working :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cython/cython/issues/2308#issuecomment-394491620, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAdqga_mi2p_JnIwE6DrON38zEg8Q-yTks5t5ZsegaJpZM4UX0AD
.

I have now confirmed that disabling the test named numpy_memoryview makes it build and run the tests on aarch64. I wouldn't say that it does it "fine" because building of the program takes about an hour on one core on my raspberry. But running the tests takes about 14.5 hours (52068 seconds) for the last run. Which sounds like it hits timeouts and stuff somewhere.

On Debian with cython 0.28.2 we have this test failing on many architectures:
arm64, ppc64el, ppc64, powerpc, alpha
See https://buildd.debian.org/status/package.php?p=cython&suite=experimental&ver=0.28.2-2 for the build logs.
The list of architectures is similar to the numpy issue with float128, so it might be just this issue:
https://github.com/numpy/numpy/issues/11103

We were all talking about a specific test failing in this issue. If you have other tests failing on other architectures, you should probably open a new issue for that.
For the record, we were talking about this:
======================================================================
FAIL: test_coerce_to_numpy (numpy_memoryview)
Doctest: numpy_memoryview.test_coerce_to_numpy
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nix/store/qdcj04rnk8kb34zjrfzg8gfgl6svfcr8-python3-3.6.5/lib/python3.6/doctest.py", line 2199, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for numpy_memoryview.test_coerce_to_numpy
  File "/tmp/nix-build-python3.6-Cython-0.28.1.drv-0/Cython-0.28.1/TEST_TMP/memoryview/c/numpy_memoryview/numpy_memoryview.cpython-36m-aarch64-linux-gnu.so", line unknown line number, in test_coerce_to_numpy

----------------------------------------------------------------------
File "/tmp/nix-build-python3.6-Cython-0.28.1.drv-0/Cython-0.28.1/TEST_TMP/memoryview/c/numpy_memoryview/numpy_memoryview.cpython-36m-aarch64-linux-gnu.so", line ?, in numpy_memoryview.test_coerce_to_numpy
Failed example:
    test_coerce_to_numpy()
Expected:
    [97, 98, 600, 700, 800]
    deallocating...
    (600, 700)
    deallocating...
    ((100, 200), (300, 400), 500)
    deallocating...
    (97, 900)
    deallocating...
    99
    deallocating...
    111
    deallocating...
    222
    deallocating...
    333
    deallocating...
    11.1
    deallocating...
    12.2
    deallocating...
    13.25
    deallocating...
    (14.4+15.5j)
    deallocating...
    (16.5+17.7j)
    deallocating...
    (18.8125+19.9375j)
    deallocating...
    22
    deallocating...
    33.33
    deallocating...
    44
    deallocating...
Got:
    [97, 98, 600, 700, 800]
    deallocating...
    (600, 700)
    deallocating...
    ((100, 200), (300, 400), 500)
    deallocating...
    (97, 900)
    deallocating...
    99
    deallocating...
    111
    deallocating...
    222
    deallocating...
    333
    deallocating...
    11.1
    deallocating...
    12.2
    deallocating...
    0.0
    deallocating...
    (14.4+15.5j)
    deallocating...
    (16.5+17.7j)
    deallocating...
    (0+0j)
    deallocating...
    22
    deallocating...
    33.33
    deallocating...
    44
    deallocating...

I just tried with Numpy 1.14.4 rather than 1.14.3 on ppc64el. This version was supposed to fix https://github.com/numpy/numpy/issues/11103
The output indeed changed. Now it just fails because the "+" is missing for the complex number.
======================================================================
FAIL: test_coerce_to_numpy (numpy_memoryview.__test__)
Doctest: numpy_memoryview.__test__.test_coerce_to_numpy
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for numpy_memoryview.__test__.test_coerce_to_numpy
  File "/home/thansen/cython-0.28.2/build/work-dir/4/memoryview/c/numpy_memoryview/numpy_memoryview.so", line unknown line number, in test_coerce_to_numpy

----------------------------------------------------------------------
File "/home/thansen/cython-0.28.2/build/work-dir/4/memoryview/c/numpy_memoryview/numpy_memoryview.so", line ?, in numpy_memoryview.__test__.test_coerce_to_numpy
Failed example:
    test_coerce_to_numpy()
Expected:
    [97, 98, 600, 700, 800]
    deallocating...
    (600, 700)
    deallocating...
    ((100, 200), (300, 400), 500)
    deallocating...
    (97, 900)
    deallocating...
    99
    deallocating...
    111
    deallocating...
    222
    deallocating...
    333
    deallocating...
    11.1
    deallocating...
    12.2
    deallocating...
    13.25
    deallocating...
    (14.4+15.5j)
    deallocating...
    (16.5+17.7j)
    deallocating...
    (18.8125+19.9375j)
    deallocating...
    22
    deallocating...
    33.33
    deallocating...
    44
    deallocating...
Got:
    [97, 98, 600, 700, 800]
    deallocating...
    (600, 700)
    deallocating...
    ((100, 200), (300, 400), 500)
    deallocating...
    (97, 900)
    deallocating...
    99
    deallocating...
    111
    deallocating...
    222
    deallocating...
    333
    deallocating...
    11.1
    deallocating...
    12.2
    deallocating...
    13.25
    deallocating...
    (14.4+15.5j)
    deallocating...
    (16.5+17.7j)
    deallocating...
    (18.812519.9375j)
    deallocating...
    22
    deallocating...
    33.33
    deallocating...
    44
    deallocating...

DOUBLE_DOUBLE printing on powerpc was improved in https://github.com/numpy/numpy/pull/11104 but they know that it's still broken.


[xml-hint addon] Prevent extraneous quote

If typing attribute=" and by closetag it becomes attribute="" and user presses Ctrl-Space to autocomplete and the completion is "x" then by this fix  it nicely gives attribute="x" instead of obnoxious attribute="x"".
Intentionally coded to cause least change to previous behavior, to deal with the frequent most obnoxious case.
Resisted temptation to make changes affecting other cases too.

Thanks!


hybridcompute - update readme.md for SDK commands generation

When releasing SDK for different languages, there are some API operations that we don't want to release to customers.
This PR adds updates in readme to make sure that what gets released aligns with our .NET SDK.

/azp run

https://github.com/Azure/sdk-release-request/issues/5041


feat: cancel download

Hello,
I added a new IPC handler cancel-download to cancel the download completely, because closing the modal only doesn't stop the app from continuing the download in the background.
Thanks everyone for this project.

@adnanlah  but,  after canceling the update, another attempt to continue the update will result in the following error (without restarting the exe):
Cannot download differentially, fallback to full download: Error: cancelled
I haven't found the reason for this error yet.

Cannot download differentially, fallback to full download: Error: cancelled

True. I have fixed by generating a new CancellationToken after firing cancel(). It works fine for me now.
Thank you very much.

Cannot download differentially, fallback to full download: Error: cancelled

True. I have fixed by generating a new CancellationToken after firing cancel(). It works fine for me now.
Thank you very much.

After I performed the update cancel update-operation-update(repeated multiple timesupdate-operation), he still received an error. (Sorry, I don't quite understand this convenience. I'm not sure if it's due to operational issues.


fix: Improve performance of updateChildrenListItemValue

The algorithm used to update ListItemNode values was quadratic when called in a loop as it interrogated all previous siblings. Calculating all expected values from the ListNode ahead of time is simpler and much more performant.
Since a transform was already in place for ListItemNode it seemed reasonable to implement one for ListNode and consolidate all of the updateChildrenListItemValue usage there to avoid repeating the same list traversals for each item. This also lead the way to removing code from other places that attempted to renumber lists at each mutation where that might be necessary.
RE #5589

The test-e2e-ci-webkit test failures are the same ones I can reproduce locally on main, they do not appear to be regressions:
    7 failed
    [webkit] › packages/lexical-playground/__tests__/e2e/HorizontalRule.spec.mjs:30:3 › HorizontalRule › Can create a horizontal rule and move selection around it
    [webkit] › packages/lexical-playground/__tests__/e2e/Keywords.spec.mjs:231:3 › Keywords › Can type "congrats Bob!" where " Bob!" is bold
    [webkit] › packages/lexical-playground/__tests__/e2e/Links.spec.mjs:1605:3 › Links › Can convert part of a text node into a link with forwards selection
    [webkit] › packages/lexical-playground/__tests__/e2e/Links.spec.mjs:1721:3 › Links › Can convert part of a text node into a link with backwards selection
    [webkit] › packages/lexical-playground/__tests__/e2e/Markdown.spec.mjs:1026:3 › Markdown › can adjust selection after text match transformer
    [webkit] › packages/lexical-playground/__tests__/e2e/Tab.spec.mjs:20:3 › Tab › can tab + IME ───
    [webkit] › packages/lexical-playground/__tests__/e2e/TextFormatting.spec.mjs:733:3 › TextFormatting › Can select multiple text parts and format them with shortcuts
  29 skipped
  379 passed (2.3m)


Planning599hadolint

The initial implementation of hadolint with the following logic:
The implementation assumes that user will have '.hadolint' configuration file by default in the root. If the user passes a different hadolint configuration file name and/or path, it should be passed via configMap to the task, the task will refer to such custom filename/filepath. If the task cannot find either, it assumes that no hadolint configuration is available, it will skip the task.
If user provides a custom hadolint name, but does not input it in configMap, unfortunately, task assumes that there's no hadolint configuration in the task.
The task displays the configuration file on the logs (cat hadolint file), as well, if there is any lint error, then the pipeline run stops.
The sample configMap, where hadolint configuration is saved under 'myhadolint.yaml' file, should look like this:
kind: ConfigMap
apiVersion: v1
metadata:
name: hadolint-config
data:
HADOLINT_CFG: myhadolint.yaml
Todos:
Please note that starter kit templates are not yet updated, so I would recommend accepting this change after starter kits are updated (that's my responsibility, working on it).
I will also write the FAQ for the cloudnativetoolkit.dev to explain how to configure this task.

this work was done in this PR https://github.com/IBM/ibm-garage-tekton-tasks/pull/99

hadolint is already integrated


chore: add read deadline

This commit adds a read deadline to ensure that read never hangs. Even
though the connection is closed by the server, calls to conn.Read
sometimes hang indefinitely. It's unclear why. In most cases, the call
to conn.Read will return an io.EOF.
Fixes #1309.

This is the wrong approach. I'll update it now that I understand the actual problem.


0.4.1 is missing archives

Hi
0.4.1 was released but only contains source code.
Is it expected ?
We are using renovate to get latest version of jv tool (and many other) and our pipeline detected this issue.
Thanks

Thanks for raising the issue! It's not expected that the archived files are missing. Currently diagnosing why is it missing while the GitHub actions logs are all successfull for this tag 🤔
Just triggered 0.4.2 to watch the release behavior

https://github.com/jenkins-infra/jenkins-version/releases/tag/0.4.2 should unblock your renovabot process: can you confirm?

Thanks. Yes all good, 0.4.2 works now.

Found the problem : i tried to update the release body while it was being updated by the release process. Which resulted in breaking the release. It was a one time fix so should not happen.
many thanks for raising the issue


Pause a simulation and check if it has to continue from Python?

We can't support this since Nek5000 simply starts from case0.f00001 everytime it restarts and overwrites the existing files. That's why I made this session mechanism in the first place.

Originally posted by @ashwinvis in https://github.com/exabl/snek5000/issues/90#issuecomment-967561595
We would need to be able to stop a simulation as soon as something had happen (in practice the growth of a linear instability). It is very easy to check from Python with the data saved in the .his file (history points). I guess we could add some code in the .usr file (subroutine userchk) to pause Nek5000 and act depending on a response of Python.
@ashwinvis Any thoughts on the feasibility of this plan? How should the 2 processes communicate? Just stdin in Fortran?

Note: this would be the first step towards a tighter interaction between Nek and Snek. With such mechanism, we would be able to do some tasks (for example on the fly output) in Python during the simulation. For output, we would need to transfer data from Nek to Snek, which is not necessary for the application described in the issue (check if the simulation has to continue).

Very cool idea.
A simple way I can see this happening is to launch a simulation in the background sim.make.exec("run"), and have the Python loop inspecting the output in an event loop. If the output is on the filesystem, it is relatively easy to implement. We need to know the process ID somehow if we have to correctly issue a "kill" / "terminate" signal.
To integrate directly, there is this approach used by the AMUSE framework. For example see this article on how DALES (a Fortran LES code) exposes a Python interface using AMUSE / OMUSE.
https://www.sciencedirect.com/science/article/pii/S2352711020303216

We now have a nice example of how it could be done with "the simple way" : https://github.com/exabl/snek5000-cbox/pull/10
This depends on #112  (now ready to be merged).
This depends on an mpirun option only available with OpenMPI. To activate this option, we now need to use a particular config file with MPIEXEC_FLAGS: "--oversubscribe --report-pid PID.txt".
It would be much better to be able to specify this environment variable in the script as explained in #109.
This method uses the signal SIGTERM to terminate the Nek processes. I didn't find a gentler way to tell Nek to stop the simulation. Unfortunately, with SIGTERM, Nek just suddenly stops and does not run its finalization process.
Remark:
This is indeed a first step on controlling a simulation from Python. I think there are 3 methods:

Communication through hard disk and signals as in https://github.com/exabl/snek5000-cbox/pull/10

Communication between Python and Nek processes via a dedicated library. Since Nek already uses MPI, it makes sense to use MPI also to send data. I think it would be doable but it requires some additional code in the .usr file.

Calling Nek Fortran API from Python (like with the AMUSE framework if I understand correctly). Much finer control but a lot of work to implement.

is simple and already a very nice step.

would allow Snek to process Nek data during the simulation without IO on disk.

is in my point of view a too ambitious project for Nek5000.

I think we can close this issue as soon as we find a good solution for #109. Maybe we can then open a new long-term issue on solution 2.

Closed by #113 and #112


[hotfix][test] Disable Google PubSub Emulator e2e test on aarch64

What is the purpose of the change
This PR disabled google pubsub emulator e2e test on aarch64, since there is no cloud-sdk docker image for aarch64.
This PR also fixed the indent nit.
Brief change log

Disable google pubsub emulator e2e test on aarch64.
Change space to tab to satisfy Flink code style.

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Looks good, thanks!
Merging this...


cross-platform development environment in docker

The following changes implement a cross-platform development environment using Docker. Developers can build and spin up the image defined in Dockerfile.dev, mount the molecule source-code and develop inside the container.
Usage
cd molecule
docker-compose up -d
# select molecule container in vscode remote explorer OR
docker-compose exec dev /bin/bash
# OR
docker exec -it molecule_dev_1 /bin/bash

See:

Container development in Visual Studio Code.

PR Type

Feature Pull Request

We already have a Dockerfile for building molecule in the root. Why you are not able to re-use that one?
Please use global .gitignore for user-specific files (aka editor specific ones). While I use vscode myself and like it, allowing it to be added would open a pandora box of other excludes. For example OpenStack project has a strict policy regarding what can get into gitignores, for the same reasons: to avoid noise. I kinda like that policy.

We already have a Dockerfile for building molecule in the root. Why you are not able to re-use that one?

I'm playing with this some more now. The current image doesn't start the Docker daemon so we would still require an alternate entrypoint script to do so. Additionally, we will still want to mount the source-code and do a pip install -e . .
I see 3 approaches to the divergence from the Dockerfile in the root:

modify the existing Dockerfile to include an alternate entrypoint script that can be targeted in the docker-compose file
include a separate development Dockerfile with the alternate entrypoint extending public latest image (not possible until #2656)
mount and target an alternate entrypoint in the docker-compose configuration

Please use global .gitignore for user-specific files (aka editor specific ones). While I use vscode myself and like it, allowing it to be added would open a pandora box of other excludes. For example OpenStack project has a strict policy regarding what can get into gitignores, for the same reasons: to avoid noise. I kinda like that policy.

Thank you, this is a feature I was unaware of and will certainly help me moving forward. I've removed the entry.

With the latest revision the user can execute the dev container through:
docker-compose run --rm dev /bin/bash

Closing as it seems not to pass CI/CD, lacks updates and it does not appear to be clear what it solves. We can reopen if needed.


Variants aren't properly compared

here is the mistake, instead of finding all variants at once in the whole "command", find them line by line

dxhd parser errors on this:
# super + {a, b} + {c, d}
echo {a, b} and {c, d}
echo {foo, bar} and {baz, qux}

that's why it has to check variants line by line in the command string

okay, fixing that requires a lot of modification for replicate function, which must be a hard work. I might start parser rewrite, as #12 requires splitting parser into a lot of functions, so it's kinda like rewrite anyway.

parser should also be ok with:
# { a, b }
echo "I'm either a or b"


The ListCallback of AlertDialogWrapper#setAdapter() has position, but no ID

Hi,
Since the new version <IP_ADDRESS>,
the listview callback obtained with dialogZonas.getListView() doesn't work.
It was useful since I could use the Id of the cursor to perform updates.
It's possible add this parameter in the adapter's callback?
Thanks.

You mean the callback has no ID? The current way matches up with what the stock AlertDialog does.
But I'll make it so that you can still set your own callback to the ListView directly.

You can set a click listener directly now and it won't be overridden.
https://github.com/afollestad/material-dialogs/releases/tag/v<IP_ADDRESS>


Feature: Add Favourites page.

To implement favourite button on shopping items list and add a favourites page on navigation.

@yuvrajsinghgmx Please Assign this to me along with labels.

@Amit-4186 please refer to #4 for further improvements and changes.

@yuvrajsinghgmx  if no one is working you can assign this issue to me

@Shreyansh1563  Any Update ??

@yuvrajsinghgmx I'm currently working on it. Please label it as Level 3, as I need to design not only the UI but also the logic for the favorites page

Go Ahead ☺👍

@yuvrajsinghgmx I've already submitted this. If you'd like me to make any changes, please let me know; otherwise, you can close this issue.


[HPC] [N-Ways] Multi-GPU Programming

All the training materials currently covers converting a sequential code to GPU using N-Ways. This feature request is to add bootcamp material related to Multi-GPU programming. Ideally the tutorial should cover different methods like
-- NVSHMEM
-- NCCL
-- MPI+X
Also the impact of system and network topology on scaling. The tutorial should also look into tools that help in getting more insights on scaling and bottlenecks.

Reviews partly done : https://docs.google.com/document/d/1RR6uSBAp8sTFZUllFMTxWuu43n_-RNsk7w_USTmkoWM/edit


Support for (adding) explicit width/height attributes

Some applications have issues with SVGs that do not specify explicit width/height attributes (e.g. Madcap Flare). For those, it would be helpful if those attributes could be added if not present.
So, this would basically be the opposite of the --enable-viewboxing feature. Maybe both could be supported, resulting in explicit width/height and a viewBox attribute.

Why should it be a renderer-workaround?
librsvg, resvg, Inkscape, Chrome/Chromium, Firefox, Adobe Illustrator render them correctly if it cotains a viewBox, therefore it is a imho just a waste of bytes. Adding width/height for files without viewBox might be too complicated.
svgo (or SVGOMG) has the opposite options:
--enable=removeViewBox
--enable=removeDimensions

The problem here is that we'd have to guess a size for the SVG - which we really can't.
We can assume px or mm or ... but it would all be arbitrary.
Furthermore we'd modify the SVG, as the default for width/height is "100%", and a standard compliant renderer tries to scale the SVG to fill the whole view-port when rendering it.
Last but least - considering Scour is an SVG optimizer and not an "SVG compatibility fixer" - such an implementation would be out of scope and actually against the goals of Scour, as it increases file size as Johannes mentioned.
At this point I'm afraid I have to close this as wontfix, as the whole point of the feature seems to be to workaround issues in other software.

Some applications have issues with SVGs that do not specify explicit width/height attributes (e.g. Madcap Flare). For those, it would be helpful if those attributes could be added if not present.
So, this would basically be the opposite of the --enable-viewboxing feature. Maybe both could be supported, resulting in explicit width/height and a viewBox attribute.
Or maybe --renderer-workaround could be changed to add those attributes?

@bllngr I kind of did not read that you provied Madcap Flare as an example.
However you could use sed:
sed -ri 's/<svg([-[:alnum:]=\" .\/\:,\(\)_#]+) viewBox="([-[:digit:]\.]+) ([-[:digit:]\.]+) ([-[:digit:]\.]+) ([-[:digit:]\.]+)"([-[:alnum:]=\" \.\/:\,\(\);#]*)>/<svg\1 viewBox="\2 \3 \4 \5"\6 width="\4" height="\5">/' filename.svg
I did not test it, but I assume it works.


Documentation did not build on docs.rs

On my local machine, cargo doc works perfectly.
But the documents did not appeared on docs.rs.
numext-constructor is proc-macro library for constructors.
And I use those constructors to construct numext-fixed-uint and numext-fixed-hash (depend on numext-fixed-uint).
First, I published numext-constructor; then, numext-fixed-uint; the last is numext-fixed-hash.
BUT, the document for numext-fixed-hash wasn't built, the documents for other two crates are ok.
I do cargo yank then cargo publish again, the document for numext-fixed-hash still wasn't built.

And, is there a way to rebuild missing version of crates (numext-fixed-hash v0.1.1 and v0.1.2)?

Hi @yangby-cryptape, thanks for reporting. I am seeing this error in the build log:
2018/12/18 04:48:48 [INFO] cratesfyi::docbuilder::chroot_builder: Building package numext-fixed-hash-0.1.2
2018/12/18 04:48:48 [DEBUG] cratesfyi::utils::build_doc: Getting package with cargo
    Updating crates.io index
 Downloading crates ...
  Downloaded numext-fixed-hash v0.1.2
2018/12/18 04:48:48 [ERROR] cratesfyi::docbuilder::queue: Failed to build package numext-fixed-hash-0.1.2 from queue: failed to parse manifest at `.../.cargo/registry/src/github.com-1ecc6299db9ec823/numext-fixed-hash-0.1.2/Cargo.toml`

This is the generated file when you upload your crate to crates.io:
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g. crates.io) dependencies
#
# If you believe there's an error in this file please file an
# issue against the rust-lang/cargo repository. If you're
# editing this file be aware that the upstream Cargo.toml
# will likely look very different (and much more reasonable)

[package]
edition = "2018"
name = "numext-fixed-hash"
version = "0.1.2"
authors = ["Boyu Yang<EMAIL_ADDRESS>"zhangsoledad<EMAIL_ADDRESS>description = "Fixed-size hash types."
homepage = "https://yangby-cryptape.github.io/rust-numext/"
keywords = ["fixed", "hash", "bignum"]
categories = ["algorithms", "data-structures"]
license = "Apache-2.0 OR MIT"
repository = "https://github.com/yangby-cryptape/rust-numext"
[dependencies.failure]
version = "0.1"

[dependencies.faster-hex]
version = "0.1"
optional = true

[dependencies.heapsize]
version = "0.4"
optional = true

[dependencies.nfuint]
version = "~0.1.2"
package = "numext-fixed-uint"

[dependencies.numext-constructor]
version = "~0.1.2"

[dependencies.rand]
version = "0.5"
optional = true

[dependencies.serde]
version = "1.0"
optional = true

[features]
default = []
support_all = ["support_rand", "support_heapsize", "support_serde"]
support_heapsize = ["heapsize", "nfuint/support_heapsize"]
support_rand = ["rand", "nfuint/support_rand"]
support_serde = ["serde", "faster-hex", "nfuint/support_serde"]
[badges.travis-ci]
repository = "yangby-cryptape/rust-numext"

I am not sure why docs.rs is failing to parse this file. Maybe @QuietMisdreavus can figure out.

I'm not totally sure what happened either. That kind of "failed to parse manifest" error can come if you're using a feature in cargo that's newer than the cargo library we're using - i remember hitting that when people were using the edition feature right after it stabilized on nightly.
However, i just tried building it myself to get a build log, and it worked for me >_> https://docs.rs/numext-fixed-hash/0.1.2/numext_fixed_hash/
I think we just updated our cargo library via https://github.com/rust-lang/docs.rs/pull/271 so the problem may have fixed itself.

Thanks!
I will check if it works when I publish again.
In the near future, I have no plan to publish new versions of these crates.
So, I close this issue, for now.

Hi, the documents for both numext-fixed-hash v0.1.1 and numext-fixed-hash v0.1.2 are missing, again.

@yangby-cryptape recent server migration rolled back your crates (and maybe few others recently built manually in the old server). I rebuilt your crates and it should work fine now.


Remove IPos typedef.

This typedef seems like legacy compiler optimization from K&R days and leads to more casting warnings.

Looks good to me.


Revamp database notifications UI

[ ] Fix re-rendering notifications after marking as read.

Is this ready for review?

Sort of, the task in description still needs to be completed. I couldn't find the fix, so maybe you have an idea?

What specifically isnt working?

When marking database notifications as read, the notification component doesn't re-render as unread.

You need to get the notification object and pass ->unread() to it, when it is marked

So need to explicitly set it instead of getting the status from the notification object>

@danharrin, I guess I'm blind, but I really can't see why the component does render but doesn't update the notification styling when marking all as read.

I don't know how to fix but its coming from configureTransitions() in the notification JS Alpine Component. You are probably caching the styles for that element somewhere so that they end up not updating? But I don't know how that all works, you probably do.
If you cant fix, maybe mount the notification Alpine onto a wrapper of the actual notification element? so that the styles arent retained

Hmm, how did you find it's caused by the configureTransitions() function?
There's no caching there.
It only retrieves the styles to use the correct display value (since it could be different if someone provides their own notification Blade component).

Hmm, how did you find it's caused by the configureTransitions() function?

I removed it from the JS and the classes updated. Its those Livewire hooks

Oh configureAnimations you meant instead of configureTransitions. 😅

i think its configureTransitions... you can check

Closing for now.


Add MLJText and its model to the registry

cc: @pazzo83
julia> models("MLJText")
1-element Vector{NamedTuple{(:name, :package_name, :is_supervised, :abstract_type, :deep_properties, :docstring, :fit_data_scitype, :hyperparameter_ranges, :hyperparameter_types, :hyperparameters, :implemented_methods, :inverse_transform_scitype, :is_pure_julia, :is_wrapper, :iteration_parameter, :load_path, :package_license, :package_url, :package_uuid, :predict_scitype, :prediction_type, :supports_class_weights, :supports_online, :supports_training_losses, :supports_weights, :transform_scitype, :input_scitype, :target_scitype, :output_scitype), T} where T<:Tuple}:
 (name = TfidfTransformer, package_name = MLJText, ... )

julia> info("TfidfTransformer")
Build TF-IDF matrix from raw documents
→ based on [MLJText](https://github.com/JuliaAI/MLJText.jl).
→ do `@load TfidfTransformer pkg="MLJText"` to use the model.
→ do `?TfidfTransformer` for documentation.
(name = "TfidfTransformer",
 package_name = "MLJText",
 is_supervised = false,
 abstract_type = MLJModelInterface.Unsupervised,
 deep_properties = (),
 docstring = "Build TF-IDF matrix from raw documents\n→ based on [MLJText](https://github.com/JuliaAI/MLJText.jl).\n→ do `@load TfidfTransformer pkg=\"MLJText\"` to use the model.\n→ do `?TfidfTransformer` for documentation.",
 fit_data_scitype = Tuple{Union{AbstractVector{_s6464} where _s6464<:AbstractVector{ScientificTypesBase.Textual}, AbstractVector{_s6437} where _s6437<:(ScientificTypesBase.Multiset{_s6176} where _s6176<:(Tuple{Vararg{ScientificTypesBase.Textual, _s6464}} where _s6464)), AbstractVector{_s6175} where _s6175<:ScientificTypesBase.Multiset{ScientificTypesBase.Textual}}},
 hyperparameter_ranges = (nothing, nothing, nothing),
 hyperparameter_types = ("Float64", "Float64", "Bool"),
 hyperparameters = (:max_doc_freq, :min_doc_freq, :smooth_idf),
 implemented_methods = [:clean!, :fitted_params, :fit, :transform],
 inverse_transform_scitype = Union{AbstractVector{_s6464} where _s6464<:AbstractVector{ScientificTypesBase.Textual}, AbstractVector{_s6437} where _s6437<:(ScientificTypesBase.Multiset{_s6176} where _s6176<:(Tuple{Vararg{ScientificTypesBase.Textual, _s6464}} where _s6464)), AbstractVector{_s6175} where _s6175<:ScientificTypesBase.Multiset{ScientificTypesBase.Textual}},
 is_pure_julia = true,
 is_wrapper = false,
 iteration_parameter = nothing,
 load_path = "MLJText.TfidfTransformer",
 package_license = "MIT",
 package_url = "https://github.com/JuliaAI/MLJText.jl",
 package_uuid = "7876af07-990d-54b4-ab0e-23690620f79a",
 predict_scitype = ScientificTypesBase.Unknown,
 prediction_type = :unknown,
 supports_class_weights = false,
 supports_online = false,
 supports_training_losses = false,
 supports_weights = false,
 transform_scitype = AbstractMatrix{ScientificTypesBase.Continuous},
 input_scitype = Union{AbstractVector{_s6464} where _s6464<:AbstractVector{ScientificTypesBase.Textual}, AbstractVector{_s6437} where _s6437<:(ScientificTypesBase.Multiset{_s6176} where _s6176<:(Tuple{Vararg{ScientificTypesBase.Textual, _s6464}} where _s6464)), AbstractVector{_s6175} where _s6175<:ScientificTypesBase.Multiset{ScientificTypesBase.Textual}},
 target_scitype = ScientificTypesBase.Unknown,
 output_scitype = AbstractMatrix{ScientificTypesBase.Continuous},)

Codecov Report

Merging #413 (aee21dc) into dev (d2aaea1) will decrease coverage by 2.11%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #413      +/-   ##
==========================================
- Coverage   75.04%   72.93%   -2.12%
==========================================
  Files          12       12
  Lines        1006      824     -182
==========================================
- Hits          755      601     -154
+ Misses        251      223      -28

Impacted Files
Coverage Δ

src/MLJModels.jl
100.00% <100.00%> (ø)

src/registry/src/info_dict.jl
75.00% <0.00%> (-25.00%)
:arrow_down:

src/model_search.jl
69.01% <0.00%> (-3.28%)
:arrow_down:

src/builtins/Transformers.jl
93.78% <0.00%> (-2.65%)
:arrow_down:

src/builtins/Constant.jl
60.71% <0.00%> (-2.15%)
:arrow_down:

src/utilities.jl
0.00% <0.00%> (ø)

src/GaussianProcesses.jl
0.00% <0.00%> (ø)

src/registry/src/Registry.jl
0.00% <0.00%> (ø)

src/registry/src/check_registry.jl
0.00% <0.00%> (ø)

src/metadata.jl
81.81% <0.00%> (+0.44%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2aaea1...aee21dc. Read the comment docs.


[New Dashboard] Update recent content section when content changes in Home

Related to https://github.com/CartoDB/cartodb/issues/14662

retest this please


refactor(ng-update): do not copy gesture config if only standard HammerJS events are used

Currently the HammerJS v9 migration always copies the gesture config
if HammerJS events are used. This is not necessary for projects where
only standard HammerJS events are used. These events are provided
by the default hammer gesture config that comes with the HammerModule.

I intentionally used refactor since it technically didn't fix anything, but just avoided some unnecessary code. Either way though, I didn't feel too strong about it, so fix sounds good too.


malformed key exception in buzz

Describe the bug
flutter: SEVERE|2022-08-13 13:29:52.260256|LoadingScreen|Failed to load data : Exception: _latestNotificationIdv2.buzz is not well-formed key
To Reproduce
Steps to reproduce the behavior:

point buzz app to at_client trunk branch
onboard with any atsign
Expected behavior
no exception during onboard

@murali-shris Does the test in stats_notification_sync_test.dart make any sense to you? I don't really understand what it's trying to do, and in any event I don't believe we are syncing the notification ID any more (see SyncUtil.shouldSync

From NotificationServiceImpl._getLastNotificationTime()
    var lastNotificationKeyStr =
        '$notificationIdKey.${_atClient.getPreferences()!.namespace}${_atClient.getCurrentAtSign()}';
    var atKey = AtKey.fromString(lastNotificationKeyStr);

AtKey.fromString is the only place that generates a "is not well-formed key" message
The error is valid - it is indeed not a well-formed key - but it shouldn't be happening, because currentAtSign() should not be null.
It can only be happening because, for some reason, current atSign is not set when _getLastNotificationTime() is being called (which can be in the initial startMonitor, or a restart of the monitor for example)
@yahu1031 can you verify please what the currentAtSign is set to? I've pushed a branch in at_client_sdk called logCurrentAtSignAtVariousTimes which will log the current atSign at various times

@yahu1031 @murali-shris where is the Buzz source code?

Lemme update the code with new branch @gkc

Can you point me at the source code please @yahu1031 ?

shouldSync

it triggers a sync and asserts that notificationKey is not synced. But I think the check should happen in the onDone call back instead of main()

@murali-shris Does the test in stats_notification_sync_test.dart make any sense to you? I don't really understand what it's trying to do, and in any event I don't believe we are syncing the notification ID any more (see SyncUtil.shouldSync)

it triggers a sync and asserts that notificationKey is not synced. But I think the check should happen in the onDone call back instead of main()

@gkc When I call
await ServiceInstances.atClientManager
    .setCurrentAtSign(
        context.read<UserData>().currentAtSign,
        BuzzEnv.appNamespace,
        context.read<UserData>().atOnboardingPreference);

in the app, I get this exception

Related code is here

This doesn't happen all the time IDK why.

Is context.read() async?

Nope @gkc it is not. And also there is no way of missing awaits.  So no awaits will be missing.
Thanks.

👍
Well for whatever reason, the problem seems to be that

context.read().currentAtSign

Is returning empty string

Thanks @gkc for the help and guidance. I fixed the issue and it is from the app as you mentioned.

Closing the issue as fixed from the app side.


added option to sw:install:vcs for excluding demodata

this commit adds a new option to install:vcs. now it is possible to disable the installation of the demo data

Looks fine for me, thanks @mitelg.
@dnoegel May you merge and deploy a new internal version.


FFMpeg issue on MacOS

youve got a typo in your autoProcess.ini??
the script is looking for ffmpeg in /usr/local/Cellar/ffmpeg/3.3.2/bin/ffmpeg**e**
if you type   which ffmpeg what is the outcome?
also what is the settings you put in your autoProcess.ini for ffmpeg?
[MP4]
ffmpeg =
ffprobe =

When I type ffmpeg in terminal it outputs:
$ which ffmpeg
/usr/local/bin/ffmpeg
the location has an alias file made by homebrew. If I run ffmpeg in terminal the output:
$ ffmpeg
ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.0.0 (clang-8<IP_ADDRESS>)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.3.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --disable-lzma --enable-vda
libavutil      55. 58.100 / 55. 58.100
libavcodec     57. 89.100 / 57. 89.100
libavformat    57. 71.100 / 57. 71.100
libavdevice    57.  6.100 / 57.  6.100
libavfilter     6. 82.100 /  6. 82.100
libavresample   3.  5.  0 /  3.  5.  0
libswscale      4.  6.100 /  4.  6.100
libswresample   2.  7.100 /  2.  7.100
libpostproc    54.  5.100 / 54.  5.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
my autoProcess.ini is:
[Radarr]
host = localhost
port = 7878
web_root =
ssl = False
apikey = *********************
[MP4]
ffmpeg = /usr/local/Cellar/ffmpeg/3.3.2/bin/ffmpege
ffprobe = /usr/local/Cellar/ffmpeg/3.3.2/bin/ffprobe
threads = auto
output_directory =
copy_to =
move_to =
output_extension = mp4
output_format = mp4
delete_original = True
relocate_moov = True
video-codec = h264,x264
video-bitrate =
video-crf =
video-max-width =
h264-max-level =
use-qsv-decoder-with-encoder = True
ios-audio = True
ios-first-track-only = False
ios-audio-filter =
max-audio-channels =
audio-codec = acc
audio-language =
audio-default-language =
audio-channel-bitrate =
audio-filter =
subtitle-codec = mov_text
subtitle-language =
subtitle-default-language =
subtitle-encoding =
fullpathguess = True
convert-mp4 = False
tagfile = True
tag-language = en
download-artwork = Poster
download-subs = False
embed-subs = True
sub-providers = addic7ed,podnapisi,thesubdb,opensubtitles
permissions = 0777
post-process = False
pix-fmt =
aac_adtstoasc = False
postopts =
preopts =
[SABNZBD]
convert = True
sickrage-category = sickrage
sonarr-category = sonarr
bypass-category = bypass
couchpotato-category = couchpotato
sickbeard-category = sickbeard
radarr_category = radarr
output_directory =
radarr-category = radarr
[Plex]
host = localhost
port = 32400
refresh = True
token =
[Sonarr]
ssl = False
host = localhost
apikey =
port = 8989
web_root =
[Deluge]
username =
sonarr-label = sonarr
output_directory =
host = localhost
radarr-label = radarr
password =
port = 58846
convert = True
sickbeard-label = sickbeard
remove = false
couchpotato-label = couchpotato
bypass-label = bypass
sickrage-label = sickrage
[uTorrent]
username =
action_after = removedata
sonarr-label = sonarr
output_directory =
host = http://localhost:8080/
radarr-label = radarr
webui = False
password =
convert = True
sickbeard-label = sickbeard
couchpotato-label = couchpotato
action_before = stop
bypass-label = bypass
sickrage-label = sickrage
[CouchPotato]
username =
delay = 65
host = localhost
apikey =
web_root =
delete_failed = False
ssl = False
password =
port = 5050
method = renamer
[SickBeard]
username =
password =
ssl = False
host = localhost
web_root =
api_key =
port = 8081
[Sickrage]
username =
password =
ssl = False
host = localhost
web_root =
api_key =
port = 8081

I was able to fix the original issue. There was a typo in my autoprocess.ini. One in ffmpeg and the second for the audio. I was able to manually run the script in terminal and the video was able to convert from mkv to mp4 w/aac. The second issue I'm having is when sabnzbd runs the post processing script. The error that is appearing is:
traceback (most recent call last):
File "/Volumes/DATA HD/sabnzb/mp4convert/sickbeard_mp4_automator-master/SABPostProcess.py", line 5, in
from autoprocess import autoProcessTV, autoProcessMovie, autoProcessTVSR, sonarr, radarr
File "/Volumes/DATA HD/sabnzb/mp4convert/sickbeard_mp4_automator-master/autoprocess/autoProcessMovie.py", line 11, in
import requests
ImportError: No module named requests
In terminal when I type the command pip3 list, requests are there
$ pip3 list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
asn1crypto (0.22.0)
babelfish (0.5.5)
beautifulsoup4 (4.6.0)
certifi (2017.4.17)
cffi (1.10.0)
chardet (3.0.4)
click (6.7)
cryptography (1.9)
dogpile.cache (0.6.4)
enzyme (0.4.1)
guessit (1.0.3)
idna (2.5)
pbr (3.1.1)
pip (9.0.1)
pycparser (2.18)
pyOpenSSL (17.1.0)
pysrt (1.1.1)
python-dateutil (2.6.1)
qtfaststart (1.8)
requests (2.18.1)
requests-cache (0.4.13)
setuptools (32.2.0)
six (1.10.0)
stevedore (1.19.1)
subliminal (1.1.1)
urllib3 (1.21.1)
wheel (0.29.0)
These files are located in: /usr/local/lib/python3.6/site-packages

I posted about this earlier in the week. Issue765
Change SABPostProcess.py scripts first line.
Original:   #!/usr/bin/env python
New:  #!/usr/local/bin/python
I think the underlying issue is SAB's python  scripts used in the OSX precompiled dmg file has its shebang (the first line with the  !#/path/ ) set to use OSX's version of python (/usr/bin/python).
That patch is being caried through to the post processing script via the "Original" path's use of /usr/bin/env.
Telling it to specifically use the homebrew python in that single script is a workaround for now.

It worked. Everything is working properly now. Thank you for helping.


[Feature] Refactor inherent LLM to LLM router

This document outlines the steps in the AIOS roadmap for refactoring the LLM structure. Feel free to discuss any of the following topics, and add any other topics you'd like to talk about in this issue.
Kernel

Set up base LLM router with customizable features
Create task-specific routers

Cost Router
Complexity Router
Efficiency Router

Agents

Connect agent workflow to router to allow for automatic agent creation

A basic router with customized features gives the flexibility and eventually it will be going to be a specific router. Feels this is a step one and two for a specific router. One doesn't have any capability. Two is specific capabilities. Both eventually will be specific routers.


Issues on mobile

The site looks pretty good on mobile, but there are a few small issues:
Small images
Images are too small and thus illegible. For the diagrams, could we use the full screen width? The logos could also be a tad bigger, though this isn't a huge deal.

Text too narrow due to floating ToC or goose
On some pages, the table of contents takes up too much room, leaving only a very narrow strip for text. Same is true for the goose on the community page. For the ToC, making it collapsed by default might be a good idea (not sure)?

Note that the ToC other pages look OK, e.g. Specification.

Specification sub-pages require horizontal scrolling
The specification sub-pages require horizontal scrolling, which is confusing especially since the ☰ menu cannot be easily found.

Menu ▾ difficult to see
In the drop-down menu, the grey ▾ arrow next to "Specifications" is grey on dark green, which is very difficult to see. A different color should be used.

Thanks Mark, on it

Looks like the first two are fixed but the second two are not. The page http://localhost:4000/specifications/v0.1/levels exhibits both problems: scrolling and the grey triangle. You can simulate mobile on Chrome, or you can use a real phone.

Hi @MarkLodato

Specification sub-pages require horizontal scrolling

So I've patched the layout so that the menu is always visible when checked on a device, see:

But the current site (e.g. slsa.dev/requirements) also exhibits horizontal scrolling due to rendering the markdown tables. It's the only way to be able to view >3 or so columns. I.e. some x-scrolling is acceptable as long as the menu is not pushed off the screen.
However, I have no idea why it's behaving this way when simulated in Chrome.
Can you can confirm the menu issue is fixed on your real phone? If so, I would suggest this bug is not a release blocker.

Menu ▾ difficult

Just got it! Fix coming soon

@MarkLodato I believe I've fixed the scrolling issue, could you help me check in-browser on Android? Much appreciated!

Both fixed. Thanks!


Assign reviewers when PR is not draft

Hello there!
I recently updated one of my workflows from v1.5.0 to the latest version of this action and found out that now assignees are also tagged as reviewers, which is pretty nice. The issue is, should they be tagged when the PR is still draft?

Ho @mariajgrimaldi. For this kind of use case, I would use a filter in the workflow so that the action doesn't run for draft PRs like this:
https://github.com/orgs/community/discussions/25722
Would this work?


💡 [Feature Request]: discord link failure

Feature
discord.wow-cms.com
Link failure
Feature Purpose
Hoping to fix it
Additional information
No response

The invitation to the WoW-CMS discord has been restored, thanks for reporting.


Non-clickable links in a comment summary

Describe the bug
Some links are rendered without a href attribute, what makes them unclickable. Also note the oddly placed quotation marks around the link, but this might be "as intended", although looking a bit off.
To Reproduce
Go to Profile/Discourse, and search for comment headers with unclickable links. One kind of such links is in comments titled as SomeUser commented on ""Some Kata Title" Java Translation".
Expected Behavior
Link is clickable and redirects to the discussed item.
Screenshots

This is probably a duplicate of #68.
I don't know if this is intentional or not at the moment. If it's intentional and it's meant to prevent random users from accessing, then it shouldn't be shown on the dashboard to begin with. But I never know because Codewars has lots of irrational stuff from over the years.

Ah what drove off is a fact that it did not happen on a dashboard but on a "Discourse" view of my profile, and it's a link to my own translation - where I think is no reason to limit this possibility.
But as you say, the reason and logic behind such behavior is probably the same as in case of #68

I also encountered it yesterday and today, but it was even more strange - Unnamed commented on "" kata.

Unnamed commented on "" kata seems to be different, but I assume that it's just a weird consequence of timing out. It was supposed to be here, but it appeared on dashbord and in my profile, but not at the discuss page.


Access violation on Winlibs MinGW-w64 Clang on Windows

Doctest version: 2.4.11
OS: Windows 10 N LTSC 21H2 x64
Compiler: Winlibs MinGW-w64 Clang 17.0.6 (posix threads)
Hello, I've encountered a problem while trying to build a test with Clang. The program immediately terminates with an access violation:
Loaded 'D:\Users\novoseiversia\Desktop\source\repo\novoseiversia\piece-of-shit\build\test\piece-of-shit-test.exe'.
Loaded 'C:\Windows\System32\ntdll.dll'.
Loaded 'C:\Windows\System32\kernel32.dll'.
Loaded 'C:\Windows\System32\KernelBase.dll'.
Loaded 'C:\Windows\System32\ucrtbase.dll'.
Loaded 'C:\apps\winlibs-mingw-w64-ucrt-11.0.1-r5\bin\libgcc_s_seh-1.dll'.
Loaded 'C:\apps\winlibs-mingw-w64-ucrt-11.0.1-r5\bin\libstdc++-6.dll'. Module was built without symbols.
Loaded 'C:\apps\winlibs-mingw-w64-ucrt-11.0.1-r5\bin\libwinpthread-1.dll'. Module was built without symbols.
Exception thrown at 0x00007FF7A680B1E5 in piece-of-shit-test.exe: 0xC0000005: Access violation writing location 0x00000238105C3728.

Doctest compiles fine on Winlibs MinGW-w64 GCC 13.2.0 (posix threads) and MSVC latest.
I'm not sure what the exact problem is (GDB is not catching any of the breakpoints I've set in the doctest.h), but I've found a similar problem with an open pull request: #560.
Attached are my buildsystem files and the test source file.
CMakeLists.txt
CMakeLists.txt
main.txt

I don't think it's related to doctest... can you compile a simple example without doctest with clang?
Also I love that your test is called piece-of-shit-test :D

A simple hello world program does compile and run successfully with Clang.
Also piece-of-shit is the name of the project hehe

I didn't notice this before but building using Clang requires target_link_libraries(... Threads::Threads) otherwise I get a bunch of undefined reference errors:
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/bin/ld: CMakeFiles\xxinit-test.dir/objects.a(main.cxx.obj): in function `__gthread_mutex_init_function(long long*)':
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/include/c++/13.2.0/x86_64-w64-mingw32/bits/gthr-default.h:733:(.text+0xfc31): undefined reference to `pthread_mutex_init'
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/bin/ld: CMakeFiles\xxinit-test.dir/objects.a(main.cxx.obj): in function `__gthread_mutex_lock(long long*)':
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/include/c++/13.2.0/x86_64-w64-mingw32/bits/gthr-default.h:749:(.text+0x101ed): undefined reference to `pthread_mutex_lock'
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/bin/ld: CMakeFiles\xxinit-test.dir/objects.a(main.cxx.obj): in function `__gthread_mutex_unlock(long long*)':
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/include/c++/13.2.0/x86_64-w64-mingw32/bits/gthr-default.h:779:(.text+0x102dd): undefined reference to `pthread_mutex_unlock'
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/bin/ld: CMakeFiles\xxinit-test.dir/objects.a(main.cxx.obj): in function `__gthread_mutex_destroy(long long*)':
[build] C:/apps/winlibs-mingw-w64-ucrt-11.0.1-r5/include/c++/13.2.0/x86_64-w64-mingw32/bits/gthr-default.h:740:(.text+0x10aed): undefined reference to `pthread_mutex_destroy'

This doesn't happen with the simple hello world program, only with doctest.

Try #define DOCTEST_CONFIG_NO_MULTITHREADING before including the header or adding it as a define through CMake - that might help - link to docs

That worked, thanks! Now I've got to figure out what the problem is with threads on Clang lol

I assume that if you make a minimal example that uses atomics or anything related to threads (but without doctest included at all) that you'd have problems compiling that as well - so I'm closing this issue

Ah, I forgot to close the issue, thanks.


How to using  multiplier in masonry...

as you know , I want to using multiplier masonry, but i dont't find excample in demo...

@ccguo there is an example in the issue related to aspect fit https://github.com/Masonry/Masonry/issues/131


Contribution Request | MacOS Support

Would you welcome a PR to add support for MacOS?

I just took a quick look, and besides hardcoded paths and variable names, i2p-zero is the hardest part, I can work on that too, in the repo you already cloned. Dunno if is something you are open to support.

@albe-rosado yes that would be helpful. Lmk what you need on my end. I don't have a Mac to verify anything so if you could get another user to check that would be great. Thanks.

Nothing so far, I will start working on it soon. Thanks! Is there any place online you hang out, in case we want to discuss stuff? Thanks

Is there any place online you hang out, in case we want to discuss stuff?

No, I try to limit my exposure to the clear as much as possible. There is a monero ecosystem matrix channel I think. There is a nevekonet i2p irc server address in the i2p release server home page. http://c2m66oddrzozztxyzjegbdwtgbeiibq5vz2tpchmqamrzcahcfoq.b32.i2p/index.txt
I can join the i2p irc server there at 1200UTC daily and check messages. I will also check the GH daily if I have time.

Just wanted to touch base, I have been pretty busy recently, but I have been working on Rust bindings for i2pd , the reason I went with i2pd is because is pretty fast, and is very easy to integrate with, as is just a portable binary supported on all major platforms and the team behind seems to be very active. i2p-zero in MacOS relies on an older version of JDK, I'm having a hard time even downloading, also it seems to not have a lot of work recently (maybe is defunct) .  Next I will remove the hardcoded paths in the bindings package and have a POC of neveko using such bindings, which in my opinion would be great as we wont have to execute binaries from code. Regarding monerod, I will take a look to see if there is any Rust implementation or existing bindings. I will cut a CR once I have i2pd integrated. I'm open to any other idea.

@albe-rosado there has been mention of i2pd in other issues. The Rust bindings for i2pd would be a great and welcome addition. As for Rust bindings for Monero, I'm not aware of any. In the future it might be worth considering migrating to Cuprate, but that wouldn't be anytime soon. Thanks for the effort. Look forward to the POC.


[Bug] llama2 7b android compilation is giving "Can only handle constant size stack allocation for now" error

Model: https://huggingface.co/mlc-ai/Llama-2-7b-chat-hf-q4f16_1-MLC
Compiling above model with cmd:
" mlc_llm compile ./dist/Llama-2-7b-chat-hf-q4f16_1-MLC/mlc-chat-config.json --device android -o ./dist/Llama-2-7b-chat-hf-q4f16_1-MLC/llama-2-7b-chat-hf-q4f16_1-android.tar "
giving below errors:
File "/home/test/Ramees/relax/src/target/source/codegen_c.h", line 104, in tvm::codegen::CodeGenC::PrintStmt(tvm::tir::Stmt const&)
void PrintStmt(const Stmt& n) { VisitStmt(n); }
File "/home/test/Ramees/relax/src/target/source/codegen_c.cc", line 989, in tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AllocateNode const*)
ICHECK_GT(constant_size, 0) << "Can only handle constant size stack allocation for now";
tvm.error.InternalError: Traceback (most recent call last):
6: operator()
at /home/test/Ramees/relax/src/driver/driver_api.cc:531
5: tvm::TIRToRuntime(tvm::runtime::Map<tvm::Target, tvm::IRModule, void, void> const&, tvm::Target const&)
at /home/test/Ramees/relax/src/driver/driver_api.cc:514
4: tvm::codegen::Build(tvm::IRModule, tvm::Target)
at /home/test/Ramees/relax/src/target/codegen.cc:73
3: tvm::codegen::BuildOpenCL(tvm::IRModule, tvm::Target)
at /home/test/Ramees/relax/src/target/source/codegen_opencl.cc:619
2: tvm::codegen::CodeGenC::AddFunction(tvm::GlobalVar const&, tvm::tir::PrimFunc const&)
at /home/test/Ramees/relax/src/target/source/codegen_c.cc:167
1: tvm::codegen::CodeGenC::PrintStmt(tvm::tir::Stmt const&)
at /home/test/Ramees/relax/src/target/source/codegen_c.h:104
0: tvm::codegen::CodeGenC::VisitStmt_(tvm::tir::AllocateNode const*)
at /home/test/Ramees/relax/src/target/source/codegen_c.cc:989
File "/home/test/Ramees/relax/src/target/source/codegen_c.cc", line 989
InternalError: Check failed: constant_size > 0 (0 vs. 0) : Can only handle constant size stack allocation for now
Any idea why it is happening??  @tqchen

I am not sure what was happening on this case, perhaps it is related to come stale variant of compiler. we recently updated our android SDK https://llm.mlc.ai/docs/deploy/android.html please try follow the new instructions

This is not just happening on the Android SDK, I am using this on a CUDA device, but this error still happens. I tried to use a built from source mlc_llm, the error would happen, but when I use the pre-built the mlc_llm, this error goes away. It is hard to locate where is the problem

Glad this is resolved. likely it was due to old ersion of the tvm


Strange behavior in modal

I tried to use component in the bootstrap modal. The component behaved strangely. An axis show minus values when dragging.
...
I found out that is due to the visibility of the component.

I do not know regarding modal, but I had this issue in case of using <div class="col"> (or <v-col> in case of using "bootstrap-vue"), which by default has position: relative. position: static (or class="position-static" in case of bootstrap) fixed the problem.

Unfortunately not work for me. I made custom invisible but not "display: none" modal and all work fine.

@shchegol Im pretty sure without demo no one can help you. You can take for example this code, as a basis, and show where the trouble is. Then I can check if I can help you.


Add alias option to every analysis node

Right now, the name of the tables created by the analysis nodes changes every time the parameters of the analysis change. An alias will be a -unique- name that you assign to an analysis node, so you can query that node always in the same way.
In Terms of UI this would be simple. Just adding an extra block for the alias on the analysis config and putting that alias on the layer list as the id of the node.

+1

@rochoa we need to discuss this at API level

+1, definitively. This is a breakthrough improvement.

A couple of questions:

Do you expect the alias to be the same when you change some of the params?
Do you expect the alias to remain the same when some filters (from a widget) are applied?

Imagine you have a metro station points dataset.
You create a couple of different visualisations where you end having the same buffer, let's say 500m over each station, the nodes are in both visualisations are:

[a0: stations] -> [a1: buffer 500m] -> [a2: intersection a1, b0].
[b0: stations] -> [b1: buffer 500m] -> [b2: intersection b1, c0].

Now, from your proposal: what would you expect as the alias for the buffer in both cases?

In general, and related to @javisantana comment, is the alias something persistent or is something you want to have temporarily as you are working with it?

From my understanding it should be persisted.
On Aug 1, 2016 1:14 PM, "Raul Ochoa"<EMAIL_ADDRESS>wrote:

A couple of questions:

Do you expect the alias to be the same when you change some of the
params?
Do you expect the alias to remain the same when some filters (from a
widget) are applied?

Imagine you have a metro station points dataset.
You create a couple of different visualisations where you end having the
same buffer, let's say 500m over each station, the nodes are in both
visualisations are:

[a0: stations] -> [a1: buffer 500m] -> [a2: intersection a1, b0].
[b0: stations] -> [b1: buffer 500m] -> [b2: intersection b1, c0].

Now, from your proposal: what would you expect as the alias for the buffer
in both cases?
In general, and related to @javisantana https://github.com/javisantana
comment, is the alias something persistent or is something you want to have
temporarily as you are working with it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CartoDB/cartodb/issues/9113#issuecomment-236553555,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIENVY0Vsm96D1tUzsfxWLuC9thBy2_ks5qbdT_gaJpZM4JVBbi
.

Then, what about the rest of my questions?

 So nothing stops us from having an abstraction at client side to ask for the query associated with a node. But that should be also considered when we define the API at client side.
I don't think it's possible to make it persistent and at the same time keep the same alias when params and filters change.
For me, it starts to look like Named Analyses, à la Named Maps, so as @javisantana was pointing, we should think about the problem we want to solve at API level because this is a higher level abstraction over what we have now.

Hi everybody from Geographica!
Any further progress / thoughs / planning about this? IMHO, this is a great improvement to mix Builder tools with custom SQL. Also great to test new and potential Builder tools algorithms.

This should be though with Analysis API -> https://github.com/CartoDB/Windshaft-cartodb/issues/559


Configuration not saving

Describe the bug
Configuration not saving in google sheets.
To Reproduce
Steps to reproduce the behavior:

Go to google sheets
Click on configure
Enter API information
Click save and never updates the information just sits on a gray save button.

Expected behavior
Should have saved information in configuration. Just sits there not updating anything.
Screenshots

Desktop (please complete the following information):

OS: Windows 10
Browser Latest Chrome
Version 80.0.3987.132

@kentico-ericd  - could you take a look?

@Isetnofret1 I'm afraid I couldn't reproduce the issue on Chrome 80 and Win10- it works every time. Can you check your browser's Console for errors, or the Network tab to see if there's any stuck/failed requests?
Also, can you try in an Incognito window without extensions?


Amplify frontend build returns  Starting SSR Build...  [ERROR]: AccessDenied: Access Denied

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
d3dh32c6fp1s0y
Region
eu-cental-1
Amplify Console feature
Not applicable
Describe the bug
building next js app in amplify returns  Starting SSR Build...  [ERROR]: AccessDenied: Access Denied.
Expected behavior
the starting SSR build should happen without errors
Reproduction steps
deploy a next.js SSR app using amplify console and in build steps the error comes in.
Build Settings
version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm install
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: .next
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

Additional information
i  think the amplify is not letting me to create cloudfront distributions due to 25 number of limit of cloudfronts but i have deleted them and maybe the amplify is not detecting is my only guess.

I have also been hit with this issue. Since I use PR previews with SSR, every PR creates a new CloudFront distribution. Thus, after ~25 PRs, we get build failures.
In Amplify, there is a setting to delete old deployments once the branch has been merged. In my opinion, this should delete the underlying resources that are created during those merges.
Manually disabling then deleting the CloudFront distributions that were associated with Lambda@Edge functions worked for me. At this point, I am tempted to write some custom scripts to either delete old distributions (and potentially other resources) on cron-job or to delete on merge in a GitHub Action.
I'd recommend renaming this issue to something related to "Merging SSR PR previews doesn't clean up created resources" or something like that as I believe that's the underlying issue.
Good luck!

Thank you for your kind information but i have already manually deleted Cloudfront distributions associated with Lambda@Edge functions. And still it's throwing me access denied.. i have also tried the same code with the same build settings on another Aws account and it was successfully deployed.So for now, i have requested a quota increase rate for Cloudfront distributions and Lambda@Edge.

Hi @zugger14, does this help? https://github.com/aws-amplify/amplify-console/blob/master/FAQ.md#error-accessdenied-access-denied

For me, it was that the standard Amplify - Backend Deployment service role didn't have all the permissions required for an SSR Nextjs deployment. I had to attach the AdministratorAccess policy to my Amplify service role to make it work.
I think this issue is a duplicate of https://github.com/aws-amplify/amplify-console/issues/2192

In my situation I was first not able to clone the repo during the BUILD step getting fatal: unable to access 'https://git-codecommit.us-east-2.amazonaws
I was able to resolve that by adding AWSCodeCommitReadOnly permission to my service role
However, I then ran into the issue described in this Jira [ERROR]: AccessDenied: Access Denied
The odd thing is that even adding AdministratorAccess permission did not make a difference this time.

Just wanted to bump this since it's been a couple weeks and I am still experiencing this.

@oste not ideal but we gave up in the end and re-init all of Amplify config from scratch -- thankfully we're still in MVP / proof of concept stage and could afford to do that.
Still had to provide AdministratorAccess -- don't like providing elevated privilges but it is what it is until Amplify team become more responsive and do some root cause analysis.

I actually fixed this for one Amplify project in one AWS account, but now I'm facing the issue again in a different account and even assigning AdministratorAccess isn't fixing it this time.
For me and many others, this basically means that deploying SSR Next.js apps on Amplify does not work.

Spent some more time on this, and I think that the Access Denied is happening when the Amplify role tries to update the s3 bucket to use transfer acceleration.
The operation - PutAccelerateConfiguration should be allowed given that the service role has full AdministratorAccess to all resources.
However, if I go to the s3 console myself (again, with full Admin access) and create a bucket, then even I am unable to update the bucket to use transfer acceleration.

Hi @cpv123 can you please share your appId so I can take a closer look, thanks!

I was able to successfully set the transfer acceleration for both new and existing buckets.
It does look like it fails on an S3 related task based on the error.

@Athena96 my Amplify appId is d23o5sr1zur08j
I spoke with AWS technical support this morning and walked them through the full problem whilst screen-sharing. I showed them both the Amplify failure and the S3 bucket problem alone. They're reaching out to another internal team and will get back to me.
I'll update in here if I get back anything good 🙏

I'm facing the same issue. My app id is d1zbf83kttrm4k, I've given the full AdministratorAccess to my user role Amplify - Backend Deployment and it still barfs with an Access Denied error. It's extremely frustrating to deal with and after searching for hours and looking at other people complaining on this open issue as well, it seems to me that there is no clear fix to this nor any guidelines on how to work around it.
This leaves me with no option but to switch to a different hosting platform.

Having same issue. Tried both adding full admin access to service account and enabling transfer acceleration. Interestingly dev environment is deployed without any issues but prod always fails at this step. Environments are identical except few different ENV vars. In my case issue seems like related to cloud frount:

at Request.extractError (/root/.//node_modules/@sls-next/aws-cloudfront/node_modules/aws-sdk/lib/protocol/rest_xml.js:53:29)

I'm facing the same issue. My app id is d1zbf83kttrm4k, I've given the full AdministratorAccess to my user role Amplify - Backend Deployment and it still barfs with an Access Denied error. It's extremely frustrating to deal with and after searching for hours and looking at other people complaining on this open issue as well, it seems to me that there is no clear fix to this nor any guidelines on how to work around it.
This leaves me with no option but to switch to a different hosting platform.

Yes.. still happened to me at 23/12/2021. My existing project which deployed couple of times with no problem, suddenly having Access Denied error. Having the service role with AdminstratorAccess, attached the newly created policy with all the permission stated here https://github.com/aws-amplify/amplify-console/blob/main/FAQ.md#error-accessdenied-access-denied, still having Access Denied. I totally fed up on this, create a sudden "IT SHOULD WORK, CAUSE IT WORK BEFORE, BUT NOW IT NOT WORK ANYMORE" problem for me
My app details:
appId: dcuydyh2ys8c4
region: ap-southeast-1
Full document here: https://github.com/aws-amplify/amplify-console/issues/1876#issuecomment-999690059

Spent some more time on this, and I think that the Access Denied is happening when the Amplify role tries to update the s3 bucket to use transfer acceleration.
The operation - PutAccelerateConfiguration should be allowed given that the service role has full AdministratorAccess to all resources.
However, if I go to the s3 console myself (again, with full Admin access) and create a bucket, then even I am unable to update the bucket to use transfer acceleration.

Yes noticed this, I also cant enable the Transfer Acceleration for ANY bucket even though I am the Root admin of the account inside the AWS Management Console.

Having same issue. Tried both adding full admin access to service account and enabling transfer acceleration. Interestingly dev environment is deployed without any issues but prod always fails at this step. Environments are identical except few different ENV vars. In my case issue seems like related to cloud frount:

at Request.extractError (/root/.//node_modules/@sls-next/aws-cloudfront/node_modules/aws-sdk/lib/protocol/rest_xml.js:53:29)

In my case this was account limitation issue. Recently my account was blocked due to suspicious activity. Account have been restored but seems like completely and I still don't have full rights to my own cloudfront distribution and this is causes Amplify deployment error.

So how you solve this ya? Like where can I get help? I also have a same problem too

@kenchoong I have contacted AWS support twice. First time they added permissions to enable  transfer acceleration and second time they lifted account restrictions after suspension. It took quite some time for them to fix issue though....

@aleksey-shmatov I see. thank you very much. Seems like no other way except contact the support.

https://github.com/aws-amplify/amplify-hosting/issues/1876

We have since resolved this issue so closing this.


feat: add linode SD failure count metric

This commit introduces a new metric to count the number of failed
requests to Linode's API when using Linode SD. Resolves #10672, inspired
by #10476.
Note: this doens't count failures when polling the /account/events
endpoint, as a 401 there is how we determine if the supplied token has
the needed API scopes to do event polling vs full refreshes each
interval.
Signed-off-by: TJ Hoplock<EMAIL_ADDRESS>

cc: @roidelapluie for when you get time!

LGTM, can you please fix linting?

My bad! Should've let CI runs complete before pinging you, sorry about that. I'll fix it!

TIL about local package prefixes for goimports command.
Looking better now, @roidelapluie!


Automated resources generation

Automated resources generation: before execution  of :backend:run task :frontend:browserDistribution task is run and its result is placed in :backend subproject resource directory.

This is a bit outdated


DirectoryAnalyzer and DirectoryWatcher should support multiprocessing (or GPU)

I love the idea of DirectoryAnalyzer and DirectoryWatcher. But as long as they only run on one CPU core, it's not usable for me.
I helped myself with multiprocessing, but that should only be a temporary solution as long as it isn't built directly into birdnetlib.
import os
from multiprocessing import Pool
from birdnetlib import Recording
from birdnetlib.analyzer import Analyzer
from datetime import datetime

def analyze_file(file_path):
    try:
        analyzer = Analyzer()
        recording = Recording(
            analyzer,
            file_path,
            lat=35.4244,
            lon=-120.7463,
            date=datetime(year=2022, month=5, day=10),
            min_conf=0.25,
        )
        recording.analyze()
        print(f"Processed {file_path}")
        return recording
    except Exception as e:
        print(f"Error processing {file_path}: {e}")
        return []

def analyze_directory(dir_path):
    pool = Pool()
    recordings = []
    for subdir, dirs, files in os.walk(dir_path):
        for file in files:
            if file.lower().endswith(".wav"):
                file_path = os.path.join(subdir, file)
                recordings.append(pool.apply_async(analyze_file, args=(file_path,)))
    pool.close()
    pool.join()

    return recordings

if __name__ == "__main__":
    dir_path = "./workdir/"
    recordings = analyze_directory(dir_path)
    print(f"Finished analyzing {len(recordings)} files")

Thanks @LimitlessGreen for opening the issue and providing your workaround. I've edited the title to represent the change that needs to be made to the project.
I plan to add support for multiprocessing, but it will need to be optional. I know of several users that require one-process-per-analyzer as part of their larger workflow. In those cases, the workflow uses a database as the queue, and runs a separate process for each available CPU core. I believe the rationale for that requirement is to isolate out-of-memory failures from taking down the entire queue.
Adding multiprocessing to DirectoryAnalyzer should be the next step in this issue. Would an interface like the following work for your use case?
directory = DirectoryAnalyzer(
    "/Birds/mp3_dir",
    patterns=["*.mp3", "*.wav"],
    processes=4
)
directory.on_analyze_complete = on_analyze_complete
directory.process()

Adding multiprocessing to DirectoryAnalyzer should be the next step in this issue. Would an interface like the following work for your use case?
directory = DirectoryAnalyzer(
    "/Birds/mp3_dir",
    patterns=["*.mp3", "*.wav"],
    processes=4
)
directory.on_analyze_complete = on_analyze_complete
directory.process()

Yes, this interface looks good to me :octocat: :smiley:

This feature is currently parked. The issue is that data has to be pickleable when passed in and out of the Pool child processes. This isn't possible at the moment in the DirectoryAnalyzer class because the process method needs access to the Analyzer class(es) in particular. Analyzer loads and runs the actual TensorFlow models, and the initializing takes a non-trivial time on start-up.
Currently, in birdnetlib analyzer start-up only happens once no matter how many audio files are eventually processed.
In the example above, the data passed to and returned from analyze_file is indeed pickleable, but at the cost of having to initialize the Analyzer with every single file.
Notes to read up on:
https://towardsdatascience.com/multiprocessing-and-pickle-how-to-easily-fix-that-6f7e55dee29d
Also see: https://pypi.org/project/dill/
I'll investigate more as time allows.
As a current workaround, one option is to split up large groups of files or use a centralized queue, then run multiple instances of python with something like supervisor.

On a second look, this is possible by using a Manager.Queue for the file path listing. Each process will still have to initialize Tensorflow and the Analyzer class(es), but that is preferable to the current method.
My thoughts are to add a dedicated DirectoryMultiProcessAnalyzer, as I'm inclined to leave DirectoryAnalyzer as a single process implementation.

More notes to myself on this issue ...
The primary use-case for DirectoryWatcher was to provide an easy implementation for handling files as they were recorded. I'm planning to leave DirectoryWatcher as a single-process, since even an older Raspberry Pi 3 can keep up with realtime recording saved into a watched directory.
The implementation for DirectoryMultiProcessAnalyzer is in PR #52. I'll take another review of it later this week, then package it for release in 0.5.

And here's an example of using DirectoryMultiProcessAnalyzer.

DirectoryMultiProcessAnalyzer has been released in 0.5.0.
I'm open to reviewing a PR for a multiprocessing implementation of DirectoryMultiProcessWatcher down the road. If anyone is interested in taking a stab at it, please open a new issue before submitting the pull request.


Shaking the reference assemblies tree

@AArnott Best I can figure, the registry of assemblies used during code generation is only ever glommed on to. That is to say, the list is read, the loaded assemblies are appended, and it is re-saved. See LoadAssembly and SaveGeneratorAssemblyList. As internal and external dependencies adjust, grow, shrink, etc, over the course of a project, does this tree ever get shaken in order to discard stale code generation references?

Possibly GetLastModifiedAssemblyTime is working around that concern. However, I am pretty sure the assembly registry would still grow and never be shaken.

Are you asking about incremental build and the inputs/outputs check that it does?
If so, you're quite possibly right. It's been so long since I looked at it. I probably didn't consider it a priority to make sure that incremental build over the long haul (with no clean/rebuilds along the way) didn't grow heavy on the regenerate-unnecessarily side.

We're talking about a list, not a tree
GetLastModifiedAssemblyTime is only used to regenerate the file if the source didn't change but generators did.
Nothing is discarded.

It probably requires fixing. Although whole incremental build support definitely needs fixing, and quite honestly, while using the generators I've come to a conclusion that the incremental build just doesn't work for me. Often my code generation depends not only on a given source file, but also on the base  class's source, leading me to conclusion that it should be at least possible to completely disable incremental-ish computations inside the tool, or even drop them completely since they have so many issues.
The tool is invoked anyway, and the time of doing the Compilation is considerable. It'll probably need profiling, but I'd guess that creating Compilation (loading all Roslyn DLLs etc.) is considerably more expensive than any following code generation.
Sorry, I think I derailed. 😅😇👀

@amis92 No worries. I'm recasting the execution on the implementation anyway, so this is quite informative, something I can keep my eyes open for.


Email message options

With the changes to the email quarantine nightly email, I'm not certain if the 'release' button in email reports the message to Microsoft or not (there is no report option anymore). Can you clarify if people can use the release button in email to send feedback to Microsoft or if they have to go into the portal to review and release to have the option to provide feedback?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2cfc15d1-833b-3139-6bc0-1acb97d507c8
Version Independent ID: 31e24618-c2e6-4fe4-9651-04d6007548a4
Content: Find and release quarantined messages as a user in Office 365 - Office 365
Content Source: microsoft-365/security/office-365-security/find-and-release-quarantined-messages-as-a-user.md
Service: o365-seccomp
GitHub Login: @chrisda
Microsoft Alias: chrisda

@cchambers-cap, you are correct. The Release button does just that with no reporting. For reporting, you need to click Review to go into the Quarantine portal in the Security & Compliance Center where users can release and report their messages.

@cchambers-cap - Thank you for submitting feedback.
From our understanding, the issue you raised has been answered by chrisda  so we will close this issue.
Thank you for your contribution to make the docs better! Much appreciated!


Add verification config types from RFC

Currently, the Subject enum types use both #[serde(deny_unknown_fields)] and #[serde(flatten)] , which is not supported, and is supposed to fail on deserialization. Strangely, it works here, as the tests pass.
Also tested by consuming verify::read_verification_file() from a simple crate, instead of consuming this branch from policy-server and policy-evaluator.
Part of https://github.com/kubewarden/policy-server/issues/142.

Force pushed to rebase main changes.

rebased main again.

Please re-review, I found out I misread the RFC, and I wasn't covering the case where any_of is not listed.

Superseded by https://github.com/kubewarden/policy-fetcher/pull/54


Update ws dependency

Summary

Our ws dependency is super outdated, and is insecure. It is used for the websocket example code in RNTester. This PR updates the dependency, and removes undefined console.logs.
Changelog

[General] [Security] - Updates ws dependency to 6.4.1
Test Plan

run RNTester & ensure works, check integration test too.

And more specifically addresses this comment: https://github.com/react-native-community/discussions-and-proposals/issues/64#issuecomment-444822438

CLI, Metro and Detox depends on ^1.1.x.
They should be also updated. Doesn't they?
Multiple different versions makes bundle bloated, because every package have own dependencies versions bundled within.
When will be WS updated for RN it should be updated for CLI and Metro also. But Detox is little bit complicated because there was initiation for update, but it breaks theirs Jenkins CI.

I honestly think that we should not even have ws as a dependency in this repo for the only purpose of serving up a demo in RNTester, seems a little silly. Could we perhaps just remove instead?

I would also be happy to find another way to deal with the websockets, it’s a very small surface area.

@ericlewis i agree with you. I also think if it is not possible it should be moved into devDependencies because it is not necessary in production. Or is it required for CI with --production flag?