Unable to Deploy Stacks if Image Didn't Previously Exist

Rancher Versions:
Server: 1.5.4
healthcheck: rancher/healthcheck:v0.2.3
ipsec: rancher/net:v0.11.2
network-services: rancher/network-manager:v0.6.6, rancher/metadata:v0.9.1
scheduler: rancher/scheduler:v0.7.5
Docker Version:
1.12.6
OS and where are the hosts located? (cloud, bare metal, etc):
Ubuntu 16.04 on AWS
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
Single node rancher master
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
cattle
Steps to Reproduce:
Deploy a stack with a service that has an invalid docker image tag with rancher-compose, such as a typo or incorrectly generated tag.
Attempt to re-deploy stack with corrected docker image tag.
Results:
With the incorrect docker image tag, Rancher will report the image doesn't exist and fail to launch the stack. However if the stack is re-deployed with a known working docker image tag it never fully starts and hangs indefinitely. Deleting and re-deploying the stack does not fix the issue. Manually starting a service via the UI does bring the service up, but deploying via rancher-compose does not work.
It appears that the incorrect image continually sticks around in the delayed process tab. This seems to affect all stacks deploying, even those without the incorrect image tag.

Logs in rancher-server show continual errors for the incorrect image tag
2017-04-11 23:17:40,796 ERROR [:] [] [] [] [ecutorService-1] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16963
2017-04-11 23:17:40,801 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16962
2017-04-11 23:17:41,735 ERROR [7204d830-d429-4023-89fa-d28f8bc9b10d:16964] [genericObject:260] [genericobject.create] [] [ecutorService-2] [c.p.e.p.i.DefaultProcessInstanceImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16964
2017-04-11 23:17:41,759 ERROR [:] [] [] [] [ecutorService-2] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16964
2017-04-11 23:17:41,805 ERROR [a1bcfa78-4b62-4bdd-ab7d-d9a0c18966ca:16966] [genericObject:259] [genericobject.create] [] [ecutorService-3] [c.p.e.p.i.DefaultProcessInstanceImpl] Agent error for [compute.instance.pull.reply;agent=2]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16966
2017-04-11 23:17:41,831 ERROR [13f37547-66fa-4610-be72-dd4988cc6a49:16965] [genericObject:261] [genericobject.create] [] [ecutorService-1] [c.p.e.p.i.DefaultProcessInstanceImpl] Agent error for [compute.instance.pull.reply;agent=2]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16965
2017-04-11 23:17:41,843 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=2]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16966
2017-04-11 23:17:41,860 ERROR [:] [] [] [] [ecutorService-1] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=2]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16965
2017-04-11 23:17:42,771 ERROR [04059b48-cd81-4512-a0c5-2fea5ac76e1c:16967] [genericObject:262] [genericobject.create] [] [ecutorService-2] [c.p.e.p.i.DefaultProcessInstanceImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16967
2017-04-11 23:17:42,794 ERROR [:] [] [] [] [ecutorService-2] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-api:develop-bed50capull-16967
2017-04-11 23:18:09,692 ERROR [fca6d757-e1de-4876-8b92-6887a6ed4133:16692] [genericObject:241] [genericobject.create] [] [ecutorService-3] [c.p.e.p.i.DefaultProcessInstanceImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-web:d8bd03b-newrelic-developmentpull-16692
2017-04-11 23:18:09,726 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Agent error for [compute.instance.pull.reply;agent=13]: Error response from daemon: No such image: demandbase/dashboard-web:d8bd03b-newrelic-developmentpull-16692

rancher-compose command used to deploy:
rancher-compose --verbose \
        -f docker-compose-rancher.yml \
        --env-file .env.export \
        -p stack-name \
        up \
        --confirm-upgrade \
        --pull \
        -d

From one of the bad genericobjects:
Database changed                                                                                [2/42]
mysql> select * from generic_object where id = 262;
+-----+------+------------+----------+--------------------------------------+-------------+------------+---------------------+---------+-------------+-------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------+------+
| id  | name | account_id | kind     | uuid                                 | description | state      | created             | removed | remove_time | data
                                                                                | key  |
+-----+------+------------+----------+--------------------------------------+-------------+------------+---------------------+---------+-------------+-------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------+------+
| 262 | NULL |          7 | pullTask | b1dad7b8-03c5-4f85-89f4-2c6d6fb31ba9 | NULL        | activating | 2017-04-11 21:57:12 | NULL    | NULL        | {...} | NULL |
+-----+------+------------+----------+--------------------------------------+-------------+------------+---------------------+---------+-------------+-------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------+------+
1 row in set (0.04 sec)

@ecliptik I'll look into what is happing, but this can be avoided by not doing -p.  Nothing has changed in this area for a bit, so not too sure whats up.

@ecliptik I think I'm wrong, can you share what rancher-compose version you are using?

@ibuildthecloud this is a newish install from last week when we split our development master away from production master and now have two endpoints to test newer versions of Rancher before rolling to our production master.
It was originally 1.5.3 but I upgraded it to 1.5.4 last night after it was released. Before 1.5.4 I don't remember seeing this issue. Although we did have problems were stacks would go into upgraded and we had to manually finish them instead of them automatically completing like before.
Version of rancher-compose is v0.12.3

I'm not exactly sure where these incorrect tag names are coming from either, since I don't see anywhere in our Jenkins jobs that adds a pull-17138 or anything of that type to a image tag for the two stacks that are reporting an incorrect image tag name.
It does appear that pull-17138 could refer to PullTaskCreate and process id 17138.

Update from slack convo with @ibuildthecloud  and @ecliptik :
pull-XXX is intentionally added by rancher to make the pull task idempotent. We temporarily tag the image like that and then delete the tag subsequently.
The bug is centered around why we cannot delete the temporary tag after the pull succeeds.

This issue seems to be a regression from https://github.com/rancher/cattle/pull/2593 which was introduced in v1.5.4.  That PR is to address a memory leak caused by how we used topics in Hazelcast.  While addressing that issue we caused a scenario in which a single message can be delivered N times.  During testing this bug was not discovered because we have as a general characteristic of our system is that all operations are idempotent and assume concurrent events will happen.  So for pretty much everything the fact a double message is delivered doesn't matter.  Except..... image pulling from rancher-compose pull or rancher-compose up -p.  It was found that while the operation is idempotent it can not be ran concurrently for the same message.  This was not discovered in our testing unfortunately as its a pretty specific failure scenario.  The scenario to reproduce is to restart the rancher-agent 2-3 times with about 5 seconds in between each restart.  Then do a rancher-compose pull or rancher-compose up -p and it would fail.

Also, just a bit more context.  This issue requires HA to be enabled.  If you run HA w/ multiple servers you get a 1/N chance of hitting this bug, but HA with 1 node is 100% chance of hitting it.

Attempted to re-produce with v1.5.6.-rc1 and wasn't able to. Looks good on our end.

Rancher server: v1.5.6-rc1
HA setup with only 1 node
I don't see the problem when trying to deploy a stack with broken image, and then trying to redeploy the same stack with a working docker image tag.
Also i tested restarting rancher-agent 2-3 times with 5 seconds between each restart, and i can see rancher-compose pull and rancher-compose up -p works.


Instructions

What happened to the instructions on loading your own model and send HTTP requests or curl to the server?

that's moved here


Setting of Stereotypes at Notifications to be checked

To be checked, whether writeAllowed stereotypes at notifications have been properly set

Corrected in UML and YANG (e.g. MacInterface_1.0.0-tsp.200123.1415+yang.4).

Corrected in UML and YANG (e.g. MacInterface_1.0.0-tsp.200123.1415+yang.4).


fix: handle extension with multiple dots

fixes: https://github.com/nvim-tree/nvim-web-devicons/issues/206
I took @gegoune's approach on this comment (https://github.com/nvim-tree/nvim-web-devicons/issues/206#issuecomment-1436045476)
Before

After

(Note that I have set the custom icon on my config for .stories.tsx )

I've done these

[ ]  extract API if appropriate
[x]  resolve merge conflicts
[x]  extract a small recursive function

Looks good; I just cannot remember the original problem. I can't replicate the before/after in the description.
What's the test case? I'm trying icons_by_file_extension
  ["stories.tsx"] = {
    icon = "X",
    color = "#519aba",
    cterm_color = "74",
    name = "StoriesTsx",
  },
``

Steps to reproduce
This is the minimum steps to reproduce the original problem:

Set strict=true on nvim-web-devicon.

require'nvim-web-devicons'.setup {
 strict = true,
}

Install telescope.nvim, one of the most popular fuzzy finder plugin. This plugin supports integration with nvim-web-devicons.
Add foo.test.js in your project. (stories.tsx has nothing to do with reproducing the problem. Apologies for the confusion.)
Search "test.js" by using :Telescope find_files.

Expected behavior:
Since "test.js" is in the icons_by_file_extension table, foo.test.js should be displayed with the test.js icon.

Actual behavior:
foo.test.js appears with js icon, not test.js icon.

You can reproduce this behavior with many other plugins (such as lualine.nvim) that support integration with nvim-web-devicons.
Please note that this is not a issue on nvim-tree because it has already been solved within the plugin itself after this PR (https://github.com/nvim-tree/nvim-tree.lua/pull/423).
My test cases
As I have written as an example in my description, I have tested the following filenames:

foo.ts => Showing ts icon.
foo.rc.lua => Showing lua icon since there is no "rc.lua" in the icon table. (This was raised in https://github.com/nvim-tree/nvim-web-devicons/issues/205 with my original implementation https://github.com/nvim-tree/nvim-web-devicons/pull/199)
foo.stories.ts => Tested if the user configured icons are resolved correctly. This also works fine.
foo.some.long.compound.ext.py => This filename has more than 2 dots, but it resolves to the py icon correctly.

(This is off-topic, but maybe we should have a test code.)
Relevant issue not mentioned above.
https://github.com/nvim-tree/nvim-web-devicons/issues/36

Set strict=true on nvim-web-devicon.

Thank you; that seems to be the bit I was missing.

foo.test.js appears with js icon, not test.js icon.

I cannot replicate this on master. I can replicate it on your fork's master c2c2317 however the test.js and js icons are the same; test.js was updated at 87c3ded
I think I understand now:
local result = ""

local function tst(name, ext)
	local icon, hl = require('nvim-web-devicons').get_icon(name, ext, { strict = true })
	result = string.format("%s, %s (%s) %s %s", result, name, ext, icon, hl)
end

tst("test.js", "js")
tst("test.js", nil)

tst("foo.test.js", "test.js")
tst("foo.test.js", "js")
tst("foo.test.js", nil)

print(result)

master:
test.js (js)  DevIconJs, test.js (nil)  DevIconJs, foo.test.js (test.js)  DevIconTestJs, foo.test.js (js)  DevIconJs, foo.test.js (nil)  DevIconJs
branch:
test.js (js)  DevIconJs, test.js (nil)  DevIconJs, foo.test.js (test.js)  DevIconTestJs, foo.test.js (js)  DevIconJs, foo.test.js (nil)  DevIconTestJs


feat(trajectory_follower): enable keeping steer control until converged by default

PR Type

Improvement

enable keeping steer control until converged by default for checking pull_over geometric parking in scenario tests.
Related Links

https://github.com/autowarefoundation/autoware.universe/pull/1672
Description

Review Procedure

Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[x] Code follows coding guidelines
[x] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[x] Commits are properly organized and messages are according to the guideline
[x] Code follows coding guidelines
[ ] (Optional) Unit tests have been written for new behavior
[x] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets
[ ] Write release notes

CI Checks

Build and test for PR: Required to pass before the merge.
Check spelling: NOT required to pass before the merge. It is up to the reviewer(s). See here if you want to add some words to the spell check dictionary.

@kosuke55 Please change the title. It is not clear.

@TakaHoribe
thanks you are right. I changed the title.

@TakaHoribe
Could you please approve again?


NtpTimeProvider ignores NTP data

Thank you for this nice library!
I happen to run my app in a few devices that don't have properly sync'd clocks, which caused TOTP codes to fail.
It seemed pretty easy, just using NtpTimeProvider should do it!
Unfortunately it didn't work:
Turns out that NtpTimeProvider currently uses timeInfo.getReturnTime() which is the the local timestamp for the reply -- No NTP data is used at all!
The fixed code looks like this (Maybe ntpOffset could be cached?)
    @Override
    public long getTime() throws TimeProviderException {
        long ntpOffset = 0;
        try {
            TimeInfo timeInfo = client.getTime(ntpHost);
            timeInfo.computeDetails();
            if (timeInfo.getOffset() != null) {
                ntpOffset = timeInfo.getOffset();
                Log.i(TAG, "NTP offset to calculate TOTP is " + ntpOffset + "ms");
            } else {
                Log.w(TAG, "Unable to calculate NTP offset");
            }
        } catch (Exception e) {
            Log.w(TAG, "Failed to get NTP offset", e);
        }

        return (System.currentTimeMillis() + ntpOffset) / 1000;
    }

Thanks for raising this issue @paulo-raca  - it's my understanding that getReturnTime will return the remote timestamp that the server recieved the request - is this not the case? Can you link me to some docs explaining what this method actuallty returns?

Would be happy to accept a PR on this btw - but would like to throw a TimeProviderException if there is any failure to obtain the offset/time - thoughts?

Hello, @samdjstevens. Unfortunately this is not the case.
I don't know of any documentation I can point you to, but in the code you can see:
class NTPUDPClient {
    ...
    public TimeInfo getTime(InetAddress host, int port) {
       ...
        long returnTime = System.currentTimeMillis();
        // create TimeInfo message container but don't pre-compute the details yet
        TimeInfo info = new TimeInfo(recMessage, returnTime, false);

        return info;
    }

I've created a PR with these changes

Just tagged & released 1.7.1 with your fix, should hopefully be on Maven Central in a couple of hours. Thanks!


[hack week] Add containerd runwasi wasmtime shims

Proposed Changes

Refactor the build scripts a bit
Add containerd runwasi wasmtime shims
Add RuntimeClass manifests for all supported runtimes (including the autodetected nvidia runtimes)

Note that this currently balloons the k3s build time by several minutes, due to the extra time spent building the rust crates.
Types of Changes
new feature
Verification
Run a WebAssembly image:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-wasi
  labels:
    app: example-wasi
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example-wasi
  template:
    metadata:
      labels:
        app: example-wasi
      annotations:
        module.wasm.image/variant: compat-smart
    spec:
      runtimeClassName: wasmtime
      containers:
      - name: example-wasi
        image: index.docker.io/wasmedge/example-wasi:latest
        imagePullPolicy: IfNotPresent

kubectl logs -l app=example-wasi --tail=-1

Testing
Linked Issues

User-Facing Change
K3s now supports execution of  wasm images via the wasmtime engine.

Further Comments
Caveats:

runwasi also supports wasmedge, but that requires llvm and lld (the llvm linker) which isn't available on s390x, so I have skipped it for now.
Only the wasmtime runtime works, wasmtimed (single runtime process shared by all the pods) appears to be broken:
E0201 23:38:33.301847      53 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err=<rpc error: code = Unknown desc = failed to create containerd task: failed to start shim: start failed: containerd-shim-wasmtimed-v1: Ttrpc(RpcStatus(code: NOT_FOUND message: "/runwasi.services.sandbox.v1.Manager/Connect is not supported")): exit status 1: unknown > pod="default/example-wasi-6b754d67db-htqwc"
I'm poking upstream about that.

Hi @brandond
Very excited to see k3s supporting Wasm containers!
I am with the WasmEdge project. Just to clarify that WasmEdge has no runtime dependency on LLVM. In fact, one could build WasmEdge using GCC, which is available on s360x. We are using LLVM for AOT compilation. But the "Wasm image" typically only contains the Wasm bytecode (for interpreter) or the AOT compiled native code (done on another machine). In either case, the deployment machine does not need LLVM.
One of the unique features of WasmEdge is that it enables developers to build complete microservices -- HTTP servers, web service clients, pooled database connections, messaging queue clients etc. All using libraries they are already familiar with (eg tokio / hyper on Rust, node / fetch() on JavaScript). See examples here:
https://github.com/docker/awesome-compose/tree/master/wasmedge-mysql-nginx
https://github.com/docker/awesome-compose/tree/master/wasmedge-kafka-mysql
WasmEdge provides first-class support for AI inference frameworks, such as Tensorflow, PyTorch, and OpenVINO.
https://github.com/second-state/WasmEdge-WASINN-examples
Finally, WasmEdge can run JavaScript apps. It supports many node.js APIs and even AI inference APIs.
https://wasmedge.org/book/en/write_wasm/js.html
I could go on ... We would love to see WasmEdge supported in k3s. Thank you!
Michael Yuan - maintainer of WasmEdge

@juntao thanks for stopping by! Do you have any tips on building without a working llvm/lld for the target arch? We aim to ship static binaries for arm/arm64/amd64/s390x and I was having a very hard time finding a distro with working static libs for lld on s390x. My preference would be to build in either buildroot or alpine, given that's where we're doing our existing work.

Do you have any tips on building without a working llvm/lld for the target arch?

Hi, the only thing you need is to set WASMEDGE_BUILD_AOT_RUNTIME=OFF in the cmake stage. This will disable the LLVM/LLD dependencies.

@hydai thanks! I was looking at https://wasmedge.org/book/en/contribute/build_from_src.html and https://wasmedge.org/book/en/contribute/build_from_src/linux.html but it wasn't clear from there that turning off WASMEDGE_BUILD_AOT_RUNTIME eliminates the LLVM dependencies, but that makes sense.
It looks like I'll need to use a gnu toolchain for rust on arm and s390x, so I'lll probably have to move the runwasi build out to another pipeline... which is fine, as they were probably too time consuming to do in the main K3s build anyway.

It looks like I'll need to use a gnu toolchain for rust on arm and s390x, so I'lll probably have to move the runwasi build out to another pipeline... which is fine, as they were probably too time consuming to do in the main K3s build anyway.

I am not familiar with the s390x arch. Please let me know if you encounter any issues :-)
We(WasmEdge) would like to support more arches if possible.

@hydai I have given up on s390x due to lack of rust support for musl libc on that platform, but running into problems building for 32bit arm. I opened https://github.com/WasmEdge/WasmEdge/issues/2254 if you wouldn't mind taking a look.

Hi @brandond
Thanks for the report. I am going to dig into this issue.
I have a question: Does k3s use musl libc only? If so, I should also need to check the compatibility between our current implementation (based on glibc) and the musl libc.

@hydai yes, we use musl for everything. Our "root" (userspace binaries that ship with k3s) are derived from buildroot, while the golang bits are built on alpine. Everything is statically linked.

It sounds like it would be helpful if runwasi provided prebuilt bins with some variety of configurations. What would help most? We'd be happy to help make this process easier.
Also have you consider the spin and slight shims via https://github.com/deislabs/containerd-wasm-shims? We are including these in AKS and in Cluster API (CAPZ documentation is forthcoming, but the bits are in image-builder via https://github.com/kubernetes-sigs/image-builder/pull/1037).

@devigned yes that would be great. I ended up moving the build portion of this out to a couple patches on top of a fork to the upstream repo at https://github.com/k3s-io/runwasi. Unfortunately I ran into multiple issues with arch support and producing statically linked builds with musl; it appears that all we can reliably build at this point is wasmtime on amd64 and arm64.
With respect to other shims, I might take a look at those at some point. I look forward to the ecosystem settling down a bit so there aren't a half-dozen-odd competing runtimes each with their own weird build requirements.


Ceasar cypher

Added Cryptography category and the Caesar Cipher algorithm.
Here's a sample of the encryption part:

There's decryption as well, I just cut it from the gif to not make it too long. First tracer encrypts the string and then the second one decrypts it back to the original.
Since the algorithm requires to update the array (by rotating characters in-place) while traversing it, I added code (the setValue function) to set a single value in Array1DTracer, Array2DTracer and Tracer, since there was previously only the setData function which sets all the data. I figure this feature will be useful for any algorithm that needs to make updates while executing.
Another thing, since this algorithm uses strings instead of numbers in the arrays, I refactored the calls to refineNumber to call refinePrimitiveByType which will perform the needed operation based on the argument type (only numbers and strings for now).

Side note, let me know if you guys don't want me to apply js-beautify whenever I make a pull request. I figure it doesn't hurt but still want to hear your thoughts on it. Perhaps we should add a linter (js-hint or es-lint) as part of the project to standardize code formatting?

I just saw that tracer._notify already had the functionality to update individual values via second argument.
I will close this pull request and remove the changes related to setValue before making a second pull request.

I love that Cryptography category is added!
I think we need to put the documentation beside the scratch paper to clarify the methods.
For refinePrimitiveByType method it makes much sense, but how do you think about renaming it to refine since it is going to be used a lot?
Thanks for your work!

Yessir, documentation, at least for the main methods, would definitely be useful.
As far as refinePrimitiveByType, the reason I like that (or something as verbose) better is because useful function names eliminate (or at least reduce) the need for documentation since the name itself describes what the code does.
I would go furthermore and suggest to update some other functions and variables with more meaningful names: for example, tm inside /js/module/tracer.js can be called manager, or even tracerManager. A few extra characters don't hurt the code efficiency but can make a difference when somebody new is looking at the code for the first time.
Of course, adding modules would make all of this much cleaner and easier, and remove the global space pollution that currently exists 😉

You are totally right.
I will see if there is any other variable with unclear name such as tm.
Also, I will work on rewriting in ES6 soon.
Thank!


LIME-508 - Added DVA ThirdParty Endpoint to ParameterStore Parameter …

What changed
Added DVA ThirdParty Endpoint to ParameterStoreParameters and ConfigurationService
Why did it change
To be able to use the DVA endpoint parameter when needed

Closing as this is out of date


firebase.configure crash in second target in workspace

[REQUIRED] Step 1: Describe your environment

Xcode version: _____ 11.0
Firebase SDK version: _____Latest
Firebase Component: _____Messaging, instanceID, analytics
Component version: _____Latest
Installation method: CocoaPods | Carthage | Zip file (CocoaPod)
  pod 'FirebaseInstanceID' pod 'Firebase/Analytics' pod 'Firebase/Messaging'

[REQUIRED] Step 2: Describe the problem
Steps to reproduce:
I have two targets in the workspace, one of my targets that i implemented firebase in first works fine, the  crashes giving this log
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[FIRInstanceIDCheckinPreferences preferencesFromKeychainContents:]: unrecognized selector sent to class 0x10f525108'
Inside this function : (FIRInstanceIDCheckinPreferences *)cachedCheckinPreferences
This file: FIRInstanceIDCheckinStore.m

I have same problem ,too
I was use firebase version 6.14.0, second target was fine, but after I update to 6.17.0,
app will crash after launch.
btw, I use   auth、 storage 、 analytics、 dynamicLink、 message


feat/category management

feat: add category-form component
feat: add category-management component
feat: add supplier-account component
feat: add supplier-account component
feat: add new routes

:tada: This PR is included in version 1.7.0 :tada:
The release is available on:

1.7.0
GitHub release

Your semantic-release bot :package::rocket:


Discriminated extended interfaces

Discriminated types are great, however they work only when the parent type is defined as union of child types. There is a disadvantage that all shared properties needs to be declared in all child types.
Can we extend it to support the following case, where the parent type is also an interface?
TypeScript Version:  2.0.3
Code
interface Shape {
    type: string;
    color: string;
}

interface Circle extends Shape {
    type: "Circle";
    radius: number;
}

const shape = {
    type: "Circle",
    color: "red",
    radius: 5,
} as Shape;

if (shape.type === "Circle") {
    console.log(shape.radius);
}

Expected behavior:
No error.
Actual behavior:
Property 'radius' does not exist on type 'Shape'.

That would require the type-checker to linearly search through every type it has ever indexed to see if they extend the Shape interface, or keep track of all type hierarchies of each type, the first will hugely impact performance and the second will probably choke the RAM.
However there can be an alternative. If somehow you tag Shape in a way that reflects it's subtypes then this could be implemented without any impacts. This would make the language more complex though.
I'm guessing you should embrace for Working as Intended 😅 😅

Yeah I can implement it by having separate parent interface and union type. But it doesn't look clean.

Not to mention a type from some other union could show up with a Circle discriminant and now your code breaks, which is hardly ideal.


Added support for functions in addition to strings in the relativeTime object

More information at evoL@c7edf55e4b02d878e497df14284b8e44e5ccfc91.
Also I've updated the Polish language file to reflect those changes.

Hmm, I think other languages will likely need something similar to this. There's no way of knowing what every language will need, so we should probably pass as much information to the function as possible. I'm thinking of adding the key as well to the token functions.
function(number, key, withoutSuffix) {
    ...
}

Then, you could do something like this :
var reusable = function(number, key, withoutSuffix) {
    switch(key) {
        case 'm':
            return withoutSuffix ? 'minuta' : 'minutę';
        case 'mm':
            return plural(number) ? 'minuty' : 'minut';

    }
}
...
relativeTime : {
    ...
    m : reusable,
    mm : reusable,
    ...
},

I think this might be a little more readable.

Thanks, I've cleaned up my language file according to the docs.


Crash on workspace selection

First of all: very useful package, thanks!
But there seems to be a slight problem (at least on my system):
When I open up the TODO pane with the default scope "workspace", Atom first uses up all of my RAM and then crashes. Searching for TODOs in open files or in the current file works well, it is just the search in the whole workspace that causes problems. When I select one of the working options, the RAM goes back to normal and all works well. When I then select workspace again, the problem shows again.
Out of curiosity: What exactly do you define as "workspace"? I don't really get it, is workspace the same as the current project? And that again would be the current folder I am working in, right?
I have no clue on what is causing the Issue, but I would really appreciate a fix.

Thank you.
By workspace we mean the currently opened project(s) root and up. If you have a veery huge codebase, opened a large number of projects or have some binary files that does not get marked as binary in your opened project we have seen that the search can take a long time and even make Atom non-responsive.
Inside todo-show we use the same mechanism as used by find-and-replace to search for tags like TODO and FIXME. If you open the project you are having problems with and use Project Find: Show you are probably seeing somewhat the same behaviour (although less severe, because you only search for one keyword).
We are working on a way to search for all tags with a single roundtrip / regex query (and still have the regex customisable), which would majorly improve this issue for very large projects.

Next release has a major performance improvement, which should help on what you are observing. I will close this issue and reopen if requested again.


Question about the train step of DDPM schedule

Why your step is set as 10, I mean, it is so small, how you choose to set this param? Thanks

Good question - Using only 10 denoising steps allows our policy to run on edge compute (NVIDIA Jetson Orin). The minimal quality gain in the actions by adding more denoising steps (~100) was not worth it for our task.


flarectl: fix duplicate alias in zone subcommand

Fixes: #1465
Description
check and certs were both aliased to "c". Changed certs alias from "c" to "ct".
This would've been way more easier to debug if a log were introduced before os.Exit(1). I'd be happy to add it here but I'm unsure if not logging is the intended behavior.
https://github.com/cloudflare/cloudflare-go/blob/c3ac69e2e73752b34b57aeba74e6a3fbb9cbc600/cmd/flarectl/flarectl.go#L715-L718
Just checked v0.82.0 where the zone command worked and zone c use the create command instead of the certs command. It looks like the c alias was changed from certs to create since the create command was added (d0f7110d28f7c9f322c104b64a432923074216c5 )
Has your change been tested?
Yes, flarectl zone now works
Screenshots (if appropriate):
Not appropriate
Types of changes
What sort of change does your code introduce/modify?

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[ ] This change is using publicly documented in cloudflare/api-schemas
and relies on stable APIs.

This would've been way more easier to debug if a log were introduced before os.Exit(1). I'd be happy to add it here but I'm unsure if not logging is the intended behavior.

this is an unfortunate side effect of the design of the CLI tool here. all errors end up caught here and duplicated in the output since the commands will return an error.
thanks for fixing this one!


Fix not being able to update entities

Description:
Note: This is a copy of #19344, but against dev branch, all credit for the fix goes to @glentakahashi
When editing an entity in the frontend dialog, pressing save causes a "save failed: Entity is already registered" error. This is because the frontend always sets name and new_entity_id in the websocket command even if they haven't been changed. This adds a check that the new_entity_id is actually different from entity_id before erroring that the new_entity_id is already registered.
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.

The same issue exists for other components (tested with switch.mqtt), and also if the entity has been autodiscovered instead of added manually.


CORS Issue

Library

[ ]<EMAIL_ADDRESS>or<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>
Description
Trying to generate token from registered app to access another api. Permission is granted in active directory app registration.
http://localhost:3001/ added to redirect URI under SPA but getting following error
"'https://login.microsoftonline.com/xxxxxx/oauth2/v2.0/token' from origin 'http://localhost:3001' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource"
Source
const msal = require('@azure/msal-node');

 const config = {
  auth: {
  clientId: "sdsdwewewew-b98c-4bd4-a3d7-c3da07753a52",
  authority: "https://login.microsoftonline.com/xxxx.onmicrosoft.com",
  clientSecret: "Uxpsdwwewewewewo_4HqFk",
  redirectUri: "http://localhost:3001/"
 },
system: {

  piiLoggingEnabled: false,
  logLevel: msal.LogLevel.Verbose,
},
cache: {
  cacheLocation: "localStorage", // Needed to avoid "User login is required" error.
  storeAuthStateInCookie: false // Recommended to avoid certain IE/Edge issues.
  }

}
const authenticate = (): any => {

  const cca = new msal.ConfidentialClientApplication(config);
   const clientCredentialRequest = {
   scopes: ["api://zsdsdsdsdsd/.default"],
 };

 cca.acquireTokenByClientCredential(clientCredentialRequest).then((response: any) => {
   console.log("Response: ", response);
 }).catch((error: any) => {
   console.log(JSON.stringify(error));
 });

}
export default authenticate;

[ ] Internal (Microsoft)
[ ] Customer request

@tharaka86 Since you are using @azure/msal-node and have a ConfidentialClientApplication, you shouldn't need to register your redirect URI as a SPA. If your app is a SPA application, you should be using the @azure/msal-browser library. If you still run into CORS issues we can investigate further.

msal-browser

@pkanher617 I do have the redirect URL set up as SPA. With msal-browser, can I get a token as the app without user involvement.
My scenario is app getting a token for it self which is valid to access another API
Thanks

@tharaka86 The app model is important here. Is your app a Single-Page application, with no server-side component? Or does it communicate with a backend server that will be retrieving data?
If it is the former, there are no confidential client flows available, you will be required to get a token with user interaction. If it is the latter, you can use the confidential client flow on that backend server to retrieve tokens.
Browsers cannot be confidential clients, as they cannot keep secrets. They can only be used to access web APIs on behalf of a user. See here for more info.

@tharaka86 The app model is important here. Is your app a Single-Page application, with no server-side component? Or does it communicate with a backend server that will be retrieving data?
If it is the former, there are no confidential client flows available, you will be required to get a token with user interaction. If it is the latter, you can use the confidential client flow on that backend server to retrieve tokens.
Browsers cannot be confidential clients, as they cannot keep secrets. They can only be used to access web APIs on behalf of a user. See here for more info.

@pkanher617  this is for a single page app that tried to call back end api. for it to access backend api, im tryig to generate token

this is for a single page app that tried to call back end api. for it to access backend api

@tharaka86 I think there is a misunderstanding here. I'm not asking what resource you are communicating with. I'm asking which component are you trying to authenticate?
Regardless of where the API that you are retrieving data from is located, an application environment restricted to the browser (AKA a Single-Page Application) can only authenticate on behalf of a user - it cannot authenticate the app itself.
If your application is a Single-Page Application BUT it also communicates with a backend server that you own then you can use the backend server that you own to make confidential client requests without user interaction, and then the backend server that you own can make requests to the backend api you are trying to access.
Does this make sense?

this is for a single page app that tried to call back end api. for it to access backend api

@tharaka86 I think there is a misunderstanding here. I'm not asking what resource you are communicating with. I'm asking which component are you trying to authenticate?
Regardless of where the API that you are retrieving data from is located, an application environment restricted to the browser (AKA a Single-Page Application) can only authenticate on behalf of a user - it cannot authenticate the app itself.
If your application is a Single-Page Application BUT it also communicates with a backend server that you own then you can use the backend server that you own to make confidential client requests without user interaction, and then the backend server that you own can make requests to the backend api you are trying to access.
Does this make sense?

@pkanher617  thanks, its clear now. Thanks for the help

this is for a single page app that tried to call back end api. for it to access backend api

@tharaka86 I think there is a misunderstanding here. I'm not asking what resource you are communicating with. I'm asking which component are you trying to authenticate?
Regardless of where the API that you are retrieving data from is located, an application environment restricted to the browser (AKA a Single-Page Application) can only authenticate on behalf of a user - it cannot authenticate the app itself.
If your application is a Single-Page Application BUT it also communicates with a backend server that you own then you can use the backend server that you own to make confidential client requests without user interaction, and then the backend server that you own can make requests to the backend api you are trying to access.
Does this make sense?

@pkanher617 thanks, its clear now. Thanks for the help

Absolutely! If there is nothing else can I close this issue?


havingRaw is not working

I have a very strange issue with a query in laravel 5.
This part:
->havingRaw('MIN(pr.landed_price) > (pro.price + pro.shipping)')

throws an error saying
column not found

This is strange because

the column definitely exists
the query is running with laravel 4 without a problem
I have checked both raw query outputs (L4 & L5) with ->toSql(); and they are completely identical.
Running the query without laravel works, too.

Is there a known bug, anything else I don't see - and how can I solve this?

Give me the version of L4 you tested.
Generated SQL: select price from pro having MIN(pr.landed_price) > (pro.price + pro.shipping)
The pr table isn't included on FROM clausure


[schemas] PlacesApi PlaceAutocompleteStructuredFormat

The PlaceAutocompleteStructuredFormat response looks different than the documentation says
Environment details
PlaceAutocompleteStructuredFormat specification
Steps to reproduce
Documentation

According to documentation can see that secondary_text is required
Result from endpoint

What for Guernsay doesn't make a sense

@wokayme Thank you for opening this issue. 🙏
Please check out these other resources that might help you get to a resolution in the meantime:

Support Console - If you have a support contract.
Discord - chat with other developers
StackOverflow - use the google-maps tag
Google Maps Platform issue trackers

This is an automated message, feel free to ignore.


[camera] Camera starts lagging on android after startImageStream & stopImageStream (A resource failed to call release.)

Camera
Details
On Android, camera preview starts lagging after starting and stopping image stream a few times, and the "await cameraController.stopImageStream()" never completes, and I see this message in the logs "W/System  ( 1327): A resource failed to call release. ".
Here is the video
I'm experiencing this even on camera app example.
Iv'e added a button that starts/stops the image stream. After few tries to start and stop the image stream, the preview starts lagging.
https://user-images.githubusercontent.com/17802092/193770604-0b078425-f82a-480e-a9aa-65fff3126164.mp4
Here is the code to reproduce

Replace this code in the camera sample in the main.dart class (camera 0.10.0+3).
https://gist.github.com/RobertApikyan/f9bc5a212272d955ff8a069de10d7fd4
Run the app on Android device.
Toggle the start/stop stream button a few times.
Finally, left the stream enabled, and you will see the lagging screen similar to video

Platform
Target Platform: Android
Target OS version/browser: Android 11
Devices: Galaxy A70
Logs

Logs
Performing hot restart...
Syncing files to device SM A705FN...
Restarted application in 2,742ms.
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/Camera  ( 5336): close
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client io.flutter.plugins.cameraexample API Level 2
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.close()(CameraDeviceImpl.java:1334) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onCaptureStarted(android.hardware.camera2.impl.CaptureResultExtras, long) for 317ms
I/Camera  ( 5336): open | onClosed
W/Camera  ( 5336): The selected imageFormatGroup is not supported by Android. Defaulting to yuv420
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_OPEN for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): startPreview
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): startPreviewWithImageStream
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 314ms
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/Camera  ( 5336): CameraCaptureSession onClosed
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
W/System  ( 5336): A resource failed to call release.
I/Camera  ( 5336): CameraCaptureSession onClosed
W/System  ( 5336): A resource failed to call release.
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/Camera  ( 5336): startPreview
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 337ms
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/Camera  ( 5336): CameraCaptureSession onClosed
W/System  ( 5336): A resource failed to call release.
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): startPreviewWithImageStream
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 337ms
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onResultReceived(android.hardware.camera2.impl.CameraMetadataNative, android.hardware.camera2.impl.CaptureResultExtras, android.hardware.camera2.impl.PhysicalCaptureResultInfo[]) for 328ms
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/Camera  ( 5336): CameraCaptureSession onClosed
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
W/System  ( 5336): A resource failed to call release.
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/Camera  ( 5336): startPreview
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onCaptureStarted(android.hardware.camera2.impl.CaptureResultExtras, long) for 335ms
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 336ms
I/Camera  ( 5336): CameraCaptureSession onClosed
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): startPreviewWithImageStream
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onCaptureStarted(android.hardware.camera2.impl.CaptureResultExtras, long) for 356ms
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 356ms
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/Camera  ( 5336): CameraCaptureSession onClosed
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
W/System  ( 5336): A resource failed to call release.
W/System  ( 5336): A resource failed to call release.
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/Camera  ( 5336): startPreview
W/libc    ( 5336): Unable to set property "debug.sf.dequeuebuffer" to "1": connection failed; errno=13 (Permission denied)
E/BufferQueueProducer( 5336): ImageReader-720x480f23m1-5336-11 waitForFreeSlotThenRelock: timeout
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 1.300s
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onCaptureStarted(android.hardware.camera2.impl.CaptureResultExtras, long) for 1.231s
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/Camera  ( 5336): CameraCaptureSession onClosed
W/System  ( 5336): A resource failed to call release.
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): startPreviewWithImageStream
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Camera  ( 5336): Updating builder settings
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 348ms
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onResultReceived(android.hardware.camera2.impl.CameraMetadataNative, android.hardware.camera2.impl.CaptureResultExtras, android.hardware.camera2.impl.PhysicalCaptureResultInfo[]) for 340ms
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
I/Camera  ( 5336): CameraCaptureSession onClosed
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
W/System  ( 5336): A resource failed to call release.
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 0
I/ViewRootImpl@657baa0FlutterActivity: ViewPostIme pointer 1
I/Camera  ( 5336): startPreview
W/libc    ( 5336): Unable to set property "debug.sf.dequeuebuffer" to "1": connection failed; errno=13 (Permission denied)
E/BufferQueueProducer( 5336): ImageReader-720x480f23m1-5336-11 waitForFreeSlotThenRelock: timeout
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_IDLE for client io.flutter.plugins.cameraexample API Level 2
I/Camera  ( 5336): CameraCaptureSession onConfigured
I/Choreographer( 5336): Skipped 76 frames!  The application may be doing too much work on its main thread.
I/Camera  ( 5336): Updating builder settings
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=0 in void android.hardware.camera2.impl.CameraDeviceImpl$4.run() for 1.288s
D/Camera  ( 5336): Updating builder with feature: ExposureLockFeature
D/Camera  ( 5336): Updating builder with feature: ExposurePointFeature
D/Camera  ( 5336): Updating builder with feature: ZoomLevelFeature
W/s.cameraexampl( 5336): Long monitor contention with owner main (5336) at void android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle()(CameraDeviceImpl.java:1283) waiters=1 in void android.hardware.camera2.impl.CameraDeviceImpl$CameraDeviceCallbacks.onCaptureStarted(android.hardware.camera2.impl.CaptureResultExtras, long) for 1.222s
D/Camera  ( 5336): Updating builder with feature: AutoFocusFeature
D/Camera  ( 5336): Updating builder with feature: NoiseReductionFeature
I/Camera  ( 5336): CameraCaptureSession onClosed
I/Camera  ( 5336): updateNoiseReduction | currentSetting: fast
D/Camera  ( 5336): Updating builder with feature: FocusPointFeature
D/Camera  ( 5336): Updating builder with feature: ResolutionFeature
D/Camera  ( 5336): Updating builder with feature: SensorOrientationFeature
D/Camera  ( 5336): Updating builder with feature: FlashFeature
D/Camera  ( 5336): Updating builder with feature: ExposureOffsetFeature
D/Camera  ( 5336): Updating builder with feature: FpsRangeFeature
I/Camera  ( 5336): refreshPreviewCaptureSession
I/CameraManagerGlobal( 5336): Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_ACTIVE for client io.flutter.plugins.cameraexample API Level 2
W/System  ( 5336): A resource failed to call release.

Roberts-MacBook-Pro:example robert$ fvm flutter doctor -v
[✓] Flutter (Channel stable, 3.3.2, on macOS 12.5 21G72 darwin-x64, locale en-AM)
• Flutter version 3.3.2 on channel stable at /Users/robert/desktop/projects/flutter_versions/3.3.2
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision e3c29ec00c (3 weeks ago), 2022-09-14 08:46:55 -0500
• Engine revision a4ff2c53d8
• Dart version 2.18.1
• DevTools version 2.15.0
[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
• Android SDK at /Users/robert/Library/Android/sdk
✗ cmdline-tools component is missing
Run path/to/sdkmanager --install "cmdline-tools;latest"
See https://developer.android.com/studio/command-line for more details.
✗ Android license status unknown.
Run flutter doctor --android-licenses to accept the SDK licenses.
See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 13F100
• CocoaPods version 1.11.3
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2021.2)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
[✓] IntelliJ IDEA Community Edition (version 2019.3.5)
• IntelliJ at /Applications/IntelliJ IDEA CE.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin version 193.7547
[✓] Connected device (3 available)
• SM A705FN (mobile) • R58MB1CZHER • android-arm64  • Android 11 (API 30)
• macOS (desktop)    • macos       • darwin-x64     • macOS 12.5 21G72 darwin-x64
• Chrome (web)       • chrome      • web-javascript • Google Chrome 105.0.5195.125
[✓] HTTP Host Availability
• All required HTTP hosts are available
! Doctor found issues in 1 category.

Hi @RobertApikyan
I tested with my two devices and see that there is only lagging at the time start/stop the image stream. I didn't have the same behavior as your demo video (from 0:30-0:35). Are there any other logs in the console at that time?

Pixel 3a, Android 12
Samsung Galaxy Tab A7 Lite, Android 12 OneUI 4.1, SM-T225

May be related/similar based on the logs: #108941

I have it on Samsung galaxy A70 model, I will try to grab some more logs, but in general the main logs are there

@RobertApikyan I think this issue is duplicated of #108941 when its OP has also reproduced it by start and stop image stream, please see: https://github.com/flutter/flutter/issues/108941#issuecomment-1205074639


Clean code comments to match wording and order in readme

Woo-hoo cleanup! I think you'll need to resolve merge conflicts first before I can merge.

@jdan fixed.


Building and running with GPUs

Hi,
I'm trying to get amuse up and running with GPUs but haven't had any success. Specifically I want petar and fastkick to run on GPUs. I've been using this script to build amuse:
#! /bin/bash

module purge
module load foss/2022a
module load CUDA/11.7.0
module load GSL/2.7-GCC-11.3.0
module load Miniconda3/23.1.0-1

#
# Change path below to where you want this installed:
#
export INST_DIR=$HOME/soft/amuse-gpu
#
#
export ENV_DIR=$INST_DIR/amuse-env

echo "This script will install AMUSE in the directory: $INST_DIR"
read -r -p "Are you sure? [y/N]" -n 1
echo
if [[ "$REPLY" =~ ^[Yy]$ ]]; then

	mkdir -p $INST_DIR
	cd $INST_DIR

	conda create -y --prefix $ENV_DIR --copy python=3.10
    conda init bash
	conda activate $ENV_DIR

	conda install -y mpi4py docutils numpy pytest h5py matplotlib scipy astropy pandas seaborn

    # edit from here to instead install amuse from source so we can configure it with GPU eventually
    cd $INST_DIR
    git clone -b feature/galaxy-cluster https://github.com/fredt00/amuse.git
    cd amuse
    pip install --upgrade pip
    pip install -e . --no-cache-dir
    ./configure --enable-cuda
    make framework
    make petar.code
    make bhtree.code
    make fastkick.code
    make halogen.code
    make hop.code
    make fi.code

fi

But it always fails at ./configure, complaining that configure: error: cannot find cuda runtime libraries in /apps/system/easybuild/software/CUDA/11.7.0/lib /apps/system/easybuild/software/CUDA/11.7.0/lib64.
This slightly convoluted installation seems to be the only way to get MPI working correctly for the non GPU installation which only works at runtime if I use miniconda as above.
I tried running just ./configure and then manually editing config.mk to
CUDA_ENABLED=yes
NVCC=/apps/system/easybuild/software/CUDA/11.7.0/bin/nvcc
NVCC_FLAGS=
CUDA_TK=/apps/system/easybuild/software/CUDA/11.7.0
CUDA_LIBS=-L/apps/system/easybuild/software/CUDA/11.7.0/targets/x86_64-linux/lib/stubs -lcuda  -L/apps/system/easybuild/software/CUDA/11.7.0/lib64 -lcudart

And the GPU versions of the codes built successfully. However then running them with this script:
#!/bin/bash -l

#SBATCH -J galaxy-cluster
#SBATCH -o galaxy-cluster.%J.out
#SBATCH -e galaxy-cluster.%J.err
#SBATCH --partition=devel
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=28
#SBATCH --gpus=4
#SBATCH --mem-per-cpu=4000
#SBATCH --time=00:10:00

export OMPI_MCA_rmaps_base_oversubscribe=yes
export OMPI_MCA_mpi_warn_on_fork=0
export OMPI_MCA_rmaps_base_oversubscribe=yes
export OMP_STACKSIZE=128M
export OMP_NUM_THREADS=2
ulimit -s unlimited

module purge
module load foss/2022a
module load GSL/2.7-GCC-11.3.0
module load Miniconda3/23.1.0-1
conda activate /home/oxfd1327/soft/amuse-gpu/amuse-env
nvidia-smi
mpirun python -u $@

And I get the error:
/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/core.py:964: UserWarning: MPI (unexpectedly?) not available, falling back to sockets channel
  warnings.warn("MPI (unexpectedly?) not available, falling back to sockets channel")

**********************************************************

mpiexec does not support recursive calls

**********************************************************
Traceback (most recent call last):
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/channel.py", line 1778, in accept_worker_connection
    return server_socket.accept()
  File "/home/oxfd1327/soft/amuse-gpu/amuse-env/lib/python3.10/socket.py", line 293, in accept
    fd, addr = self._accept()
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oxfd1327/soft/amuse-gpu/amuse/examples/fred/galaxy_cluster_master.py", line 349, in <module>
    main(**o.__dict__)
  File "/home/oxfd1327/soft/amuse-gpu/amuse/examples/fred/galaxy_cluster_master.py", line 207, in main
    cluster = star_cluster(code=petar,code_converter=converter_petar, W0=W0, r_tidal=r_tidal,r_half=r_half, n_particles=N_cluster, M_cluster=M_cluster, field_code=FastKick,field_code_number_of_workers=1,code_number_of_workers=2)
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/ext/derived_grav_systems.py", line 94, in __init__
    self.bound=code(self.converter, mode='gpu',number_of_workers=code_number_of_workers)
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/community/petar/interface.py", line 409, in __init__
    petarInterface(**keyword_arguments),
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/community/petar/interface.py", line 38, in __init__
    CodeInterface.__init__(
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/core.py", line 748, in __init__
    self._start(name_of_the_worker = name_of_the_worker, **options)
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/core.py", line 776, in _start
    self.channel.start()
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/channel.py", line 1962, in start
    self.socket, address = self.accept_worker_connection(server_socket, self.process)
  File "/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/channel.py", line 1782, in accept_worker_connection
    raise exceptions.CodeException('could not connect to worker, worker process terminated')
amuse.support.exceptions.CodeException: could not connect to worker, worker process terminated
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[27418,1],0]
  Exit code:    1
--------------------------------------------------------------------------

Is there anything obviously wrong with this process? Any help would be greatly appreciated!
Cheers,
Fred

Hi Fred,
There are a few things that jump out to me in your scripts:

neither of them load an MPI module,
the compile script loads the CUDA module but the run script doesn't,
you're starting Python using mpirun.

If MPI is available on the machine without a module and that's the one you want to use, then point 1. should be okay.
For 2., this could be causing you to use a different CUDA when compiling (the one from the module) than when running (some other version on the system), and that tends to cause problems. It's important to run in the same environment that you've compiled in.
I think 3. is the cause of the mpiexec does not support recursive calls message. AMUSE uses MPI in a different way than most applications: instead of having many copies of your script running in parallel, there's only one copy, which will dynamically create parallel community code instances as needed within your allocation. So you should start your script without mpirun, AMUSE will call it itself if needed (it has other ways of starting models too).

Thanks for the advice! In terms of your suggestions:

So foss/2022a is actually a bunch of modules, sorry I should have provided the full list:

  1) GCCcore/11.3.0                 4) GCC/11.3.0                      7) libxml2/2.9.13-GCCcore-11.3.0     10) OpenSSL/1.1                     13) libfabric/1.15.1-GCCcore-11.3.0  16) OpenMPI/4.1.4-GCC-11.3.0    19) FFTW/3.3.10-GCC-11.3.0       22) ScaLAPACK/2.2.0-gompi-2022a-fb
  2) zlib/1.2.12-GCCcore-11.3.0     5) numactl/2.0.14-GCCcore-11.3.0   8) libpciaccess/0.16-GCCcore-11.3.0  11) libevent/2.1.12-GCCcore-11.3.0  14) PMIx/4.1.2-GCCcore-11.3.0        17) OpenBLAS/0.3.20-GCC-11.3.0  20) gompi/2022a                  23) foss/2022a
  3) binutils/2.38-GCCcore-11.3.0   6) XZ/5.2.5-GCCcore-11.3.0         9) hwloc/2.7.1-GCCcore-11.3.0        12) UCX/1.12.1-GCCcore-11.3.0       15) UCC/1.0.0-GCCcore-11.3.0         18) FlexiBLAS/3.2.0-GCC-11.3.0  21) FFTW.MPI/3.3.10-gompi-2022a

2 and 3 are both good points. I've removed mpirun and loaded CUDA and now I just get the warning:
/home/oxfd1327/soft/amuse-gpu/amuse/src/amuse/rfi/core.py:964: UserWarning: MPI (unexpectedly?) not available, falling back to sockets channel
  warnings.warn("MPI (unexpectedly?) not available, falling back to sockets channel")

And my code runs, although I don't see any speed up compared to when I configured without GPUs so I'm wondering if it is configured correctly. Do you know of a way to confirm the GPU utilisation? Running nvidia-smi before the python call shows that I am being allocated the requested GPUs but I can't see any information about their usage with seff for example.
In my script petar is called with
self.bound=code(self.converter, mode='gpu',number_of_workers=code_number_of_workers)

Is this the correct way to get petar to use GPUs? I can't see any mention of GPUs in the petar interface files.

PeTar in AMUSE currently doesn’t use the GPU, this would require at least manually modifying the Makefile but probably more modifications.

Ah ok, that makes sense. I've been trying to see if FastKick will run on the GPUs but strangely I get this error every few bridge timesteps:
Traceback (most recent call last):
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/examples/fred/galaxy_cluster_master.py", line 353, in <module>
    main(**o.__dict__)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/examples/fred/galaxy_cluster_master.py", line 268, in main
    integrator.evolve_model(time)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/couple/bridge.py", line 598, in evolve_model
    return self.evolve_joined_leapfrog(tend, timestep)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/couple/bridge.py", line 624, in evolve_joined_leapfrog
    self.kick_codes(timestep / 2.0)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/couple/bridge.py", line 756, in kick_codes
    de += x.kick(dt)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/couple/bridge.py", line 478, in kick
    self.kick_with_field_code(
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/couple/bridge.py", line 516, in kick_with_field_code
    ax,ay,az=field_code.get_gravity_at_point(
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/couple/bridge.py", line 146, in get_gravity_at_point
    return code.get_gravity_at_point(radius, x, y, z)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 168, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 166, in __call__
    object = self.precall()
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 215, in precall
    return self.definition.precall(self)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/interface.py", line 373, in precall
    transition.do()
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/state.py", line 123, in do
    self.method.new_method()()
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 168, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 168, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 168, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 170, in __call__
    result = self.convert_result(result)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/methods.py", line 209, in convert_result
    return self.definition.convert_result(self, result)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/interface.py", line 682, in convert_result
    return self.handle_return_value(method, result)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/interface.py", line 614, in handle_as_unit
    unit.append_result_value(method, self, value, result)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/interface.py", line 70, in append_result_value
    self.convert_result_value(method, definition, value)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/interface.py", line 80, in convert_result_value
    definition.handle_errorcode(errorcode)
  File "/cosma/home/dp016/dc-thom14/soft/amuse-gpu/amuse/src/amuse/support/interface.py", line 586, in handle_errorcode
    raise exceptions.AmuseException(
amuse.support.exceptions.AmuseException: Error when calling 'commit_particles' of a '<class 'amuse.community.fastkick.interface.FastKick'>', errorcode is -3

It seems to happen randomly but usually after the third bridge timestep Any Idea what could be causing this? Is it a GPU configuration problem? It doesn't seem to happen with mode='cpu'


Explain nickname property in version object

Coverage remained the same at 100.0% when pulling 0ef1a43950f14c65a760fe8123b4401e2b315bc3 on paul-v-m:patch-1 into 27a93dc8057ae0b315e90d8e79cb499d27044552 on WhichBrowser:master.

Thanks Paul!


Dolphin specific http headers should begin with "X-"

This would make the use of a reverse proxy a lot easier because usually these headers beginning with X- are automatically forwarded.

Fix with https://github.com/canoo/dolphin-platform/pull/849


DDC attaches redundant member signatures to generic classes

Member type signatures can be inherited from the superclass if they remain unchanged.  This is broken for generics in DDC right now though - the comparison is done on the unsubstituted type, so a redundant signature is added at every subclass level.  See the below Dart code and the generated js: in particular the redundant signature for B.f.
class A<T> {
  T f(T x) {}
  int g(int x) {}
}

class B<T> extends A<T> {
  T f(T x) {}
  int g(int x) {}
}

  tmp__test.A$ = dart.generic(T => {
    class A extends core.Object {
      f(x) {
        T._check(x);
      }
      g(x) {}
    }
    dart.addTypeTests(A);
    dart.setSignature(A, {
      methods: () => ({
        f: dart.definiteFunctionType(T, [T]),
        g: dart.definiteFunctionType(core.int, [core.int])
      })
    });
    return A;
  });
  tmp__test.A = A();
  tmp__test.B$ = dart.generic(T => {
    class B extends tmp__test.A$(T) {
      f(x) {
        T._check(x);
      }
      g(x) {}
    }
    dart.setSignature(B, {
      methods: () => ({f: dart.definiteFunctionType(T, [T])})
    });
    return B;
  });

cc @munificent


Disallow empty strings as identifiers

I expect you meant 'task name' and 'file identifier' (when uploading a file). These are solved.

I expect you meant 'task name' and 'file identifier' (when uploading a file). These are solved.

Yes. Thanks!


Windows. Local directory and system package (?) confusion.

Nuitka is running on the Windows 10 in VirtualBox.
Python installed from the official site, nuitka installed via pip.

python -m nuitka --version

2.0.5
Commercial: None
Python: 3.11.8 (tags/v3.11.8:db85d51, Feb  6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]
Flavor: CPython Official
Executable: C:\Program Files\Python311\python.exe
OS: Windows
Arch: x86_64
WindowsRelease: 10
Version C compiler: ~\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86_64\13.2.0-16.0.6-11.0.1-msvcrt-r1\mingw64\bin\gcc.exe (gcc 13.2.0).

I have two files:

cat main.py

from lib.locale.bar import bar

bar()

cat lib\locale\bar.py

def bar():
        print("Hi")

Build command:
python.exe -m nuitka --standalone --report=report.txt --onefile .\main.py

When running with python "Hi" is printed, when running built version for MacOS and Linux, it's printed also. But windows built version gives an error.

.\main.exe

Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\Temp\OND049~1\main.py", line 1, in <module>
    from lib.locale.bar import bar

ModuleNotFoundError: No module named 'lib.locale.bar'; 'lib.locale' is not a package

Report file is attached.
report.txt
Seems, the problem in the "locale" directory name. Looks like Nuitka confuses it with some system module.
However, python interpreter doesn't gives this error. So, it may be Nuitka problem.

this isn't a nuitka thing but more of a python problem, you'll need an empty init file within locale, next to bar.py

This fixes the issue.
Thanks for additional explanation!


Not Able To Download from Microsoft Store

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

WARNING
All invalid issues will be rejected!!

Before going further

If your problem is a bug with youtube-dl or a request for new site support please report it here

Make sure you are using the latest yt-dlg version (Click the Settings icon and then About to view the current version)

Make sure you are using the latest youtube-dl version (Click the Settings icon and then Update to update to the latest youtube-dl version)

Make sure you searched the bugtracker for similar issues including closed ones

Make sure to read the FAQs file

[x] I think my problem is NOT with youtube-dl
[ ] I've verified and i assure that I'm running yt-dlg 1.X.Y
[ ] I assure that i am using the latest version of youtube-dl
[x] Searched bugtracker
[x] I've read the FAQs file

What is the purpose of your issue?

[x] Bug report
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Please remove any sections between (---) if they are not related to your issue

Bug report
What operating system do you use ?
Windows 10.0.19045 Build 19045
List of actions to perform to reproduce the problem:

Go to https://apps.microsoft.com/store/detail/ytdlg/XP9CCFSWS911F5

Launch Windows Store thru magnet link

Store will open to home page and not download page

Alternatively:

Open Windows Store
Search for yt-dlg as is the webpage
Store says no results found

What is the expected behaviour ?
Windows store allows download
What happens instead ?
App cannot be found on windows store
No adblock or anti-trackers enabled. Is there a direct portable download available on this repo? Thanks!

The app is in Revision in this days and will be available by region soon, you can download the same installer from the GitHub repo the MSI: https://github.com/oleksis/youtube-dl-gui/releases/download/v1.8.5/yt-dlg-20230224.1.msi

Thank you!

The update is live in the Store: https://apps.microsoft.com/store/detail/XP9CCFSWS911F5

When you can check and is OK for you Region feel free to close the issue. Thanks for the feedback!

I imagine it will be ready in a few days for me. Thanks!

Maybe this should be closed by now?

@jonathonchavez You can confirm if this issue has been solved?

Yes. I did the following:

Viewed the app in the store
Installed it
Clicked the gearbox then "Update" to download the CLI backend
Used it to download  a video from YT

So in the store it's viewable, installable, and functional :thumbsup:


Weird error when passing the string '6.1.13'

The library returns with a "[Error: Offset out of range]" error when evaluating the string:
'6.1.13'
To try it:
var mmm = require('mmmagic');
var Magic = mmm.Magic;
var magic = new Magic( mmm.MAGIC_MIME_TYPE );

magic.detect( new Buffer('6.1.13'), function( err, magic ){
  console.log("ERROR:", err );
})

This only seems to happen on my server (not on the local machine) where I have GCC (via devtoolset-3 on CentOS):
gcc (GCC) 4.9.1 20140922 (Red Hat 4.9.1-10)
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

And node: v4.4.0
Interestingly, it only seems to be doing it when the LAST number is a double-digit one. So, 6.3.10 fails, but 6.3.9 or 6.3.2 work.
Any hints?

There was a regression upstream that was fixed post-libmagic v5.26. I have merged the fix that was made upstream now, can you please test the master branch?

Hi,
Confirmed as fixed -- thank you!
Out of curiosity, can you tell by looking at the patch what actually caused the "Offset out of range" error?!?
If not all good -- but yes, please close.
Merc.

Thanks for the confirmation.
I basically just looked upstream and saw some offset-related commits. There was one that was an off-by-one bug and another related to DER parsing. I'm guessing it was one of those commits.
I will publish a new version soon.