Linux-compatible FMU generation

Hi there.
I've been trying to export the BouncingBall example in FmiExport.jl but this fails with the following error:
[ Info: Saving example files at: /tmp/fmibuildjl_test_pT3ZdP
[ Info: [Build FMU] Automatically determined build file at: `/home/cas/tmp/FMIExport.jl/example/FMI2/BouncingBall/src/BouncingBall.jl`.
[ Info: [Build FMU] Generating package ...
[ Info: [Build FMU] Source package is /home/cas/tmp/FMIExport.jl/example/FMI2/BouncingBall, deployed at /tmp/fmibuildjl_axGCIh/merged_BouncingBall
[ Info: [Build FMU] Relative src file path is src/BouncingBall.jl
[ Info: [Build FMU] ... reading FMU template file at /home/cas/.julia/packages/FMIBuild/y2TgX/src/../template/ME/FMU2/src/FMU2_content.jl
[ Info: [Build FMU] ... reading old FMU source file at /tmp/fmibuildjl_axGCIh/merged_BouncingBall/src/BouncingBall.jl
[ Info: [Build FMU] Removing `FMIBUILD_NO_EXPORT_*` blocks ...
[ Info: [Build FMU] ... removed `FMIBUILD_NO_EXPORT_*` blocks.
[ Info: [Build FMU] Adding/removing dependencies ...
  Activating project at `/tmp/fmibuildjl_axGCIh/merged_BouncingBall`
[ Info: [Build FMU]    > Most recent version of `FMIExport` already checked out, is `/home/cas/.julia/packages/FMIExport/Fq22B`.
   Resolving package versions...
    Updating `/tmp/fmibuildjl_axGCIh/merged_BouncingBall/Project.toml`
  [8af89139] + FMICore v0.17.2
  No Changes to `/tmp/fmibuildjl_axGCIh/merged_BouncingBall/Manifest.toml`
[ Info: [Build FMU]    > Added `FMICore`
    Updating `/tmp/fmibuildjl_axGCIh/merged_BouncingBall/Project.toml`
  [226f0e26] - FMIBuild v0.1.15
    Updating `/tmp/fmibuildjl_axGCIh/merged_BouncingBall/Manifest.toml`
  [226f0e26] - FMIBuild v0.1.15
  [9b87118b] - PackageCompiler v2.1.7
  [05181044] - RelocatableFolders v1.0.0
  [6c6a2e73] - Scratch v1.2.0
  [a5390f91] - ZipFile v0.10.1
  [4af54fe1] - LazyArtifacts
[ Info: [Build FMU]    > Removed FMIBuild
  Activating project at `~/tmp/FMIExport.jl/example/FMI2/BouncingBall`
[ Info: [Build FMU] ... adding/removing dependencies done.
[ Info: [Build FMU] ... generating new FMU source file at /tmp/fmibuildjl_axGCIh/merged_BouncingBall/src/BouncingBall.jl
[ Info: [Build FMU] ... generating package done.
[ Info: [Build FMU] Compiling FMU ...
[ Warning: PackageCompiler: This does not look like an official Julia build, functionality may suffer.
PackageCompiler: bundled artifacts:
  ├── Libiconv_jll - 3.674 MiB
  └── XML2_jll - 5.872 MiB
  Total artifact file size: 9.545 MiB
✔ [02m:15s] PackageCompiler: compiling base system image (incremental=false)
✔ [02m:59s] PackageCompiler: compiling nonincremental system image
Precompiling project...
  11 dependencies successfully precompiled in 6 seconds
[ Info: PackageCompiler: Executing /tmp/fmibuildjl_axGCIh/merged_BouncingBall/src/BouncingBall.jl => /tmp/jl_packagecompiler_OKjrQk/jl_RcdzTu
[ Info: PackageCompiler: Done
✔ [01m:18s] PackageCompiler: compiling incremental system image
In file included from /home/cas/.julia/packages/FMIBuild/y2TgX/src/../template/ME/header/FMU2_init.c:19:
/home/cas/.julia/packages/FMIBuild/y2TgX/src/../template/ME/header/FMU2_init.h:104:10: fatal error: WinDef.h: No such file or directory
  104 | #include <WinDef.h>
      |          ^~~~~~~~~~
compilation terminated.
 In template/ME/FMU2_init.h it says the code is windows specific (I'm using Fedora38) and I think that is the problem? Are there any plans to make the fmu-generation linux compatible? Or am I missing something obvious?
EDIT: I have tried generating BouncingBall.fmu on a Windows machine and this did work

This is exactly the problem, currently this only works on Windows because we are using a Windows-library (DLL-specific) in the considered file.
This could be fixed by adding a Linux-equivalent for the FMU2_init.h-file and using the Linux-Version for islinux() == true during export.
Further a solution for this is on todo, but with lower priority, so if this is urgent I have no problem and would appreciate if anyone else wants to contribute :)

Alright, thanks for the clarification. It's currently not very urgent and my C++ skills are not up to the task, but maybe later a solution will be coming your way :)


Tab visibility conditions

Added tab visibility conditions and check on the label

Closing. This fix was handled in another PR.


Fix BUILDPLATFORM redefinition issue in some Containerfiles

Removed the hardcoded ARG BUILDPLATFORM=linux/amd64 from
Containerfile.bpfman-agent{.openshift} and Containerfile.bpfman-operator{.openshift}.
This change addresses an issue where Podman fails to build the images if
BUILDPLATFORM is redefined. By removing the redefinition, the build
process is made compatible with Podman.
This change remains compatible with Docker, which is more lenient with
ARG redefinitions.
I didn't bisect which version of Podman made this a failure; it doesn't occur with:
% podman version
Client:       Podman Engine
Version:      5.0.3
API Version:  5.0.3
Go Version:   go1.22.5
Built:        Tue Jan  1 00:00:00 1980
OS/Arch:      linux/amd64

but does with:
% podman version
Client:       Podman Engine
Version:      5.2.2
API Version:  5.2.2
Go Version:   go1.22.6
Built:        Wed Aug 21 01:00:00 2024
OS/Arch:      linux/amd64

Fixes: https://github.com/bpfman/bpfman-operator/issues/120

@msherif1234 I didn't change either of:

Containerfile.bpfman-agent.openshift
Containerfile.bpfman-operator.openshift

These two files also redefine BUILDPLATFORM twice. Should we change the OpenShift files too?

Is there any issue with my change w.r.t. multi-arch builds?

@msherif1234 I didn't change either of:

Containerfile.bpfman-agent.openshift
Containerfile.bpfman-operator.openshift

These two files also redefine BUILDPLATFORM twice. Should we change the OpenShift files too?

Yes make sense while u there to remove the duplicate as well for openshift version

I've seen the following warning when building the operator and agent images:
WARN: RedundantTargetPlatform: Setting platform to predefined $TARGETPLATFORM in FROM is redundant as this is the default behavior (line 27)

Is this a similar issue?

I've seen the following warning when building the operator and agent images:
WARN: RedundantTargetPlatform: Setting platform to predefined $TARGETPLATFORM in FROM is redundant as this is the default behavior (line 27)

Is this a similar issue?

I was wary of "fixing" this in the same way in case that breaks multi-arch builds. But if you're game we can find out.

@msherif1234 I didn't change either of:

Containerfile.bpfman-agent.openshift
Containerfile.bpfman-operator.openshift

These two files also redefine BUILDPLATFORM twice. Should we change the OpenShift files too?

Yes make sense while u there to remove the duplicate as well for openshift version

Done in https://github.com/bpfman/bpfman-operator/compare/092556513f397958167b9dcf2cf9014eb7a71946..7b708a9045d54bb99f8a2b549f9b91c55aff518d.

I wonder if this PR fixes https://github.com/bpfman/bpfman-operator/pull/122

@Billy99 do you have any concerns with merging this PR?

I pushed changes today - need to test again with docker. But I think we
should augment the CI workflow to test both docker and podman. It’s much
more efficient than manual testing.
On Thu, 5 Sep 2024 at 19:21, Billy McFall @.***> wrote:

@Billy99 https://github.com/Billy99 do you have any concerns with
merging this PR?
@frobware https://github.com/frobware Did you test with docker? I
didn't try podman, but a similar change in bpfman repo causes a problem:
$ cat Containerfile.bpfman
ARG BUILDPLATFORM
FROM --platform=$BUILDPLATFORM redhat/ubi9-minimal AS bpfman-build
:
$ docker buildx build --progress=plain --platform linux/arm64 -f ./Containerfile.bpfman . -t quay.io/billy99/bpfman:test
#0 http://quay.io/billy99/bpfman:test#0 building with "default" instance using docker driver
#1 [internal] load build definition from Containerfile.bpfman
#1 transferring dockerfile: 2.02kB done
#1 DONE 0.0s
Containerfile.bpfman:4
2 |     ARG BUILDPLATFORM
3 |
4 | >>> FROM --platform=$BUILDPLATFORM redhat/ubi9-minimal AS bpfman-build
5 |
6 |     # The following ARGs are set internally by docker/build-push-action in github actions
ERROR: failed to solve: failed to parse platform : "" is an invalid component of "": platform specifier component must match "^[A-Za-z0-9_-]+$": invalid argument
Still playing with it.
—
Reply to this email directly, view it on GitHub
https://github.com/bpfman/bpfman-operator/pull/121#issuecomment-2332373810,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHGOHYAYE55MVMCOISD223ZVCOLXAVCNFSM6AAAAABNPYE3A6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZSGM3TGOBRGA
.
You are receiving this because you were mentioned.Message ID:
@.***>

There is a comment in my bpfman Containerfiles about not using --platform to fill in this ARG, but I didn't write it:
# We do not use --platform feature to auto fill this ARG because of incompatibility between podman and docker
ARG BUILDPLATFORM=linux/amd64

Also, I moved the ARG BUILDPLATFORM to after the FROM statement and it compiled for both docker and podman, but I don't know if for the FROM --platform=$BUILDPLATFORM ...  line, if --platform is set properly or not.
$ cat Containerfile.bpfman
FROM --platform=$BUILDPLATFORM redhat/ubi9-minimal AS bpfman-build

ARG BUILDPLATFORM
:

Today, images are only built when the PR merges (build and push) or if image-build.yaml is updated (build). So your green CI for this PR didn't test docker or podman, unless I misunderstood your comment.

It would have tested Docker because the make build-images step is executed prior to running the integration tests. Therefore, even though images are only pushed after a PR merges, the image build itself is still performed during the CI process before the integration tests are executed.
I updated the CI steps in commit to run:

make build-images
make OCI_BIN=podman build-images

This goals was to ensure we validate the image builds with both Docker and Podman. However, the version of Podman that gets installed from the Ubuntu archives (v3.4) is outdated and doesn't catch the redefinition issue, which only appears in Podman 5.2. As a result, while the CI change is valid, it wouldn't have detected the failure due to the older Podman version.
The CI output shows that podman is now used in addition to docker for the build-images step.
I was trying to avoid manual testing for future PRs by ensuring that make OCI_BIN=podman build-images works on every pull request. It's debatable whether it's worth pursuing without having a means to install newer versions of podman on ubuntu.

/hold
I want to further understand whether we should be setting any buildplatform/targetplaform/targetarch related ARGS in global scope. Perhaps we only need to define these in build stages and rely on the defaults.

/hold
I want to further understand whether we should be setting any buildplatform/targetplaform/targetarch related ARGS in global scope. Perhaps we only need to define these in build stages and rely on the defaults.

/hold cancel
I simplified the change in https://github.com/bpfman/bpfman-operator/compare/9f219922bbdc20f27a92518189a449cc96825ad7..1dcd8577e793acd8813ca2c0ba45eb39aa4957c4.
We still need to decide whether to keep the CI commit that uses both Podman and Docker for building images. At the very least, it provides some CI coverage for Podman compatibility, even though the version of Podman is very outdated.

I wonder if this PR fixes konflux CI issues in #122 /LGTM

Seems like the same issue looking the build failure from: https://github.com/bpfman/bpfman-operator/pull/122/checks?check_run_id=29757802714.

I pulled the CI change out as #130. Given how old the version of Podman is on Ubuntu, I'm doubtful of the value of running Podman per the build-images step. This simplifies this PR to focus solely on fixing day-to-day development when using Podman.

I suspect it's not related. https://github.com/bpfman/bpfman-operator/pull/122 passed multiple integration test cases, then failed on the application one.

I was focused on the integration test failure, which is cleared up now, but there are still two Konflux failures, so those may be related.


fix TextField compile error for haxe 3.2 rc2

https://github.com/openfl/lime/issues/380

Forgot to close this after making the fix, thank you again! :smile:


Show and Edit commands should be able to accept multiple arguments

In other words:

funky foo1 foo2 ... fooN should show the funk definitions for foo1 foo2 ... fooN.
funky -e foo1 foo2 ... fooN should open up the definitions for foo1 foo2 ... fooN (either individually or in the same file) in an editor, ready to be modified.

Closing stale issue.


Maven and heavy dependency requirements make installation impossible

Hi. I don't want to be that guy, but I've been trying all night to get the official jnats client "installed" at all.
Using Eclipse (4.6.1) on Ubuntu 16.04, the Maven dependency in the global repo imports a bundle type dependency:
   <dependency>
  	<groupId>io.nats</groupId>
  	<artifactId>jnats</artifactId>
  	<version>0.7.3</version>
  	<type>bundle</type>
   </dependency>

Eclipse will report "Missing artifact io.nats:jnats:bundle:0.7.3". I followed the trail to Maven Central which has 1.0 as the latest version. The Maven global repo in Eclipse was re-indexed and updated 2 days ago (that was a pain...) so I doubt the issue is a recently replaced version.
I then tried the manual approach:
git clone git@github.com:nats-io/java-nats.git
cd java-nats
mvn install

So I install Maven and everything that entails, then try "mvn install":
...
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ jnats ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 17 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @ jnats ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 46 source files to /home/user/java-nats/target/test-classes
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Some input files use or override a deprecated API.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java uses unchecked or unsafe operations.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Recompile with -Xlint:unchecked for details.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,33] package javafx.scene.input does not exist
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,1] static import only from classes and interfaces
[INFO] 2 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46.019 s
[INFO] Finished at: 2017-02-06T21:43:14-08:00
[INFO] Final Memory: 37M/349M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:testCompile (default-testCompile) on project jnats: Compilation failure: Compilation failure:
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,33] package javafx.scene.input does not exist
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,1] static import only from classes and interfaces

There's a javafx dependency in your test code that's not included in any Maven dependency. Should it be? Is it possible this is just related to me using OpenJDK 1.8?
So I commented out the static import for javafx in test LogVerifier.java, re-ran "mvn install" and now I have everything built.
I have no clue if this is how I should be accessed/importing jnats into my project. modifying the project and manually installing with Maven was the only way I could get the project built. There seems to be a lot of dependency overkill for a client interface to a comparatively straight forward messaging server.
I'm not here to complain. I'm interested in any thoughts on how to properly work with jnats in my project. Thanks.

It's OK, you can be that guy :) especially if I screwed something up in the
latest release (which was Thursday), in which case I'm that guy! ;)
I apologize for your frustration., I'll look at it right now.
On Mon, Feb 6, 2017 at 11:07 PM duhmojo<EMAIL_ADDRESS>wrote:

Hi. I don't want to be that guy, but I've been trying all night to get the
official jnats client "installed" at all.
Using Eclipse (4.6.1) on Ubuntu 16.04, the Maven dependency in the global
repo imports a bundle type dependency:

io.nats
jnats
0.7.3
bundle

Eclipse will report "Missing artifact io.nats:jnats:bundle:0.7.3". I
followed the trail to Maven Central which has 1.0 as the latest version.
The Maven global repo in Eclipse was re-indexed and updated 2 days ago
(that was a pain...) so I doubt the issue is a recently replaced version.
I then tried the manual approach:
git clone git@github.com:nats-io/java-nats.git
cd java-nats
mvn install
So I install Maven and everything that entails, then try "mvn install":
...
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ jnats ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 17 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @ jnats ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 46 source files to /home/user/java-nats/target/test-classes
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Some input files use or override a deprecated API.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java uses unchecked or unsafe operations.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Recompile with -Xlint:unchecked for details.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,33] package javafx.scene.input does not exist
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,1] static import only from classes and interfaces
[INFO] 2 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46.019 s
[INFO] Finished at: 2017-02-06T21:43:14-08:00
[INFO] Final Memory: 37M/349M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:testCompile (default-testCompile) on project jnats: Compilation failure: Compilation failure:
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,33] package javafx.scene.input does not exist
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,1] static import only from classes and interfaces
There's a javafx dependency in your test code that's not included in any
Maven dependency. Should it be? Is it possible this is just related to me
using OpenJDK 1.8?
So I commented out the static import for javafx in test LogVerifier.java,
re-ran "mvn install" and now I have everything built.
I have no clue if this is how I should be accessed/importing jnats into my
project. modifying the project and manually installing with Maven was the
only way I could get the project built. There seems to be a lot of
dependency overkill for a client interface to a comparatively straight
forward messaging server.
I'm not here to complain. I'm interested in any thoughts on how to
properly work with jnats in my project. Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nats-io/java-nats/issues/98, or mute the thread
https://github.com/notifications/unsubscribe-auth/AGq4LO5bXJrvHxA-tari2ALDWx_zbkW9ks5raAoggaJpZM4L5F_8
.

--
Larry McQueary | NATS.IO Product Management
@mcqueary | github.com/mcqueary

The import in question seems to have been inserted by my IDE while I wasn't
looking. IntelliJ IDEA is sometimes a tad more helpful than I would like :)
The good news is that it's just a test dependency and the import was not
even necessary, therefore your workaround was fine.
I'll look into adding OpenJDK to our test matrix in Travis.
On Tue, Feb 7, 2017 at 5:28 AM Larry McQueary<EMAIL_ADDRESS>wrote:

It's OK, you can be that guy :) especially if I screwed something up in
the latest release (which was Thursday), in which case I'm that guy! ;)
I apologize for your frustration., I'll look at it right now.
On Mon, Feb 6, 2017 at 11:07 PM duhmojo<EMAIL_ADDRESS>wrote:
Hi. I don't want to be that guy, but I've been trying all night to get the
official jnats client "installed" at all.
Using Eclipse (4.6.1) on Ubuntu 16.04, the Maven dependency in the global
repo imports a bundle type dependency:

io.nats
jnats
0.7.3
bundle

Eclipse will report "Missing artifact io.nats:jnats:bundle:0.7.3". I
followed the trail to Maven Central which has 1.0 as the latest version.
The Maven global repo in Eclipse was re-indexed and updated 2 days ago
(that was a pain...) so I doubt the issue is a recently replaced version.
I then tried the manual approach:
git clone git@github.com:nats-io/java-nats.git
cd java-nats
mvn install
So I install Maven and everything that entails, then try "mvn install":
...
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ jnats ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 17 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.6.1:testCompile (default-testCompile) @ jnats ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 46 source files to /home/user/java-nats/target/test-classes
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Some input files use or override a deprecated API.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java uses unchecked or unsafe operations.
[INFO] /home/user/java-nats/src/test/java/io/nats/client/ConnectionImplTest.java: Recompile with -Xlint:unchecked for details.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,33] package javafx.scene.input does not exist
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,1] static import only from classes and interfaces
[INFO] 2 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46.019 s
[INFO] Finished at: 2017-02-06T21:43:14-08:00
[INFO] Final Memory: 37M/349M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:testCompile (default-testCompile) on project jnats: Compilation failure: Compilation failure:
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,33] package javafx.scene.input does not exist
[ERROR] /home/user/java-nats/src/test/java/io/nats/client/LogVerifier.java:[9,1] static import only from classes and interfaces
There's a javafx dependency in your test code that's not included in any
Maven dependency. Should it be? Is it possible this is just related to me
using OpenJDK 1.8?
So I commented out the static import for javafx in test LogVerifier.java,
re-ran "mvn install" and now I have everything built.
I have no clue if this is how I should be accessed/importing jnats into my
project. modifying the project and manually installing with Maven was the
only way I could get the project built. There seems to be a lot of
dependency overkill for a client interface to a comparatively straight
forward messaging server.
I'm not here to complain. I'm interested in any thoughts on how to
properly work with jnats in my project. Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nats-io/java-nats/issues/98, or mute the thread
https://github.com/notifications/unsubscribe-auth/AGq4LO5bXJrvHxA-tari2ALDWx_zbkW9ks5raAoggaJpZM4L5F_8
.
--
Larry McQueary | NATS.IO Product Management
@mcqueary | github.com/mcqueary

--
Larry McQueary | NATS.IO Product Management
@mcqueary | github.com/mcqueary

I'll leave this issue open pending a stable release that closes it, but the 1.1-SNAPSHOT just pushed an hour ago should resolve this, if you care to give it a try.

Note:  we now test against openjdk8 in Travis CI.

Closed by ancillary changes contained in #100


read-only filesystem check

I had problems to save a changed configuration and it takes some time to find the curiosity: the overlay filesystem was mounted readonly, but I need to check this over ssh.
However a short message in the right corner like the field "unsaved changes" would be nice.
Even the mount points page within luci didn't show the readonly status and therefore did not show a button to remount this as readwrite.

Was it actually overlayfs being mounted r/o or rather OpenWrt's preinit deciding to use overlayfs with upperdir==tmpfs due to other options being unavailable?
I'm asking this because preinit should make sure we NEVER end up with read-only rootfs and in case rootfs_data being unavailable it should use tmpfs instead. I could create an additional attribute in ubus call system board to allow detecting that state more comfortably (if that's actually what you are observing).

No, the overlayfs was mounted r/o. I don't know why this happened. I rebooted the device, but this didn't happened again.
root@openwrt:~# mount
proc on /proc type proc (ro,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
tmpfs on /dev type tmpfs (ro,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (ro,relatime,mode=600,ptmxmode=000)
mountd(pid2330) on /tmp/run/blockd type autofs (ro,relatime,fd=7,pgrp=1,timeout=30,minproto=5,maxproto=5,indirect)
[reboot]
root@openwrt:~# mount
/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
/dev/mtdblock4 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
tmpfs on /dev type tmpfs (rw,nosuid,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,mode=600,ptmxmode=000)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)
mountd(pid2192) on /tmp/run/blockd type autofs (rw,relatime,fd=7,pgrp=1,timeout=30,minproto=5,maxproto=5,indirect)

I guess that something went horrible wrong. I can't reproduce that issue, but nevertheless, maybe a warning symbol would be useful

Hm, maybe the filesystem holding the upperdir got corrupted and was automatically re-mounted read-only, and that change then propagated to overlayfs.

Sorry, no. I do not have any further information about this

because preinit should make sure we NEVER end up with read-only rootfs and in case rootfs_data being unavailable it should use tmpfs instead.

I think that there is an exception: if the overlay exists but is too full (preventing full overlayfs working), overlayfs mounts it as read-only.
Refernce to:
https://github.com/openwrt/openwrt/commit/18691109df5a590e9914ae73740f145beec847ce

That should not be a problem here:
root@openwrt~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 8.3M      8.3M         0 100% /rom
tmpfs                    60.7M    148.0K     60.6M   0% /tmp
/dev/mtdblock4            6.1M    468.0K      5.7M   7% /overlay
overlayfs:/overlay        6.1M    468.0K      5.7M   7% /
tmpfs                   512.0K         0    512.0K   0% /dev


Counts only data layers

Contributes to #6219
CR this change please, @juanignaciosl
cc/ @luisbosque

Review count and space Hound reccomendations. Also, might it be interesting not only adding the 'kind' count but also keeping the total one? Up to you anyway :+1:


Provisioning fails with 'gzip: stdout: No space left on device'

I have updated Ansible requirements (sudo ansible-galaxy install -r provisioning/requirements.yml --force) and am trying to provision, but it is failing. I have retried a few times.
It is failing sometimes at geerlingguy.mailhog | Install sSMTP and sometimes at geerlingguy.postfix | Ensure postfix is installed (Debian). Error log follows:
failed: [drupalvm] => {"failed": true}
stderr: Running depmod.
update-initramfs: deferring update (hook will be called later)
The link /initrd.img is a dangling linkto /boot/initrd.img-3.16.0-57-generic
vmlinuz(/boot/vmlinuz-3.16.0-57-generic
) points to /boot/vmlinuz-3.16.0-57-generic
 (/boot/vmlinuz-3.16.0-57-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-3.16.0-57-generic.postinst line 491.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.16.0-57-generic /boot/vmlinuz-3.16.0-57-generic
run-parts: executing /etc/kernel/postinst.d/dkms 3.16.0-57-generic /boot/vmlinuz-3.16.0-57-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.16.0-57-generic /boot/vmlinuz-3.16.0-57-generic
update-initramfs: Generating /boot/initrd.img-3.16.0-57-generic

gzip: stdout: No space left on device
E: mkinitramfs failure cpio 141 gzip 1
update-initramfs: failed for /boot/initrd.img-3.16.0-57-generic with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-3.16.0-57-generic.postinst line 1025.
dpkg: error processing package linux-image-3.16.0-57-generic (--configure):
 subprocess installed post-installation script returned error exit status 2
dpkg: dependency problems prevent configuration of linux-image-extra-3.16.0-57-generic:
 linux-image-extra-3.16.0-57-generic depends on linux-image-3.16.0-57-generic; however:
  Package linux-image-3.16.0-57-generic is not configured yet.

dpkg: error processing package linux-image-extra-3.16.0-57-generic (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-image-generic-lts-utopic:
 linux-image-generic-lts-utopic depends on linux-image-3.16.0-57-generic; however:
  Package linux-image-3.16.0-57-generic is not configured yet.
 linux-image-generic-lts-utopic depends on linux-image-extra-3.16.0-57-generic; however:
  Package linux-image-extra-3.16.0-57-generic is not configured yet.

dpkg: error processing package linux-image-generic-lts-utopic (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-generic-lts-utopic:
 linux-generic-lts-utopic depends on linux-image-generic-lts-utopic (= <IP_ADDRESS>.48); however:
  Package linux-image-generic-lts-utopic is not configured yet.

dpkg: error processing package linux-generic-lts-utopic (--configure):
 dependency problems - leaving unconfigured
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.16.0-56-generic /boot/vmlinuz-3.16.0-56-generic
run-parts: executing /etc/kernel/postinst.d/dkms 3.16.0-56-generic /boot/vmlinuz-3.16.0-56-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.16.0-56-generic /boot/vmlinuz-3.16.0-56-generic
update-initramfs: Generating /boot/initrd.img-3.16.0-56-generic

gzip: stdout: No space left on device
E: mkinitramfs failure cpio 141 gzip 1
update-initramfs: failed for /boot/initrd.img-3.16.0-56-generic with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1
dpkg: error processing package linux-image-extra-3.16.0-56-generic (--configure):
 subprocess installed post-installation script returned error exit status 1

Postfix configuration was untouched.  If you need to make changes, edit
/etc/postfix/main.cf (and others) as needed.  To view Postfix configuration
values, see postconf(1).

After modifying main.cf, be sure to run '/etc/init.d/postfix reload'.

Running newaliases
 * Stopping Postfix Mail Transport Agent postfix
   ...done.
 * Starting Postfix Mail Transport Agent postfix
   ...done.
Errors were encountered while processing:
 linux-image-3.16.0-57-generic
 linux-image-extra-3.16.0-57-generic
 linux-image-generic-lts-utopic
 linux-generic-lts-utopic
 linux-image-extra-3.16.0-56-generic
E: Sub-process /usr/bin/dpkg returned an error code (1)

stdout: Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  libyaml-0-2 linux-headers-3.16.0-51 linux-headers-3.16.0-51-generic
  linux-headers-3.16.0-52 linux-headers-3.16.0-52-generic
  linux-headers-3.16.0-53 linux-headers-3.16.0-53-generic
  linux-image-3.16.0-30-generic linux-image-3.16.0-51-generic
  linux-image-3.16.0-52-generic linux-image-3.16.0-53-generic
  linux-image-extra-3.16.0-30-generic linux-image-extra-3.16.0-51-generic
  linux-image-extra-3.16.0-52-generic linux-image-extra-3.16.0-53-generic
  python-crypto python-httplib2 python-jinja2 python-markupsafe
  python-paramiko python-pkg-resources python-support python-yaml sshpass
Use 'apt-get autoremove' to remove them.
Suggested packages:
  procmail postfix-mysql postfix-pgsql postfix-ldap postfix-pcre sasl2-bin
  dovecot-common postfix-cdb mail-reader postfix-doc
The following packages will be REMOVED:
  ssmtp
The following NEW packages will be installed:
  postfix
Preconfiguring packages ...
0 upgraded, 1 newly installed, 1 to remove and 85 not upgraded.
5 not fully installed or removed.
Need to get 0 B/1084 kB of archives.
After this operation, 3600 kB of additional disk space will be used.
(Reading database ... 249158 files and directories currently installed.)
Removing ssmtp (2.64-7) ...
Processing triggers for man-db (<IP_ADDRESS>-1ubuntu1) ...
Selecting previously unselected package postfix.
(Reading database ... 249140 files and directories currently installed.)
Preparing to unpack .../postfix_2.11.0-1ubuntu1_amd64.deb ...
Unpacking postfix (2.11.0-1ubuntu1) ...
Processing triggers for ufw (0.34~rc-0ubuntu2) ...
Processing triggers for ureadahead (0.100.0-16) ...
ureadahead will be reprofiled on next reboot
Processing triggers for man-db (<IP_ADDRESS>-1ubuntu1) ...
Setting up linux-image-3.16.0-57-generic (3.16.0-57.77~14.04.1) ...
Setting up linux-image-extra-3.16.0-56-generic (3.16.0-56.75~14.04.1) ...
Setting up postfix (2.11.0-1ubuntu1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.6) ...

msg: '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"   install 'postfix'' failed: Running depmod.
update-initramfs: deferring update (hook will be called later)
The link /initrd.img is a dangling linkto /boot/initrd.img-3.16.0-57-generic
vmlinuz(/boot/vmlinuz-3.16.0-57-generic
) points to /boot/vmlinuz-3.16.0-57-generic
 (/boot/vmlinuz-3.16.0-57-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-3.16.0-57-generic.postinst line 491.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.16.0-57-generic /boot/vmlinuz-3.16.0-57-generic
run-parts: executing /etc/kernel/postinst.d/dkms 3.16.0-57-generic /boot/vmlinuz-3.16.0-57-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.16.0-57-generic /boot/vmlinuz-3.16.0-57-generic
update-initramfs: Generating /boot/initrd.img-3.16.0-57-generic

gzip: stdout: No space left on device
E: mkinitramfs failure cpio 141 gzip 1
update-initramfs: failed for /boot/initrd.img-3.16.0-57-generic with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-3.16.0-57-generic.postinst line 1025.
dpkg: error processing package linux-image-3.16.0-57-generic (--configure):
 subprocess installed post-installation script returned error exit status 2
dpkg: dependency problems prevent configuration of linux-image-extra-3.16.0-57-generic:
 linux-image-extra-3.16.0-57-generic depends on linux-image-3.16.0-57-generic; however:
  Package linux-image-3.16.0-57-generic is not configured yet.

dpkg: error processing package linux-image-extra-3.16.0-57-generic (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-image-generic-lts-utopic:
 linux-image-generic-lts-utopic depends on linux-image-3.16.0-57-generic; however:
  Package linux-image-3.16.0-57-generic is not configured yet.
 linux-image-generic-lts-utopic depends on linux-image-extra-3.16.0-57-generic; however:
  Package linux-image-extra-3.16.0-57-generic is not configured yet.

dpkg: error processing package linux-image-generic-lts-utopic (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of linux-generic-lts-utopic:
 linux-generic-lts-utopic depends on linux-image-generic-lts-utopic (= <IP_ADDRESS>.48); however:
  Package linux-image-generic-lts-utopic is not configured yet.

dpkg: error processing package linux-generic-lts-utopic (--configure):
 dependency problems - leaving unconfigured
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 3.16.0-56-generic /boot/vmlinuz-3.16.0-56-generic
run-parts: executing /etc/kernel/postinst.d/dkms 3.16.0-56-generic /boot/vmlinuz-3.16.0-56-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 3.16.0-56-generic /boot/vmlinuz-3.16.0-56-generic
update-initramfs: Generating /boot/initrd.img-3.16.0-56-generic

gzip: stdout: No space left on device
E: mkinitramfs failure cpio 141 gzip 1
update-initramfs: failed for /boot/initrd.img-3.16.0-56-generic with 1.
run-parts: /etc/kernel/postinst.d/initramfs-tools exited with return code 1
dpkg: error processing package linux-image-extra-3.16.0-56-generic (--configure):
 subprocess installed post-installation script returned error exit status 1

Postfix configuration was untouched.  If you need to make changes, edit
/etc/postfix/main.cf (and others) as needed.  To view Postfix configuration
values, see postconf(1).

After modifying main.cf, be sure to run '/etc/init.d/postfix reload'.

Running newaliases
 * Stopping Postfix Mail Transport Agent postfix
   ...done.
 * Starting Postfix Mail Transport Agent postfix
   ...done.
Errors were encountered while processing:
 linux-image-3.16.0-57-generic
 linux-image-extra-3.16.0-57-generic
 linux-image-generic-lts-utopic
 linux-generic-lts-utopic
 linux-image-extra-3.16.0-56-generic
E: Sub-process /usr/bin/dpkg returned an error code (1)

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/Users/rob/playbook.retry

drupalvm                   : ok=13   changed=1    unreachable=0    failed=1

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

It looks like:
gzip: stdout: No space left on device

Can you log in (vagrant ssh) and run df -h and paste the output? It looks like you've hit 20+ GB on the VM disk image.

vagrant@drupalvm:/tmp$ df -h
Filesystem                                         Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-root                         78G   12G   63G  16% /
none                                               4.0K     0  4.0K   0% /sys/fs/cgroup
udev                                               988M  4.0K  988M   1% /dev
tmpfs                                              201M  432K  200M   1% /run
none                                               5.0M     0  5.0M   0% /run/lock
none                                              1001M     0 1001M   0% /run/shm
none                                               100M     0  100M   0% /run/user
/dev/sda1                                          236M  229M     0 100% /boot
/dev/loop1                                          56M   56M     0 100% /tmp/vbox
<IP_ADDRESS>:/Users/rob/work/_drupalvm/_db-dumps_  931G  632G  300G  68% /db-dumps
vagrant                                            931G  632G  300G  68% /vagrant

Yeah, it looks like the /boot volume is full (/dev/sda1), and that's causing the error.
See http://askubuntu.com/questions/89710/how-do-i-free-up-more-space-in-boot for more info—also, I typically blow out my Drupal VM instances on a weekly basis (vagrant destroy -f) and rebuild them from scratch, so this kind of problem is avoided :)
Drupal VM is structured so it can be destroyed and rebuilt as easily as possible, so it might be good when you run into issues like this to just destroy it and rebuild; otherwise you'll have to do some of the cleanup as suggested in the above thread on Ask Ubuntu.


Editor layout not restored after restart.

System information:

Windows 10
DBeaver 6.2.5
w/ Office integratiom

Connection specification:

Database: Firebird or MySQL but doesn't matter

Describe the problem you're observing:
When selecting the vertical layout in the editor (having the script on the left side and the results on the right) and closing/opening DBeaver again, the layout is horizontal again. When selecting the context menu and looking into the "Layout" submenu it still states the layout should be vertical.
To get the vertical layout back again, "horizontal" must be selected first and "vertical" second.
Steps to reproduce, if exist:
See problem desctiption.
Include any warning/errors/backtraces from the logs

Thank you for the bug report.

Fixed (we'll use global prefs store for this option)

verified


PPYR_10586-PA gene-model is screwed up

Missing 5 C-terminal amino acids (ELSKL)

Resolved with ec1042826db44958014263d21c9990cf74eafa9d


Logo in navbar is not displayed

Logo in Navbar component renders perfectly fine locally, but on the website it's not seen. I believe we need to require/import logo.svg before using it.

Hello, I think I might have a solution for this problem.
According to the docs pertaining to deploying on render, you perform these two steps
- Build Command: npm run build
- Publish Directory: dist

This means that code is being run from the dist directory which probably doesn't have the logo.svg and hence can't find it.

Logo in Navbar component renders perfectly fine locally

It works fine locally because you are running npm run dev which runs the website from the top directory where logo.svg is located.
Solution:

You copy the logo.svg in the dist directory. Maybe you can add it as a script to be run before you run  npm run build if you have a lot of assets.
Or you can change the import path so instead of

it becomes

but this would break locally.

Sorry these are the solutions I can come up with. I have had a similar issue as I have used render for one of my projects.
Hope this helps.

Thank you for you response! What worked for me was installing file-loader for webpack and importing logo to the Navbar component :)


Fix dropdown menu position in playground when scroll

Summary
This Pull Request introduces a fix to maintain the dropdown menu's position relative to the toolbar during page scroll. Previously, the dropdown menu was using position: fixed, causing it to remain stationary in the viewport instead of moving along with the toolbar during scrolling.
How to reproduce
adding content in the playground editor untill a scroll bar appear, and then open any dropdown menu in the toolbar and scroll the page.
Details
The dropdown menu's positioning logic has been updated to become fixed or absolute based on the toolbar's position.
Important Note
Obtaining an accurate toolbar position is crucial for this logic. However, due to asynchronous loading of elements (like the Logo), directly extracting the toolbar's position in a useEffect hook can result in inaccurate values. To ensure precision, we're extracting the toolbar position asynchronously, which guarantees the correct calculation after all page elements have fully loaded. This is done by using setTimeout which could be consider a bad paractise, so any advice on extracting the toolbar position without using it is very appreciated.

Thank you, while this works it makes the Dropdown module lose it's reusability aspect as you're including the toolbar height in the computations.
We should be able to fix this retaining the position: fixed and a transform: translate(x, y) that moves as you scroll, looking at the nearest scroll parent and intersection (to handle visibility).

Hello, I believe that in the DropDown.tsx file, the dependency on toolbar height was already established prior to this Pull Request.
useEffect(() => {
    const button = buttonRef.current;
    const dropDown = dropDownRef.current;

    if (showDropDown && button !== null && dropDown !== null) {
      const {top, left} = button.getBoundingClientRect();
      dropDown.style.top = `${top + 40}px`;
      dropDown.style.left = `${Math.min(
        left,
        window.innerWidth - dropDown.offsetWidth - 20,
      )}px`;
    }
  }, [dropDownRef, buttonRef, showDropDown]);

it is the useEffect hook in DropDown component and it is used for calculating the position of dropdown menu when it is shown, and 40 here is the height of toolbar. I can try to solve that as well if needed.
I will also take a look at solving it using transform: translate(x, y), thank you for the advice!

the dependency on toolbar height was already established prior to this Pull Request.

You're right

we need toolbar's initial position in order to solve that case

But do we need a strong dependency with the toolbar? My hunch is that we can build a component that is able to render a dropdown {above|before|after|below} a HTML ref. The dropdown should reposition when you scroll or resize the page to stick to the same position as the component is.

the dependency on toolbar height was already established prior to this Pull Request.

You're right

we need toolbar's initial position in order to solve that case

But do we need a strong dependency with the toolbar? My hunch is that we can build a component that is able to render a dropdown {above|before|after|below} a HTML ref. The dropdown should reposition when you scroll or resize the page to stick to the same position as the component is.

Hello, I realized that the initial position is indeed unnecessary and I've been able to simplify the solution. The latest changes have been committed, where the dropdown component no longer depends on the button's height being set to 40px and can be accurately positioned in relation to the button element.
I also took note of issue #4516 and concur that components such as dropdowns, modals, and buttons aren't central to a rich text editor framework. Until the core team settles on the direction to take - be it utilizing external libraries or developing in-house components - alterations to the playground code should be primarily directed towards rectifying existing bugs, instead of expanding the functionalities of current components. This PR is meant for a quick fix to improve the lexical playground user experience so I believe that addressing the dropdown's positioning within this very PR should suffice for the moment.


How to traverse tree-like structures recursively?

The docs say:

A note about object safety: It is currently not possible to wrap a ParallelIterator (or any trait that depends on it) using a Box or other kind of dynamic allocation, because ParallelIterator is not object-safe. (This keeps the implementation simpler and allows extra optimizations.)

This basically rules out recursive transformation of parallel iterators. Cannot call a recursive function returning an impl ParallelIterator because it leads to recursive type, which is obviously not allowed. With standard iterators one can use Box<dyn Iter<...>> im such case but not with rayon.
What is the recommended way of doing recursion with rayon iterators?
How to implement this:
/// Return iterator over all items in a tree.
fn traverse<I, Node, F>(root: Node, children: F) -> I
where
  I: ParallelIterator<Item=Node>,
  F: (Fn(Item) -> I) + Sync + Send,
  Node: Sync + Send

If it is complex to implement, could it be added as a new feature (probably with more options to determine traversal order)?

Nothing concrete, but for some trees, it might be possible to use rayon::iter::split

Yeah, I think split could work for a tree, or at least it may be a good model of implementing an unindexed producer.
I've seen the Box<dyn Iterator> approach used before where it basically traverses the entire tree into an iterator tree, before the consumer gets to do anything at all. I really dislike that, because I feel like iterators ought to be as lazy as possible, and more so if you're hoping to parallelize it. Otherwise, you might as well collect that pre-traversal into a Vec, and then you can very quickly iterate that, in serial or parallel. Maybe you have a more dynamic approach though...

I realized that the question is quite general and applies not only to trees. Another example could be chaining multiple iterators, where we don't know the number of input iterators statically.


[Spike] Ruby Agent - APM missing information for creation of APM to Cloud service relationships

Description
This is a timeboxed 2 week discovery exercise related to https://new-relic.atlassian.net/browse/NR-185329 intended to uncover what is the missing data preventing APM entities from being mapped to AWS services. Scope of what is needed is not currently clear.
Engineering work to remediate identified areas will be planned for future quarters.
An assumption is made that whatever data gaps exist for the Ruby Agent in this story would likely apply to all APM language agents, so that any remediative work would have to be considered for those agents too.
Acceptance Criteria
TBD
Design Consideration/Limitations
TBD
Dependencies
TBD
Additional context
Business impact

Focus on AWS only for now. Will expand out to other providers later.


llvm/arm-clang fails to build STM32 platforms

when building with arm-clang or llvm for arm we get the following error:
/home/galak/git/zephyr/drivers/interrupt_controller/intc_exti_stm32.c:231:9: error: initializer element is not a compile-time constant

To reproduce, you can grab the ARM LLVM prebuilt here:
https://github.com/ARM-software/LLVM-embedded-toolchain-for-Arm/releases
export ZEPHYR_TOOLCHAIN_VARIANT=llvm
export LLVM_TOOLCHAIN_PATH=<PATH_TO_LLVM>

west build -p -b nucleo_f767zi  -T tests/kernel/common/kernel.common

Issue is coming from the method used to fill this array:
https://github.com/zephyrproject-rtos/zephyr/blob/fdb631c5d703e11e07f8d10e9c079b49135832fe/drivers/interrupt_controller/intc_exti_stm32.c#L209-L211
which is the way I've found to access the values of the following property:
			line-ranges = <0 1>, <1 1>, <2 1>, <3 1>,
				      <4 1>, <5 5>, <10 6>;

This could be replaced by something like:
	static const struct stm32_exti_range line_range_##idx = {	\
		DT_PROP_BY_IDX(node_id, line_ranges, UTIL_BY_TWO(idx)), \
		DT_PROP_BY_IDX(node_id, line_ranges, UTIL_INC(UTIL_BY_TWO(idx))) \
	};

This would require the addition of
#define UTIL_INC(x) UTIL_PRIMITIVE_CAT(UTIL_INC_, x)
#define UTIL_BY_TWO(x) UTIL_PRIMITIVE_CAT(UTIL_BY_TWO_, x)
with :
#define UTIL_INC_0 1
#define UTIL_INC_1 2
#define UTIL_INC_2 3
...
#define UTIL_BY_TWO_0 0
#define UTIL_BY_TWO_1 2
#define UTIL_BY_TWO_2 4
...

UTIL_INC used to be present, but was removed sometime ago in https://github.com/zephyrproject-rtos/zephyr/pull/32692 with no obvious replacement.

maybe we should re-introduce UTIL_INC.

Unrelated: @galak, note that the LLVM compilation didn't complete, as I got the following error:
lang-16: warning: argument unused during compilation: '--specs=nosys.specs' [-Wunused-command-line-argument]
~/st/stm32cubeclt_1.11.1/GNU-tools-for-STM32/bin/arm-none-eabi-ld.bfd: zephyr/libzephyr.a(cbprintf_complete.c.obj): in function `extract_conversion':
~/zephyrproject/zephyr/lib/os/cbprintf_complete.c:645: undefined reference to `__aeabi_memclr8'
~/st/stm32cubeclt_1.11.1/GNU-tools-for-STM32/bin/arm-none-eabi-ld.bfd: zephyr/libzephyr.a(cbprintf_complete.c.obj): in function `encode_uint':
~/zephyrproject/zephyr/lib/os/cbprintf_complete.c:802: undefined reference to `__aeabi_uldivmod'
~/st/stm32cubeclt_1.11.1/GNU-tools-for-STM32/bin/arm-none-eabi-ld.bfd: zephyr/drivers/serial/libdrivers__serial.a(uart_stm32.c.obj): in function `LL_USART_SetBaudRate':
~/zephyrproject/modules/hal/stm32/stm32cube/stm32f4xx/drivers/include/stm32f4xx_ll_usart.h:1029: undefined reference to `__aeabi_uldivmod'
~/st/stm32cubeclt_1.11.1/GNU-tools-for-STM32/bin/arm-none-eabi-ld.bfd: ~/zephyrproject/modules/hal/stm32/stm32cube/stm32f4xx/drivers/include/stm32f4xx_ll_usart.h:1029: undefined reference to `__aeabi_uldivmod'
~/st/stm32cubeclt_1.11.1/GNU-tools-for-STM32/bin/arm-none-eabi-ld.bfd: modules/stm32/stm32cube/lib..__modules__hal__stm32__stm32cube.a(stm32f4xx_hal_rcc.c.obj): in function `HAL_RCC_GetSysClockFreq':
~/zephyrproject/modules/hal/stm32/stm32cube/stm32f4xx/drivers/src/stm32f4xx_hal_rcc.c:916: undefined reference to `__aeabi_uldivmod'
clang-16: error: ld.lld command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.


Fix unrotate method

This PR fixes the handling of the longitudes in unrotate().

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Coding questions

I'm beginner in AI world, I'm trying to learn how to train my own model with this example but, i don't understand where am i supposed to use and write these codes ?
"
CUDA_VISIBLE_DEVICES=0 python3 test.py
--eval_data data_lmdb_release/evaluation --benchmark_all_eval
--Transformation None --FeatureExtraction VGG --SequenceModeling BiLSTM --Prediction CTC
--saved_model saved_models/None-VGG-BiLSTM-CTC-Seed1111/best_accuracy.pth
..."
I use Pycharm, never tried Cuda. (I understand CPU, GPU but I don't understand how and where to use these codes..)
Thanks you sooo very much guys!!

Download these codes & datasets as given.
Open the terminal in the root directory of this repo (containing test.py).
Type the above command in the terminal & testing will begin.
PS - If you do not have GPU on your machine, you can just remove the "CUDA_VISIBLE_DEVICES=0" from the command and run the rest, i.e., "python3 test.py --eval_data data_lmdb_release/evaluation --benchmark_all_eval --Transformation No..."


Implement AsRawFd for PollFd

Implement the trait on PollFd, providing an as_raw_fd() accessor to
get the RawFd associated with the PollFd.
Subsumes #1147, #1286.  Closes #1146.
Test: cargo test --test test test_pollfd_fd

Will have a CHANGELOG.md conflict with #1515, which is one of the annoying aspects of maintaining a Changelog file in-repo.

Another changelog idea I've seen is formatting commits such that a tool can construct the document from them.  E.g., https://www.conventionalcommits.org/en/v1.0.0/ .

I've seen that one.  But I don't think it would work.  Enforcing consistent commit message is harder than enforcing consistent changelog entries, especially on a project with so many different contributors.

I think it should be enforceable via bors / pre-commit CI check, but I haven't investigated very hard.


Input buffer

Some commands like 'bb', 'lo' etc. should accept input after invoking. Currently, there's no such thing in Gitu. Also handling autocomplete inside it might get messy (skip in first iteration?).
As seen in Magit:

First iteration implemented for git checkout -b with 08d995e1814b1c607f6582054b56760b8af50549


Modify a typing error of README.md (English)

The printing errors of the words used in the subhead and body have been corrected and integrated. (README.md, English)
Suppoted Logs => Supported Logs
Conguration => Configuration

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks!


Detect if we are receiving too few or too much data per hour

This has been migrated from https://github.com/ooni/pipeline/issues/265

[ ] Improve monitoring around space usage and failed pipeline run and point to the pipeline handbook
[ ] Detect if we are receiving too few or too much data per hour

Implemented in https://github.com/ooni/backend/issues/437


Terraform 12.x is not supported

Describe the bug:
Any Terraform version after 0.11.14 is not working
To Reproduce:

install Terrafrom as prerequisite (0.12.1 or 0.12.3)
follow the instruction here to deploy Bedrock simple app
run  terraform init

Expected behavior:
Bedrock should work normally with any Terraform version or documentations state the supported versions.
Screenshots:

Additional context:
I downgrade Terraform to 11.14 and everything went well
Does this require updates to documentation?:
Yes

Fixed awhile ago.  Closing.


V2 improve anonymizer engine

move text handling to entities.

Approved the production code, tests will be done next iteration


[Feature] Separate Creation of Example Submissions from Exercise Edit

Checklist

[X] I tested all changes and all related features with different users (student, tutor, editor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[X] Client: I followed the coding and design guidelines.
[X] Client: I added unit tests (Jest) related to the features (with a high test coverage)
[X] Client: I added authorities to all new routes and check the course groups for displaying navigation elements (links, buttons)
[X] Client: I documented the TypeScript code using JSDoc style.
[X] Client: I translated all the newly inserted strings into German and English

Motivation and Context
Previously creation of example submissions was not intuitive for the instructors, because they had to go to edit exercise screen in order to add a new example submission.
Now I have provided a dedicated dashboard where the user can also create the example submissions.
Description
Removed the creation of the example submissions from the exercise edit screen.
Provide an exercise example submissions table/dashboard for modelling and text exercises for both regular and exam.
Steps for Testing (normal)

Log in to Artemis
Navigate to Course Management
Pick a course
Click on exercises
Create a text/modelling exercise
Click on example submissions

Steps for Testing (exam)

Log in to Artemis
Navigate to Course Management
Pick a course
Click on exams
Create an exam
Create an exercise group
Create a text/modelling exercise
Click on example submissions

Screenshots

@joschahenningsen Thank you for the review 😃

This doesn't seem related to your changes, right?

Yes this PR has nothing to do with the update of example submission.

When editing the modeling exercise there is only a "Create Example Solution" button even though there already is an example solution. It would be nice to see it or at least have some sort of indicator that there is one.

I think you are confusing Example Solution with Example Submissions.
There are many Example Submissions that an Instructor can create and this PR makes possible to create them through a dedicated example submissions dashboard. (previously the user had to create them on the Edit screen of an exercise)
However, this PR does not concern the creation of an Example Solution.
There is only 1 Example Solution an Instructor can create.
For Text Exercises this is done during the creation of the exercise itself.
While for Modeling Exercise the process is more complicated since it is only possible to create it during the Edit of the Exercise.
If @krusche  wants to separate this process like we did for example submissions, then we can discuss about it and create a new PR.
After creating the example solution, the button is replaced by the link that redirects to the created example solution.(screenshots below)
Before creating the example solution: [edit screen for model exercise]

After creating the example solution: [edit screen for model exercise]

Looks good, but the "Delete" button does not work and the error is shown on the "Exercises" page, not on the specific exercise example submission page.

Hey @ArtemTsurikov thanks for the review. I have tested the delete button many times and it always worked fine for me. However, you are right about the alert being shown not in the right screen. I fixed that now.

Code looks good so I'm approving :)
The only "issue" is: Should the example submissions really be accessible from the course management screen? Why not in the exercise detail page? Otherwise it would bloat up the tables with another button.

@ivanchimeno thank you for the review.
The button is also added to the exercise detail page. Screenshot is below

Ah yes, my bad. 🤦 Ignore that comment.

What Obstkernes status regarding the editor role comments from @FrankeLukas ?

What is the status regarding the editor role comments from @FrankeLukas ?

I incorporated his feedback. @jpbernius


Creating an ndarray from ragged nested sequences is deprecated

This should fix the new test failures popping up in github actions.

@MridulS I think this is now looking good, should I merge it so you can test the pyproject PR?


Semantic search BERT model to TFLITE

0
I have this code for semantic search engine built using the pre-trained bert model. I want to convert this model into tflite for deploying it to google mlkit. I want to know how to convert it. I want to know if its even possible to convert this into tflite. It might be because its mentioned on the official tensorflow site :https://www.tensorflow.org/lite/convert. But I dont know where to begin
code:
`from sentence_transformers import SentenceTransformer
Load the BERT model.
Each sentence is encoded as a 1-D vector with 78 columns
sentence_embeddings = model.encode(sentences)
print('Sample BERT embedding vector - length', len(sentence_embeddings[0]))
print('Sample BERT embedding vector - note includes negative values', sentence_embeddings[0])
#@title Sematic Search Form
code adapted from https://github.com/UKPLab/sentence-transformers/blob/master/examples/application_semantic_search.py
query = 'Nobody has sane thoughts' #@param {type: 'string'}
queries = [query]
query_embeddings = model.encode(queries)
Find the closest 3 sentences of the corpus for each query sentence based on cosine similarity
number_top_matches = 3 #@param {type: "number"}
print("Semantic Search Results")
for query, query_embedding in zip(queries, query_embeddings):
distances = scipy.spatial.distance.cdist([query_embedding], sentence_embeddings, "cosine")[0]
results = zip(range(len(distances)), distances)
results = sorted(results, key=lambda x: x[1])

print("\n\n======================\n\n")
print("Query:", query)
print("\nTop 5 most similar sentences in corpus:")

for idx, distance in results[0:number_top_matches]:
    print(sentences[idx].strip(), "(Cosine Score: %.4f)" % (1-distance))

``

@ali9653,
If you have saved your model to a file, then you can use the TFLite converter Python API to convert your model to TFLite. Please follow this official guide to convert your saved model. Thanks!

@ali9653,
If you have saved your model to a file, then you can use the TFLite converter Python API to convert your model to TFLite. Please follow this official guide to convert your saved model. Thanks!
How can I save this model its a pre-trained model. The link does not have instructions of converting a bert model

@ali9653,
You can select the BERT model of your choice from this official TF Hub link and follow the user guide given. Thanks!

@ali9653,
You can select the BERT model of your choice from this official TF Hub link and follow the user guide given. Thanks!

I'm sorry but I think you are not understanding the question. I know how to import a model from tensorflow hub be it bert or universal sentence encoder. I want to convert it into TfLite is the question. There is no user guide for conversation

@ali9653,
Please take a look at this example for end to end model conversion.

@ali9653,
Please take a look at this example for end to end model conversion.

I'm sorry you are still not understanding. The model they have used Mobilenet_v2 is directly supported. Mine is imported from tensorflow hub and there is no documentation regarding that.

Have you tried using pre-trained BERT models Mobile Bert, Albert.

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!


Getting OOM while reading large file on Lollipop devices

I tried to read and encrypt video file (though I used my own encryption algorithm), I got OOM exception.
protected byte[] readFile(final FileInputStream stream) {
//...
}
Can you please help in that?

I did a Google search for "OOM Exception". It is Out Of Memory Exception. Here are the first two responses ...
http://www.javacoffeebreak.com/faq/faq0027.html
http://stackoverflow.com/questions/19720956/get-oom-exception-when-sending-message-with-a-high-speed-with-netty
Mick

I know what OOM exception is, I am just informing you that when I tried to read video file (around 20MB using device having only 512MB RAM and running Android 5.1.1), you code gave me OOM exception. You can check and fix the issue in your code.


Automatic scaling not starting at zero

Hi, I have the following code for a bar chart that has a very simple data set of integers:
<html>
  <head>
    <title>ChartJs Test</title>
    <script src="http://www.chartjs.org/dist/2.6.0/Chart.bundle.js"></script>
  </head>

  <body>
      <div style="width: 75%">
        <canvas id="canvas"></canvas>
      </div>

      <script>
        var barChartData = {
          labels: ["January", "February", "March"],
          datasets: [{
            label: 'Dataset 1',
            data: [2, 3, 1]
          }]
        };

        window.onload = function() {
          var ctx = document.getElementById("canvas").getContext("2d");

          window.myBar = new Chart(ctx, {
            type: "bar",
            data: barChartData
          });
        };

      </script>
  </body>
</html>

You can see in the js fiddle that there are two visual issues. Firstly the scale does not start at zero giving a false impression of the data, and also the scale on the left has decimals places even though the data is only integers. Why is this? Am I missing something? Cheers

@burt202 You can add beginAtZero to your scales configuration, which will allow you to specify that the scale starts at 0. Here is an updated jsfiddle: https://jsfiddle.net/9bgeu5rx/1/
options: {
     scales: {
         yAxes: [{
             ticks: {
                 beginAtZero: true
             }
         }]
     }
 }

Thanks for the reply. I did know about beginAtZero option but the data feeding the chart is dynamic so I cant really use it. Is there any other way of achieving this? I suppose I could analyse the data coming in to see if it has values close to 0 and only set it when it is?

@burt202 you would have to detect that on your end before feeding the data to Chart.js.

OK thanks. Giving this more thought, i think the issue is not necessarily to do with starting at zero because you can change the data to be 20, 21, 22 and you would have the same problem (scale would start at 20). I appreciate its difficult to add in automatic lower and upper scale values that is not one of the data values because its hard to know the context of the data, I think I was just surprised that this was the default. Closing for now as I'll work out a way of using the options to dynamically achieve this. Cheers

https://github.com/chartjs/Chart.js/issues/4537#issuecomment-316820751
This solution is not working


Aether not taking newest SNAPSHOT-artifact

We are using the current master Spark Notebook (Scala 2.11) and having an issue with the resolution of snapshot dependencies. Our current configuration is
"customLocalRepo": "/home/myuser/.m2/repository"

 "customRepos": [
    "nexus % default % http://repo.myorg/content/repositories/hosted.xxx.snapshots % maven"
  ]

We often deploy SNAPSHOTS (i.e. always version = 1.0-SNAPSHOT) of our own libraries to our nexus repository (which is our cooperate repository).
Now if we deploy a new artifact to nexus (i.e. same version but with another timestamp), Spark-Notebooks just takes the latest from our local maven repo (if there is one) instead of checking whether there is a newer version on nexus.
I've configured  value to "always" for my nexus repo (in settings.xml), but this does not help.
The only solution I've found so far is to delete the corresponding old files in my local maven repo and then restarting the notebook.
Is this the expected behavior? Maybe it could be fixed with this : http://stackoverflow.com/questions/33084282/how-to-use-aether-to-get-the-latest-timestamped-snapshot-dependency

Now if we deploy a new artifact to nexus (i.e. same version but with another timestamp), Spark-Notebooks just takes the latest from our local maven repo (if there is one) instead of checking whether there is a newer version on nexus.

maybe there's still something to be improved (e.g. by your link),
but either case why should you have the same artifact version both in nexus-repository and in local maven repo? it introduces lots of ambiguity...
Does it respect the timestamps if snapshot in ONLY in remote nexus (and not present locally)?
I'd even expect local repo to simply get higher priority over remote (e.g. as it seems happens with local ivy / sbt publishLocal in sbt).

Well, I don't install any artifact in the local repository, they seem to be cached automatically in there by aether/maven during dependency resolution

Some observations; If I don't specify  "customLocalRepo": "/home/myuser/.m2/repository", aether completely fails to download dependencies, even though "/home/myuser/.m2/repository" should be the default location for local local maven repository anyways.
I'm also wondering why the local maven repo is defined as "org.sonatype.aether.repository.RemoteRepository"  and not "org.sonatype.aether.repository.LocalRepository" in Deps.scala? Although I'm not familiar with aether, this seems a little strange.

btw, we have some wish to switch to newer/faster/better coursier, but so far this is only at discussion level.
ideally, we'd switch to coursier both build.sbt for faster SN builds, and customDeps downloading in kernel.


Scheduled workflow raising exception: where to observe re-execution?

The code below shows a scheduled workflow that raises an exception on first execution. In order to identify the executing workflow a log statement logs the workflow id (sched-print_log-2024-11-05T03:31:50+00:00).
When another workflow is scheduled, it will not raise an exception due to the workflow_failed Boolean variable set to True when the first workflow raises an exception.
My expectation was that the failed workflow with id sched-print_log-2024-11-05T03:31:50+00:00 will be re-executed as it did not successfully complete on first execution. Where in the DBOS schema can I observe that the failed workflow is re-executed and successful on 2nd try?
Thank you.
import sys
import time
import traceback
from datetime import datetime

from dbos import DBOS

DBOS()

workflow_failed: bool = False

@DBOS.scheduled("*/10 * * * * *")
@DBOS.workflow()
def print_log(scheduled_time: datetime,
              actual_time: datetime) -> None:
    DBOS.logger.info(f"Workflow with id {DBOS.workflow_id} start ...")
    global workflow_failed
    if not workflow_failed:
        workflow_failed = True
        raise Exception()
    DBOS.logger.info(f"... workflow with id {DBOS.workflow_id} completed")

def main(arguments: list[str]) -> None:
    DBOS().launch()

    while True:
        time.sleep(5)
        DBOS.logger.info('alive')

if __name__ == "__main__":
    try:
        main(sys.argv)
    except KeyboardInterrupt:
        DBOS.logger.warning('KeyboardInterrupt received')
    except Exception as e:
        DBOS.logger.error(traceback.format_exc())
    finally:
        DBOS.destroy()

Example log:
"C:\Users\miso\VirtualEnvironments\drag\Scripts\python.exe" "C:\Users\miso\Workspaces\drag\z_research\infinite_loop\infinite_loop.py"
19:31:46 [    INFO] (dbos:_dbos.py:266) Initializing DBOS
19:31:48 [    INFO] (dbos:_dbos.py:394) DBOS launched
19:31:50 [    INFO] (dbos:infinite_loop.py:17) Workflow with id sched-print_log-2024-11-05T03:31:50+00:00 start ...
19:31:50 [   ERROR] (dbos:_core.py:236) Exception encountered in asynchronous workflow: Traceback (most recent call last):
  File "C:\Users\miso\VirtualEnvironments\drag\Lib\site-packages\dbos\_core.py", line 234, in _execute_workflow_wthread
    return _execute_workflow(dbos, status, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\miso\VirtualEnvironments\drag\Lib\site-packages\dbos\_core.py", line 192, in _execute_workflow
    output = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\miso\Workspaces\drag\z_research\infinite_loop\infinite_loop.py", line 21, in print_log
    raise Exception()
Exception

19:31:53 [    INFO] (dbos:infinite_loop.py:30) alive
19:31:58 [    INFO] (dbos:infinite_loop.py:30) alive
19:32:00 [    INFO] (dbos:infinite_loop.py:17) Workflow with id sched-print_log-2024-11-05T03:32:00+00:00 start ...
19:32:00 [    INFO] (dbos:infinite_loop.py:22) ... workflow with id sched-print_log-2024-11-05T03:32:00+00:00 completed
19:32:03 [    INFO] (dbos:infinite_loop.py:30) alive
19:32:08 [    INFO] (dbos:infinite_loop.py:30) alive
19:32:10 [    INFO] (dbos:infinite_loop.py:17) Workflow with id sched-print_log-2024-11-05T03:32:10+00:00 start ...
19:32:10 [    INFO] (dbos:infinite_loop.py:22) ... workflow with id sched-print_log-2024-11-05T03:32:10+00:00 completed
19:32:12 [ WARNING] (dbos:infinite_loop.py:37) KeyboardInterrupt received

Process finished with exit code -1

Hi @chbussler, I think the expectation isn't quite right.
A @DBOS.workflow() that throws an exception does not get retried automatically. There may be some confusion between workflows and steps. If you have a @DBOS.step() function called from a workflow, then you can configure retries using something like this:
@DBOS.step(retries_allowed=True, max_attempts=3)
def step_foo():
  #... throw

@DBOS.workflow()
def wf_bar():
  #...
  step_foo()

If you have code like the above then you can observe step_foo() retrying with a warning message printed to the app's log.
Does this help?

Hi @apoliakov, thanks for the super-quick response, that's great!

No confusion :-): pdf
I created a setup based on your input and added a try, except around the step invocation; this proves that DBOS retries the step as specified in the step annotations; and in the workflow itself it is possible to catch the case that the retries are exhausted. This means if retrying fails, the try and catch has the next opportunity to deal with the failure. Great, thanks for outlining as it made me implement this case as code (added below)!
However, my focus was not on step failures and re-execution, but on workflow failures and re-execution.

I need a bit more time to make my argument as the system isn't behaving according to my expectations based on the DBOS documentation. I'd like to create one or more use cases around workflows in code so that I can explain what I am trying to address. Please bear with me in this discussion and I'll get back to you.
Thank you!
import sys
import time
import traceback
from datetime import datetime

from dbos import DBOS

DBOS()

@DBOS.step(retries_allowed=True, max_attempts=2)
def step_foo():
    DBOS.logger.info(f"Step start ...")
    raise Exception()
    DBOS.logger.info(f"... step completed")

@DBOS.scheduled("*/30 * * * * *")
@DBOS.workflow()
def workflow_steps_exceptions(scheduled_time: datetime,
                              actual_time: datetime) -> int:
    DBOS.logger.info(f"Workflow with id {DBOS.workflow_id} start ...")

    try:
        step_foo()
    except Exception as e:
        DBOS.logger.error(f"Exception: {e}")

    DBOS.logger.info(f"... workflow with id {DBOS.workflow_id} completed")
    return 5

def main(arguments: list[str]) -> None:
    DBOS().launch()

    while True:
        time.sleep(5)
        DBOS.logger.info('alive')

if __name__ == "__main__":
    try:
        main(sys.argv)
    except KeyboardInterrupt:
        DBOS.logger.warning('KeyboardInterrupt received')
    except Exception as e:
        DBOS.logger.error(traceback.format_exc())
    finally:
        DBOS.destroy()

Thanks @chbussler!
One related case to consider is the Programming Guide example here:
https://docs.dbos.dev/python/programming-guide
Here we show how workflows continue to execute where they left off if the app process is stopped and restarted. Note that process restart is not the same as throwing an exception.
In our view, if your code throws an exception, retry may or may not be warranted. We can imagine many cases like "throw on invalid input" where a workflow retry would never succeed. That's why our step decorator lets you choose the appropriate behavior for your code.
Does this make sense? Let us know how else we can help.

@apoliakov, before going back to exceptions within a workflow, I want to implement the standard case first to observe how a workflow recovers.
In below code there is a workflow with two steps step_foo and step_bar. The two steps are called one after the other, and there is a sleep(10) statement between those. What I wanted to accomplish is the following:

Start workflow from a first terminal
Observe the execution of step_foo
Wait for the log statement announcing the sleep statement
Close the first terminal (aka, simulating server crash) during the sleep before step_bar starts
Restart the workflow in a new, second terminal
Observe the recovery of the workflow started in the first terminal

And that does not happen, I do not see that the crashed workflow recovers, meaning, I would expect the workflow to start, print the log statements (except for those within step step_foo's implementation), and see the sleep again (this time with step_foo skipped), and continue with step_bar. I only observe new workflows being executed.
So no exceptions within a workflow, "just" a server crash by closing the first terminal. What in the below code would I have to change to have it recover? Thanks.
import sys
import time
import traceback
from datetime import datetime

from dbos import DBOS

DBOS()

@DBOS.step(retries_allowed=True, max_attempts=2)
def step_foo():
    DBOS.logger.info(f"Step foo start ...")
    DBOS.logger.info(f"... step foo completed")

@DBOS.step(retries_allowed=True, max_attempts=2)
def step_bar():
    DBOS.logger.info(f"Step bar start ...")
    DBOS.logger.info(f"... step bar completed")

@DBOS.scheduled("*/30 * * * * *")
@DBOS.workflow(max_recovery_attempts=50)
def workflow_steps_exceptions(scheduled_time: datetime,
                              actual_time: datetime) -> int:
    try:
        DBOS.logger.info(f"Workflow with id {DBOS.workflow_id} start ...")

        try:
            step_foo()
        except Exception as e:
            DBOS.logger.error(f"Exception in step foo: {e}")

        DBOS.logger.info(f"Workflow with id {DBOS.workflow_id} "
                         f"sleeping for 10 seconds")
        time.sleep(10)

        try:
            step_bar()
        except Exception as e:
            DBOS.logger.error(f"Exception in step bar: {e}")

        DBOS.logger.info(f"... workflow with id {DBOS.workflow_id} completed")
    except Exception as e:
        DBOS.logger.error(f"Exception in workflow: {e}")
    return 5

def main(arguments: list[str]) -> None:
    DBOS().launch()

    while True:
        time.sleep(5)
        DBOS.logger.info('alive')

if __name__ == "__main__":
    try:
        main(sys.argv)
    except KeyboardInterrupt:
        DBOS.logger.warning('KeyboardInterrupt received')
    except Exception as e:
        DBOS.logger.error(traceback.format_exc())
    finally:
        DBOS.destroy()

Hi @chbussler! Good to hear from you.
This example does work for me. I suspect it may have to do with what OS/terminal you use and how you start the app? One possibility is that the, perhaps, the process does not get stopped when you close the terminal.
This is what I see. I use a mac and launch the app using python3 my_app/main.py. I then open another terminal and get the pid of the process like this:
$ ps ax | grep -i my_app
19539 s023  S+     0:00.46<EMAIL_ADDRESS>my_app/main.py

Then, when the first terminal window goes into sleeping in the second window I run something like kill 19539. Holding down CTRL+C also works but creates some more verbose output.
My output looks like this. At 12:34:30 we run one healthy cycle. Then at 12:35:00 we start again, step foo finishes and then I kill the process. I wait about a minute and then start the app again at ~12:36:05. The first thing the app does is resume the workflow that started at 12:35:00. This workflow is tagged  id sched-workflow_steps_exceptions-2024-11-10T17:35:00+00:00 which confirms it started at 12:35 (after time zone adjustment). We see the "sleeping," then the "step bar start" then "step bar completed". Then the next cycle starts at 12:36:30.
12:34:28 [    INFO] (dbos:main.py:55) alive
12:34:30 [    INFO] (dbos:main.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:34:30+00:00 start ...
12:34:30 [    INFO] (dbos:main.py:13) Step foo start ...
12:34:30 [    INFO] (dbos:main.py:14) ... step foo completed
12:34:30 [    INFO] (dbos:main.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:34:30+00:00 sleeping for 10 seconds
12:34:33 [    INFO] (dbos:main.py:55) alive
12:34:38 [    INFO] (dbos:main.py:55) alive
12:34:40 [    INFO] (dbos:main.py:19) Step bar start ...
12:34:40 [    INFO] (dbos:main.py:20) ... step bar completed
12:34:40 [    INFO] (dbos:main.py:44) ... workflow with id sched-workflow_steps_exceptions-2024-11-10T17:34:30+00:00 completed
12:34:43 [    INFO] (dbos:main.py:55) alive
12:34:48 [    INFO] (dbos:main.py:55) alive
12:34:53 [    INFO] (dbos:main.py:55) alive
12:34:58 [    INFO] (dbos:main.py:55) alive
12:35:00 [    INFO] (dbos:main.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:35:00+00:00 start ...
12:35:00 [    INFO] (dbos:main.py:13) Step foo start ...
12:35:00 [    INFO] (dbos:main.py:14) ... step foo completed
12:35:00 [    INFO] (dbos:main.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:35:00+00:00 sleeping for 10 seconds
Terminated: 15                    ##<<--- this i where I killed the app. Restarting....

(.venv) badger:my-app apoliakov$ python3 my_app/main.py
12:36:05 [    INFO] (dbos:_dbos.py:266) Initializing DBOS
12:36:05 [    INFO] (dbos:_dbos.py:394) DBOS launched
12:36:05 [    INFO] (dbos:main.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:35:00+00:00 start ...
12:36:05 [    INFO] (dbos:main.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:35:00+00:00 sleeping for 10 seconds
12:36:10 [    INFO] (dbos:main.py:55) alive
12:36:15 [    INFO] (dbos:main.py:55) alive
12:36:15 [    INFO] (dbos:main.py:19) Step bar start ...
12:36:15 [    INFO] (dbos:main.py:20) ... step bar completed
12:36:15 [    INFO] (dbos:main.py:44) ... workflow with id sched-workflow_steps_exceptions-2024-11-10T17:35:00+00:00 completed
12:36:20 [    INFO] (dbos:main.py:55) alive
12:36:25 [    INFO] (dbos:main.py:55) alive
12:36:30 [    INFO] (dbos:main.py:55) alive
12:36:30 [    INFO] (dbos:main.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:36:30+00:00 start ...
12:36:30 [    INFO] (dbos:main.py:13) Step foo start ...
12:36:30 [    INFO] (dbos:main.py:14) ... step foo completed
12:36:30 [    INFO] (dbos:main.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-10T17:36:30+00:00 sleeping for 10 seconds
12:36:35 [    INFO] (dbos:main.py:55) alive
12:36:40 [    INFO] (dbos:main.py:55) alive
12:36:40 [    INFO] (dbos:main.py:19) Step bar start ...
12:36:40 [    INFO] (dbos:main.py:20) ... step bar completed
12:36:40 [    INFO] (dbos:main.py:44) ... workflow with id sched-workflow_steps_exceptions-2024-11-10T17:36:30+00:00 completed
12:36:45 [    INFO] (dbos:main.py:55) alive

What output do you see? And if you see something different, maybe let us know what OS you use, how you launch dbos and check to make sure you terminate the process?

Hi @apoliakov thanks for this alternative approach to kill the process, should have thought about it myself. I tried to reproduce the same way (you see Killed) in the below log, and the workflow does not recover (I also assume that all recover is completed first before normal processing resumes).
My environment is DBOS v0.13, Windows 11 Pro, 23H2, OS build 22631.4460
If there is anything you'd like me to do, please let me know, happy to try.
The log:
miso@SILVERPOWER MINGW64 ~/Workspaces/drag/z_research/workflow_two_steps_workflow (next)
$ python workflow_two_steps.py
18:54:57 [    INFO] (dbos:_dbos.py:266) Initializing DBOS
18:54:58 [    INFO] (dbos:_dbos.py:394) DBOS launched
18:55:01 [    INFO] (dbos:workflow_two_steps.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-16T02:55:00+00:00 start ...
18:55:01 [    INFO] (dbos:workflow_two_steps.py:13) Step foo start ...
18:55:01 [    INFO] (dbos:workflow_two_steps.py:14) ... step foo completed
18:55:01 [    INFO] (dbos:workflow_two_steps.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-16T02:55:00+00:00 sleeping for 10 seconds
18:55:03 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:08 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:11 [    INFO] (dbos:workflow_two_steps.py:19) Step bar start ...
18:55:11 [    INFO] (dbos:workflow_two_steps.py:20) ... step bar completed
18:55:11 [    INFO] (dbos:workflow_two_steps.py:44) ... workflow with id sched-workflow_steps_exceptions-2024-11-16T02:55:00+00:00 completed
18:55:13 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:18 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:23 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:28 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:30 [    INFO] (dbos:workflow_two_steps.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-16T02:55:30+00:00 start ...
18:55:30 [    INFO] (dbos:workflow_two_steps.py:13) Step foo start ...
18:55:30 [    INFO] (dbos:workflow_two_steps.py:14) ... step foo completed
18:55:30 [    INFO] (dbos:workflow_two_steps.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-16T02:55:30+00:00 sleeping for 10 seconds
Killed
(drag)
miso@SILVERPOWER MINGW64 ~/Workspaces/drag/z_research/workflow_two_steps_workflow (next)
$ python workflow_two_steps.py
18:55:44 [    INFO] (dbos:_dbos.py:266) Initializing DBOS
18:55:45 [    INFO] (dbos:_dbos.py:394) DBOS launched
18:55:50 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:55:55 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:56:00 [    INFO] (dbos:workflow_two_steps.py:28) Workflow with id sched-workflow_steps_exceptions-2024-11-16T02:56:00+00:00 start ...
18:56:00 [    INFO] (dbos:workflow_two_steps.py:13) Step foo start ...
18:56:00 [    INFO] (dbos:workflow_two_steps.py:14) ... step foo completed
18:56:00 [    INFO] (dbos:workflow_two_steps.py:35) Workflow with id sched-workflow_steps_exceptions-2024-11-16T02:56:00+00:00 sleeping for 10 seconds
18:56:00 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:56:05 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:56:10 [    INFO] (dbos:workflow_two_steps.py:19) Step bar start ...
18:56:10 [    INFO] (dbos:workflow_two_steps.py:20) ... step bar completed
18:56:10 [    INFO] (dbos:workflow_two_steps.py:44) ... workflow with id sched-workflow_steps_exceptions-2024-11-16T02:56:00+00:00 completed
18:56:10 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:56:15 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:56:20 [    INFO] (dbos:workflow_two_steps.py:55) alive
18:56:25 [ WARNING] (dbos:workflow_two_steps.py:62) KeyboardInterrupt received
(drag)
miso@SILVERPOWER MINGW64 ~/Workspaces/drag/z_research/workflow_two_steps_workflow (next)
$

Hello @chbussler! I also couldn't reproduce this behavior on my laptop (macOS Sonoma). Two things might be helpful:

Update to the latest DBOS release version.
Replace time.sleep(10) in the workflow with DBOS.sleep(10) which makes sure it doesn't oversleep. Otherwise, during recovery, the workflow would be put to sleep again.
Check the workflow status table: https://docs.dbos.dev/explanations/system-tables#dbosworkflow_status See if the workflow is in PENDING or other state. The workflow recovery code only recovers workflows in the PENDING state. It's possible that the workflow actually succeeded before it was killed.

Hi @qianl15, thanks for joining the discussion.
ad 1.: My system is already on Python 0.13
ad 2.: I replaced the time.sleep as you indicated
ad 3.: I deleted the rows from the workflow_status table to be able to better observe what is ongoing
The result is: it worked this time: clearly the DBOS.sleep() is the function to use.
Thank you.
@apoliakov, with this reproduction being successful, I will come back to the original question/comment I had. I'll try to get back here with a clear statement on what I am looking for in a bit. Thank you.

Hi @apoliakov , at this point I was able to establish that an interrupt of a scheduled workflow is recovered (with your and @qianl15's help - thank you very much).
Going all the way back to the beginning of this discussion, from my viewpoint as a user, there shall be no difference on why an interrupt happened, either by a crash of the VM, or a crash of the process (what we reproduced), or any other exception from within the workflow implementation.
I believe that an exception that interrupts a scheduled workflow should not be treated differently, but also cause the retry of the workflow. We have two cases:

the exception is inherent and will happen again at every retry. This is not a problem as the maximum retry setting will prevent an endless restart behavior
the exception was intermittent, and a retry will execute the workflow to the end upon retry

For a scheduled workflow there is no way to catch exceptions from the outside, aka, a try-catch block around a scheduled workflow. The only other option I have is to put the entirety of the scheduled workflow implementation in a try-catch.
The latter, however, would not allow my code to drive the workflow execution to conclusion as my exception handling in the catch cannot ask DBOS to retry the workflow (I think).
As a consequence,
(a) I would have to implement backward compensation in the workflow implementation, and that will differ based on where the exception took place in the implementation. This in turn would require my code to keep track of all step executions in the implementation.
(b) Or, I would have to implement forward recovery in my exception handling invoking all steps that failed and that have not yet been executed. Again, that would depend on where the exception took place and that would require logic to follow the workflow implementation.
There might be additional options, please indicate in this case. Thank you.
PS @apoliakov, I met @qianl15 in person at a meetup and she kindly explained to me why it has to DBOS.sleep(). Very important learning for me, thank you @qianl15


Feature Request: Dynamic Service Registration via Reverse Connection

Expected Behavior / New Feature
It would be great if Ocelot would support a mechanism where microservices in the system, could connect "outbound" to a service running Ocelot. As part of this outbound connect the microservice would register the routes it is offering and Ocelot would start handling this route. If multiple services register the same route, Ocelot would load balance the calls between them.
The important aspect of this feature is that the Microservice would pre-establish a pool of connections to Ocelot which will then be used to handle the calls. Idea behind this is to achieve special LAN/DMZ separation of the services where no manual firewall configuration is needed where ports are opened from the DMZ into the LAN.
See this architectural chart:

Actual Behavior / Motivation for New Feature
Please note that this feature request might be quite specific to what we're doing in our software but I guess it is a topic general enough to be potentially adressed by Ocelot.
First of all this feature brings the benefit that Ocelot would be able to dynamically expose service (as standalone service) depending on their availability in the network. This allows you do scale dynamically but also publish new services in your architecture without the need of changing anything on Ocelot.
The second aspect is security and the related costs maintaining it. Opening ports in the firewall towards your private network is always risky. If your backend is dynamic, (e.g. via spawning new VMs or containers), it needs again administrative effort to change the firewall accordingly. Some of our customers have very restricted network policies and opening ports is from the DMZ to LAN is not easily possible.
Steps to Reproduce the Problem / How to implement the New Feature
Our old proxy service was a specific development to our APIs we are exposing. Basically a dynamically generated WCF proxy for our contract but we want to switch to an HTTP based reverse proxy system. We started with a custom development with this approach when we started on some new HTTP based APIs.
In the backend services we have an ASP.net core service which is having a custom IServer implementation. This IServer implementation is connecting outbound to the proxy service via WebSockets. The gateway service detects the special service connect and starts communicating via a special WebSocket messages to exchange some metadata. For Ocelot the route configurations would be exchanged. Maybe also some metadata like the number of data connections could be agreed on this channel. This client part could be either ASP.net core specific but also it could be integrated to other services as long as there is some callback for the application to handle the request.
On the gateway side, the special control and data connections would need to be handled. Once a service registers the gateway would accept some data connections from this service. As soon a client calls an API, the gateway would use one of these available connections to handle the request. In our case we also handled this via a WebSocket based protocol that exchanges HTTP headers and data in a streaming fashion. In best case connections can be re-used.
Another approach could be a message-bus system. There would be one Ocelot Host on the DMZ side, and a Ocelot Client on the LAN side. The Host would just forward all HTTP requests to this service bus, and each microservice would listen to HTTP requests relevant for it.

Or last but not least a setup like this would also be an alternative: The microservices would only register themselves on a gateway in the LAN. This registration is just for initial setup and heartbeating if services are still around. This LAN gateway can connect then directly to the individual services to execute requests. But the 2 gateways would be connected via some outbound connection pool. Benefit on this solutions is that the service registration is an own special feature within Ocelot and does not require any client application adjustment. You just need an additional service in your overall architecture.

From what I saw the host part could be done via https://ocelot.readthedocs.io/en/latest/features/delegatinghandlers.html which use a connection from the pool to forward the and handle the requests. The service registration would be a new special endpoint services would call. This would be an extension to the service discovery https://ocelot.readthedocs.io/en/latest/features/servicediscovery.html#service-discovery
The client part would be a likely a compeltely new module like an AddOcelotGatewayServer(o=> o.GatewayAddress = "https://gateway/"; o.ServiceAuthenticationKey = "Secret to allow service registration" ) which registers an IServer for handling requests.
I'm eager to get some feedback on this feature request and which solution to this problem you guys prefer.
Specifications
Not relevant.

outbound connect the microservice would register the routes it is offering and Ocelot would start handling this route. If multiple services register the same route, Ocelot would load balance the calls between them
Maybe I'm reading this wrong but this is literally describes definition of a service discovery, which Ocelot already supports out of the box. Sounds like all you trying to achieve is to have a custom service discovery provider that handles node/service registrations via network zone availability.

@Danielku15 Not bad description! 😉
So, are you ready to develop this feature?
Are your fingertips resting on the keyboard? 😄


Feature: fetch pancake contract prices

Fixes #36, solution is 90% reusable for any AMM cross-chain. Once approved I will refactor #37 to pull data from contracts and getNyanswopPrice.js to prevent redundant RPC calls.
Leading pairs in cakeLpPools.json have been reorganised to respect price exploration order: BUSD -> WBNB -> CAKE -> other coins.


Add time buffer for the package to get installed

Issue #, if available:
Description of changes:
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

trivy will be added back once https://github.com/aws/eks-anywhere-packages/pull/217 get merged