[Bug]: Deployment fails, error "no service selected" when using Consistent Container Names with private GitHub repo

Description
After Coolify upgraded to 4.0.0-beta.237, I started getting the following deployment error on a resource based on a private GitHub repo that was configured to use the Consistent Container Names feature;
[2024-Mar-16 08:35:52.864917] Oops something is not okay, are you okay? 😢
[2024-Mar-16 08:35:52.872094] no service selected
[2024-Mar-16 08:35:52.874557] Deployment failed. Removing the new version of your application.

Issue still occurs with version 4.0.0-beta.239. Reverting to 4.0.0-beta.236 using curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash -s 4.0.0-beta.236 resolves the issue with no changes to the resource.
I also tested this with a git repo hosted in Gitea, and got the same error in the same versions.
Minimal Reproduction (if possible, example repository)
Deploy a private GitHub repo, or a git repo hosted in Gitea, with the Consistent Container Names feature enabled.
Exception or Error
[2024-Mar-16 08:35:51.781415] No configuration changed & image found (ek4ksso:bc7a0fd86bb08d13bff6a6104cb64a25504c6f79) with the same Git Commit SHA. Build step skipped.
[2024-Mar-16 08:35:51.943313] ----------------------------------------
[2024-Mar-16 08:35:51.948655] Consistent container name feature enabled, rolling update is not supported.
[2024-Mar-16 08:35:51.950286] Removing old containers.
[2024-Mar-16 08:35:52.752662]
COMMAND:
docker exec vowo4sw bash -c 'SOURCE_COMMIT=bc7a0fd86bb08d13bff6a6104cb64a25504c6f79 docker compose --project-directory /artifacts/vowo4sw -f /artifacts/vowo4sw/docker-compose.yml up --build -d'

OUTPUT:
no service selected
[2024-Mar-16 08:35:52.864917] Oops something is not okay, are you okay? 😢
[2024-Mar-16 08:35:52.872094] no service selected
[2024-Mar-16 08:35:52.874557] Deployment failed. Removing the new version of your application.

Version
beta.237-beta.239

This will be fixed in the upcoming version.


Which version of scrapy should we use?

I am facing this weird issue while running the crawler on a docker instance:
Scrapy 2.9.0 - no active project

Unknown command: crawl

Use "scrapy" to see available commands

I also tried the 2.11.2 version and the same issue persists, can someone share which version works the best.

You're missing the scrapy.cfg config, as otherwise you will get that. https://github.com/ahmia/ahmia-crawler/blob/master/ahmia/scrapy.cfg

It is now Scrapy==2.11.2 in requirements.txt.
https://github.com/ahmia/ahmia-crawler/blob/master/requirements.txt


Doesn't allow me to use scrollpad on laptop to scroll on Chrome

OSX version: 10.11.6
Macbook air
Google Chrome Version: 53
When I use the plug in I simply can't scroll using the scrollwheel on the laptop... I can only scroll using the mouse.

Sorry for the late reply, I didn’t have much time to work on my personal projects lately.
I can’t reproduce this issue, if this is still relevant, please provide a live reproduction (e.g. link to the site where this happens).


Add support for an XML renderer

Prerequisites
Put an x into the box that applies:

[ ] This issue describes a bug.
[x] This issue describes a feature request.

For more information, see the Contributing Guidelines.
Description
We would like to add support for an additional XML renderer. If you are interested in making a contribution, please provide suggestions on the intended XML format here. The resulting pull request will need to contain an XML schema to support output validation.

@jocgir

Hi there. Thanks for this project. Extracting the inputs and outputs into a table has made my life a little easier writing documentation.
I wonder if there are any plans to support .adoc as an output, or to adopt a plugin pattern so more outputs can be supported?

I just created #149 as an interim ticket to track the plugin development. After release of v0.8.0 I'm gonna add the support for xml and yaml and most probably attempt the plugin system in v0.9.0


Calling Array on an ODE solution of ComponentArrays is slow

I think this is due to the fact that vcat/hcat on a VectorOfArray of ComponentArrays splats out the vector. It would be better to call everything on the inner array types and just add axes after the fact.

See if cc26c7f helps this.

Fixed by 6006bfa


Nix Piston: extra build commands

How do we handle extra commands that are to be executed after building a Nix package in Nix Piston? Like the ones that are used to install Pip and NuGet packages. I see a packages attribute that piston.mkRuntime takes and returns, is that intended to be used for extra commands?

That is ment to be for a nix package set as nix can handle installing packages from many different languages.
It does nothing now, but I plan to enable it for some packages so we can get dynamic package installs.

Okay so what about extra build commands? Like installing PIP and NuGet packages?

yeah but I haven't thought that far ahead yet - afaik python packages work fine when installed with nix, same with Ruby and haskell

Got it. Actually now I see the suggested extra commands attribute wouldn't be of much use.

Some runtimes need this, so I implemented it in the pull request referenced above

Turns out, most of the time, Nix provides ways achieve this.


cli: deprecated quit --decommission

PR: https://github.com/cockroachdb/cockroach/pull/45903
From release notes:

The --decommission flag for cockroach quit is now deprecated. It will be removed altogether in a next stable release. Deployments should use cockroach node decommission followed by either cockroach quit or an equivalent form of server shut down. [#45903][#45903] {% comment %}doc{% endcomment %}

Update decommissioning guidance to use cockroach node decommission followed by cockroach quit (or equivalent server shutdown) and never cockroach quit --decommission
Clarify distinction between decommissioning and stopping a node
Update cockroach quit documentation to match changes in https://github.com/cockroachdb/docs/pull/6825/files

@jseldess Working through terminology here. I understand when a node is decommissioning, but when it is officially “decommissioned” seems unclear to me.
Do we consider a node decommissioned when replicas=0 independently of the is_draining value, or do we consider a node decommissioned (like the UI) once it is stopped?
It is also possible the way node status is interpreted in the UI is open to change, but I'm not sure yet.

These are excellent questions, @taroface, that I don't think we have consistent answers to. I think the technical answer is that we consider a node decommissioned when replicas=0, regardless of whether it has been stopped or not. The node could still be live and able to serve as a gateway, though without any of its own data. That said, I don't know if that's the way we should frame it externally. Let me ask around.

Hi @knz, do I have the following correct?
Regarding #7422

cockroach quit has a new flag in 19.1-20.1, and the command is being deprecated in v20.2.

we nonetheless aren't recommending use of cockroach quit in v20.1.

when decommissioning a node, cockroach node drain should be run after cockroach node decommission successfully completes. (or can it be run simultaneously?)
the combination of cockroach node drain + node shutdown via process manager supplants cockroach quit for the purposes of shutting down a node.
the "drain" in cockroach node drain refers to SQL connections and not range removal.

Regarding #7384 and this thread

cockroach node recommission is used while cockroach node decommission is running/hanging and effectively cancels the latter process

(is there a way to cancel cockroach node drain in a similar way, or would you have to wait for it to finish and then manually restart the node?)

cockroach quit has a new flag in 19.1-20.1, and the command is being deprecated in v20.2.

correct

   we nonetheless aren't recommending use of cockroach quit in v20.1.

correct

when decommissioning a node, cockroach node drain should be run after cockroach node decommission successfully completes. (or can it be run simultaneously?)

after, yes (not simultaneously)

the combination of cockroach node drain + node shutdown via process manager supplants cockroach quit for the purposes of shutting down a node.

correct

the "drain" in cockroach node drain refers to SQL connections and not range removal.

SQL connections + rebalancing of ranges + execution of distributed SQL queries on behalf of other nodes

cockroach node recommission is used while cockroach node decommission is running/hanging and effectively cancels the latter process

yes

  (is there a way to cancel cockroach node drain in a similar way, or would you have to wait for it to finish and then manually restart the node?)

no node drain cannot be cancelled. You have to wait and restart.

As of https://github.com/cockroachdb/cockroach/pull/49668, we'll have marked the entirety of cockroach quit as deprecated. We're now advising users to instead use cockroach node drain and send out termination signals to the crdb process, in order to replicate what was previously done via cockroach quit.

To that end I think what we want to happen here, in this issue, is to start scrubbing our docs of any mention of cockroach quit.

I see we already have https://github.com/cockroachdb/docs/pull/7304 inflight to change our 20.1 and earlier docs. Should I file separately to track this scrubbing of cockroach quit from our v20.2 docs?

I don't think so, @irfansharif. We have https://github.com/cockroachdb/docs/issues/6951, and with the first 20.2 alpha, I'm sure we'll generate other related issues.


{PropertyName} adds spaces between words

Hi
I'm using version 5.6.2, I have a base class where I use custom error messages with {PropertyName} in them to have a uniform generic way to show the validation errors, my problem is that, for example, I have a property called "CountryCode", and in the message it shows the property name as "Country Code".
Is a way to configure the validator to respect the property names?, I know about the method .WithName but if I have to pass the name in all my rules the purpose of having a base class or use {PropertyName} is lost .
Thanks for your time

So this is the default behaviour - when building error messages, PascalCased property names are split. You can override this by setting the global DisplayNameResolver property in your app startup routine:
ValidatorOptions.DisplayNameResolver = (type, member, expr) => {
  return member.Name;
}

Thanks for your help


chore: whitespace change

Summary
Testing why https://github.com/facebook/jest/pull/9397 is partially red on CI. I suspect it is not related to the PR.
Test plan
Debugging why some CI builds are red.

Codecov Report

Merging #9637 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #9637   +/-   ##
=======================================
  Coverage   65.09%   65.09%
=======================================
  Files         287      287
  Lines       12144    12144
  Branches     3009     3009
=======================================
  Hits         7905     7905
  Misses       3604     3604
  Partials      635      635

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e3aa106...976ef65. Read the comment docs.

@noomorph any conclusions? My PR has seemingly unrelated failures as well.

@royra, looks like my issue was temporary. See #9397.


统一error类型，确保所有异常都有对应的错误类型，引入gerror模块来实现

统一error类型，确保所有异常都有对应的错误类型
export default class WAError extends Error {

  constructor (type: WAErrorType, message: string) {
    super(`${type} ${message}`)
  }

}

Currently the gerror module is used with the Wechaty ecosystem, it's very simple and straightforward, read the source code of it is recommend before continue working on this issue.
GError source code: https://github.com/huan/gerror/blob/main/src/gerror/gerror.ts

Thanks for your suggestions. We will try it asap.
And happy to see Wechaty ecosystem become better and better.


Move userId check into the signing action

Description
Checking if userId is null in processengine seems to be wrong. Made the field nullable and moved the check into the SigningAction.
I'm not happy with the magic string for code, so suggestions are welcome
Related Issue(s)

#{issue number}

Verification

[ ] Your code builds clean without any errors or warnings
[ ] Manual testing done (required)
[ ] Relevant automated test added (if you find this hard, leave it and we'll help out)
[ ] All tests run green

Documentation

[ ] User documentation is updated with a separate linked PR in altinn-studio-docs. (if applicable)

I'd just add a new enum field to UserActionResult to store the system readable information. Why not just use the ProcessErrorType we already have?

PS: I also noticed that ProcessChangeResult.ErrorType is declared nullable, instead of just adding NoError as the 0th value in the enum.
I don't know what is the most recognised pattern, but it kind of makes sense to not carry the nullable wrapper everywhere.

Another idea!
Can we remove the userId from the UserActionRequest altogether and just fetch it from the cookie in the SigningAction instead? It seems like something that would only be required there, and the signing is very dependent on the cookie (and in the future possible other cryptographic data).

I'd just add a new enum field to UserActionResult to store the system readable information. Why not just use the ProcessErrorType we already have?

Sounds like a good solution!

PS: I also noticed that ProcessChangeResult.ErrorType is declared nullable, instead of just adding NoError as the 0th value in the enum.
I don't know what is the most recognised pattern, but it kind of makes sense to not carry the nullable wrapper everywhere.

I'm seeing that the whole ProcessChangeResult class might be replaced by ServiceResult<ProcessStateChange, ProcessError> where ProcessError is a new object holding ErrorMessage and ProcessErrorType. Then we don't need to have it nullable as that is the use case for ServiceResult. So we can make a change after that is merged.

Another idea! Can we remove the userId from the UserActionRequest altogether and just fetch it from the cookie in the SigningAction instead? It seems like something that would only be required there, and the signing is very dependent on the cookie (and in the future possible other cryptographic data).

I think we should leave it to the Api project to work on the httpContext/cookies and just require that core receives the objects it needs from those from the Api project. IMO the core project ideally shouldn't need to be changed if we remove the cookie or change protocol down the road.

I think we should leave it to the Api project to work on the httpContext/cookies and just require that core receives the objects it needs from those from the Api project. IMO the core project ideally shouldn't need to be changed if we remove the cookie or change protocol down the road.

Yes, that change would include moving the signature functionality to Api.

I was think that I should as mentioned rewrite the ProcessEnginge to use the new ServiceResult. But decided that since we have a rewrite of the process engine in another commit that would just create conflicts.
I suggest we just create an issue for it and the refactor that after those changes are merged.


feature difficulty check

#12
isValid() now checks also for difficulty and prints if and where the difficulty doesn't match.

Ja, so kann man das machen!


Argus considers a long-dead job running

http://<IP_ADDRESS>/test_run/c1e6c71c-ed32-48ed-a93a-b321a5ad746b
The job has ended ten days ago (as seen in https://jenkins.scylladb.com/job/scylla-master/job/longevity/job/longevity-200gb-48h/) but argus still reports that it's running, pointing to a runner that doesn't exist any longer.

@k0machi


Visual Motivation Conversion

Now that motivation can be consumed in visual format. I think it time to convert other motivations to become visual as well.

[x] Test Failures

[x] Startup Greeting

[ ] Exit Salutation

Can we include the "Motivate Me" (alt + M) action to the conversion?

Hmm. I thought about that. I'm just trying to come up with images that fit Motivation. What motivates you and motivates me could be different. Just as long as its SFW and fits the theme I think we should be good! With that said, I think a lot of the pre-existing images might count as motivational. I'll go through and add 'MOTIVATION, and you should too.
Let's try not to collide. I'll let you go first.
I'll also add some more images and tag them as motivational as well, once you are done.

I think a lot of the pre-existing images might count as motivational.

I agree, As long as it's cute that'll motivate me lol

What do you think about having Motivate Me display an image with no title? Also what sound should be played?

Hmmm, For motivation, I prefer to play something that is common to anime sounds and basically a waifu cheering someone.

... display an image with no title
We can do this as well, but I'm thinking if we can retain it, we will have consistent alerts for other events. But, without the title, it'll have a cleaner effect as it's a user request. Can you share the reason?

Can you share the reason?

Yeah, so what I had planned is that many category of images/reaction-gifs can be used as Motivation. So I think that is really difficult to come up a one size fits all message, with the current way it implemented.
I think if we could change the asset definition to
  {
    "imagePath": "celebration/megumin_yes.gif",
    "imageAlt": "",
    "imageDimensions": {
      "width": 480,
      "height": 270
    },
    "primaryCategory": "CELEBRATION"
    "subcategories": [
      "MOTIVATION"
    ]
  }

Then we could easily support a relevant message works with the image being displayed.


DPS Resets

Hello again,
first let me tell you sorry for bothering you so much lately. Second awesome job you did with this Casual Meter.
Third I have found a bug that in Wonderholm when boss spawns totems with Colors the DPS on my meter usually resets. Is that only bug on my side or is it general bug?
Thanks for reply and keep up the good job!

https://github.com/lunyx/CasualMeter/wiki/Settings-(Advanced)#inactivity-reset-duration

Oh...yea I should probably read more carefuly next time..
Big thx for help and sorry for bothering you, guys!


Remove extra OEAssignImplicitHydrogens which could result in protonation state changes

Attempts to resolve #59
@hjuinj - can you test this out?

He notes (via Slack) that this works now, so merging.


UCS min_sstable_size should not be lower than target_sstable_size low…

…er bound
Thanks for sending a pull request! Here are some tips if you're new here:

Ensure you have added or run the appropriate tests for your PR.
Be sure to keep the PR description updated to reflect all changes.
Write your PR title to summarize what this PR proposes.
If possible, provide a concise example to reproduce the issue for a faster review.
Read our contributor guidelines
If you're making a documentation change, see our guide to documentation contribution

Commit messages should follow the following format:
<One sentence description, usually Jira title or CHANGES.txt summary>

<Optional lengthier description (context on patch)>

patch by <Authors>; reviewed by <Reviewers> for CASSANDRA-#####

Co-authored-by: Name1 <email1>
Co-authored-by: Name2 <email2>

The Cassandra Jira

@ethan-brown2022 is this what you had in mind?

@smiklosovic I think I fixed the test issue.  Please try again.


Auto-launched containers shown as test containers and are disabled

When the plugin launches new agent containers, they all seem to automatically be marked as 'test' and disabled as per Known-limitations
How does the plugin determine that its a test container?  It was not started via the test functionality in the plugin configuration.  However, I had previously started that docker image as a test container and then created a new image from it - is that the reason?

That was it.
The image I was using was created off a running container which was in test mode. 😃

Ok :-) This behavior is indeed expected. When committing an image, all the container settings such as environment variables and labels will be persisted in the new image, including those used by the plugin to track managed containers. If the committed container was a "test"container, the newly started instance will then be recognized as such by the test controller, and it will be immediately disabled (test containers should not accept any build jobs).
That being said, committing a test container to use it as an agent image can be considered a legitimate use case. The possible improvement I see would be to remove all prior, plugin-related, meta-data when starting new containers. This would prevent the described scenario.
I created a new issue to track this: #14
Thank you for reporting


Is there any way to know a frame is a compiled frame or an interpreted frame?

Our process runs slowly after couple of days.  It maybe caused by some method run in interpreter mode instead of compiled mode. Is there anyway to know a frame is run in interpreter mode/compiled mode?

I'm investigating how async-profiler can supplement output with JIT/Interpreter information.
This looks difficult, but not impossible.
Meanwhile I committed a change that adds dummy "Interpreter" frame above Java stack trace, if the topmost Java frame runs in the interpreter.

Check interpreter branch.
However, the scenario you've described is highly unlikely. Methods that run long/often enough, are almost always JIT compiled. Especially if the application has already been running for a couple of days. Other ("cold") methods that run in the interpreter, do not typically affect performance. A notable exception is when the Code Cache becomes full, but this is easy to check with any JMX capable tool like Visual VM / JConsole / Mission Control.

Another possibility could be generated method is too large and exceed the HugeMethodLimit.
By the way is it possible that code cache get too fragmented to hold a large method?

@guanggenzhang yes it is possible. In older versions of JVM there was also a bug https://bugs.openjdk.java.net/browse/JDK-8244278 which could casue a lot of fragmentation. With this bug the JVM could compile simple method few times a day. Here is a part of the JVM log with such a situation
[09:04:52.835] 214973       4       java.util.HashMap::<init> (11 bytes)
[10:49:55.579] 214973       4       java.util.HashMap::<init> (11 bytes)   made not entrant
[10:49:57.884] 228194       4       java.util.HashMap::<init> (11 bytes)
[11:31:19.272] 228194       4       java.util.HashMap::<init> (11 bytes)   made not entrant
[11:31:19.853] 231080       4       java.util.HashMap::<init> (11 bytes)
[14:06:29.779] 231080       4       java.util.HashMap::<init> (11 bytes)   made not entrant
[14:06:30.992] 241011       4       java.util.HashMap::<init> (11 bytes)
[14:49:54.579] 241011       4       java.util.HashMap::<init> (11 bytes)   made not entrant
[14:49:57.921] 242993       4       java.util.HashMap::<init> (11 bytes)

If you want to know if specified method is compiled or not you can run:
jcmd <pid> Compiler.codelist

at runtime.

The feature has been integrated. async-profiler now shows the compilation state of the top frames:

Light green are interpreted;
Cyan are inlined;
Green are regular (specifically, green top frames are compiled; the state of non-top frames is unknown).


Flight computer migration

Time estimate: 20 hours
Deadline:  NA
Description of task: Since the control stack needs to run on a separate computer (not the Xavier), we need to migrate all required code to this new platform. We also have the choice between TX2 and RPI4 8GB.
The RPI solution seems a nice first step, since the GPIO is fairly similar to the Xavier. A new breakout board will likely have to be purchased for the TX2 if we want a smooth transition.

Addressed in #324


Repack(or webpack) produces invalid JS bundle?

Ask your Question
Hi all!
I'm trying to replace metro with repack in a fresh React native 0.70 app and without additional dependencies everything works perfect. But after adding the library camelcase-keys to the app, I started getting following runtime errors:

In JS bundle from local dev server (http://localhost:8081/index.bundle?platform=android&dev=true&minify=false&app=com.repack&modulesOnly=false&runModule=true) we see, that module ./node_modules/camelcase-keys/index.js uses plain require function, instead of __webpack_require__ , which does not exists in Hermes runtime and throwing error. What do you think is going on here? Is this some misconfiguration in my app or Repack issue?
Reproduction
Repo: https://github.com/NepeinAV/repack_repro
Steps:

Generate fresh RN 0.70.6 app
Follow Repack installation guide
Install camelcase-keys lib
Import it in App.js file
Run app
Get the error

Hi @NepeinAV
I've investigated the issue and this problem occurs because camelcase-keys is a pure ESM library, therefore we need to handle it a little bit different. We need to add a separate rule in our webpack.config.mjs to handle ESM libraries as shown below:
diff --git a/webpack.config.mjs b/webpack.config.mjs
index 467d188..d55aa84 100644
--- a/webpack.config.mjs
+++ b/webpack.config.mjs
@@ -165,6 +165,24 @@ export default (env) => {
             options: {},
           },
         },
+        {
+          test: /\.[jt]sx?$/,
+          include: [
+            /node_modules(.*[/\\])+camelcase-keys/,
+            /node_modules(.*[/\\])+quick-lru/,
+          ],
+          use: {
+            loader: 'babel-loader',
+            options: {
+              presets: [
+                [
+                  'module:metro-react-native-babel-preset',
+                  {disableImportExportTransform: true},
+                ],
+              ],
+            },
+          },
+        },
         /**
          * Here you can adjust loader that will process your files.
          *

Besides adding camelcase-keys, we also needed to add quick-lru as it's a dependency of camelcase-keys and is also a pure ESM library.
If you would like to know more, you can check out this answer to a similar issue: https://github.com/callstack/repack/issues/359#issuecomment-1559509051


Is it possible to play a MJPG stream with media_kit?

Hi,
is it possible to display mjpg stream in media_kit? The stream is provided by a local backend app sending data with content type 'multipart/x-mixed-replace'.
player.open(Media(
      'http://localhost:55551/stream',
      httpHeaders: {
        'Content-Type': 'multipart/x-mixed-replace; boundary=frame',
      },
    ));

When I pass the URL to the Media object, it seems as if the player is loading something permanently but nothing is displayed. Is there a parameter I'm missing or are mjpg streams not supported by media_kit?
Thanks for your help!
Vera

I'm not very sure, didn't work with MJPEG streams before.
However, it should work, we do have MJPEG decoder/demuxer enabled.
NOTE: On the web, things may be bit limited since implementation depends upon the browser.

Should we close this ticket?

I can send you a little program that stream the camera as a MJPG stream on mac if helpful. Unfortunately I didn't find any open mjpg streams.

Hi,
confirm cannot work with mjpeg stream.
test url: http://<IP_ADDRESS>:21700/cgi-bin/mjpeg?
work fine on VLC:

log:
Launching lib\main.dart on Joy 1 in debug mode...
Running Gradle task 'assembleDebug'...
√  Built build\app\outputs\flutter-apk\app-debug.apk.
Installing build\app\outputs\flutter-apk\app-debug.apk...
Debug service listening on ws://<IP_ADDRESS>:24023/xZ3znaSeyGQ=/ws
Syncing files to device Joy 1...
I/media_kit( 4138): com.alexmercerind.media_kit_video.VideoOutputManager.create:<PHONE_NUMBER>48
I/media_kit( 4138): flutterJNIAPIAvailable = true
I/media_kit( 4138): com.alexmercerind.media_kit_video.VideoOutput: id = 0
I/flutter ( 4138): {id: 0}
I/flutter ( 4138): media_kit: wakelock: _count = 1
I/media_kit( 4138): com.alexmercerind.media_kit_video.VideoOutputManager.createSurface:<PHONE_NUMBER>48
I/flutter ( 4138): {wid: 9894}
I/flutter ( 4138):  mpv: cplayer: error: Failed to recognize file format.

Stumbled accross this issue too, maybe mjpeg should be removed from supported formats until this is fixed?

@CasperVerswijvelt
MJPEG streams should ideally work. I'm unable to figure out what's wrong in our compilation configuration.

@alexmercerind any updates? the same problem here, can't play mjpg streams with media-kit :(

@alexmercerind I have the same problem, black video with mjpeg stream

Faced the same problem, my flutter logs looks ok:
I/media_kit(10248): com.alexmercerind.media_kit_video.VideoOutputManager.create: -5476376677156710352 I/media_kit(10248): flutterJNIAPIAvailable = true I/media_kit(10248): com.alexmercerind.media_kit_video.VideoOutput: id = 24 I/flutter (10248): {id: 24} I/flutter (10248): media_kit: wakelock: _count = 1 I/media_kit(10248): com.alexmercerind.media_kit_video.VideoOutputManager.createSurface: -5476376677156710352 I/flutter (10248): {wid: 15658}
I tested few different ways to produce test MJPEG stream and every time the same behavior, everything works well in my local browser, but media-kit Video widget only shows constant loading animation without any errors or messages. I'm not a network problem but something with media-kit package.
Could you please take care on that

I am facing the same issue. Media kit works wonderfully for various video formats but we are currently forced to use a separate package to handle MJPEG. It would be ideal to be able to just use media kit and migrate to a single implementation.

Stumbled accross this issue too, maybe mjpeg should be removed from supported formats until this is fixed?

I agree, or perhaps add a small note about this known issue? Could save people some debugging time.


Initial version of a json schema for the service specification [NRTM]

As discussed in #1083, adding a JSON Schema file.
This is NOT ready to merge. This file currently only implements ADR-02 but needs to be expanded to support ADR-05
In this file, I've left namespace, description, labels and annotations as optional and everything else required. You can't add extra keys anywhere except directly inside the metadata object.
@freben  @Rugvip

Regarding ADR005, we'll be heavily focusing on the Component kind for the first iteration. So don't worry too much about the other ones just yet.

Or maybe make a separate definitions schema e.g. for the entire metadata block that anybody can ref? https://json-schema.org/understanding-json-schema/structuring.html

Yeah so it looks like we could make a individual kind schema be allOf both a ref to the base schema as well as its own additions
https://json-schema.org/understanding-json-schema/structuring.html#id5

I think probably it would be possible to have a schema that applies to everything.
json-schema has the idea of using a key value to discriminate between schemas. The one thing almost sure you will want to have is different versions in different files.

I think probably it would be possible to have a schema that applies to everything.
json-schema can express having or not properties depending on the value of other properties.
The one thing almost sure you will want to have is different versions in different files.

@freben Moved the file, renamed it to alpha1, added owner and lifecycle and made spec structure dependent of kind. So I think this file can apply to everything, you just need to add new spec definitions when they come.
I do think different versions should be different files, else this will be ginormous and difficult to read.


Perplexity per sentence implementation?

I need to get ppl per sentence for millions of lines. Splitting them into files each containing one sentence would be time consuming. Is it possible to achieve this by modifying dataloader? For example, give the model input as (num_sentences, num_tokens, max_characters_per_token) . The problem is how to pad sentences that doesn't have enough tokens. If this would work, will such padding affect state for next batch? If not, any other suggestions?

add batch_losses to append losses can get one batch sentences ppl


[WIP] implement run cmd and allow running processes without shell wrapper

NODE this is a draft opened to facilitate discussion
implementation of run command lacks ability to pass additional
arguments to the main process (which is useful when executing
test runners to, for example, pass a test filter which results
in only a subset of tests being ran)
implements #88 and #83

Happy to extract https://github.com/F1bonacc1/process-compose/issues/88 into separate PR.
regarding https://github.com/F1bonacc1/process-compose/issues/83, the way I made it work is to silence std(out|err) output for all processes apart from the main one passed to run as an argument, and the connecting os.std(in|out|err) directly to the main process
note that I had to skip p.command.SetCmdArgs() since otherwise things would break in some cases (like running tests with yarn jest --watch  in a node project). I guess this means that the main process will receive signals (Ctrl+C) directly. This is ok in my case as I set exit_on_end on the main process (which I guess would be a good default anyway).

note that I'm not well versed in go so don't hesitate to criticise the implementation :)

run command can be tested using the config file:
shell:
  shell_command: "bash"
  shell_argument: "-c"

processes:
  main:
    command: |
      echo say hi
      read -r in
      echo $in
      echo by
    depends_on:
      dep:
        condition: process_started
    availability:
      exit_on_end: true

  dep:
    command: while true; do echo sleeping... && sleep 2; done

run the main process:
./result/bin/process-compose run main -f process-compose.yaml

the dep will start in background but it's logs will not be output by the command above
instead you can view its logs with:
./result/bin/process-compose process logs -f dep

and here's an example of how to configure a process to run without a shell wrapper:
processes:
  main:
    no_shell: true
    command: yarn
    args:
      - jest
      - --watch

We could potentially default to no_shell: true if args are set. Technically you could pass extra args to a shell, as in:
> bash -c 'echo name: $0 arg1: $1' my-name my-arg
name: my-name arg1: my-arg

but not sure if that would of use to anyone

and here's an example of how to configure a process to run without a shell wrapper:
processes:
  main:
    no_shell: true
    command: yarn
    args:
      - jest
      - --watch

We could potentially default to no_shell: true if args are set. Technically you could pass extra args to a shell, as in:
> bash -c 'echo name: $0 arg1: $1' my-name my-arg
name: my-name arg1: my-arg

but not sure if that would of use to anyone

Not sure this is going to be a widely used notation :)
I think that if the args list is not empty, the configured shell can be ignored and run cmd ...args

Are there use cases where it only makes sense to run a process as part of the run command? In other words, should the interactive process be disabled when ran as part of the full process-compose.yaml?
I tested the code (as is) with vim and it creates some unwanted visual issues when Tui is disabled.

@adrian-gierakowski
That's a more than welcome PR!
Having STDIN is a super useful feature, that was asked by other users before.
I was trying to crack the TUI approach, but I like your, more practical and actually working direction, more.
Thank you for that!

I think that if the args list is not empty, the configured shell can be ignored and run cmd ...args

👍

Are there use cases where it only makes sense to run a process as part of the run command? In other words, should the interactive process be disabled when ran as part of the full process-compose.yaml?

There might be some, but in my case it works both with run and up, the latter with or without TUI. With extra args passed from cli, I would setup main process as yarn jest and then pass —watch when running with run for interactive experience. And if I wanted to do a one shot execution with outputs from all service interleaved I’d use up.

I tested the code (as is) with vim and it creates some unwanted visual issues when Tui is disabled.

could you post a minimal config I could use to reproduce? Or a nix expression? Thanks!

I tested the code (as is) with vim and it creates some unwanted visual issues when Tui is disabled.

could you post a minimal config I could use to reproduce? Or a nix expression? Thanks!

version: "0.5"
log_level: debug

processes:
  vim:
    command: vim
    args:
      - compose.yaml
    depends_on:
      dep:
        condition: process_started

  dep:
    command: while true; do echo sleeping... && sleep 0.5; done

To reproduce:

Run: process-compose -t=false
Exit
Run again

I think this can be avoided if interactive processes are disabled when run with others.

I guess vim doesn’t have non-interactive mode so it would only make sense to run it via run. But many cli tools detect tty and work with and without it.
I think we can leave it up to the user to choose the correct command to make their config work. We could also add a new flag which would disable the service when ran via up, but then we’d have to decide if it’s deps should also be disabled.

I guess vim doesn’t have non-interactive mode so it would only make sense to run it via run. But many cli tools detect tty and work with and without it.
I think we can leave it up to the user to choose the correct command to make their config work. We could also add a new flag which would disable the service when ran via up, but then we’d have to decide if it’s deps should also be disabled.

I agree, We can start by letting the users manage the configuration if they want to skip the interactive processes. Since it's an edge case, I think it's a reasonable compromise.
Once we have more feedback and more specific use cases (debuggers, wizards, setup) we can decide on a flag or other behavior.

So, do you think adding the capability to pass additional args from cli to the main process ran with run cmd would be enough for you to accept this PR? Do you have any comments or improvements suggestions regarding implementation?

So, do you think adding the capability to pass additional args from cli to the main process ran with run cmd would be enough for you to accept this PR? Do you have any comments or improvements suggestions regarding implementation?

Yes, it looks good. There are a few things that I would have done differently, some small typos and no documentation, but for a WIP PR, it's in a great shape :)

Yes, it looks good. There are a few things that I would have done differently, some small typos and no documentation, but for a WIP PR, it's in a great shape :)

Ah yes, I’ll definitely add documentation. I’ll also double check for typos. Happy to make any further changes if you provide specific pointers. Thanks!

@F1bonacc1 apologies for the long silence but I’ve been quite busy after coming back from holidays. I should have some time to work on this in about a week as I’ve got a couple of flights to take 😄

@F1bonacc1 apologies for the long silence but I’ve been quite busy after coming back from holidays. I should have some time to work on this in about a week as I’ve got a couple of flights to take 😄

@adrian-gierakowski,
Don't worry about it. I know the feeling all too well 😉

@F1bonacc1 I believe I now got the desired behaviour implemented. I've updated PR description.
Still missing documentation, but I'll add this once the implementation is finished (in case you have any further comments you'd like me to address).
Not sure why tests are failing (haven't ran them before pushing), will have a look at it tomorrow.

@F1bonacc1 not quite sure how to run the tests locally. There doesn't seem to be a way to do it via nix. And I also wasn't able to run the command which is used in github workflow from within nix develop. Would be great if instructions were added on the Readme about how to setup a dev environment, or at least allow the tests to be ran via nix flake check

@adrian-gierakowski,
You are right, I should have made it clearer in the Readme. Especially now when when there are more and more amazing contributions coming our way. Thank you for that!
If you have make you can run make testrace.
If not, go test -race ./src/...  will do the trick.

@adrian-gierakowski, You are right, I should have made it clearer in the Readme. Especially now when when there are more and more amazing contributions coming our way. Thank you for that! If you have make you can run make testrace. If not, go test -race ./src/...  will do the trick.

no luck:
> nix develop
> make testrace
go test -race ./src/...
go: inconsistent vendoring in /home/adrian/code/process-compose:
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
	github.com/InVisionApp/go-health/v2: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
	github.com/cakturk/go-netstat: is replaced in go.mod, but not marked as replaced in vendor/modules.txt

	To ignore the vendor directory, use -mod=readonly or -mod=mod.
	To sync the vendor directory, run:
		go mod vendor
make: *** [Makefile:53: testrace] Error 1

@adrian-gierakowski, You are right, I should have made it clearer in the Readme. Especially now when when there are more and more amazing contributions coming our way. Thank you for that! If you have make you can run make testrace. If not, go test -race ./src/...  will do the trick.

no luck:
> nix develop
> make testrace
go test -race ./src/...
go: inconsistent vendoring in /home/adrian/code/process-compose:
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
<EMAIL_ADDRESS>is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
	github.com/InVisionApp/go-health/v2: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
	github.com/cakturk/go-netstat: is replaced in go.mod, but not marked as replaced in vendor/modules.txt

	To ignore the vendor directory, use -mod=readonly or -mod=mod.
	To sync the vendor directory, run:
		go mod vendor
make: *** [Makefile:53: testrace] Error 1

@adrian-gierakowski,
It looks like you need to get those packages first:
git clone https://github.com/F1bonacc1/process-compose.git
cd process-compose
go get ./...    # probably the missing step, please notice the 3 dots
make testrace

@F1bonacc1 tests are fixed now
do let me know if you have any comments, improvement suggestions or if you'd like me to add some tests, thanks!
if you're happy with the implementation (or at least the interface), I'll start working on docs

Everything looks great @adrian-gierakowski!
I will wait for the documentation before merging.
I will need to move it to the new location that I plan to publish after the merge.
Spoiler: https://f1bonacc1.github.io/


[ZEPPELIN-1941] Fix cron job with release resource option dead lock

What is this PR for?
There is a deadlock in concurrent cron job execution with release resource option.
Scenario:
Two notebook run with cron job that release resource after job finished.
In Notebook.CronJob.execute() method:
T1. note.runAll(); // locked paragraphs(lock) and wait to interpreterSettings(lock)
T2. notebook.getInterpreterFactory().restart() //locked(interpreterSettings) and wait for paragraphs(lock) during jobAbort.
This will trigger a deadlock that cause zeppelin hang.
What type of PR is it?
[Bug Fix ]
Todos

[x] - Fix this by avoid acquire lock in job abort method.

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-1941

How should this be tested?
Outline the steps to test the PR here.
Screenshots (if appropriate)
Questions:

Does the licenses files need update?
Is there breaking changes for older versions?
Does this needs documentation?

Thanks for the contribution. Can you make a test case for it?

we already a test case for this https://github.com/apache/zeppelin/blob/9e75f5b50ac1573d423fa937b801df77d6a36f3c/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java#L734

It seems CI has a network issue
[ERROR] Error requesting archive.
[ERROR] Status: 503
[ERROR] Request options: {
[ERROR]   "uri": "https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2",
[ERROR]   "encoding": null,
[ERROR]   "followRedirect": true,
[ERROR]   "headers": {
[ERROR]     "User-Agent": "yarn/0.18.1 npm/? node/v6.9.1 linux x64"
[ERROR]   },
[ERROR]   "strictSSL": true
[ERROR] }
[ERROR] Response headers: {
[ERROR]   "cache-control": "no-cache",
[ERROR]   "connection": "close",
[ERROR]   "content-type": "text/html"
[ERROR] }
[ERROR] Make sure your network and proxy settings are correct.

Does this PR resolve flaky test that current master encounters?

I'm not sure, this PR is intent to fix the dead lock issue.

Flaky test may be another concurrency issue in scheduler execute jobs.

@Leemoonsoo @jongyoul Any comments for this PR ?

LGTM, I saw your description and it makes sense to me.


Fix forkAff example.

Couldn't figure out why it wouldn't compile for a good ten minutes, maybe this will help a noob going forward :D

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the PR! I've fixed this error in #106, but I'll go ahead and merge since I don't know exactly when we will merge that PR (though hopefully very soon).

Missed #106 sorry. Thanks for merging!


Created a java solution for the [189.RotateArray]#116

I am raising a pull request for issue you mentioned Leetcode (189).

As per your request Changes are done.


Update: Meld.Meld version 3.20.4

Removing old versions to reduce clutter
Microsoft Reviewers: Open in CodeFlow

/azp run


Update agent_windows.go

Log EVERYTHING in the windows agent on startup

Codecov Report

Merging #1631 (d499f32) into master (9d3d10b) will decrease coverage by 10.34%.
The diff coverage is 31.84%.

@@             Coverage Diff             @@
##           master    #1631       +/-   ##
===========================================
- Coverage   63.31%   52.97%   -10.35%
===========================================
  Files         170      181       +11
  Lines       14250    15427     +1177
===========================================
- Hits         9023     8172      -851
- Misses       4292     6336     +2044
+ Partials      935      919       -16

Flag
Coverage Δ

kind-e2e-tests
38.14% <22.04%> (-17.24%)
:arrow_down:

unit-tests
40.48% <23.00%> (-0.80%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

cmd/antrea-agent/agent.go
0.00% <0.00%> (ø)

.../agent/apiserver/handlers/networkpolicy/handler.go
58.33% <ø> (ø)

...gent/controller/noderoute/node_route_controller.go
46.06% <ø> (-0.41%)
:arrow_down:

pkg/agent/proxy/proxier_linux.go
0.00% <0.00%> (-25.00%)
:arrow_down:

pkg/agent/proxy/types/groupcounter.go
0.00% <0.00%> (-95.00%)
:arrow_down:

pkg/agent/proxy/types/types.go
0.00% <0.00%> (-84.62%)
:arrow_down:

pkg/agent/stats/collector.go
92.04% <ø> (-5.69%)
:arrow_down:

pkg/antctl/antctl.go
100.00% <ø> (ø)

pkg/antctl/transform/addressgroup/transform.go
0.00% <0.00%> (ø)

pkg/antctl/transform/appliedtogroup/transform.go
0.00% <0.00%> (ø)

... and 113 more

ok, i fixed the ...s, no big deal either way to me, @tnqn @antoninbas  thanks ! this will help us alot when testing windows

ive lost touch with this PR, closing will reopen another one at some point

ive lost touch with this PR, closing will reopen another one at some point


Add TLS1.2 workaround for code coverage script

PR Summary
GitHub needs TLS1.2. Invoke-WebRequest defaults do not support TLS1.2 and hence code coverage runs are blocked as they cannot get artifacts from GitHub.
This PR adds in the workaround to change the SecurityProtocol to add TLS1.2
Also added -ErrorAction SilentlyContinue so it does not throw a non-terminating error when pwsh.exe is not found.
PR Checklist

[X] PR has a meaningful title

Use the present tense and imperative mood when describing your changes

[X] Summarized changes
[X] Change is not breaking
[X] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
[X] This PR is ready to merge and is not Work in Progress.

If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

User-facing changes

[X] Not Applicable
OR
[ ] User-facing Documentation needed

[ ] Issue filed - Issue link:

Testing - New and feature

[X] Not Applicable or can only be tested interactively
OR
[ ] Make sure you've added a new test if existing tests do not effectively test the code changed

[ ] Add [feature] if the change is significant or affects feature tests

Hmm... I think we should look at add this more globally. I was under the impression that Start-PSBootstrap would be run in all of the CIs before any commands and we wouldn't need to make this change in every location. One the recent PRs should have addresses this.
If we have a common entry point for all activities in all environments, we should make this setting change there. perhaps setting it during import of build.psm1?
Separate issue: consider a patch to Windows PowerShell 5.1 to enable TLS 1.1 and TLS 1.2 by default.

@markekraus This particular script does not run in CI, but a different test machine as the execution time for code coverage runs exceed CI run time limits.
Also, this script does not use Start-PSBootstrap as it does not build PowerShell but uses the packages generated by AppVeyor nightly build.


Removed unnecessary length argument that has been removed from usb.util.get_string.

This fixes "micronfcboard-python doesn't work with latest version of pyusb".

Thanks for the fix!


V20CredFilterIndy possibly wrong

validator for V20CredFilterIndy seems off. See also #55 #56 #57 #58. Simply skipping this validation enables a successful workflow (given correct input, produces correct output against aca-py)
   @validator("cred_def_id")
    def cred_def_id_pattern(cls, value):
        # Property is optional
        if value is None:
            return

        pattern = r"^([123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz]{21,22}):3:CL:(([1-9][0-9]*)|([123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz]{21,22}:2:.+:[0-9.]+)):(.+)?$"
        if not re.match(pattern, value):
            raise ValueError(
                f"Value of cred_def_id does not match regex pattern ('{pattern}')"
            )
        return value

see also https://github.com/didx-xyz/aries-cloudcontroller-python/tree/hack/any-return-type

I think this is fixed now. What was the endpoint you were getting this error with?


Add Source Map to Package

Bug Report

Version

svix 1.21.0
svix-react 1.11.0
Platform

Description

The source maps need to added to the package. I currently have 157 warnings whenever I build my client because of the lack of source mapping. I've seen other people bring up this issue as well.
I know I can turn source mapping off, but that makes debugging in production more difficult.
Please update the package to contain the source maps so these warnings will go away.

Whoops, sorry about that. Also sorry for missing this ticket. Will take a look asap!

I think the reason why you're seeing such warnings is that in svix we are publishing the package with sourceMaps but without the corresponding TS files. I'll send a PR to publish src as well as dist, so the TS files are in the location pointed by the source maps.
svix-react does not publish sourceMaps (yet), so it shouldn't have the same issue.


Remove code in binders related to original use in Spring XD

In https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream-binders/spring-cloud-stream-binder-rabbit/src/main/java/org/springframework/cloud/stream/binder/rabbit/RabbitMessageChannelBinder.java#L469  there is code that creates a new classloader that was originally done to avoid errors such as https://jira.spring.io/browse/XD-2509
As there are not multiple classloaders in spring-cloud-stream's runtime, it is quite likely this code should be removed.

Actually this is not necessary since it has been fixed in spring-amqp anyway.

I am going to remove this with #439


Are There Any Examples Besides the Two in the README?

Love the project! Wondering (as my title suggests) if there are any other examples usages of this library or documentation somewhere? I've successfully hacked my way to getting a value slider working the way I wanted, but I'm having a hard time figuring out the range slider.
Thank you for this project!

Hi @alexwhb,
There are more examples in the Previews. Let me know if this helps.
https://github.com/spacenation/swiftui-sliders/blob/d5a7d856655d5c91f891c2b69d982c30fd5c7bdf/Sources/Sliders/RangeSlider/RangeSlider.swift#L81-L181

@ay42 This is exactly what I was looking for! Thank you so much!


Write pdf file

Hello, i'm using your module to write a pdf on file. I use axios to call API and your module to write the file.
The problem is the final file is incomplete.. it is only 67byte instead the 87kb.
If i open the pdf with a text editor i see that only few rows are written.
is there a fix or i wrong something?

Could you provide an example?  It is hard to say without seeing some code.

Hello,
i use this to write the pdf string to file
await FileSystem.writeFile(path, pdfstring)
the pdf string is about this: (i truncate now to not flood the post)

%PDF-1.7
%����
1 0 obj
<<
/Type /Catalog
/Pages 2 0 R
/PageMode /UseNone
/ViewerPreferences <<
/FitWindow true
/PageLayout /SinglePage
/NonFullScreenPageMode /UseNone

endobj
5 0 obj
<<
/Length 2404
/Filter [ /FlateDecode ]

stream
x��XYs��~ǯ@H\s(Y�E뢽�%�v6�<ɛ����v󐿟��R�U�RI������1C�~�jN��g3m��~����*":��W��f"�t+��~��F��h���Z	��>l��#k�MD?G�?y���;;a����R��ى�`��&]��юƞ$=/?ET53�ܟ3#C�r��ɍ��;2
r?F�\j-5��^
����V
in the final file i have this string ending here x��XYs��~ǯ
all the other content is truncate. I suppose there is some weird character that stop the function?

RN sometimes has trouble passing strings containing binary data between js-side and native-side.  Try base 64 encoding it (something like https://www.npmjs.com/package/base-64 can help), then specifying encoding while writing: FileSystem.writeFile(path, pdfstring, 'base64').
If you have performance issues, you could try using FileSystem.fetch() to download the pdf directly to disk (instead of loading in memory js side, encoding, then passing to native to write to disk).

Thanks for tip.
at this momenti am using your fetch function and work fine, but for my flow was better using axios.
well, i will try with base64 but think it will work.
thanks.

Assuming resolved.  Please open a new issue if not.


Normalization in Golub-Welsch for Gauss-Jacobi correct?

First of all -- thanks for this amazing package; I've been using it quite a bit. However, I have the feeling that there is something fishy with the Golub-Welsch algorithm in gaussjacobi():
For $n<=4000$ and $max(a,b)>=5$, the function JacobiGW(n, a, b) is called, see https://github.com/ajt60gaibb/FastGaussQuadrature.jl/blob/bd0b69e6af7dde596bc831d4beffd8c8f2d76224/src/gaussjacobi.jl#L29
In the function JacobiGW(n, a, b) the Golub-Welsch algorithm is called, i.e. the nodes and weights are computed based on the eigenvalues/-vectors of the Jacobi matrix. The weights are the square of the first entry of the normalized eigenvector times the integral of the weight over the support, $w_j = \mu_0 \v_1^{j}^2$. In gaussjacobi.jl the weights are computed here:
https://github.com/ajt60gaibb/FastGaussQuadrature.jl/blob/bd0b69e6af7dde596bc831d4beffd8c8f2d76224/src/gaussjacobi.jl#L385
with $\mu_0 = 2^(ab+1)*gamma(a+1)*gamma(b+1)/gamma(2+ab)$. But then comes the next line:
https://github.com/ajt60gaibb/FastGaussQuadrature.jl/blob/bd0b69e6af7dde596bc831d4beffd8c8f2d76224/src/gaussjacobi.jl#L386
Why is the division necessary? In fact, this normalization leads to the weights returned by JacobiGW() satisfing $\sum_{i=1} w_i = 1 != \mu_0$ hence a contradiction, in general; the integral of the Jacobi weight is not one in general.
In short: shouldn't line 386 be commented out?
happy holidays! :)

You are correct:
julia> x,w = gaussjacobi(10,6,6)
([-0.815935, -0.66479, -0.492055, -0.302324, -0.101975, 0.101975, 0.302324, 0.492055, 0.66479, 0.815935], [0.000286535, 0.00719355, 0.05063, 0.161847, 0.280042, 0.280042, 0.161847, 0.05063, 0.00719355, 0.000286535])

julia> sum(w) # should be 0.681984681984682 from Mathematica
0.9999999999999999

Would you mind making a PR?

Was this closed by #67?

Yes, it was. Thanks for the book-keeping.


RackspaceLBDriver: vips and sourceaddresses

WHAT
Currently Rackspace has the option of returning multiple virtual IP addresses as part of the return load balancer details.
This adds extra["privateVips"] and extra["publicVips"] containing all the SERVICENET/PUBLIC virtual IPs (respectively).
This also adds extra["ipv6PublicSource"], extra["ipv4PublicSource"], extra["ipv4PrivateSource"].
WHY
Information not otherwise accessible.
I used 'extra' to mirror what was done for the Node object.
Tests all pass + added tests for all new functionality.

merged into trunk with modifications, thanks!


Wrap .NET docs to 80 chars

Draft for now, might want to wait for https://github.com/open-telemetry/opentelemetry.io/pull/1041 to be merged first

PR #1041 has been merged. There are now conflicts with this PR. I'm glad to submit an updated PR if you'd like. Let me know @cartermp what you prefer.

@chalin I'll get this one over the finish line. Conflicts done, just want to re-run the tool on the relevant docs.

@chalin all done now!

\o/


Crashes not working on UWP

Even though I have added Analytics and Crashes nuggets in my .NETStandard project and all 3 head projects (Android, iOS, UWP) and then in App.xaml.cs OnStart() of my .NETStandard project, added below as you suggested by replacing my app secret appropriately:
AppCenter.Start("ios={Your App Secret};android={Your App Secret};uwp={Your App Secret}", typeof(Analytics), typeof(Crashes));
The Crashes are not reported on UWP at all but they are reported on Android and IOS.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5d72f6fe-74d6-696e-e709-5f6d577c0a58
Version Independent ID: b3d0ea8b-5a1f-bf71-7dc2-9685cf23c00b
Content: Get Started with Xamarin - Visual Studio App Center
Content Source: docs/sdk/getting-started/xamarin.md
Service: vs-appcenter
GitHub Login: @elamalani
Microsoft Alias: emalani

@dinobu For support issues, can you please reach out to App Center support by clicking on the blue icon on the lower right of your screen? Someone from our team should be able to help you. Thanks!

@dinobu The comments feature of our documentation is intended to enable customers to suggest changes to the docs or ask clarifying questions about the docs.
App Center support is provided directly from within the App Center portal. Any time you need help, just click the blue chat button in the lower-right corner of any page and our dedicated support team will respond. For additional information, see the App Center Help Center.


:wrench: Created Dockerfile and docker-compose.yml to deploy the project

 Set up a docker-compose to run MongoDB and Spring in one command


feat: Sends emails on Account approve or reject

Two new emails are send after account approval or account rejection. I was not sure about email content so I have improvised a lot, but everything is at templates and locales so super simple to change it at future.
I have added new variable called FRONTEND_URL so I can provide link to user at email -- tbh. because I am not sure about emails content, I am not sure if this is required at this point :).
I also had to update gem "rspec-rails", "6.0.0.rc1", because current version does not work properly with Rails 7 --> have_enqueued_mail  is broken.
Testing instructions
Reject or approve any user via Backoffice -- such user should receive appropriate email. Please test only for accounts that you own their user emails :).
Tracking
https://vizzuality.atlassian.net/browse/LET-514

Please @agnessa , I will need your assistance here :). We need to either add FRONTEND_URL variable to stage or rewrite email of text that this variable is not needed <-- text of the emails is huge brainstorming and I hope a bit that somebody will polish them later on :D


Frontend Project "paragraph-generator" Improved Styling

Title and Issue number
Frontend Project "paragraph-generator" Improved Styling
Close #3632
Output
Before

After

Checklist:

[x] I have mentioned the issue number in my Pull Request.
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md

@MohitGupta121 Kindly review this pull request.

Hi @MohitGupta121
Kindly review this pull request so that I can work on other issues.


Disposer is not called when promise is cancelled

Bluebird version: 3.1.8
Node.js: 7.2.1
Consider the following example code:
const Promise = require('bluebird');
Promise.config({
    cancellation: true
});
const acqPromise = new Promise(function (resolve, reject) {
    setTimeout(resolve, 1000);
});
const disposer = acqPromise.disposer(function () {
    console.log('DISPOSER CALLED');
});
const promise = Promise.using(disposer, function () {
    return Promise.resolve();
});
promise.cancel();

DISPOSER CALLED is never printed in this example (and it is printed when promise.cancel() is not called).
A real-world example of this behavior impact is Knex.js, a database access library that uses Bluebird promises; Disposer interface is used to acquire and release database connections from the pool, so when a query promise is created and canceled like this, the connection gets actually acquired and never released, draining the pool and hanging up the application.

Promise.using(disposer, f)

should desugar to
disposerPromise.then(f).finally(disposeFn)

and in this case, it looks like it doesn't.

Some debugging: using.js has this:
    Disposer.prototype.resource = function () {
        if (this.promise().isFulfilled()) {
            return this.promise().value();
        }
        return NULL;
    };

The disposer is not invoked unless the resource acquiring promise has been fulfilled, and in the example provided it's not yet fulfilled, though it would in be one second, and then something has to release the resource.

And it's not actually pending, it's cancelled - the resource allocator promise is cancelled when the main promise is cancelled. Which means that if resource allocator promise doesn't handle onCancel, it gets "forgotten" as 3.x cancellation semantics suggest. If this behavior is considered correct, whoever uses disposer interface must handle onCancel event in the allocator promise, otherwise there's no guarantee it would be disposed properly. Guess it's worth mentioning in the docs.

@xmirya yeah I was originally going to suggest using onCancel, but on second thought, its better that the disposer is called even if the resource was not fully acquired, and the code provided by the user can hopefully handle that case too.

The problem i see is that resource passed to disposer is what acquiring promise returns. And if it's cancelled, it can't return anything despite the fact it might have been allocated (cancelling the acquiring promise itself seems like a sane idea to me). Thus both allocator and disposer need to track the resource via some external state, not managed by promises API.

@xmirya isn't this already cancelling the acquiring promise? The problem is you cannot use disposer-using libraries that are not cancellation aware; they have to specifically use new Promise. And so the idea would be, "if the promise on which disposer was set up doesn't have an explicit onCancel callback defined, it will not be cancelled from the perspective of the disposer; instead, the disposer will wait for it to be fulfilled or rejected then it will run if there is a fulfilment value

Mean it is already cancelling the acquiring promise and that's ok in case the acquiring promise is aware of onCancel, which i would just implement in my downstream code to make it work w/o any bluebird changes. But the solution looks good.

The disposer cannot be called because there is no resource acquired by the promise.


Fixed #461 : Include timestamp in history

Description
Included timestamp in history
Related issues and discussion
#461
Screen-shots, if any

@l-o-b-s-t-e-r   But it is according to the mocks  by @teolemon . Refer the issue for more info .

@l-o-b-s-t-e-r  Thanks for the suggestion .

How about something like 15 minutes ago, 5 hours ago, 5 days ago ?
And perhaps a tiny barcode icon next to the barcode, a tiny clock/history next to the barcode
And removing the share icon, and replacing it by a color dot for the Nutriscore ?

I have added time as you desired.
I have also added icon of barcode and clock/history as you desired.

Removing the share icon, and replacing it by a color dot for the Nutriscore is another issue #840. We will solve it there.

Hi @PrajwalM2212 thanks for the work, however, could you make the barcode and the clock icon the same size? It looks like the clock is a larger icon.
And we are not working on the share icon in this pull request, could you make sure that you do not edit the original icon? It looks like the scaling was changed...

Share icon edit was by mistake , I have made it right again.

Thank you, @PrajwalM2212, could you make the other changes too and then we can review the PR again?

The clock icon comes from android itself and cannot be changed . The only workaround is to increase size of barcode . Do you want this change ?

Can we perhaps get the same icon as the History one in the menu ?

I will look into it .

@teolemon  I have set the same icon as the one in the menu . I had to create a new drawable as the one used in menu was blue in colour.

Merged, thanks @PrajwalM2212!


Design and implement quorum balancing

As a developer, I need to know what algorithm to follow for quorum assignment in order for the protocol to prevent censorship while being open.
Description
We can consider two degenerate cases for quorum balancing:

When there is a single quorum with all nodes in it, safety is maximized, but so is communication overhead;
When each node live in their own quorum, safety does not exists but there is no communication: a less degenerate version would be having a ring where every node belongs to two quorums, each with another node (or: two two nodes quorums).

The goal of quorum balancing is to produce a network graph with optimal safety & overhead. Note that the nodes that are quorum intersections are likely to be higher-staking nodes, as they have more to lose.
Example of rules:

There is no quorum with < 3 nodes;
There is no quorum with > 7 nodes;
A quorum economic value must be >= 350,000 BOA;
The top X (X=5?) stakers are placed in different quorums and used as 'seed';

Definition of done:

[ ] A new validator is assigned a quorum immediately;
[ ] Quorums get regularly shuffled (how often?);
[ ] There is a sufficient (how much?) quorum intersection;
[ ] Validators are distributed among quorums in an unpredictable manner;
[ ] Validators are distributed among quorums in a fairly balanced way;
[ ] Quorum distribution only depends on globally available state;

Tentative approach:

Come up with a simple algorithm based on the example rules posted above;
Build test case for working and non-working (byzantine) validators;
Compute the economic penalty (low & high bounds) => If node A cheats, home much does it cost ? Is it aligned with the power this node has in the system ?;
Optimize
Repeat

Depends on #209

Some research notes:
This post describes the new quorum config in Stellar, and also has a general overview of how quorums in Stellar work: https://medium.com/stellar-developers-blog/why-quorums-matter-and-how-stellar-approaches-them-547336c1275
The two PRs implementing semi-automatic quorum configuration and an "intersection-checker"
https://github.com/stellar/stellar-core/pull/2125
https://github.com/stellar/stellar-core/pull/2127
Here's a paper linked to from the blog post: https://arxiv.org/pdf/1902.06493.pdf. From the paper:

The Disjoint Quorums Problem answers the question whether a given instance of Federated
Byzantine Agreement System contains two quorums that have no nodes
in common. We show that this problem is NP-complete.

In our case we wouldn't mark any nodes as belong to an "organization" and the "quality" of the nodes would not be hardcoded, but instead we would just use the amount of stake as the marker of quality. After all, if a node misbehaves then it will get slashed, and its staked amount becomes lower (and therefore the "quality" of the node drops).
The blog post describes a commit we're not using yet (https://github.com/stellar/stellar-core/releases/tag/v11.2.0) which was released in June 28th 2019. We're using a commit from May 20th 2019.
It might be possible to reuse / adapt these Stellar routines (Config::generateQuorumSet / Config::generateQuorumSetHelper), if they don't depend on any Stellar-specific state. But we do have to take into account the random value and the quorum re-shuffling too.

Apparently the quorum intersection checker is so expensive, that they've implemented an interrupt mechanism to halt the worker thread while it's calculating the intersection of quorums: https://github.com/stellar/stellar-core/pull/2454
In any case, I will try to use the code in v11.2.0. Although there's still an issue because we're using a slightly older commit, and there seem to be quite a few files which were touched between 324c1bd61b0e9bada63e0d696d799421b00a7950 (commit we're using) and v11.2.0.
So as a first step, I'll see if we can update to v11.2.0 first.

This commit is very sad: https://github.com/stellar/stellar-core/commit/f6a6567d1c33a732e3870612bfa7aa39ea17ecd9

It looks like the code already depended on App and Herder, so instead of removing those dependencies, they just moved the checker into the Herder module and made it more stellar-dependent. It's a shame..
But then again, stellar-core was never meant to be a library.

Ah yeah, I looked at it a while back (as you can see).
From what I remember the dependency is limited to the config struct. If you invert the dependency, a la BanManager, it could work.

So far I've managed to almost make it work by removing the Config dependency. But then I ran into this:
ource/scpp/build/QuorumIntersectionCheckerImpl.o: In function `std::__detail::_Hash_code_base<stellar::PublicKey, std::pair<stellar::PublicKey const, unsigned long>, std::__detail::_Select1st, std::hash<stellar::PublicKey>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, false>::_M_hash_code(stellar::PublicKey const&) const':
/usr/include/c++/9/bits/hashtable_policy.h:1292: undefined reference to `std::hash<stellar::PublicKey>::operator()(stellar::PublicKey const&) const'
source/scpp/build/QuorumIntersectionCheckerImpl.o: In function `std::__detail::_Hash_code_base<stellar::PublicKey, std::pair<stellar::PublicKey const, unsigned long>, std::__detail::_Select1st, std::hash<stellar::PublicKey>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, false>::_M_bucket_index(std::__detail::_Hash_node<std::pair<stellar::PublicKey const, unsigned long>, false> const*, unsigned long) const':
/usr/include/c++/9/bits/hashtable_policy.h:1304: undefined reference to `std::hash<stellar::PublicKey>::operator()(stellar::PublicKey const&) const'
source/scpp/build/QuorumTracker.o: In function `std::__detail::_Hash_code_base<stellar::PublicKey, std::pair<stellar::PublicKey const, std::shared_ptr<stellar::SCPQuorumSet> >, std::__detail::_Select1st, std::hash<stellar::PublicKey>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, false>::_M_hash_code(stellar::PublicKey const&) const':
/usr/include/c++/9/bits/hashtable_policy.h:1292: undefined reference to `std::hash<stellar::PublicKey>::operator()(stellar::PublicKey const&) const'
source/scpp/build/QuorumTracker.o: In function `std::__detail::_Hash_code_base<stellar::PublicKey, std::pair<stellar::PublicKey const, std::shared_ptr<stellar::SCPQuorumSet> >, std::__detail::_Select1st, std::hash<stellar::PublicKey>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, false>::_M_bucket_index(std::__detail::_Hash_node<std::pair<stellar::PublicKey const, std::shared_ptr<stellar::SCPQuorumSet> >, false> const*, unsigned long) const':
/usr/include/c++/9/bits/hashtable_policy.h:1304: undefined reference to `std::hash<stellar::PublicKey>::operator()(stellar::PublicKey const&) const'

It looks like it needs an operator() for hashing support. And we seem to have removed / not ported the hashing code from stellar-core. It might be this one https://github.com/stellar/stellar-core/blob/f3857733a9b67da4528df59bb616ea84ba539a1a/src/crypto/ECDH.cpp#L71, but I'll have to see.

Ah I found it: https://github.com/bpfkorea/agora/blob/ab8b39f475a29d584f736fbdd4662edb8f2b87a3/source/scpp/src/crypto/SecretKey.h#L131
It's declared, but never defined.

Status update of this feature
What's been done

Add ability to provide a custom genesis block.
#662 - Add bindings to unordered_map
#663 - Add bitset implementation
#545 - Workaround for SCPEnvelope memory handling
#546 - Support hashing SCPEnvelopes
#659 - Add missing hashing support in SCP bindings
#655 - Updated SCP bindings to v11.2.0 as it was required to be able to include the quorum intersection checker tool.
#645 - Add registerListener() API so nodes not in a node's quorum also receive SCP messages.
#643 - Connect to all quorum set validators. This is a safety vs liveness issue. For safety reasons a validator node will attempt to connect to all of its quorum nodes. It resolves some test-suite failures.
#637 - Added pretty-printing of SCPEnvelope - This improves the debugging experience a lot.
#629 - Reject outdated SCP messages. If a block was externalized, we do not care about nomination / balloting for blocks of that height or older.
#618 - Refactor network manager to later allow easier retrieval of quorum set hashes (for #621)
#648 - Fixed a broken code path of network discovery
#549 - Improved isQuorumSetSane() to make debugging easier
#577 - Decoupled Nominator and Ledger classes.

PRs that need review

#712 - Custom genesis block. Needed in order to add Enrollments that define the preimages, and we need the preimages in order to derive the random seed for the quorum balancing algorithm.
#664 - Quorum intersection checker. Needed to verify the generated quorums have good intersection properties.
#621 - (needs a rebase) Fix missing quorum set hashes when reaching consensus. Nodes need to be able to look up hash => quorum set, otherwise they may reject messages outright.

What still needs to be worked on

SCPEnvelopes are likely not propagated properly. #716 has a failing test-case, and adding proper propagation seems to resolve part of the issue.
Fixing SCPEnvelope propagation reveals a secondary issue with timers firing out of order. SCP panics when an outdated timer fires, leading it to throw with an error: https://github.com/bpfkorea/agora/blob/5ffc4574b99ebafc7817801ad2770ea2253cd126/source/scpp/src/scp/BallotProtocol.cpp#-L640-L642. We've had these errors sporadically before, but with the propagation fix they happen frequently. I think it's related to how we implemented timers. We spawn fibers and then call fiber.sleep, but that's not an ideal way to write timers.
#684 will need more work. Integration tests have been delayed by the issues above. Additionally the algorithm might need to be more fine-tuned as the integration tests are written.
#723 need to examine the usage of toVec() because it returns pointers to either stack-allocated or GC-allocated memory, and this is a problem for memory management across the C++ boundary.

PRs still in progress, but may be reviewed.
[PR #684](Implement quorum balancing algorithm).

Currently the only part that is not fully implemented is:

Quorums get regularly shuffled (how often?);

Right now the quorums only get reshuffled when the validator set changes (new enrollment, expired enrollment). But we may want to shuffle quorums regularly, for example every N blocks (N would be defined by the protocol).
However I think this will require more preimage support. In our unittests we have tests such as:

Set validation cycle to 20
Generate genesis
Create 19 blocks
Start the nodes

The nodes won't have any preimages on startup except the commitment in the enrollment in the genesis block. If we set the "periodic shuffle" N parameter to 10, then on boot-up the nodes would generate the quorums for block height 0 and then on block height 10. But it's not possible to test this right now because the preimages are missing for height 10, so this would lead to an assertion failure.


Queue Manager Redeploy after Helm Chart Upgrade

Hi Callum,
as discussed today: I switched the MQ helm chart in our repository from version 3.0.0 to latest version 5.0.0 lately. After push to main ArgoCD picked it up but deployment did not succeed after several tries. Only thing that helped to get it deployed was to delete the Queue Manager and redeploy it from scratch.
The error message I got in ArgoCD:
"StatefulSet.apps "t01n01k-ibm-mq" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden."
Thank you,
Uwe

Hi Uwe,
Thanks for raising this issue. The above problem is occurring as we are including metadata labels within the volumeClaimTemplates section of the statefulset. One of these labels include the version of the helm chart, which then prevents the upgrade as Kubernetes will not allow any modifications to this part of the statefulset. Moving forward we will remove any labels within the volumeClaimTemplates that include product or helm chart versions to avoid this occurring in new deployments. Unfortunately as existing deployments already have this present there is little we can do for historically deployed queue managers. In this case the kubernetes resources will need to be deleted, but leaving the PVC claims, and then another deployment which can utilize the same storage. It is worth saying that as we increase major helm versions it may not always be possible to complete a helm upgrade as backward compatibility is not always possible, but in this case it should be.

Closing as addressed in latest helm chart release


Export to PDF issue

Preflight Checklist

[x ] I agree to follow the Code of Conduct that this project adheres to.
[ x] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

You must agree to search and the code of conduct. You must fill in this entire template. If you delete part/all or miss parts out your issue will be closed.
If you are technical, you should reporting bugs along the lines of https://marker.io/blog/how-to-write-bug-report. If you are not technical, we will make allowances, please try to make an effort to understand the process.
Describe the bug
The Export to PDF option seems to be encountering some error, the only message the end user are getting is a simple "Error!" when looking at https://convert.diagrams.net/node/export
To Reproduce
Steps to reproduce the behavior:
Tested the export option from within the normal chrome browser and the draw.io desktop app and both are returning the same behaviour when trying to perform a Export to pdf.
Expected behavior
PDF to be created but it doesnt.
Screenshots

draw.io version (In the Help->About menu of the draw.io editor):

v23.1.1

Desktop (please complete the following information):

OS: Windows10
Browser Chrome
Browser Version 121.0.6167.140 (Official Build) (64-bit)

I tested the problem in incognito/private mode with all browser extensions switched off, write "yes" below:

yes

Additional context
None

Fixed, thanks for report.
The desktop app cannot be doing this, though, what version are you using?

And Thank you issue seems resolved.

OK, I think that's the chrome app. I thought Google discontinued support for those?


[BUG] Can not run npm run build

Information

Version: 5.x
Packages:
"@tsed/common": "5.67.0",
"@tsed/core": "5.67.0",
"@tsed/multipartfiles": "5.67.0",
"@tsed/swagger": "5.67.0",
"@tsed/typeorm": "5.67.0",
"@tsed/platform-express": "5.67.0",
"@tsed/exceptions": "5.67.0",
"@tsed/di": "5.67.0",

I run "npm run build", then it not work with error typescript. See image below

Try to upgrade your typescript please

Try to upgrade your typescript version please

Thank you, it work ok!


Dogecoin Core v1.10 takes up ~2gb of RAM

I was planning on running a Dogecoin Core full node on a Raspberry Pi alongside a Bitcoin Core node. However, I just noticed that Dogecoin Core takes up ~2gb of RAM on my desktop as compared to ~300mb for Bitcoin Core V1.15.
Is this a know issue? Are there plans to address this? Anything I can do to help?

1.14 (over in https://github.com/dogecoin/dogecoin/tree/1.14-branding ) uses a lot less memory, although we do have problems due to sheer number of blocks.

Could you explain what was done to improve the memory consumption? Just
curious.
Also, are the blocks stored in RAM or something? How does the number of
blocks affect the memory usage?
On Feb 12, 2018 1:33 PM, "Ross Nicoll"<EMAIL_ADDRESS>wrote:

1.14 (over in https://github.com/dogecoin/dogecoin/tree/1.14-branding )
uses a lot less memory, although we do have problems due to sheer number of
blocks.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1472#issuecomment-365069439,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB2FVkQH0n6G-j3rrfvX6RNa_X_xtKbfks5tUK44gaJpZM4SBiCM
.

So one of the biggest uses of memory is a block header index. With AuxPoW we have the entire block header of the parent chain as part of our full block headers, essentially doubling memory usage. It also scales (obviously) by number of blocks, worsening the situation. 1.14 (and the 1.11 alpha we never released) remove AuxPoW data from that index; it reduces our ability to sanity check internal state, but that's considered a reasonable trade-off given the data is validated when received.

Wow so much cool. Such programming.
Gonna read up on AuxPoW now. Thanks!
On Feb 14, 2018 11:40 AM, "Ross Nicoll"<EMAIL_ADDRESS>wrote:

So one of the biggest uses of memory is a block header index. With AuxPoW
we have the entire block header of the parent chain as part of our full
block headers, essentially doubling memory usage. It also scales
(obviously) by number of blocks, worsening the situation. 1.14 (and the
1.11 alpha we never released) remove AuxPoW data from that index; it
reduces our ability to sanity check internal state, but that's considered a
reasonable trade-off given the data is validated when received.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1472#issuecomment-365721675,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB2FVuulpmvmUQCQWBLmvRoGzX5DnMbFks5tUzasgaJpZM4SBiCM
.

Hello good time
I bought  Dogecoin on Chanelle but  Dogecoin has not been deposited with me
https://changelly.com/transaction/747c60b84b42
https://dogechain.info/tx/8611f9c5f276580551f9a37a017a74a8cb248892f4c7c99a617de22a60006364
Dogecoin Address: D7nfiu8VzfQVygujFjX6eAVotJHBGWsoN9
My wallet is at www.cryptonator.com
Please check why Dogecoin is not deposited in my account  help help help me
What is the meaning of the following statement?
Not yet redeemed
What time do you deposit to my account
please check it . Thank you
Please, please, I beg you to follow my work, I swear you

A 1.14 node idles at around 880 MB resident memory for me so I'll close here for now.


The documentation doesn't says about requiring graphviz .

The documentation doesn't says about requiring graphviz.
When I tried to run it complained about not having the /usr/bin/dot

@yegor256 please dispatch this issue


Web - New Group - When creating a new group, a console error is displayed

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to URL https://staging.new.expensify.com/
Log in to NewDot with any type of account
Click on the "New group" option
Search for 4 users and add them to the group.

Expected Result:
Console error should not be displayed when creating a new group
Actual Result:
When creating a new group, a console error is displayed
Workaround:
Unknown
Platform:

Where is this issue occurring?

Web

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/199865099-42f80971-e3e3-4cf5-b740-6f67631cdecc.mp4
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

I was not able to reproduce this in newdot. No console errors are present when I create a new group. Adding Eng for a second pair of eyes.

@lschurr sorry, I'm OOO, not sure why melving added me. Can you please reassing?

Also unable to reproduce. Please reopen if we discover the additional steps (or specific account) that makes this reproducable


Cannot build for Android (version 1.3.4)

Trying to build with version 1.3.4 and getting the following error:
`FAILURE: Build failed with an exception.

What went wrong:
Could not resolve all files for configuration ':file_picker:debugCompileClasspath'.

Failed to transform file 'flutter.jar' to match attributes {artifactType=processed-jar} using transform JetifyTransform
Failed to transform '/Users/miguelruivo/DevTools/flutter/bin/cache/artifacts/engine/android-arm64/flutter.jar' using Jetifier. Reason: Cannot open a library at 'FileMapping(from=/Users/miguelruivo/DevTools/flutter/bin/cache/artifacts/engine/android-arm64/flutter.jar, to=/home/chris/.gradle/caches/transforms-1/files-1.1/flutter.jar/238118fc2d1bd61f2da6041b9800dfc1/jetified-flutter.jar)'. (Run with --stacktrace for more details.)
`

Project builds fine with version 1.3.3.

Thank you for noticing and reporting this so fast. I published for the pub.dartlang a version that had a local dependency in the build.gradle when it shouldn't and this would cause every Android build that uses 1.3.4 to fail.
A fix is already deployed with version 1.3.4+1.
Just let me know if it's all working as it should again and, once more, thank you for reporting.

Thank you for the fast fix :)
It is now working again as intended.