Added systemd

I have added support for systemd
Example repo

First, you've copied another language entry and changed (some) fields. Second, you've completely ignored the template we require users to fill out when submitting a pull-request.
Please read our contribution guide next time. Closing this in favour of #5768.


Checkboxes uncheck and recheck themselves inconsistently on iOS

Type: bug
Platform: ios 8 webview
Hi,
I have three checkboxes in my app, and when I click on them in the iOS simulator, it's as if I had clicked twice: they uncheck and recheck themselves inconsistently.
Everything's fine on Android.
To reproduce this bug, just create Checboxes in a template and test in iOS simulator (iPhone 5 portrait, iOS 8.4).

					<input type="checkbox" ng-model="dataCheckbox.type1" ng-true-value="1" ng-false-value="''"></input>
					Small Waves

					<input type="checkbox" ng-model="dataCheckbox.type0" ng-true-value="0" ng-false-value="''"></input>
					Normal Waves

					<input type="checkbox" ng-model="dataCheckbox.type2" ng-true-value="2" ng-false-value="''"></input>
			      	Step-Ups

```

in the corresponding controller:
$scope.dataCheckbox = {};
$scope.dataCheckbox.type1 = '';
$scope.dataCheckbox.type0 = '';
$scope.dataCheckbox.type2 = '';

Greetings @louisdoe!
My sensors indicate that you need to update your issue through our custom issue form. We are now requiring all issues to be submitted this way, to ensure that we have all of the information necessary to fix them as quickly as possible.
Click Here To Update Your Issue
I will have no choice but to close this issue if it is not resubmitted through the form. Please fill out the rest of the form, so that I may use my friendly robot powers to assist you.
Thank you!

Did I put enough info in this issue ? Can someone please advise me on how to write this so you can have a look at it ? I have been stuck with this for several days. Thanks

@louisdoe what version of Ionic are you using? We had a number of checkbox issues fixed recently.

Hi @mlynch I am using ionic 1.1.0
Do you think it is already fixed in 1.1.1 ?

I have just tried with ionic 1.1.1, I still see the issue.
Can you please help me on this, it's stalling the progress on my app.
Thanks

My checkboxes control (filter) a big list below in a collection repeat, it seems that if I remove this list, the issue disappear...
Whoa, it's a bit too complex for me here, can someone help me figure this out ? I could give access to my code if necessary.
Thanks

Can someone please help me here I am stuck !!

The click event fires twice, for a single tap !!! Can you please help me ?

http://stackoverflow.com/questions/30706214/prevent-ios-autofocus-first-input-when-page-opens-ionic
This is related. I have the same issue in the iOS Iphone 5S simulator (running iOS 9.2).
My use case: I open the side menu and click on logout. Right underneath where I would have clicked is my input from the login view. Causing it to auto focus and then all sorts of wierd scrolling behavior happens.
I assume this happens on our actual iPad but then the logout button doesn't appear on the same x,y cords as the input fields.

@PerfectionCSGO thanks for for backing this, I feel so alone in here ! This bug is really annoying and preventing me from publishing, I can't find a way to even workaround.
Did you find a fix yourself ?

I'd recommend switching over to ion-checkbox, or ion-toggle.
They aren't supporting label tags anyway. Check out this blog post and look at the Form Inputs and Radio and Checkbox sections.

@louisdoe I just implemented $ionicLoading when logging out as a work around.

@patrickmcd I switched to the ion-checbox directive as you recommended. And the bug is still here !
Could someone take over my code to see for himself ?
@PerfectionCSGO ok thanks. I don't think I can do this in my checkbox case. But can you give me some code so maybe I get inspired ? ;) thanks.

I reuse this code.
    function (type) {
        $ionicLoading.show({
            template: '
' + type + '
<ion-spinner icon="android"></ion-spinner>'
         });
    },

@PerfectionCSGO  thanks for your code.
I am not sure how to defer the checkbox click, do you have an idea ?
I tried on codepen but I can't really find a way to reproduce the bug. Plus, it is only on iOS webview, would it be possible to check a codepen as a webview?

Is there someone in the ionic team who could directly check our code (I'll grant access to our repo) ? This bug seems highly difficult to reproduce.
Thanks

i am getting a related issue with a modal which when clicked on a close button the underneath button is clicked as well.
concrete issue: everytime when i close my modal with a button (ng-click) on the bottom right corner the tab bar underneath the modal is clicked
Info:
Cordova CLI: 6.0.0
Ionic Version: 1.2.4
Ionic CLI Version: 1.7.14
Ionic App Lib Version: 0.7.0
ios-deploy version: 1.8.2
ios-sim version: 5.0.2
OS: Mac OS X El Capitan
Node Version: v4.0.0
Xcode version: Xcode 7.2.1 Build version 7C1002
is there some help around? getting crazy with that issue

@louisdoe, @keyban, can you provide a codepen to demonstrate/recreate the issue?
Thanks,
Dan

I did one when opening this thread
http://codepen.io/LouisDoe/pen/gPvGXe?editors=1000

Hi @louisdoe, can you give me step-by-step instructions on what to click, where, etc to recreate the issue.
Thanks,
Dan

Hi @louisdoe - we're going to close this due to lack of response.  If you do respond and provide instructions, we'll re-open and take a look.  Thanks for contributing and for using Ionic!
Thanks,
Dan


Why my superset runs so slowly?

It takes over twenty seconds after clicking a link.  Any methods to speed up?
Many thanks.
version: 0.14.1

The one who knows if your machine / database / network is slow or under stress it's you. I don't think we can help there.

Yes, it is running very slow, will take more 20s when refresh data. Did you fix that?

me2.I have installed 2 supersets.version superset-0.17.6.one is in my mac virtualbox.one is in linux vir server.But I got the same problems.When I logged into superset,it took me very long time to show the main page.The second machine even didn't show the page until I stop the service.Anyone know what the problem is .Seems like some network is wrong?

I found my problem is ah very very low bit problem.When I started the sup service,I tapped ctrl + Z,and I thought that was the correct way to let the service run in the background.oh..I forgot it.It is hangout.Just parsed the service.I know the fg! But I forgot the bg.Hope U guys won't follow my fault.

Yes...I too got this problem.
I am working with Superset 0.19.0.  I am using superset as iframe inside my project. The problem is that the superset page is taking more time to load.  Is the problem with Gunicorn webserver? I have changed the worker type of gunicorn from sync to async and tried with eventlet and gevent. Still, it takes more time to load. How can I optimize the speed. Thanks in advance.


Text doesn't appear on some devices until reloading.

Steps to repro:
Write application with text, images, adverts, buttons etc.
Build the APK.
Notice how it works fine on virtual devices and some physical devices. But on some physical devices text doesn't load until user navigates to home screen and back to the application.
|

Hi @danbudworthmead
Can you please provide your flutter doctor -v and a minimal complete reproducible code sample?
Please provide more information about the devices in which you faced this issue.
Thank you

`[√] Flutter (Channel master, 1.26.0-2.0.pre.84, on Microsoft Windows [Version 10.0.19041.685], locale en-GB)
• Flutter version 1.26.0-2.0.pre.84 at Z:\GitHub\flutter
• Framework revision cda1fae6b6 (3 days ago), 2020-12-18 04:04:02 -0500
• Engine revision 1be6f414e7
• Dart version 2.12.0 (build 2.12.0-166.0.dev)
[√] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
• Android SDK at C:\Users\acurl\AppData\Local\Android\sdk
• Platform android-30, build-tools 30.0.2
• Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
• All Android licenses accepted.
[√] Chrome - develop for the web
• Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe
[√] Android Studio (version 4.1.0)
• Android Studio at C:\Program Files\Android\Android Studio
• Flutter plugin can be installed from:
https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
[√] IntelliJ IDEA Community Edition (version 2019.1)
• IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2019.1.3
• Flutter plugin can be installed from:
https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
https://plugins.jetbrains.com/plugin/6351-dart
[√] VS Code (version 1.52.1)
• VS Code at C:\Users\acurl\AppData\Local\Programs\Microsoft VS Code
• Flutter extension can be installed from:
https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter
[√] Connected device (2 available)
• Chrome (web) • chrome • web-javascript • Google Chrome 87.0.4280.88
• Edge (web)   • edge   • web-javascript • Microsoft Edge 87.0.664.60
• No issues found!`

`import 'dart:io';
import 'package:url_launcher/url_launcher.dart';
import 'package:flutter/material.dart';
import 'package:skeld_launch_flutter/SkeldButtons/VideoAdvertButton.dart';
import 'SkeldButtons/PlayButton.dart';
import 'SkeldButtons/RevertButton.dart';
import 'InfoText.dart';
import 'TitleText.dart';
import 'package:admob_flutter/admob_flutter.dart';
void main() {
WidgetsFlutterBinding.ensureInitialized();
runApp(
MaterialApp(
home: Home(
),
),
);
}
class Home extends StatefulWidget {
@override
_HomeState createState() => _HomeState();
}
class _HomeState extends State {
AdmobBannerSize bannerSize;
AdmobInterstitial interstitialAd;
AdmobReward rewardAd;
@override
void initState() {
super.initState();
Admob.initialize();

interstitialAd = AdmobInterstitial(
  adUnitId: getInterstitialAdUnitId(),
  listener: (AdmobAdEvent event, Map<String, dynamic> args) {
    if (event == AdmobAdEvent.loaded) {
      interstitialAd.show();
    }
    handleEvent(event, args, 'Interstitial');
  }
);

rewardAd = AdmobReward(
  adUnitId: getRewardBasedVideoAdUnitId(),
  listener: (AdmobAdEvent event, Map<String, dynamic> args) {
    if (event == AdmobAdEvent.opened) {
      rewardAd.load();
    }
    handleEvent(event, args, 'Reward');
  }
);

interstitialAd.load();
rewardAd.load();

}
@override
void dispose() {
interstitialAd.dispose();
rewardAd.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return Scaffold(
body: InfoText(),
);
}
_launchURL(String url) async {
if (await canLaunch(url)) {
await launch(url);
} else {
throw 'Could not launch $url';
}
}
void handleEvent(AdmobAdEvent event, Map<String, dynamic> args, String adType) {
switch (event) {
case AdmobAdEvent.loaded:
print('New Admob $adType Ad loaded!');
break;
case AdmobAdEvent.opened:
print('Admob $adType Ad opened!');
break;
case AdmobAdEvent.closed:
print('Admob $adType Ad closed!');
break;
case AdmobAdEvent.failedToLoad:
print('Admob $adType failed to load. :(');
break;
default:
}
}`
`import 'package:flutter/material.dart';
class InfoText extends StatelessWidget {
@override
Widget build(BuildContext context) {
return Container(
alignment: Alignment.center,
child: Text(
'Make sure Among Us is closed!\n\n'
'skeld.net is the world's first custom Among Us server.\n\n'
'Play Zombies, Crazy Colors, Twins, and much more.\n\n'
'Played by Ssundee, Socksfor1, H2O Delirious, Jelly, and others.\n\n'
'These adverts help us pay for the costs of running the servers.\n\n'
'skeld.net was created by Saghetti & Arkatme. We love the original '
'Among Us game and wanted to add some new game modes!',
textAlign: TextAlign.center,
style: TextStyle(
color: Colors.black,
fontFamily: 'Arial',
fontSize: 24,
),
),
);
}
}`
This code should just display some black text on a white background. But even that doesn't work.
The black text doesn't appear on one of my staff members Android OS 8 devices and one's Android OS 10 device until they go to the home screen and back into the application.
I am a complete flutter newbie so forgive my code sample.

Upon further investigation, the example application works fine. So it must be an issue with my code.

After a whole lot of testing, I have discovered that Colors.black breaks the application.
This really is a glaring issue.

So, it's like it the physical phone sometimes is unable to load the text.

Possibly connected to https://github.com/flutter/flutter/issues/72632? Do you see a similar error in the console?

in my situation i see popup message "Device frame rate is invalid. Device returned a target frame rate of 0.0 FPS. Using 60 FPS instead.". Only web last version - all is ok.

Uhh I don't know as I don't have the device. One of my friends in another country does and they don't have a PC so are unable to connect it to a PC to check.

I think I got the same issue with master branch. But in my case, sometimes it's even worst, the text is bugged and is bad rendered.
[√] Flutter (Channel master, 1.26.0-2.0.pre.86, on Microsoft Windows [version 10.0.18363.1256], locale fr-FR)
    • Flutter version 1.26.0-2.0.pre.86 at C:\src\flutter
    • Framework revision b793285329 (3 days ago), 2020-12-18 10:14:36 -0800
    • Engine revision 7983c6fda0
    • Dart version 2.12.0 (build 2.12.0-167.0.dev)

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at C:\Users\Akiat\AppData\Local\Android\Sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = C:\Users\Akiat\AppData\Local\Android\Sdk
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • CHROME_EXECUTABLE = C:\Users\Akiat\Desktop\chrome.bat

[√] Android Studio (version 4.1.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[√] VS Code, 64-bit edition (version 1.51.1)
    • VS Code at C:\Program Files\Microsoft VS Code
    • Flutter extension version 3.17.0

[√] Connected device (3 available)
    • sdk gphone x86 (mobile) • emulator-5554 • android-x86    • Android 11 (API 30) (emulator)
    • Chrome (web)            • chrome        • web-javascript • Google Chrome 87.0.4280.88
    • Edge (web)              • edge          • web-javascript • Microsoft Edge 87.0.664.60

• No issues found!

Well, it seems like that version of the SDK was severely fucked.
Grabbing latest from the github fixed all of the issues...


Fix multus setup

The VMs are searching in the default NS instead of the kubevirt also for VM I correct the multus resource name

Thanks for the PR. However, these errors you found are actually there on purpose... there is a warning at the top of lab 8 that mentions that the lab contains errors - the ones you are reporting here.
The idea behind this is to help make sure that students review the files they are working with.
If you think this is confusing or shouldn't be there we can apply the fix - but I would also remove that warking from the lab at the same time.
@scheeles  @slintes @fabiand what do you think?
Until we agree on the best way forward let's
/hold

I liked it, it's a nice little exercise for debugging not starting VMs

Agree I like it as well. I now now how to fix it ;-)


boolean options don't seem to work

Boolean options don't seem to work the way they should. Here is an example:
package main

import (
	"fmt"
	"github.com/docopt/docopt-go"
)

var usage string = `
Mytest.

 The docopt website says:

Use two spaces to separate options with their informal description.

If you replace the tab with two (or more) spaces, it works:
package main

import (
        "fmt"
        "github.com/docopt/docopt-go"
)

var usage string = `
Mytest.

Wouldn't it make sense to permit tabs?
On Thu, Nov 30, 2023, 01:49 Marco Molteni @.***> wrote:

@tmbdev https://github.com/tmbdev it looks like that this ticket can be
closed?
—
Reply to this email directly, view it on GitHub
https://github.com/docopt/docopt.go/issues/73#issuecomment-1833426282,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACRQPZ7LTTSM3RHUPZH4NTYHBJD5AVCNFSM4LSOEOF2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBTGM2DENRSHAZA
.
You are receiving this because you were mentioned.Message ID:
@.***>


(Question) Does it will have a CLI tool beyond examples?

The Dioxus CLI should soon work with Blitz so you can run dx serve --platform desktop and your app will automatically hot-reload

@matthunz I'm aware of the Dioxus CLI, thanks!


Note about starting fig-elk when SELinux is enabled

Centos|Redhat is used in the example since that was the type of distribution I was on. Regardless, the note should point future readers, who might be in the same predicament, in the right direction.

Thanks @cdituri !

no problem, anytime ;-)


Connection Error in worker thread

Using the client, and making a basic connection, a worker thread keeps on spitting out a connection error every 5 seconds.
Reproduce by using the client, and sleeping for 1 second. This does not seem to affect the functionality though.
Version: 1.0.3
Exception in thread "Thread-1" javax.ws.rs.ProcessingException: java.net.ConnectException: Connection refused: connect 	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287) 	at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252) 	at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684) 	at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681) 	at org.glassfish.jersey.internal.Errors.process(Errors.java:315) 	at org.glassfish.jersey.internal.Errors.process(Errors.java:297) 	at org.glassfish.jersey.internal.Errors.process(Errors.java:228) 	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444) 	at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681) 	at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:437) 	at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:343) 	at org.symphonyoss.symphony.agent.invoker.ApiClient.invokeAPI(ApiClient.java:673) 	at org.symphonyoss.symphony.agent.api.DatafeedApi.v1DatafeedCreatePost(DatafeedApi.java:90) 	at org.symphonyoss.symphony.clients.impl.DataFeedClientImpl.createDatafeed(DataFeedClientImpl.java:95) 	at org.symphonyoss.client.services.MessageFeedWorkerV2.initDatafeed(MessageFeedWorkerV2.java:95) 	at org.symphonyoss.client.services.MessageFeedWorkerV2.run(MessageFeedWorkerV2.java:74) 	at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Connection refused: connect 	at java.net.DualStackPlainSocketImpl.connect0(Native Method) 	at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79) 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) 	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) 	at java.net.Socket.connect(Socket.java:589) 	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668) 	at sun.net.NetworkClient.doConnect(NetworkClient.java:175) 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) 	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264) 	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367) 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105) 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999) 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513) 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) 	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) 	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) 	at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:394) 	at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285) 	... 16 more

It actually does impact the functionality of real-time services (MessageService, ChatService, RoomService) and admittedly not the best implementation here.
Side note:  You should check your connectivity to the Agent Server as this is the root cause of the event.
The SJC real-time datafeed (long-poll) automatically tries to recover itself from a network event.
The problem has been catching both runtime exceptions and more importantly showing the health of the upstream system.   There have been some recent additions to the REST API on health checks and I believe I can bubble up the exceptions into the SJC that can be captured, leaving it up to the developer on how to handle the situation (manual shutdown / restart..etc.)
You are not the first one to report issues here, so I'm going to resolve for 1.1.0 milestone.

Hi Frank, when do you think you will get 1.1 out? Oniseijin is working on an important piece for their compliance department that needs to get out shortly.
Is there anything specific he needs to check on the connectivity? the rest of the functionalities of his code seems to be working fine.

Trying to squeeze in all the 1.48 goodies into it, so it's not this week but before Innovate.
The SNAPSHOT probably by this weekend.
So, what he can do right now with 1.0.3 is implement the health check through the REST API language binding which is exposed through SJC.  This can be put into a background thread.   At that point he can literally catch issues and restart the client (SymphonyClient.shutdown() first).
    apiClient = org.symphonyoss.symphony.agent.invoker.Configuration.getDefaultApiClient();
    apiClient.getHttpClient().register(MultiPartFeature.class);

    apiClient.setBasePath(agentUrl);

    SystemApi systemApi= new SystemApi(apiClient);

    V2HealthCheckResponse v2HealthCheckResponse =
    systemApi.v2HealthCheckGet(symAuth.getSessionToken().getToken(), symAuth.getKeyToken().getToken());

     v2HealthCheckRespons.get....(call health checks)

I'll double check the Agent connectivity... for my use cases I didn't need to send messages, just get meta data from the pod; could be agent is down, but that exception is indeed too deep.

Good news then! (I think)
v1.1.0-SNAPSHOT already implements a way to disable real-time services.  This means the client is initialized with no services (this is what's long-polling).  You will still have access to underlying client implementations (assume you are only using POD calls).
        SymphonyClientConfig symphonyClientConfig = new SymphonyClientConfig();

        //Disable all real-time services
        symphonyClientConfig.set(SymphonyClientConfigID.DISABLE_SERVICES, "True");

        //Create an initialized client
        symClient = SymphonyClientFactory.getClient(
                SymphonyClientFactory.TYPE.V4,symphonyClientConfig);

See:  https://github.com/symphonyoss/symphony-java-client/blob/develop/examples/sendmessage/SendMessage.java

Thanks! managed to configure to the proper agent URL. Consider me your chaos monkey this month, but did flesh out a problem under the covers ;)


Run Scheduler and Client from the same MPI Rank

I believe that in some simple cases, we don't need to have a rank dedicated to the Scheduler, and a rank dedicated to the Client. We should provide a way to start both within the same MPI rank in https://github.com/dask/dask-mpi/blob/master/dask_mpi/core.py#L62-L71.
Thoughts?

Seems fine to me.
On Mon, Mar 11, 2019 at 1:25 PM Guillaume Eynard-Bontemps <
<EMAIL_ADDRESS>wrote:

I believe than in some simple case, we don't need to have a rank dedicated
to the Scheduler, and a rank dedicated to the Client. We should provide a
way to start both within the same MPI rank in
https://github.com/dask/dask-mpi/blob/master/dask_mpi/core.py#L62-L71.
Thoughts?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-mpi/issues/29, or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszK-hXbWnE093kNxNhEOH1LLXd9Roks5vVru5gaJpZM4bpfR9
.

@guillaumeeb Sorry for the delay!
I think this is entirely true.  And it's an excellent idea!  In fact, I would like to go a little further than this.  I would like to make the launch algorithm work for any size MPI communicator, not just > 2!
I am considering an optional parameter like separate_scheduler_rank to place the scheduler on a rank different from the client rank.
Here's what I'm considering

If the MPI communicator size is 1, then place the scheduler and a single worker on the same rank as the Client (rank 0).  This is the "LocalCluster" option.  The separate_scheduler_rank option is ignored.
If the MPI communicator size is 2, then place the scheduler on the same rank as the client (rank 0) and start a worker on the other rank (rank 1).  Again, the separate_scheduler_rank option is ignored.
If the MPI communicator size is 3 or more, then place the client on rank 0, the scheduler on rank 0 or rank 1 (depending on separate_scheduler_rank), and the workers on all of the other ranks.

Is this reasonable?

If you think about use case where the MPI rank would be less than 3, then yes. I assumed that there was not really such a use case... But maybe for algorithm testing or debug purpose?
Otherwise I think the idea of an optional parameter is perfectly fine!

I'd like to reboot this issue :D
Is there a reason to not place a worker on each MPI rank? For example if the MPI size is 1: place worker, scheduler and client on rank 0. If the MPI size is >=2: scheduler on rank 0, client on rank 1, workers on all ranks.
If that isn't a fundamentally bad idea I'd propose adding a flag (maybe called workers_on_all_ranks=True?) that leads to this new behaviour. With a default of False so that nothing changes for current users.

Thanks for rebooting this, @betatim!
I have some cycles to devote to this today, so the timing is great!
I've been thinking about how to give the user more explicit control about placement for client, scheduler and workers within the MPI cluster.  To aid with this, I've been thinking about adding the CLI options:

scheduler_rank: The MPI rank of the scheduler.  The default value is 0.
client_rank: The MPI rank of the client, if being used in a stand-alone MPI script.  If the MPI cluster size is 1, then the client rank will be 0.  If the MPI cluster size is 2 or more, then the default value is 1.

They could be the same, and they could be different from 0/1, but they have sensible defaults (chosen for backwards compatibility).
I think the client and scheduler are the easy bits, though, because there are only ever 1 of each of them.
The workers are the harder bits.  I think the default behavior (again, backward compatibility) is for workers to "fill" the remaining MPI cluster, one worker per MPI rank.  But it's the idea of more than one client/scheduler/worker per MPI process that we are discussing, I think.  And if we allow the client and scheduler to run on the same MPI rank, then why not multiple workers?
In terms of implementation, most MPI implementations that I am aware of will not allow multiple processes to be associated with the same MPI rank (and things like forking/spawning new processes from an MPI process can actually be blocked).  I do not know if some MPI implementations will allow it, or if it is safe even if possible, but regardless, the general approach to MPI is that each rank should (ideally) be a single process.  That means that to get multiple client/scheduler/workers to run on a single MPI process will require running them asynchronously in a single process, which is very doable from a "I know how to code that up" perspective... but I don't know if it will be a stable solution.
Does anyone listening in on this conversation have thoughts on this?  In this time zone, maybe @guillaumeeb @jacobtomlinson?
Of course, I don't have a problem with implementing something that gives the user the ability to create an unstable instantiation of a Dask-MPI cluster if it also gives the user more flexibility.  However, I don't want to create general instability for other users who aren't aware of the complexity and subtlety.

In terms of a user CLI to allow complete control over worker placement, the complete solution would involve asking the user to specify a list of one int per MPI rank, with the value of the int indicating the number of workers to launch for the corresponding rank.
That is, something like:
workers_per_rank = [0,0,1,1,1,2]
would describe launching 0 workers on ranks 0 and 1, 1 worker on ranks 2, 3, and 4, and 2 workers on rank 5.  ...But for large clusters, this is untenable.
One possible solution would be to assume that the last int in the list is "repeating" for all additional ranks to the size of the MPI cluster.  For example, one might specify:
workers_per_rank = [0, 0, 1]
which would imply not launching a worker on ranks 0 and 1 and only launching a worker on ranks 2 or higher.  That's the default behavior right now.

So, currently Dask-MPI in the stand-alone script mode only works if you have an MPI comm size of 3 or larger.  This proposed change would make it possible to run Dask-MPI with MPI comm sizes 1 or 2, for which we would need to define sensible defaults.  Here are my suggestions:
MPI comm size 1:
There is only 1 possible choice here, so it doesn't really need to be said.  However, just to be pedantic: scheduler on rank 0, client on rank 0, 1 worker on rank 0.
MPI comm size 2:
I think the default arrangement should be client on rank 0, scheduler on rank 0, and worker on rank 1.  But I'm open for comment here.
MPI comm sizes 3+:
The normal defaults should exist with scheduler on rank 0, client on rank 1, and workers on all other ranks.

My desire to have a worker on the ranks 0 and 1 is that I don't know enough about the scheduler/job submission system I am using to request different types of nodes per rank. So I end up with a gigantic machine with eight GPUs for each rank. Given that I have to queue a while to get the allocation I'd like to use all of it :D Of course an alternative solution would be to figure out if it is possible to receive a smaller node for rank 0 and 1 (a bit more like you'd do in the cloud).
This means for me it is all about "workers on all the ranks". I care less/know nothing about how the scheduler and client are distributed amongst the available ranks.
I also don't use the CLI but call initialize() within my code. But I assume if a CLI option is added there will also be a way to do the same thing via initialize.
I hadn't considered the restrictions on starting new processes and such (can you tell I am a MPI newbie?). I had naively assumed that we'd just add more stuff to the event loop and that would be that. I am +1 on making it hard for people to configure things in a way that makes running unstable. My assumption is that most people "have no clue", so like me they'd just see a wasted node and see that they can configure it to do work and do it. And then have a miserable life because sometimes things work and sometimes they don't. I think it would be better to not give people that rope to hang themselves by and write a short bit of documentation explaining why you can't configure a particular setup.
For "why not more than one worker": I learnt that the word "worker" is overloaded. For example when I instantiate one CUDAWorker I will end up with as many workers (in the sense of entries in the scheduler information) as there are GPUs in the system. From that point of view I think we don't need more than one worker (aka CUDAWorker instance) per rank. From what you said above about "more than one process per rank is often not allowed" it also sounds like in the CPU setting you'd only want one worker per rank?

My desire to have a worker on the ranks 0 and 1 is that I don't know enough about the scheduler/job submission system I am using to request different types of nodes per rank. So I end up with a gigantic machine with eight GPUs for each rank. Given that I have to queue a while to get the allocation I'd like to use all of it :D Of course an alternative solution would be to figure out if it is possible to receive a smaller node for rank 0 and 1 (a bit more like you'd do in the cloud).

Understood!  Then I would lean towards making the "run workers everywhere" option easy to implement, at least.

I also don't use the CLI but call initialize() within my code. But I assume if a CLI option is added there will also be a way to do the same thing via initialize.

Yes.  I think the cli option is actually the easy option.  If we find a solution that works for the initialize() workflow, then the CLI implications will be natural.

I hadn't considered the restrictions on starting new processes and such (can you tell I am a MPI newbie?). I had naively assumed that we'd just add more stuff to the event loop and that would be that. I am +1 on making it hard for people to configure things in a way that makes running unstable. My assumption is that most people "have no clue", so like me they'd just see a wasted node and see that they can configure it to do work and do it. And then have a miserable life because sometimes things work and sometimes they don't. I think it would be better to not give people that rope to hang themselves by and write a short bit of documentation explaining why you can't configure a particular setup.

Yes.  I think that being explicit here would benefit everyone.  Then, at least, people will have a better idea of what to expect before hand.
That said, MPI is always confusing for many people.  Dask-MPI was created out of a simple abstraction that attempted to make launching a dask cluster in an MPI environment easy, but I think that just opened the trail to a steep climb for many people.  Once you're in the door, MPI can be a labyrinth!  (And that's not even going into the complications created by each implementation of MPI being a little different from each other...and from the MPI standard itself!)

For "why not more than one worker": I learnt that the word "worker" is overloaded. For example when I instantiate one CUDAWorker I will end up with as many workers (in the sense of entries in the scheduler information) as there are GPUs in the system. From that point of view I think we don't need more than one worker (aka CUDAWorker instance) per rank. From what you said above about "more than one process per rank is often not allowed" it also sounds like in the CPU setting you'd only want one worker per rank?

Yes.  You've hit on exactly the issue, I think.  That is why the initial implementations of Dask-MPI have always worked in "units of 1".  And I think (because of this overloaded "worker" word) it is safe to continue assuming only 1 worker per rank.
If that were the assumption, then I would modify my CLI/parameter suggestion above to replace the workers_per_rank option with a workers_skip_ranks option like so:
workers_skip_ranks = [0, 1, 5]
which would imply that workers would not be launched on ranks 0, 1 and 5.

For "why not more than one worker": I learnt that the word "worker" is overloaded. For example when I instantiate one CUDAWorker I will end up with as many workers (in the sense of entries in the scheduler information) as there are GPUs in the system. From that point of view I think we don't need more than one worker (aka CUDAWorker instance) per rank. From what you said above about "more than one process per rank is often not allowed" it also sounds like in the CPU setting you'd only want one worker per rank?

Yes.  You've hit on exactly the issue, I think.  That is why the initial implementations of Dask-MPI have always worked in "units of 1".  And I think (because of this overloaded "worker" word) it is safe to continue assuming only 1 worker per rank.
If that were the assumption, then I would modify my CLI/parameter suggestion above to replace the workers_per_rank option with a workers_skip_ranks option like so:
workers_skip_ranks = [0, 1, 5]
which would imply that workers would not be launched on ranks 0, 1 and 5.
Changed my mind.  I think it actually makes sense to just adopt your notion of whether to allow a worker on the same rank as the client/scheduler.  This is the notion of worker exclusivity, or whether a worker has exclusive access to an MPI rank.  Would it be sufficient to just have a boolean exclusive_workers option?  If exclusive_workers=False then workers would be launched on the same ranks as the client and scheduler.  If exclusive_ranks=True, then the default behavior of "no workers on client and scheduler ranks` would apply.

I think this makes a load of sense. Especially on large nodes.
Thinking purely from a GPU perspective there are a bunch of different configurations I would love to have (assuming the rank 0 node has 8 GPUs):

Give the Client and Scheduler their own GPU, populate the other 6 with workers
Give the Client and scheduler a shared GPU, populate the others with 7 workers
Start 8 workers, have the client share with a worker and the scheduler share with a different worker

I'm also playing around with https://github.com/jacobtomlinson/dask-hpc-runner which is based on https://github.com/dask/distributed/pull/4710 and https://github.com/dask/dask-mpi/pull/69. My main goal here is to make the dask-mpi deployment paradigm (populate an existing allocation with a Dask cluster) easier to do in non-MPI scenarios.
My motivating use case is SLURM where we have env vars to get the ranks and a shared filesystem to communicate the scheduler address. It would be nice to leave MPI out of things altogether because there can be consequences to importing mpi4py especially in GPU workloads.
There are other places like Databricks where it would also be useful to have this paradigm for side-loading a Dask cluster onto a Spark cluster.

@jacobtomlinson:  Oooh!  Excited to hear about progress on the dask-hpc-runner.  Been interested in that for a while.  And I agree that the use cases are compelling.
Thanks for chiming in on these suggestions.  It sounds like the 3 bulletted configurations you described would be satisfied by the options I suggested above.  I'm working on a PR now.

Good point about assigning one of the GPUs (each) to the scheduler and client.

Ok.  I've been playing around with this today and there are solutions, but I don't think they are as pretty as anyone was hoping.  So, I'm sharing my experiences in the hopes that someone might have better ideas than I have.
First, it's easy to launch a worker on the same MPI rank as the scheduler.  The scheduler and worker are both asynchronous tasks, and therefore they can both be run on the same asynchronous event loop, which is started with the asyncio.run_until_complete() function.
However, the complication comes from the fact that the client code is synchronous.  And launching an asynchronous task inside of synchronous code can only be done by starting an event loop with either asyncio.run_until_complete() or asyncio.run_forever(), both of which are blocking until the asynchronous tasks are canceled or completed.  If we were to try to launch the scheduler in the same process as the client, the asyncio.run_until_complete() function would block the execution of the client code indefinitely because the shutdown signal would never be sent (because it is sent by the client code which is blocked).
The only way to resolve this issue is for the event loop that runs in the client process to either: (1) start an event loop in a separate thread or (2) start an event loop in a separate process (i.e., fork or spawn).  For option 2, this violates the 1 process per MPI rank rule that I mentioned above, so I think we should exclude it from consideration.  For option 1, however, this means starting a thread pool in the client code, starting the event loop in a separate thread, continuing execution of the main thread (i.e., the client code) and then shutting down the thread when the event loop is complete.
Option 1 is technically doable, but I feel like it is messy, and running processes on MPI ranks that randomly grab extra threads without being explicit about it seems like a way of creating problems for users later on.  ...But it's possible.
What do people think about this?

This is a fun rabbit hole. I spent a ton of time thinking about async-in-sync, sync-in-async, sync-in-async-in-sync, etc recently.
To clarify the client code can either be sync or async, but we have no control over how the user decides to implement things so we have to support both.
If we are in a situation where we want to start more than one component per rank it's probably because each node is big and a single rank refers to a substantial machine. Do you think it's fair to say in these cases that the "one process per rank" rule will not be enforced? It seems bonkers for an HPC admin to enforce a single process on a machine with 8 GPUs. How on earth do they expect to saturate that hardware?

I don't think it is true that one MPI rank typically equates to one machine/node. In mixed threading/processing jobs, one MPI rank typically equates to one machine, with the assumption that the cores on the machine will be saturated with the appropriate number of threads. In plain CPU multiprocessing jobs, multiple MPI ranks are assigned to a single machine, typically one MPI rank per core.   Obviously, it depends on how the scheduler is configured,but all of my HPC experience has followed that rule.

To be clear about this, MPI typically doesn't decide where to place its ranks.  Or rather, the user does not usually need to tell MPI where to place its ranks.  This can be done with something called a machinefile, which lists the names of the machines on which to launch MPI ranks.  Such a machinefile can look like this:
node001:16
node002:16
node003:16
node004:16

which would tell MPI to launch 16 ranks on node001, 16 ranks on node002, etc.
However, in practice, this is almost never done because this process is handled by the HPC resource manager, like Slurm, LSF, PBS, etc.  Depending on what you request from the resource manager, it will essentially construct a machinesfile to place multiple MPI ranks across your hardware.  Different queue policies can be used to enforce things like "nodes need to be full" (i.e., 1 MPI rank per core) or "nodes can be shared across multiple jobs" or similar.
Like I mentioned above, some MPI implementations will actually crash if you try to spawn or fork a process from the process assigned to an MPI rank.  While some MPI implementations may allow it, it is generally considered best practice with MPI to assume 1 process per MPI rank.
And everything I've just described suggests the dilemma with using threads.  The HPC user expects MPI ranks to be placed in a way that uses the entire requested resource (i.e., "fills the nodes").  This gets hard when some processes are creating threads that the user didn't know about when originally requesting the resources.
But you make an excellent point @jacobtomlinson about the fact that the client code can be sync or async.  This problem goes away entirely if the client code is async because then the client code could be run in the same async event loop as the scheduler and/or worker.  However, to make it possible to run async client code, it seems to me that initialize methodology needs to change.  Currently, its mode of operation is via an asynchronous "pass-through function" like so:
from dask_mpi import initialize
initialize(...)  # <-- Start scheduler and workers for MPI ranks other than the client MPI rank

# Client code starts here...

where the client code is executed only on the client MPI rank because the worker and scheduler MPI ranks block due to their running event loops.  But if the client MPI rank is the same as the scheduler MPI rank, then the scheduler's event loop will block and prevent the execution of the client code.  To make it possible to run this asynchronously, one would need to change this to something like:
from dask_mpi import initialize

async def client_code():
    # Client code starts here

initialize(..., coroutine=client_code())

where the initialize method takes the client_code() coroutine and runs it on the same event loop used for the scheduler and/or worker...and only on the client MPI rank.
I don't want to spend my whole day just writing up my thoughts/notes, though.  So, let me investigate a bit about how one might change the Dask-MPI API to make this kind of operation possible.

Thanks for the info, I've only ever used machines where node==rank so this is good to know and definitely affects the design here.
I'm very keen to move away from the initialize function because it is all sorts of magic and is often a point of confusion for users. In dask-hpc-runners I'm playing around with using context managers instead, which still feels magic, but less-so. However that does open you up to async with a lot more easily.
My big concern about letting user code and Dask code share an event loop like this is that if a user accidentally does any blocking code they will very likely run into deadlocks that will be painful to debug. I think it would always be safer to start a new loop on a separate thread and run the Dask components there to avoid them getting deadlocked by user code on the main event loop.

@jacobtomlinson:  Thanks for the reply!  I agree that there is enough "magic" going on here that I don't want to charge forward with anything that makes things worse.
You are correct about the accidentally blocking user code scenario.  I hadn't really thought about that, but it is a problem.  And it would be a nightmare to diagnose.  I think you are correct that always executing the Dask code in its own event loop is the safest bet.  I'll put a crude template in place in the form of a PR and then we can discuss further.

One thought based on a conversation with @jacobtomlinson: maybe the solution isn't to try and run scheduler and client on the same rank but instead to run more than one rank per node. This would solve my problem of "wasting" a whole node on the scheduler and one more on the client. It might be differently complicated to achieve this, but wanted to mention it.
I think for the queue system/cluster I am using I can set --ntasks-per-node which I assume influences the number of MPI ranks per nodes?!

Ok. I'm working on a solution to this, but I think it needs to change the fundamental way that the dask_mpi.initialize() function works.  In fact, it's so large of a change, I think it needs to be a completely new function, maybe called dask_mpi.execute().  The idea would be this:

The client code would need to be "wrapped" in a function.  It could be an asynchronous function, but it would still need to be wrapped in a function regardless.

The dask_mpi.execute(...) function would take the client function as input (or the coroutine) and execute it on the specified client_rank with all of the additional options (e.g., scheduler_rank, exclusive_workers, ...).

To achieve the current Dask-MPI behavior, where workers, scheduler, and client are all in separate MPI ranks, the execute() function might look like:
def execute(func, *args, **kwargs):
    comm = mpi4py.MPI.COMM_WORLD
    this_rank = comm.Get_rank()

    if this_rank == 0:
        async def run_scheduler():
            async with Scheduler(...) as scheduler:
                comm.Barrier()
                await scheduler.finished()
        asyncio.run_until_complete(run_scheduler())

    elif this_rank == 1:
        comm.Barrier()

        ret = func(*args, **kwargs)

        with Client() as client:
            client.shutdown()

        return ret

    else:
        comm.Barrier()

        async def run_worker():
            async with Worker(...) as worker:
                await worker.finished()

        asyncio.run_until_complete(run_worker())

There would be an obvious modification for func being a Coroutine object to run func asynchronously.
And the above modifications to make it more customizable and make MPI rank placement easier would be easy to implement, too.
I particularly like this approach for a number of reasons:

It is much more explicit, and it removes a lot of the previous Dask-MPI "magic" which, as @jacobtomlinson has already pointed out, makes it hard for new users to use Dask-MPI and diagnose problems with Dask-MPI when they have them.

The dask_mpi.execute() function is essentially a decorator, which is an appropriate paradigm to fit the function of Dask-MPI (as opposed to a context manager).

It makes it easy to run the client code in a thread to prevent collision with a scheduler or worker event loop.

What do people think?

Using a decorator is an interesting idea. I'm not sure the code above would behave quite right though, a decorator expects to return a function, not the return value of the function. Everything in that example would be called at the time of decoration, not when the user function is called.
I wonder if the user's function should expect to take the client as an argument too?

Sorry for the delay.
Yeah. It's not a decorator yet. But it could be easily transformed into one.
As for taking the client in as input, I would assume that is the user's choice. The user could pass it in or create it inside the custom function.

Ok.  I'm looking at this a bit today, again.  First, I have created a mock-up of the solution I described above in https://github.com/dask/dask-mpi/pull/110.  I'd appreciate any comments on that solution in a review, if you have time.
In #110, the execute() function wraps the user's "client" code in a function and runs that function in its own thread on the MPI rank selected (client_rank).  Because the client code executes in its own thread, all of the rest of the Dask-MPI functionality can be executed "normally", without interference.  The user can customize the placement of the scheduler with scheduler_rank, and the user can force placement of workers on all MPI ranks (regardless of where the client and scheduler are located) with the exclusive_workers=False setting.

Now that I've had some experience playing around implementing #110, I've discovered a new way of accomplishing the same things.  And I like this approach even more.
The original intent behind the initialize() function was to make it so that a user could run a client script (inside which Dask code exists) with a Dask cluster created over the MPI ranks.  Originally, the idea was that each component--scheduler, client code, and workers--would run on its own MPI rank.  With this issue, however, we have started discussing overlapping components and running multiple components in the same MPI rank.  Since the Dask components (worker and scheduler) are asynchronous, this is possible in the same process.  And since the client code can always be run in a separate thread, it is possible to run all three components in the same process, too.  However, doing this inside initialize() function, as it was structured, was not possible.
Now, with #110, we can do all of this with the execute() function (see above), but it required changing the way Dask-MPI operates.  Namely, you have to submit a function to execute().
Now, this brings me to something new.  If we exploit Python's built-in runpy.run_path function, we can actually go back to the idea of just "running a script" with a Dask cluster built over the MPI ranks.  Basically, what this would mean is going from:
def my_client_code():
    # define client code here

dask_mpi.execute(my_client_code, **execute_kwargs)

to
import runpy

dask_mpi.execute(runpy.run_path, my_client_script, **execute_kwargs)

where my_client_script is a stand-alone script containing the code inside the my_client_code() function.
In other words, the use of dask_mpi.execute would change from:

create a script called my_dask_mpi_script.py with:
import dask_mpi

def my_client_code(*args, **kwargs):
    # define client code here

dask_mpi.execute(my_client_code, *args, ..., **kwargs)

and run this script with:
mpirun -np N python my_dask_mpi_script.py

to the new way of running things:

create a script called my_dask_mpi_script.py containing only the client code:
# define client code here

and run this script with:
mpirun -np N dask-mpi --script my_dask_mpi_script.py

This would unify the batch-mode Dask-MPI operation with the interactive (CLI-based) operation.  The dask-mpi CLI tool would "do everything", allowing the user to either launch a Dask-MPI cluster from the command-line, or run a client script in batch-mode.

Today I can call initialize from somewhere in my code, or not use it and do something else (say LocalCUDACluster). In my case my code is somewhere inside some "framework" that has its own CLI, does setup and bookkeeping stuff, etc. Somewhere in that "framework" code my code gets called "hey, time to get to work, do your thing now!". At this point I want to setup my cluster and do "my thing". This means it is super useful to have a function I can call, instead of it being the other way around with mpirun -np N dask-mpi --script my_dask_mpi_script.py.
I think mpirun -np N dask-mpi --script my_dask_mpi_script.py looks really nice and it solves a problem we've discussed above regarding naming/control flow/etc. I just think there are enough instances where the user wants to "own the __main__" instead of dask-mpi owning it.

@betatim I agree with you entirely.  My intention is to make the execute() function part of the public API of the dask_mpi package.  So, you will still have an "initialize()-like interface, but I'm not sure that is what you are asking for.

At this point I want to setup my cluster and do "my thing". This means it is super useful to have a function I can call, instead of it being the other way around with mpirun -np N dask-mpi --script my_dask_mpi_script.py.

This is really the tricky point, because if you want to use MPI, then you must execute your MPI program with mpirun/mpiexec.  Even if you try to use the MPI dynamical capabilities such as Spawn, you need to launch the main program with mpirun or mpiexec.  That is just how MPI works.
So, for your case, if you have an external framework that (I assume) is not an MPI-based framework, and you want to use Dask-MPI to launch a cluster "on demand" from within the framework, then the only way to do it is to execute a subprocess (e.g., subprocess.Popen(["mpirun", "-n", str(N), "my_mpi_program"])).  I have searched for ways to skip the need for mpirun/mpiexec for a long time, but there just isn't a way out.  At least not right now (though maybe I should check again with the most up-to-date versions of MPI and mpi4py).
Even Dask-Jobqueue uses subprocess to submit a job to a resource manager (PBS, LSF, etc.).
If anyone knows of a way to get around the need for mpirun and mpiexec and to do what you are looking for, @betatim, I am open to implementing a solution.  For I have wanted a solution like that for a long time, too!

Incidentally, the mpirical package (https://github.com/kmpaul/mpirical) is the closest success I've ever had at trying to accomplish something like "run this function with MPI".  It still uses mpirun or mpiexec under the hood, and tries to pass pickled objects back and forth between the processes so that return values can still be retrieved (even if the MPI code was run in another process).  It's not perfect, though, which I think is part of the reason why it just stagnated.

This is really the tricky point, because if you want to use MPI, then you must execute your MPI program with mpirun/mpiexec. Even if you try to use the MPI dynamical capabilities such as Spawn, you need to launch the main program with mpirun or mpiexec. That is just how MPI works.

Agreed that you need to use mpirun, and I do. The thing I was trying to say is that I don't want to mpirun -np N dask-mpi --script my_dask_mpi_script.py - here dask-mpi "owns the main". Instead what I'd like to do is mpirun -np N python some_entry_point.py --foo 42 --bar 21 make_it_so - here some_entry_point.py "owns the main" and eventually calls the code in which I want to use execute() to setup the dask cluster.

Great. Then I understand you. And that should be doable with the execute() function.  I hope that is satisfactory.


Added logical axiomatization of classes & use of reasoning

Currently not completed, just putting this out early cause I'm fully making this up, so if any sections you think needs to be added, or any red flags of stupid things I've written comes up - can make a start of it.
Not gonna ping or add any reviewers for now as I don't want to burden anyone with my half done work

@dosumis - I think this is what you meant for the logical axiomatization docs? Figured I'll ask you first in general before asking others to review in case i totally missed the mark. Thanks

@bvarner-ebi thanks for proof-reading. In the future, happy for you to just go in and make changes directly to the branch if it is trivial changes :) can easily see the diff you changed anyway. Thanks!


A generator of schedules for Genesis tests

The main motivation of this PR is to implement a generator of schedules for genesis tests.
It can be found in commit d1617a8, and a first file to look at could be
ouroboros-consensus-diffusion/test/consensus-test/Test/Consensus/PointSchedule/SinglePeer.hs
Unit tests are provided in ouroboros-consensus-diffusion/test/consensus-test/Test/Consensus/PointSchedule/Tests.hs which look for flaws in the generated schedules.
The generators have been integrated in commit fd0aece with the provisional Praos/Genesis test of a long range attack.
For the rest, this PR includes:

commits 21da791 and 5085ea2 with tests to check that our materialization of a Praos node enforces timeouts and follows rollbacks,
commit d410936 which fixes the chain schema generator so alternative chains are guaranteed to be less dense than the honest chain
commits 13472d2, 0948d8c, and 51bd716 with various improvements to the peer simulator and the chain schema generator.

I just force-pushed commits after rebasing the result of merging #434. I updated all the commit hashes in the description.

Rebased on top of more recent main (706c4b162072a4ccf4d484007130f080f5c1a183).


Enable feeding the server configuration from environment variables

Following up #762. This PR enables the substitution of certain configuration item from environment variables.
In future it can be reused as long as the configuration is loaded from dropwizard.

Coverage increased (+0.003%) to 39.077% when pulling 2c0e0618cb1a354da7656b1324518e874476e0f4 on feature/enable-env-variables into 5486030a933e0d8c846f440d555953cc3de4ad27 on master.

In the new yaml configuration, there is no more specific server config file, just a unique one.
While this mechanism could work for the new yaml config, I am trying to have something more generic using Java reflection which does not require changing the yaml file and follow the same format as for the current property file (e.g. --env GROBID__NB_THREADS=10 ).
So this PR will probably not be merged, because the yaml-config branch is almost ready.

I am trying to have something more generic using Java reflection which does not require changing the yaml file and follow the same format as for the current property file (e.g. --env GROBID__NB_THREADS=10).

Ok I did not manage to find a generic solution. I think nothing reasonable can work with yaml list. It would be probably better to forget about overriding config. parameter with environment variables? I think this can work only for trivial configuration.

No problem.
For the YAML de-serialisation perhaps using Jackson would solve some of the problems in the mapping. If it works as the JSON/XML mapper it should be rather simple to implement even with complex nested YAML config files...

For the YAML de-serialisation perhaps using Jackson would solve some of the problems in the mapping. If it works as the JSON/XML mapper it should be rather simple to implement even with complex nested YAML config files...

The problem is the mapping of environment variable (which are kind of linear serialization) to override the configuration class/object, which is nested with lists.

The solution adopted from dropwizard (we could borrow the idea.. and the code)  is to specify the variable directly in the configuration, so that the complexity is left out, without losing flexibility.

Luca, how do you see the process with the new yaml config? It would be only user-driven? I don't think we can hack the yaml file for all the possible parameter values, because it has potentially much more than hundred of possible parameter keys now (each model has its own set of nested parameters).

Luca, how do you see the process with the new yaml config? It would be only user-driven? I don't think we can hack the yaml file for all the possible parameter values, because it has potentially much more than hundred of possible parameter keys now (each model has its own set of nested parameters).

Yes, let's keep it simple. User can always define their own configuration and decide which variables can be read from the env.

I've updated this PR. Now the users can select which information can be read from environment variables by using
${VARIABLE_NAME:- $$default_value$$}
E.g. for the grobid-home:
${GROBID__GROBID_HOME:- "grobid-home"}
To keep it simple we can just leave the user to decide what to replace (or we can add some variables that can be useful e.g. for the docker container"

When I tries to write the documentation for this, I realize that there is no clear way to get it working with docker, for example:
docker run -t --rm --init -p 8080:8070 -p 8081:8071 \
    --env GROBID_MODEL_PRELOAD=true \
    --env GROBID_GROBID_HOME="/some/where/grobid-home" \
    lfoppiano/grobid:0.7.0

... because the grobid.yaml in the image does not have these environment variable definitions. Of course if we need to modify the configuration file, we are in the configuration file that is mounted when running the container, so we don't need environment variable.
So the user would need not only its own definitions in the configuration file, but also rebuild a new image with the modified configuration file.

Yes indeed. My idea would be that once you've set up your configuration file with the parameters, then you can easily run the docker by changing the parameters at the command line, instead of editing the file.
Maybe others that are using more the docker architecture could provide more useful use cases? @de-code, what do you think?

Yes, for me the main use-case would also look like that. Mainly with docker / container (but not strictly).
There would be a default configuration and environment variables make it easy to override some of those values (including setting previously unset values). This is to make deployment easier but could also make experimentation easier in the same way.
This to potentially consider:

Allow overriding only specific configuration options rather than providing everything (this could also be possible by merging multiple configuration files)
Running a single docker command setting environment variables may be easier to copy and paste by users as an example, than a multi-step example with a separate config file
Deploying container in Kubernetes would require be easier with environment variables than a more bespoke deployment config supporting the configuration file
Environment variables can almost always be set

(see also #480 - I had my previous implementation based on Airflow's config)

Sorry if I was not clear, the problem I raise is the complexity of the solution:

the idea it to have a mechanism to easily run the docker by changing the parameters at the command line, instead of editing the file,
but to realize this solution we have to

edit the config file (anyway)
rebuild an image (which takes ages for the complete one with DL)
then we can run the docker by changing the parameters at the command line

The consequences are that the solution can never be compatible with the images on dockerhub and it requires some heavy preliminary work that finally makes it look more complicated that just specifying a config file as parameters at the command line.
I initially started to document it in the Docker section, but this might need to go to the "developer notes" because the existing docker images cannot support this.

I guess you could include a default config file with all of the placeholders in Docker Hub?
Sub-projects would need to do the same, or anyone with a different default config.
But it does indeed look like a bit of effort to maintain all of those placeholders (and prone to copy and paste errors).
That is why Airflow' config opted for a more general pattern.
You mentioned that it doesn't work well with lists, which is true. One could try to make it work maybe. One challenge would be to select the right item, not based on the index (e.g. by model name).
But maybe it will be easier to revise the config to avoid lists for things that could avoid it.
For example I would maybe want to override the model parameters which are currently a list, but could also be a map (with the name as the key, using underscore instead of hyphen).
Perhaps a bit similar (although not environment variables), Helm install supports --set (and variations) to override values in the yaml file. It also has some limitations but works for many cases.

Sorry if I was not clear, the problem I raise is the complexity of the solution:
[...]
Yes, the solution is supposed to be for advanced users not for simple users of the docker version...  But IMHO I don't think a rebuild of the docker image is necessary.

I guess you could include a default config file with all of the placeholders in Docker Hub?

We can include some placeholder by default so that certain basic configurations can be modified from the docker image without supplying an additional config file. E.g. grobid home, number of threads, lazy/eager load of models... etc...
Personally, for more advance usege, I would modify the config file and add placeholders where I need to, supply such values to the docker image and I will have the possibility to run a different configuration without modifying the configuration file.

I think this is not further needed.


docker rmi doesn't remove image

Hi,
I'm probably missing something but I've a problem with removing image:
yed@rublan: ~ $ docker -v
Docker version 0.7.4, build 010d74e

yed@rublan: ~ $ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
<none>              <none>              d9bfa321bc0e        2 days ago          335.1 MB
<none>              <none>              c7d71211b4a4        2 days ago          345.2 MB
<none>              <none>              1372145c5bd5        3 days ago          183.4 MB
<none>              <none>              4f5105b30e0b        3 weeks ago         575.5 MB
<none>              <none>              d9574283b781        3 weeks ago         175.3 MB
ubuntu              12.10               b750fe79269d        9 months ago        175.3 MB
ubuntu              quantal             b750fe79269d        9 months ago        175.3 MB

now I try to remove one of these images, but it fails (not sure why):
yed@rublan: ~ $ docker rmi d9bfa321bc0e
Error: Conflict, d9bfa321bc0e wasn't deleted
2014/01/10 16:01:47 Error: failed to remove one or more images

however when I tag it and then I try to remove given tag, it gives me a bit more verbose message (which doesn't help though):
yed@rublan: ~ $ docker tag d9bfa321bc0e  foo
yed@rublan: ~ $ docker rmi foo
Error: Conflict, cannot delete d9bfa321bc0e because the container defb8317ee1e is using it
2014/01/10 16:01:55 Error: failed to remove one or more images

more of that: there's no image defb8317ee1e (at least not in the docker images output).

way


Change from ckan_sdk to ckanclient as library name and import Client directly

Change from
from ckan_sdk import client
myclient = client.Client(...)

to
from ckanclient import Client
myclient = Client(...)

Closed by #18


deliver fails when app is "Waiting for Review"

Reopening issue: https://github.com/fastlane/fastlane/issues/6010 to reinitiate the conversation.
When the app is in "Waiting for review" status on iTunes Connect, deliver will fail with error Problem processing review submission.. According to the issue #6010 this is an expected behavior. I just want to understand if this is still the case. Is there any other way to reject the version on the iTunes connect and resubmit with deliver automatically?

I'd like to help, can you provide some more information about this issue, maybe you can run in verbose the command and add it to the issue. 🚀

@minuscorp Just add deliver(submit_for_review: true, force: true) to your lane and run it when the latest version is in "Waiting for Review" status in iTunes Connect.

correct. Maybe there could be a parameter which will reject the current build on iTunes Connect and submit the new one.

Hey folks! I think thats a really great idea @RishabhTayal, it would be nice to have that option. We would definitely take a look at a PR with that added. Let us know if we can help! 🚀

Reading the stub that produces the error: this, I'm wondering if the only difference between a successful review and one that doesn't is the error key with null content... If there's no certain message that Apple uses to really confirm that the review was errored because it's being reviewed... How can we do this?

@ohayon where can I start with the implementation? I know about the YOUR_FIRST_PR.md. I am talking about the technical details about the feature?

Hey @RishabhTayal - I think this file would maybe be a good place to start tracing through some of the code. Also, if you're not familiar, you can use pry to help with debugging and stepping through code too 😄 .

@ohayon I will check that. What's a good way to mock the app store submission while development, since I don't want to keep on submitting a build for dev purpose.

@RishabhTayal I think we have some fixture json files that have mocked responses, you should be able to use WebMock and return those as necessary 👍

@ohayon Could you explain that more? Is there any link/resource I can refer to?


GameObject import does not import material names (FTV-661)

While trying to replicate @jcowles import of the Attic USD data set, I found that all imported materials are named "Standard (Clone)" despite the materials having names in the USD file.
Would be great if the name survived import!

Yeah, I was just noticing this the other day myself. We get those generic names because we clone the material and never give it a name. I think it should be pretty easy to fix, as long as we have the USD surface in hand when we clone the material.

Closing. The PRs have been merged in dev.


Version control of Design System WATT

Description
We want to create WATT as a package to make it easily accessible for products that want to use the design system.
By having it as a package a product would use a specific version we would avoid breaking changes.

Closing in favor of #1211


WS reporting incorrect amount o people in room.

Describe the bug
The WebSocket and front end room counter is reporting the wrong number of people on a room.
To Reproduce
Steps to reproduce the behavior:

Go to https://dogehouse.tv
Look at any room which is mid-large.
Observe the listener count
Join the room and count how many people are in it
The count will be different

You can also see the bug in action on this public API: https://api.dogehouse.xyz/v1/popularRooms. Observe one of the rooms and see how the numPeopleInside field is different to the actual people in the room on dogehouse.tv. not peoplePreviewList

I think this is dup of #1135

Duplicate of #1135


Update dataset_user.json

Hi, Pranav here, made some additions. 1st time using fork. Did the changes via github itself

looks gud


Email template footer can not be empty

Describe the issue:
It does not allow keeping the footer empty in email templates. The Console has a UI validation and carbon managament console throws an error if user tried to keep footer blank and update. But there could be valid cases where a footer is not required. For example, the default email templates also does not include a footer and just --- are added. So it is visible in the emails sent out.
Note the --- at the left bottom corner.

Workaround:
Use following in the footer
<!--  -->

Environment information

Product Version: Any

Closing since the changes are deployed in the product-is


ID indirections

If I rename a building's short ID, then "permanent" links change.  We have a way to redirect links?

This is implemented now.  There is a "redirects" key in the yaml file which can be used for this.  Currently everything has to be done manually.


Parse fail: AttributeError: 'NoneType' object has no attribute 'find_all'`

Hi,
Love the app—it has found me some great music over the past few years.
Recently (6 months) it has stopped working. I imagine the Bandcamp API has changed, but what do I know!
This is the output:
λ bandcamp-player.exe synthwave INFO:root:request: https://bandcamp.com/tag/synthwave Traceback (most recent call last):   File "C:\Users\rmalone\scoop\apps\python\current\Scripts\bandcamp-player-script.py", line 11, in <module>     load_entry_point('bandcamp-player==0.2.0', 'console_scripts', 'bandcamp-player')()   File "c:\users\rmalone\scoop\apps\python\3.7.4\lib\site-packages\bandcamp_player\__init__.py", line 26, in main     loop()   File "c:\users\rmalone\scoop\apps\python\3.7.4\lib\site-packages\bandcamp_player\__init__.py", line 16, in loop     album_url = tag_data.album_random().href   File "c:\users\rmalone\scoop\apps\python\3.7.4\lib\site-packages\bandcamp_parser\tag.py", line 32, in album_random     albums = self.albums()   File "c:\users\rmalone\scoop\apps\python\3.7.4\lib\site-packages\bandcamp_parser\tag.py", line 27, in albums     results = soup.find('div', attrs={'class': 'results'}).find_all('a') AttributeError: 'NoneType' object has no attribute 'find_all'
You haven't updated this Git in ages, but if you know someone might use it again I hope you wish to renew your interest.
Regards, R

hi @richardmalone, you can check out https://github.com/hopdebee/bandcamp-player-evo, which has gotten rid of the python based scraping and now uses an external js library to get the album urls.


change dependence name

Proposed changes
Change in requirement pillow to Pillow

Types of changes
What types of changes does your code introduce to this project?
Put an x in the boxes that apply 😄🚀

[x] Bugfix (non-breaking change which fixes an issue) 🐛
[ ] New feature (non-breaking change which adds functionality) ✨
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) 💥
[ ] Documentation Update (if none of the other choices apply) 📖

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code. ✅

[x] I have read the CONTRIBUTING.md 📚
[x] I have added tests that prove my fix is effective or that my feature works ✅✔
[x] I have added necessary documentation (if appropriate) 📝

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc... 💡❓
References and related issues (e.g. #1234)
N/A 📌

Will not implement chimeragpt, as it's quite suspicious.


svix-server: 1.38.0 -> 1.40.0

Diff: https://github.com/svix/svix-webhooks/compare/v1.38.0...v1.40.0
Changelog: https://github.com/svix/svix-webhooks/releases/tag/v1.40.0

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 356034

aarch64-darwin

:white_check_mark: 1 package built:

svix-server


New Feature - Advanced Request Aggregation

First of all thanks for the effort you guys put in the awesome project!
New Feature: Advanced Request Aggregation
A more advanced version of the Request Aggregation Feature to allow smarter composition of multiple request and or subsequent request.
Motivation for New Feautre
One of the problems developers face when developing applications that talks to microservices is that you will get a chatty client(lots of little request).  By using a [api gateway(http://microservices.io/patterns/apigateway.html)  we can work towards a optimized api for each client. To further support the effort for getting a optimized a api gateway should have a feature as API Composition which you guys made in to the awesome feature of Request Aggregation
Proposal
Currently there is support for creating a aggregate route by defining keys on routes and aggregating the keys into on route. It would be awesome if we could provide more intelligence to the processing of the route.
Perhaps a way of adding a config option to the aggregate route that will instantiate a instance of a IAggregateRouteComposer.  Which would get the input params of the route
public interface IAggregateRouteComposer
{
    Task<object> Process(object input);
}

public class CustomAggregateRouteComposer : IAggregateRouteComposer
{
    public Task<object> Process(object input)
    {
        //compose response
     }
}

{
    "ReRoutes": [
        {
            "DownstreamPathTemplate": "/customer",
            "UpstreamPathTemplate": "/customer/{customerId}",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 51881
                }
            ],
            "Key": "Customer"
        },
        {
            "DownstreamPathTemplate": "/orders",
            "UpstreamPathTemplate": "/orders",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 51882
                }
            ],
            "Key": "Orders"
        }
    ],
    "Aggregates": [
        {
            "Composer": "CustomAggregateRouteComposer",
            "UpstreamPathTemplate": "/product-detail/{productId}"
        }
    ]
}

@maartenmensink thank you very much for your interest in the project.
I agree with you, we need a better / smarter way to do request aggregation.
I like your approach so let's leave this issue open and I will get to it at some point. If you would like to have a go yourself I am very happy to accept PRs and support / advise in the implementation.
Once again thank you very much! 😄

I think I will make it so you have to configure the aggregates and reroutes but you will be able to provide the implementation that maps the reroutes into the reponse. Interface might be something like..
public Task<HttpResponseMessage> Map(List<HttpResponseMessage> originalResponses)
{
        //do stuff with original responses
       return new HttpResponseMessage() { Content = asdf };
}

This means you can even aggregate the headers you want returning etc...

I think if we have a way to configure all scenario's this would be a great way to quickly expand the gateway api. If we could support scenarios like the following scenario.
Customer is placing a order at webshop. Customer retrieves the basket and you want to return product suggestions for all the products in the current basket.
/basket
gives you the current basket
{
	lines:[{
		productId: 1
	},
	{
		productId: 2
	}]
}

/suggestions?productId=1&productId2
gives you the suggestion for the your basket.
{
	suggestions:[{
		productId: 1
	},
	{
		productId: 2
	}]
}

We would need a way of expressing output params and be able to provide them as input params
/basket => productIds = lines.Select(p => p.productId)
/suggestions <= productIds

Ive started hacking around in #310 will try make this nicer...it would allow you to do the scenario above but is clunky...for me there is tradeoff between clunky + what a user needs to hand roll themselves vs cleaner, less hand roll, less functionality.

@maartenmensink I have thought about the feature you need and can't think of a better way to implement it that GraphQL.
To clarify in your scenario you want to retrieve the customers basket and use the products in the basket offer them suggestions.
Ocelot doesn't really have a concept of proper types, it just pushes http (strings) around really. In order for Ocelot to be able to get the basket, parse it, then get the suggestions would require a major change in Philosophy. I just want Ocelot to be a reverse proxy with a few extra features. GraphQL is doing a much better job than I could do for that :)
Anyway I recently added a sample to show how you can easily integrate GraphQL with Ocelot and this might help you out, it is here.
In addition to this Microsoft are looking into this problem in their eShopOnContainers project, the branch is here. The approach is to use Ocelot as an API gateway to handle cross cutting concerns but pass the request on to an aggregating service that would handle the basket / suggestions logic above.
I hope this makes sense and I would like to close the issue now!

@maartenmensink  How did you solve your problem with basket and suggestions? I have similar issue right now.

At this moment we have dedicated WebApi Controllers that handle request that handle data composition. This way we can handle complex cases and later on create the right plumbing to handle it genericly.


subsys: settings: Fix missing dependency on FLASH_PAGE_LAYOUT

If CONFIG_FLASH_PAGE_LAYOUT is not enabled, building the settings code
fails at the linking stage with the following error:
libzephyr.a(settings_init.c.obj): In function `settings_init_fcb':
subsys/settings/src/settings_init.c:62: undefined reference to `flash_area_get_sectors'
collect2: error: ld returned 1 exit status

Signed-off-by: Johan Hedberg<EMAIL_ADDRESS>

Another question: NFFS does select FLASH_PAGE_LAYOUT instead of depends on FLASH_PAGE_LAYOUT. Should I also use select for SETTINGS? What's the reasoning for choosing one over the other?

@andrzej-kaczmarek could you comment on the above questions? git blame seems to point at you at least regarding the NFFS Kconfig entry :)

@jhedberg this was according to review comment [1] but tbh I do not remember the exact reason for this. but since comment was made by @carlescufi perhaps he knows the answer :)
[1] https://github.com/zephyrproject-rtos/zephyr/pull/1288#pullrequestreview-63636842

Note regarding the CI failure: it's a false-positive due to a "too long" commit message line of a verbatim copy-pasted line from the build error.

It is possible to use any FS supported by  VFS along with the settings, but so far only NFFS implements API
for file rename. Function on which compilation fails is dedicated to initialize while FCB is the settings back-end.

Another question: NFFS does select FLASH_PAGE_LAYOUT instead of depends on FLASH_PAGE_LAYOUT. Should I also use select for SETTINGS? What's the reasoning for choosing one over the other?

The only reason was to ensure that NFFS was the top-level Kconfig option to enable, and that would select automatically the flash page layout, which at least at the time was optional.