installing dask-mpi with pip and using it in interactive jobs

dask-mpi looks great!
I tried to install and use it simply but things did not work as expected.
For a minimum example I prepared a binder ready repo where installed dask-mpi and I created 2 notebooks:

Test.ipynb contains the test according to the documentation,
Interactive_Cluster.ipynb contains my attempt to use dask-mpi on interactive jobs, following the corresponding documentation.

In a second branch, called conda I tried the same but this time using conda instead of pip.
The tests are failing using pip
FileNotFoundError: [Errno 2] No such file or directory: '/srv/conda/envs/notebook/bin/mpirun'

However mpirun is available under /usr/bin/
The tests look ok using conda, but I am not able to have a running cluster.
I would like to understand where I can improve.
Thanks!

@aplamada could you share the full test output?

        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/srv/conda/envs/notebook/bin/mpirun'

/srv/conda/envs/notebook/lib/python3.8/subprocess.py:1706: FileNotFoundError
________________________ test_basic[distributed.Nanny] _________________________

loop = <tornado.platform.asyncio.AsyncIOLoop object at 0x7f6509f85400>
worker_class = 'distributed.Nanny', mpirun = ['mpirun']

    @pytest.mark.parametrize(
        "worker_class",
        ["distributed.Worker", "distributed.Nanny", "dask_cuda.CUDAWorker"],
    )
    def test_basic(loop, worker_class, mpirun):
        try:
            import_term(worker_class)
        except (ImportError, AttributeError):
            pytest.skip(
                "Cannot import {}, perhaps it is not installed".format(worker_class)
            )
        with tmpfile(extension="json") as fn:

            cmd = mpirun + [
                "-np",
                "4",
                "dask-mpi",
                "--scheduler-file",
                fn,
                "--worker-class",
                worker_class,
            ]

>           with popen(cmd):

/home/jovyan/dask-mpi/dask_mpi/tests/test_cli.py:46:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/srv/conda/envs/notebook/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/srv/conda/envs/notebook/lib/python3.8/site-packages/distributed/utils_test.py:1050: in popen
    proc = subprocess.Popen(args, **kwargs)
/srv/conda/envs/notebook/lib/python3.8/subprocess.py:858: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x7f6509f85940>
args = ['/srv/conda/envs/notebook/bin/mpirun', '-np', '4', 'dask-mpi', '--scheduler-file', '/tmp/tmp55h2f9wz.json', ...]
executable = b'/srv/conda/envs/notebook/bin/mpirun', preexec_fn = None
close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = 19
c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True
start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals, start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/srv/conda/envs/notebook/bin/mpirun'

/srv/conda/envs/notebook/lib/python3.8/subprocess.py:1706: FileNotFoundError
______________________________ test_no_scheduler _______________________________

loop = <tornado.platform.asyncio.AsyncIOLoop object at 0x7f650a180850>
mpirun = ['mpirun']

    def test_no_scheduler(loop, mpirun):
        with tmpfile(extension="json") as fn:

            cmd = mpirun + ["-np", "2", "dask-mpi", "--scheduler-file", fn]

>           with popen(cmd, stdin=FNULL):

/home/jovyan/dask-mpi/dask_mpi/tests/test_cli.py:61:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/srv/conda/envs/notebook/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/srv/conda/envs/notebook/lib/python3.8/site-packages/distributed/utils_test.py:1050: in popen
    proc = subprocess.Popen(args, **kwargs)
/srv/conda/envs/notebook/lib/python3.8/subprocess.py:858: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x7f650a180520>
args = ['/srv/conda/envs/notebook/bin/mpirun', '-np', '2', 'dask-mpi', '--scheduler-file', '/tmp/tmpzsv7y2wb.json']
executable = b'/srv/conda/envs/notebook/bin/mpirun', preexec_fn = None
close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
creationflags = 0, shell = False, p2cread = 15, p2cwrite = -1, c2pread = 19
c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True
start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals, start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/srv/conda/envs/notebook/bin/mpirun'

/srv/conda/envs/notebook/lib/python3.8/subprocess.py:1706: FileNotFoundError
________________________________ test_dashboard ________________________________

loop = <tornado.platform.asyncio.AsyncIOLoop object at 0x7f650a2adee0>
mpirun = ['mpirun']

    def test_dashboard(loop, mpirun):
        with tmpfile(extension="json") as fn:

            cmd = mpirun + [
                "-np",
                "2",
                "dask-mpi",
                "--scheduler-file",
                fn,
                "--dashboard-address",
                ":59583",
            ]

>           with popen(cmd, stdin=FNULL):

/home/jovyan/dask-mpi/dask_mpi/tests/test_cli.py:153:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/srv/conda/envs/notebook/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/srv/conda/envs/notebook/lib/python3.8/site-packages/distributed/utils_test.py:1050: in popen
    proc = subprocess.Popen(args, **kwargs)
/srv/conda/envs/notebook/lib/python3.8/subprocess.py:858: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x7f650a340d30>
args = ['/srv/conda/envs/notebook/bin/mpirun', '-np', '2', 'dask-mpi', '--scheduler-file', '/tmp/tmp_4z5i5b1.json', ...]
executable = b'/srv/conda/envs/notebook/bin/mpirun', preexec_fn = None
close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
creationflags = 0, shell = False, p2cread = 15, p2cwrite = -1, c2pread = 19
c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True
start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals, start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/srv/conda/envs/notebook/bin/mpirun'

/srv/conda/envs/notebook/lib/python3.8/subprocess.py:1706: FileNotFoundError
============================= slowest 10 durations =============================
5.02s call     dask_mpi/tests/test_cli.py::test_stale_sched[--nanny]
5.01s call     dask_mpi/tests/test_cli.py::test_stale_sched[--no-nanny]
3.08s call     dask_mpi/tests/test_core.py::test_basic
0.50s teardown dask_mpi/tests/test_cli.py::test_basic[distributed.Worker]
0.50s teardown dask_mpi/tests/test_cli.py::test_dashboard
0.50s teardown dask_mpi/tests/test_cli.py::test_stale_sched[--nanny]
0.50s teardown dask_mpi/tests/test_cli.py::test_no_scheduler
0.50s teardown dask_mpi/tests/test_cli.py::test_stale_sched[--no-nanny]
0.50s teardown dask_mpi/tests/test_cli.py::test_basic[distributed.Nanny]
0.50s teardown dask_mpi/tests/test_cli.py::test_basic[dask_cuda.CUDAWorker]
=========================== short test summary info ============================
SKIPPED [1] ../home/jovyan/dask-mpi/dask_mpi/tests/test_cli.py:31: Cannot import dask_cuda.CUDAWorker, perhaps it is not installed
SKIPPED [2] ../home/jovyan/dask-mpi/dask_mpi/tests/test_cli.py:87: Should we explicilty expose --worker-port?
SKIPPED [1] ../home/jovyan/dask-mpi/dask_mpi/tests/test_cli.py:160: Should we expose this option?
=================== 4 failed, 3 passed, 4 skipped in 17.66s ====================

Looking at the tests it seems to just specify mpirun. So I wonder if something is up with your PATH.
Can you do which -a mpirun?

Sure
$ which -a mpirun
/usr/bin/mpirun

So I tried the same also on my mac and it looks like mpirun is expected in the environment, e.g. I created a .venv and I got
FileNotFoundError: [Errno 2] No such file or directory: .... /dask-mpi/.venv/bin/mpirun
I also noticed that the test are using exclusively conda.

Interestingly if I run through the tests in a simple Python docker container I see similar errors where it can't find mpirun but it isn't looking in the Python environment for it.
$ docker run -it --rm python bash
# apt-get update && apt-get install openmpi-bin -y
# git clone https://github.com/dask/dask-mpi.git
# cd dask-mpi/
# pip install dask[distributed] requests mpi4py pytest pytest-asyncio pytest-timeout
# pip install -e .

        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/mpirun'

/usr/local/lib/python3.8/subprocess.py:1702: FileNotFoundError
________________________ test_basic[distributed.Nanny] _________________________

loop = <tornado.platform.asyncio.AsyncIOLoop object at 0x7fefc1041a30>
worker_class = 'distributed.Nanny', mpirun = ['mpirun', '--allow-run-as-root']

    @pytest.mark.parametrize(
        "worker_class",
        ["distributed.Worker", "distributed.Nanny", "dask_cuda.CUDAWorker"],
    )
    def test_basic(loop, worker_class, mpirun):
        try:
            import_term(worker_class)
        except (ImportError, AttributeError):
            pytest.skip(
                "Cannot import {}, perhaps it is not installed".format(worker_class)
            )
        with tmpfile(extension="json") as fn:

            cmd = mpirun + [
                "-np",
                "4",
                "dask-mpi",
                "--scheduler-file",
                fn,
                "--worker-class",
                worker_class,
            ]

>           with popen(cmd):

dask_mpi/tests/test_cli.py:46:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/usr/local/lib/python3.8/site-packages/distributed/utils_test.py:1050: in popen
    proc = subprocess.Popen(args, **kwargs)
/usr/local/lib/python3.8/subprocess.py:854: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x7fefc1041eb0>
args = ['/usr/local/bin/mpirun', '--allow-run-as-root', '-np', '4', 'dask-mpi', '--scheduler-file', ...]
executable = b'/usr/local/bin/mpirun', preexec_fn = None, close_fds = True
pass_fds = (), cwd = None, env = None, startupinfo = None, creationflags = 0
shell = False, p2cread = -1, p2cwrite = -1, c2pread = 18, c2pwrite = 19
errread = 20, errwrite = 21, restore_signals = True, start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals, start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/mpirun'

/usr/local/lib/python3.8/subprocess.py:1702: FileNotFoundError
______________________________ test_no_scheduler _______________________________

loop = <tornado.platform.asyncio.AsyncIOLoop object at 0x7fefc123c340>
mpirun = ['mpirun', '--allow-run-as-root']

    def test_no_scheduler(loop, mpirun):
        with tmpfile(extension="json") as fn:

            cmd = mpirun + ["-np", "2", "dask-mpi", "--scheduler-file", fn]

>           with popen(cmd, stdin=FNULL):

dask_mpi/tests/test_cli.py:61:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/usr/local/lib/python3.8/site-packages/distributed/utils_test.py:1050: in popen
    proc = subprocess.Popen(args, **kwargs)
/usr/local/lib/python3.8/subprocess.py:854: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x7fefc128c4c0>
args = ['/usr/local/bin/mpirun', '--allow-run-as-root', '-np', '2', 'dask-mpi', '--scheduler-file', ...]
executable = b'/usr/local/bin/mpirun', preexec_fn = None, close_fds = True
pass_fds = (), cwd = None, env = None, startupinfo = None, creationflags = 0
shell = False, p2cread = 14, p2cwrite = -1, c2pread = 18, c2pwrite = 19
errread = 20, errwrite = 21, restore_signals = True, start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals, start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/mpirun'

/usr/local/lib/python3.8/subprocess.py:1702: FileNotFoundError
________________________________ test_dashboard ________________________________

loop = <tornado.platform.asyncio.AsyncIOLoop object at 0x7fefc13dc970>
mpirun = ['mpirun', '--allow-run-as-root']

    def test_dashboard(loop, mpirun):
        with tmpfile(extension="json") as fn:

            cmd = mpirun + [
                "-np",
                "2",
                "dask-mpi",
                "--scheduler-file",
                fn,
                "--dashboard-address",
                ":59583",
            ]

>           with popen(cmd, stdin=FNULL):

dask_mpi/tests/test_cli.py:153:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.8/contextlib.py:113: in __enter__
    return next(self.gen)
/usr/local/lib/python3.8/site-packages/distributed/utils_test.py:1050: in popen
    proc = subprocess.Popen(args, **kwargs)
/usr/local/lib/python3.8/subprocess.py:854: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <subprocess.Popen object at 0x7fefc13dc2e0>
args = ['/usr/local/bin/mpirun', '--allow-run-as-root', '-np', '2', 'dask-mpi', '--scheduler-file', ...]
executable = b'/usr/local/bin/mpirun', preexec_fn = None, close_fds = True
pass_fds = (), cwd = None, env = None, startupinfo = None, creationflags = 0
shell = False, p2cread = 14, p2cwrite = -1, c2pread = 18, c2pwrite = 19
errread = 20, errwrite = 21, restore_signals = True, start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals, start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session, preexec_fn)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)

            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)

            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)

        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass

            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                child_exec_never_called = (err_msg == "noexec")
                if child_exec_never_called:
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
>               raise child_exception_type(errno_num, err_msg, err_filename)
E               FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/mpirun'

/usr/local/lib/python3.8/subprocess.py:1702: FileNotFoundError
============================= slowest 10 durations =============================
5.01s call     dask_mpi/tests/test_cli.py::test_stale_sched[--nanny]
5.01s call     dask_mpi/tests/test_cli.py::test_stale_sched[--no-nanny]
1.38s call     dask_mpi/tests/test_core.py::test_basic
0.50s teardown dask_mpi/tests/test_cli.py::test_no_scheduler
0.50s teardown dask_mpi/tests/test_cli.py::test_basic[distributed.Nanny]
0.50s teardown dask_mpi/tests/test_cli.py::test_dashboard
0.50s teardown dask_mpi/tests/test_cli.py::test_basic[dask_cuda.CUDAWorker]
0.50s teardown dask_mpi/tests/test_cli.py::test_basic[distributed.Worker]
0.50s teardown dask_mpi/tests/test_cli.py::test_stale_sched[--nanny]
0.50s teardown dask_mpi/tests/test_cli.py::test_stale_sched[--no-nanny]
=========================== short test summary info ============================
SKIPPED [1] dask_mpi/tests/test_cli.py:31: Cannot import dask_cuda.CUDAWorker, perhaps it is not installed
SKIPPED [2] dask_mpi/tests/test_cli.py:87: Should we explicilty expose --worker-port?
SKIPPED [1] dask_mpi/tests/test_cli.py:160: Should we expose this option?
=================== 4 failed, 3 passed, 4 skipped in 15.67s ====================

It looks like distributed.utils_test.popen is the culprit here. I'll raise an issue over there to explore this.
https://github.com/dask/distributed/blob/942f2355892117ed95723b909cbcc7a672c6b269/distributed/utils_test.py#L1053

I think heading down this testing route is getting off track from your actual problem though.
You mentioned in your original post that you can't run dask-mpi outside of pytest either.
Could you share an example of what you are doing and the errors you see?

Yes. Thanks!
I tried to follow the documentation:
$ mpirun -np 4 dask-mpi --scheduler-file ~/scheduler.json

Output.

```
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at:    tcp://<IP_ADDRESS>:37069
distributed.scheduler - INFO -   dashboard at:                     :8787
distributed.nanny - INFO -         Start Nanny at: 'tcp://<IP_ADDRESS>:44465'
distributed.nanny - INFO -         Start Nanny at: 'tcp://<IP_ADDRESS>:46441'
distributed.nanny - INFO -         Start Nanny at: 'tcp://<IP_ADDRESS>:45255'
distributed.diskutils - INFO - Found stale lock file and directory '/home/jovyan/dask-worker-space/worker-4c0vfn69', purging
distributed.diskutils - INFO - Found stale lock file and directory '/home/jovyan/dask-worker-space/worker-ba1mb5l8', purging
distributed.diskutils - INFO - Found stale lock file and directory '/home/jovyan/dask-worker-space/worker-xs12ygfl', purging
distributed.worker - INFO -       Start worker at:     tcp://<IP_ADDRESS>:37523
distributed.worker - INFO -          Listening to:     tcp://<IP_ADDRESS>:37523
distributed.worker - INFO -          dashboard at:           <IP_ADDRESS>:36903
distributed.worker - INFO - Waiting to connect to:     tcp://<IP_ADDRESS>:37069
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          1
distributed.worker - INFO -                Memory:                   2.00 GiB
distributed.worker - INFO -       Local Directory: /home/jovyan/dask-worker-space/worker-9hf54nel
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.worker - INFO -         Registered to:     tcp://<IP_ADDRESS>:37069
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:37523
distributed.core - INFO - Starting established connection
distributed.core - INFO - Starting established connection
distributed.worker - INFO -       Start worker at:     tcp://<IP_ADDRESS>:41195
distributed.worker - INFO -          Listening to:     tcp://<IP_ADDRESS>:41195
distributed.worker - INFO -          dashboard at:           <IP_ADDRESS>:33341
distributed.worker - INFO - Waiting to connect to:     tcp://<IP_ADDRESS>:37069
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          1
distributed.worker - INFO -                Memory:                   2.00 GiB
distributed.worker - INFO -       Local Directory: /home/jovyan/dask-worker-space/worker-hq3en_u5
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.worker - INFO -         Registered to:     tcp://<IP_ADDRESS>:37069
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection
distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:41195
distributed.core - INFO - Starting established connection
distributed.worker - INFO -       Start worker at:     tcp://<IP_ADDRESS>:38473
distributed.worker - INFO -          Listening to:     tcp://<IP_ADDRESS>:38473
distributed.worker - INFO -          dashboard at:           <IP_ADDRESS>:35987
distributed.worker - INFO - Waiting to connect to:     tcp://<IP_ADDRESS>:37069
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          1
distributed.worker - INFO -                Memory:                   2.00 GiB
distributed.worker - INFO -       Local Directory: /home/jovyan/dask-worker-space/worker-uaoj0551
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:38473
distributed.core - INFO - Starting established connection
distributed.worker - INFO -         Registered to:     tcp://<IP_ADDRESS>:37069
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection
^C^Cjovyan@jupyter-aplamada-2ddask-2dmpi-2dtest-2durdxljuz:~$ ^C
jovyan@jupyter-aplamada-2ddask-2dmpi-2dtest-2durdxljuz:~$ ALLOW_RUN_AS_ROOT=true
jovyan@jupyter-aplamada-2ddask-2dmpi-2dtest-2durdxljuz:~$ mpirun -np 4 dask-mpi --scheduler-file ~/scheduler.json
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at:    tcp://<IP_ADDRESS>:42467
distributed.scheduler - INFO -   dashboard at:                     :8787
distributed.nanny - INFO -         Start Nanny at: 'tcp://<IP_ADDRESS>:37515'
distributed.nanny - INFO -         Start Nanny at: 'tcp://<IP_ADDRESS>:46855'
distributed.nanny - INFO -         Start Nanny at: 'tcp://<IP_ADDRESS>:39345'
distributed.diskutils - INFO - Found stale lock file and directory '/home/jovyan/dask-worker-space/worker-uaoj0551', purging
distributed.diskutils - INFO - Found stale lock file and directory '/home/jovyan/dask-worker-space/worker-9hf54nel', purging
distributed.diskutils - INFO - Found stale lock file and directory '/home/jovyan/dask-worker-space/worker-hq3en_u5', purging
distributed.worker - INFO -       Start worker at:     tcp://<IP_ADDRESS>:38217
distributed.worker - INFO -          Listening to:     tcp://<IP_ADDRESS>:38217
distributed.worker - INFO -          dashboard at:           <IP_ADDRESS>:45855
distributed.worker - INFO - Waiting to connect to:     tcp://<IP_ADDRESS>:42467
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          1
distributed.worker - INFO -                Memory:                   2.00 GiB
distributed.worker - INFO -       Local Directory: /home/jovyan/dask-worker-space/worker-uf_og6_c
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:38217
distributed.core - INFO - Starting established connection
distributed.worker - INFO -         Registered to:     tcp://<IP_ADDRESS>:42467
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection
distributed.worker - INFO -       Start worker at:     tcp://<IP_ADDRESS>:45469
distributed.worker - INFO -          Listening to:     tcp://<IP_ADDRESS>:45469
distributed.worker - INFO -          dashboard at:           <IP_ADDRESS>:37789
distributed.worker - INFO - Waiting to connect to:     tcp://<IP_ADDRESS>:42467
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          1
distributed.worker - INFO -                Memory:                   2.00 GiB
distributed.worker - INFO -       Local Directory: /home/jovyan/dask-worker-space/worker-v9ttiup0
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:45469
distributed.core - INFO - Starting established connection
distributed.worker - INFO -         Registered to:     tcp://<IP_ADDRESS>:42467
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection
distributed.worker - INFO -       Start worker at:     tcp://<IP_ADDRESS>:40255
distributed.worker - INFO -          Listening to:     tcp://<IP_ADDRESS>:40255
distributed.worker - INFO -          dashboard at:           <IP_ADDRESS>:40961
distributed.worker - INFO - Waiting to connect to:     tcp://<IP_ADDRESS>:42467
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          1
distributed.worker - INFO -                Memory:                   2.00 GiB
distributed.worker - INFO -       Local Directory: /home/jovyan/dask-worker-space/worker-vxf9x9qk
distributed.worker - INFO - -------------------------------------------------
distributed.scheduler - INFO - Register worker
distributed.scheduler - INFO - Starting worker compute stream, tcp://<IP_ADDRESS>:40255
distributed.worker - INFO -         Registered to:     tcp://<IP_ADDRESS>:42467
distributed.core - INFO - Starting established connection
distributed.worker - INFO - -------------------------------------------------
distributed.core - INFO - Starting established connection
\```

and next
from distributed import Client
client = Client(scheduler_file='~/scheduler.json')

The second part is hanging.
By the way - I don't really have experience with using dask on distributed systems (just locally), and kind of zero experience with dask-mpi.
I used only ipyparallel + MPI (see here).
In case you have any reference I am happy to use it.

Ok so it looks like your cluster is starting correctly but the client is unable to connect.
Are you able to open the dashboard in your browser? I think from your example it would be <IP_ADDRESS>:8787.
I'm wondering if there are some firewall rules or network connectivity issues getting in the way.

I checked http://<IP_ADDRESS>:8787/info/main/workers.html  (different setup)

I guess it looks fine, isn't it?
The problem is that it is hanging!

Ok in that example you seem to be running the scheduler on your local machine. But then you are connecting a client using the scheduler file.

Can you try client = Client("tcp://localhost:8786")?
Can you have a look in the ~/scheduler.json file and see what address is listed in there?

I am running it in a container
docker run --rm -it --cpus=3 --memory=3g --shm-size=3g --memory-swap=4g    -p 8888:8888 -p 8787:8787 image

Awesome - using the address tcp://<IP_ADDRESS>:41337 from the ~/scheduler.json works as desired!

Glad things are working for you, strange that Distributed is failing to read the scheduler file itself though.

Many thanks!
It looks like using the absolute path it works!

Thanks, @jacobtomlinson!
@aplamada I'm glad it's working for you!

@aplamada ahh that makes sense.
I think the hang is happening here because os.path.exists doesn't expand the ~. I'll raise an issue/PR to resolve this and also add a timeout here.

I have opened an issue upstream so given that this problem has been worked around for @aplamada I'm going to close this out.

Thanks a lot! Yes the issue was clarified.
I was able to use it on an HPC cluster with a dask cluster that has workers on multiple nodes and it works as expected!


Firefox+HTML5+H.264+MP4+Orange PI+ARM Cortex-A7: Sound problem

What do I have:
SoC: Orange Pi+ 2 http://www.orangepi.org/orangepiplus2
CPU: H3 Quad-core ARM Cortex-A7 https://en.wikipedia.org/wiki/ARM_Cortex-A7
OS: Armbian Ubuntu Xenial https://www.armbian.com/orange-pi-plus-2
Audio subsystem: PulseAudio
Problem:
Test video HTML5 H.264/MP4: https://www.quirksmode.org/html5/tests/video.html
Firefox: video is played fine, but audio – NOT: white noise only.
Youtube on Firefox: one part of clips is played fine, second part of clips: video is played fine, but audio – NOT: white noise only.
Naturally, second part of Youtube clips have format H.264/MP4.
However:
Chromium: ALL Youtube clips is played fine.
Downloading test video in format H.264/MP4 to local disk.
mpv: test video is played fine.
kodi: test video is played fine.
Therefore, the problem NOT in hardware, NOT in PulseAudio, but EXACTLY in Firefox or its plugins.
I’m trying install plugins and addons from OS and Firefox repositories, but the problem still is not solved.
Please help me.

Openh264 repository only works on video part. it seems that the issue you met isn't covered here.
Suggest you submit this issue to a proper site. thanks!


Feature Enhancement Concept and Standardization of BP Predecessor/Successor

Overview

related feature Enhancement Concept and Standardization of BP Predecessor/Successor
This feature aims to introduce enhanced support for managing business partner relationships, including the lifecycle of business entities (predecessors and successors). The feature will allow businesses to track and maintain detailed relationship data between partner entities throughout their lifecycle changes, ensuring a comprehensive view of the evolving partnerships.
Explain the topic in 2 sentences
Enhance business partner management by adding features for tracking predecessor-successor relationships. The goal is to provide transparency in entity transitions and improve the efficiency of business partner management processes.
What's the benefit?
This feature will improve operational efficiency by providing clear insights into the business partner lifecycle and relationships. It reduces complexity and risk by enabling better alignment and communication across partners.
What are the Risks/Dependencies?

This feature introduces potential breaking changes in how business partner data is stored and managed.
It may require alignment with interface partners who depend on this data.
The architecture team will need to review the current system for compatibility.

Detailed explanation
Current implementation
Currently, the system tracks business partners as isolated entities with no built-in functionality to manage transitions like mergers, acquisitions, or restructuring.
Proposed improvements

Introduce fields and logic to track lifecycle changes, allowing the system to link predecessor and successor companies.
Improve data querying and reporting capabilities to provide a clear overview of business partner relationships over time.

Feature Team
Contributor

Contributor 1
Contributor 2

Committer

Committer 1
Committer 2

User Stories

Issue 1: Implement tracking of business partner lifecycle, linked to the repository for business partner data.

Acceptance Criteria

[ ] Criteria 1: Business partners can be linked to predecessor and successor entities.
[ ] Criteria 2: Data queries return relationship information accurately.

Test Cases
Test Case 1

Steps

Add a new business partner.
Create a lifecycle event linking predecessor and successor.
Query for business partner relationship data.

Expected Result

Business partner is stored.
Lifecycle event correctly links predecessor and successor.
Relationship data is accurately returned in queries.

Architectural Relevance
The following items are ensured (answer: yes) after this issue is implemented:

[ ] This feature aligns with our current architectural guidelines

Data Sovereignty: All data sharing activities across company boundaries follow the Catena-X Regulatory Framework, in particular the Data Exchange Governance, and the Dataspace Protocol via a compliant Connector (like the tractusx-edc or similar, see Connector KIT)

[ ] CX-0010 Business Partner Number
[ ] CX-0012 Business Partner Data Pool AP
[ ] CX-0074 Business Partner Gate AP
[ ] CX-0076 Golden Record EndtoEnd Requirements Standard

Justification: (Fill this out, if at least one of the checkboxes above cannot be ticked. Contact the Architecture Management Committee to get an approval for the justification)
Additional information

[ ] I am aware that my request may not be developed if no developer can be found for it. I'll try to contribute a developer (bring your own developer)

@HeyHardy since I would like to keep these issues here as clear product features I would like to rename this issue to "Business Partner Successor and Predecessor Relationships"
Does that sound fitting for you?


Automated linting and formatting

Creating a new issue so there's space to discuss since it's come up as asides in some other issues (#207, #147).
My personal view on the topic is easy, reasonable consistency, but I know it can be quite controversial! It seems practical to start with minimal configuration and progress from there as we find issues that annoy. Hopefully this is something we set up early and don't need to think about much after the initial pain.
Two positions to seed the discussion:

Enforce via CI: right now, we have a pylintrc, but folks appear free to ignore.
Enforce on commit: more controversial / more overhead, but saves CI from linting/formatting fails.

Some package suggestions:

Formatters: yapf and black are two good choices. Even though the former is google, I've tended towards black because it's faster (I believe yapf is quadratic) and treats non-determinism as a bug. Two dings against, however:

It requires Python 3.6+ to run, which may or may not be a problem, especially if we use pre-commit hooks (see below)
Could flax handle the transition from 2-space indent to 4-space tab?!

Linters: We already have pylint, so it may be good to leave at that.
Imports: This issue cropped up from sorting imports, so I'd be remiss to leave out; there's the ever-popular isort, but I've preferred this one just because it's a little simpler. Either is fine.
Pre-commit: I've found the pre-commit project very helpful to manage commit hooks. Here is the configuration from timecast. Two pain points:

Requires people to install pre-commit hooks (a one-liner that I'd imagine goes into dev on-boarding docs)
Failed pre-commit hooks fail a commit; good hygiene, but this might annoy people since it's not the standard git loop

Proposal
TBD. No proposal because just discussion for now.
Transition
TBD. Such a change could create a lot of merge conflicts. Once we decide on the scope of linting/formatting, first thing to do is understand how much conflict will actually arise. We can go from there (e.g., if it's really bad, either change scope or set a date by which all changes need to be in before The Linting; and we can refer to flax history as BL and AL...)

Tagging @mohitreddy1996 and @joaogui1

Thanks @danielsuo for starting this conversation! A few comments:

@levskaya had spoke of black being really unhelpful with numpy codebases (maybe because of long argument lists to functions?) -- @levskaya can you elaborate more as to how black might be suboptimal for us?

Re: enforce on commit. Do we have a sense for how long linting our current codebase would take? Seems like we could do a back-of-the-envelope for codebase growth to assess whether this would be a viable idea.

Python 3.6+ is fine.

pre-commit.com looks neat, I hadn't heard of it before.

Transition plan -- here's my proposal: Choose a particular subdirectory of flax/, set up the linter rules just for that directory and all the relevant machinery you proposed and let's see what the diff looks like. Then we can discuss whether we should make the linter rules more strict or more lenient. Once we're happy with them, we can split up the codebase and coordinate a virtual hackathon to fix the rest. @danielsuo if you want to give it a shot to get the ball rolling, that would be great.

cc @bastings who had also thought about linters for Flax before (though I don't think he is actively working on it)

Happy to get started! Responses inline:

@levskaya had spoke of black being really unhelpful with numpy codebases (maybe because of long argument lists to functions?) -- @levskaya can you elaborate more as to how black might be suboptimal for us?

No special love for black except that it's easy. Happy to configure something else. As Guideo says, black only if the team argues about style a lot.

Re: enforce on commit. Do we have a sense for how long linting our current codebase would take? Seems like we could do a back-of-the-envelope for codebase growth to assess whether this would be a viable idea.

Good point. Typically, this isn't an issue since with the right tooling, linting only operates on changes, but could be a problem in certain cases (e.g., during transition) when exclude patterns are very long, diffs are large, etc. I suppose more advanced cross-file/cross-module checks could also cause problems. Just ran flake8 on a couple of repositories in their entireties to get a rough sense for what simpler checks might look like:

repo
cores
time

flax
1
3.89

flax
16
0.69

jax
1
17.87

jax
16
2.08

tensorflow
1
3:53.70

tensorflow
16
13.29

# flax with 1 core
flake8 -q -j 1 3.85s user 0.04s system 99% cpu 3.888 total

# flax with 16 core
flake8 -q 6.33s user 1.97s system 1205% cpu 0.689 total

# jax with 1 core
flake8 -q -j 1 17.61s user 0.25s system 99% cpu 17.865 total

# jax with 16 core
flake8 -q 24.66s user 3.20s system 1338% cpu 2.081 total

# tensorflow with 1 core
flake8 -q -j 1 230.18s user 3.47s system 99% cpu 3:53.70 total

# tensorflow with 16 core
flake8 -q  458.48s user 15.71s system 3567% cpu 13.291 total

Python 3.6+ is fine.

To clarify, Python 3.6+ would be a developer requirement if we used black and formatted/linted on commit.

pre-commit.com looks neat, I hadn't heard of it before.

I love it! But...I can see if some would find it annoying. Below is an example where I intentionally remove some docstrings, force the coverage badge to regenerate, and fail the commit. In this case, the fixes are automatic, but I still need to re-commit.

Transition plan -- here's my proposal: Choose a particular subdirectory of flax/, set up the linter rules just for that directory and all the relevant machinery you proposed and let's see what the diff looks like. Then we can discuss whether we should make the linter rules more strict or more lenient. Once we're happy with them, we can split up the codebase and coordinate a virtual hackathon to fix the rest. @danielsuo if you want to give it a shot to get the ball rolling, that would be great.

Sounds good! Will throw something together later today. I'll use black, flake8, etc just as an example, but I have no religion about any of these. I will emphasize that at least once or twice more in case blood pressures rise!

I like yapf because it allows for more customization compared to black
Here's a discussion we had about that on the JAX repo

Thanks for the link! Helpful to see @levskaya's and @shoyer's responses. @joaogui1's PRs in jax (here and here) also instructive. Seems like there's quite a bit of resistance to automated Python formatting (possibly linting). I'll leave the issue open, but this might be DOA.

I don't think it's DOA, folks on JAX didn't seem to want it, but @avital seems open to the idea

Oops, I phrased too dramatically, my bad. Yes different projects, but the Jax folks did have good points that may apply here given the overlap of contributors. Just wanted to make it easier for the maintainers to reject the issue if they want to. Happy to go either way!

It seems like it is very hard to get agreement on this, and since there hasn't been much recent activity I'm closing this issue.


file download from application is 0B result

i am using mattermost  v2.19.2  but i faced problem when i try to install image using the app it download okay ,but when when  i try to install document  i got result with 0B .the @mattermost/react-native-network-client version is 1.7.2

@Abdullah2100 Would you be able to help share more details such as your OS and version, Mattermost server version, and reproduction steps?

I apologize for the late response. Saving images and videos using "@react-native-camera-roll/camera-roll" works great, but saving files is not working. Here’s the code from Mattermost React Native:
import RNUtils from '@mattermost/rnutils';

    const saveFile = async (path: string) => {
        if (mounted.current) {
            if (Platform.OS === 'android') {
                try {
                    await RNUtils.saveFile(path);
                } catch {
                    // do nothing in case the user decides not to save the file
                }
                setAction('none');
                return;
            }

            updateLocalFilePath(serverUrl, item.id, path);

            Share.open({
                url: pathWithPrefix('file://', path),
                saveToFiles: true,
            }).catch(() => {
                // do nothing
            });

            setAction('none');
        }
    };

I’m using Android 13.
react-native-cli: 2.0.1
react-native: 0.74.5
``

I found a solution that still seems weird to me, but it works. The image download works fine, but the file download doesn't work for me. After searching many times and using the emulator's file explorer, I discovered that the application is downloading files to a path that's not visible to the user, like data/user/0/packagename. Under this path, I found multiple files containing the files I downloaded. I then use this path to copy the file to a new location where I can access it.
async saveFile(appPath: string, path: string, item: GalleryItemType, isImage: boolean) {
    if (Platform.OS === 'android') {
        let fileFullName = "";  // This will store the full path to the saved file.

        try {
            // Normalize appPath in case it starts with "file://"
            const filePath = appPath.startsWith("file://")
                ? appPath.substring(7)  // Remove "file://" prefix
                : "/" + appPath;  // Ensure it starts with a slash

            // Define the public download directory based on image or document
            const downloadFolder = path + `${isImage ? this.image : this.document}` + '/' + item.name;

            // Check if the file already exists in the download directory
            const isExistInDownload = await RNFS.exists(downloadFolder);
            if (isExistInDownload) {
                return;  // File exists, no need to proceed further
            } else {
                let counter = 1;
                const fileDownloadDir = path + `${isImage ? this.image : this.document}` + '/';
                const fileExtension = item.extension || "";  // Default to empty string if extension is missing
                let baseFileName = item.name.substring(0, item.name.length - (1 + fileExtension.length));  // Remove extension from the filename

                // Construct the full file name (without counting possible duplicates yet)
                fileFullName = fileDownloadDir + baseFileName + '.' + fileExtension;

                // Check if file already exists with the same name. If so, append a counter to the name.
                while (await RNFS.exists(fileDownloadDir + baseFileName + '.' + fileExtension)) {
                    fileFullName = fileDownloadDir + baseFileName + `(${counter})` + '.' + fileExtension;
                    counter++;
                }

            }

            // Copy the file from the app's directory to the public directory
            await RNFS.copyFile(filePath, fileFullName);

            // You can uncomment the following line if you need to update the local path after copying
            // updateLocalFilePath(serverUrl, item.id, path);

        } catch (error) {
            // Catch any error and log it
            console.log(`\n\nError during file copy: ${String(error)}\n\n`);
        }
    }
}


chore: Update build script to generate flow and TS types:

#Fixes #25
Using StyleX as a starting point, overhauling the build scripts.

[x] Copy the gen-flow wrapper around flow-api-translator
[x] Set up script to generate per-file Flow and TS type-defs
[x] Update the package.json to point to the new output files
[x] Delete the existing build steps

Skipped these steps as we decided to keep flat bundles for now:

[ ] Set up a Babel translation step for publishing to NPM that doesn't pre-bundle
[ ] Set up a Babel translation step that translates to Haste modules

Nice! I kind of want to keep flat bundling for OSS

I kind of want to keep flat bundling for OSS

One downside of flat bundling is that you can take advantage of tree-shaking in modern bundlers.
However, if we stop here, we can keep the flat bundles for the JS code and only keep the separate files for the type-defs.

This isn't a modular library

If we only publish an ESM build to NPM, we can limit what we export so that all the other files are "private".

AFAICT the motivation for this change is because flow doesn't support automatically generating lib defs

The flow-api-translater package being used by the script is the Flow teams solution for generating lib-defs. The other motivation is that this lets us generate TS type-defs as well. A folder of file like this is fully supported in both Flow and TS and so we don't really gain anything by bundling them into flat files.

There's nothing to tree-shake out of this package on web, or even native really. AFAIK, tree-shaking / DCE isn't limited to what's in different files and should work if exports within a file aren't used. So we can focus on the types side of things

@necolas That makes things simple enough. This PR is now ready.
I simply changed the output path from dist/dom.js to dist/dom/index.js and not the separate type files work as expected. The dist/dom/index.d.ts and dist/dom/index.js.flow files shadow the actual file and that works for both TS and Flow.
I can probably make this set up work for internal sync as well.

Update: Made a couple of changes to ensure that no two files in the package have the same name. (Except when there are dom and native counter-parts`). This was mostly true already except:

There was one unused file
There are two versions of flattenStyle.js that work similarly but with different types.

For now, I've renamed one of the flattenStyle.js to flattenStyleXStyles.js, but eventually, it should be deleted.
This work has been done to be compatible with Haste for internal sync.

Not able to reproduce this failure locally. I don't even have the file that is erroring. :(

Not able to reproduce this failure locally. I don't even have the file that is erroring. :(

I can reproduce once this PR is rebased on main

@necolas Does this look good to merge? (I know you're on a break, so let me know if I should run this by someone else while you're away)

There's still a few things to update before that.  I'll comment over the weekend. Also left some questions above

Does this patch still allow importing of types from the main package, e.g., import type { StrictHTMLElement } from 'react-strict-dom'?

Anything that is exported from the entry point can be imported directly. If there are types that are currently not exported from the entrypoint that are supposed to be public, we can add them.

The paths to the build files also need to be updated in the performance workflow file.

^ this still needs to be done.
Also need to update the paths in the build size badges in the RSD README. They will update to the correct values once I publish the release after this patch

I pushed fixes for the path updates.
However, there's one more path in the .flowconfig - https://github.com/facebook/react-strict-dom/blob/main/.flowconfig#L15. Updating that to point to the new flow entry point results in flow errors which should show up in CI for that commit

Let me figure this out.

Removing the mapping (or mapping to a non-existent file, as was the case before in this PR) results in no flow errors...which is a bit concerning.

This is a known issue with Flow. It causes "untyped" code rather than type errors. We might need to add some checks to cache type-coverage regressions for this. We also need to update the Flow version. I'll do that after this PR has merged.

Thanks! Merged


Session in readme

I just tried to use the gem without even looking at the dummy app, just following the readme and a first problem I had was NameError: undefined local variable or method session for main:Object
What is a session? How is it related to the User? Answers to these questions could be helpful. :-)

Can you be more specific? How did you use gem's methods and were?
Readme was thought rather as general description of gem's possibilities and main features, once we establish api, we will add wiki pages.

This is not a bug. What I mean is that I couldn't easly figure out what is a session object and how to use it. I think it could be explained on examples in readme. But probably you have it already in your plans...


Letter Distance

Description
enter here if you have any details
Program Name -  Letter Distance
Issue No.- #139

Type issue number after hash(#)

Checklist(self check)

[ ] Yes, this task assigned to me.
[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] My Filename is in camelCase format
[x] I have added Problem statement, Problem link in file.

@aayushi-droid Please review and merge


Color operators like $, + and =

Particularly helps in JavaScript and jQuery and makes the theme more consistent with other highlighters.
Before

After

I think this should be submitted to https://github.com/chriskempson/base16-textmate, Spacegray just uses it for syntax.


Update README.md

Minor info about dep tool.

@chankh Closing as your updates covering the purpose of this PR.
Thanks.


JBPM-5233: Data source management

- Adds the hability of defining data base drivers that will be automatically installed during server startup.
- DriverDefaultInitializerTest
- DataSourceRuntimeManagerTest
- DBCPDataSourceProviderTest
- DBCPDriverProviderTest
- WildflyDriverProviderTest
- WildflyDataSourceProviderTest
- Tests base classes
- Copyright header removal form xml file

Hello @manstis Would you mind review this PR.
Thanks.
Depends on https://github.com/droolsjbpm/droolsjbpm-build-bootstrap/pull/283

Was the removing of the Copyright requested?
I thought every file should have the Copyright. Still I fear many of the non-Java files I have committed are missing it. Just checking if I should add them or remove them.

Hello @Rikkola, good question. I'm not sure if there's a formal request for removing copyright headers from xml files, but @csadilek suggested to not add them in xml files. And since it's a new module, I removed them.

@wmedvede @manstis Looking at the Wildfly-specific stuff, it makes me wonder... has this ever been tested on Weblogic/Websphere? Or is this the final proof that we are going the "appliance" route?
(Please note I am all for having Central be container-less appliance. I just never heard it confirmed that we're really doing that - I'll be really excited if it is finally confirmed.)

@triceo That is a good question.. and I do not know the answer. I also do not know what the "target" is for Data Sources.. whether it is a "platform" wide feature; or limiting itself to a subset of containers. I will have to defer an explanation to @wmedvede and/or @csadilek

@triceo @manstis we're not testing any WB work on WAS/WLS anymore. We've also removed the kie-wb-distributions from master a while ago, as per our agreement from May.

Hello @manstis , I've applied all the suggestions, and used the same procedure as usual for updating the open PR. But for some reason the PR wasn't updated properly and conversation weren't "closed". If you look at the commit all the modifications are there.

Jenkins please retest this.

@psiroky @mbiarnes Any ideas why this might be failing?

Jenkins please retest this.

Hello @manstis , it seems it has built now. Is it now ready for merge?

Thanks @wmedvede
A couple of JavaDoc typos, but I won't let them delay the merge further.