CPU backed failure on ARM due to XLA/LLVM (and a potential fix)

Hi,
JAX CPU backend fails on ARM architecture (e.g. NVIDIA Jetson AGX, ARMv8.2) with the following errors:
Python 3.6.12 (default, Aug 17 2020, 23:45:20)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from jax.config import config;
>>> config.update('jax_platform_name', 'cpu')
>>> import jax.numpy as jnp
>>> x = jnp.ones((1000, 1000))
2021-02-08 17:35:23.661294: W external/org_tensorflow/tensorflow/core/platform/profile_utils/cpu_utils.cc:116] Failed to find bogomips or clock in /proc/cpuinfo; cannot determine CPU frequency
>>> x*x
'carmel' is not a recognized processor for this target (ignoring processor)
'+neon' is not a recognized feature for this target (ignoring feature)
'+fp-armv8' is not a recognized feature for this target (ignoring feature)
'+crypto' is not a recognized feature for this target (ignoring feature)
'+crc' is not a recognized feature for this target (ignoring feature)
'carmel' is not a recognized processor for this target (ignoring processor)
'carmel' is not a recognized processor for this target (ignoring processor)
'+neon' is not a recognized feature for this target (ignoring feature)
'+fp-armv8' is not a recognized feature for this target (ignoring feature)
'+crypto' is not a recognized feature for this target (ignoring feature)
'+crc' is not a recognized feature for this target (ignoring feature)
'carmel' is not a recognized processor for this target (ignoring processor)
'carmel' is not a recognized processor for this target (ignoring processor)
'+neon' is not a recognized feature for this target (ignoring feature)
'+fp-armv8' is not a recognized feature for this target (ignoring feature)
'+crypto' is not a recognized feature for this target (ignoring feature)
'+crc' is not a recognized feature for this target (ignoring feature)
'carmel' is not a recognized processor for this target (ignoring processor)
'carmel' is not a recognized processor for this target (ignoring processor)
'+neon' is not a recognized feature for this target (ignoring feature)
'+fp-armv8' is not a recognized feature for this target (ignoring feature)
'+crypto' is not a recognized feature for this target (ignoring feature)
'+crc' is not a recognized feature for this target (ignoring feature)
'carmel' is not a recognized processor for this target (ignoring processor)
LLVM ERROR: 64-bit code requested on a subtarget that doesn't support it!
Aborted (core dumped)

JAX was reported to work on similar architectures (e.g. NVIDIA Jetson TX2), but I think they only used GPU backend, which works fine.
We have identified the problem as missing AArch64 statements in XLA and LLVM build files. The following seem to fix the problem:

Adding following lines to cpu_compiler:deps in org_tensorflow/tensorflow/compiler/xla/service/cpu/BUILD

  "@llvm-project//llvm:ARMCodeGen",
  "@llvm-project//llvm:AArch64CodeGen",

Changing llvm_host_triple in org_tensorflow/third_party/llvm/llvm.autogenerated.BUILD (this is required due to default fall-back architecture that is set as X86_64, which is indicated to be fixed)

llvm_host_triple = "aarch64-unknown-linux_gnu"

Adding linux_aarch64 target to llvm_all_cmake_vars in org_tensorflow/third_party/llvm/llvm.bzl

	"@org_tensorflow//tensorflow:linux_aarch64": cmake_var_string(
			_dict_add(
					cmake_vars,
					llvm_target_cmake_vars("AArch64", "aarch64-unknown-linux_gnu"),
					posix_cmake_vars,
					linux_cmake_vars,
			),
	),

With these changes the result is as follows:

>>> from jax.config import config;
>>> config.update('jax_platform_name', 'cpu')
>>> import jax.numpy as jnp
>>> x = jnp.ones((1000, 1000))
>>> x*x
DeviceArray([[1., 1., 1., ..., 1., 1., 1.],
             [1., 1., 1., ..., 1., 1., 1.],
             [1., 1., 1., ..., 1., 1., 1.],
             ...,
             [1., 1., 1., ..., 1., 1., 1.],
             [1., 1., 1., ..., 1., 1., 1.],
             [1., 1., 1., ..., 1., 1., 1.]], dtype=float32)

Thanks @girgink ! We will try to fix this internally in tensorflow tree.

https://github.com/tensorflow/tensorflow/commit/942f315def49da3a76c05076216e623142b4dfae links in AArch64 codegen and sets the cmake vars. Let us know if that's sufficient to get JAX to work on your machine.

Thanks @d0k, I will build again and inform you accordingly

@girgink Note we haven't bumped the TF version in the JAX WORKSPACE file. You'll either need to do that or point the WORKSPACE file to an up-to-date TF checkout that includes that change.

Ok!

@d0k @hawkinsp I confirm that the fix works as expected.
I run also tests with JAX_NUM_GENERATED_CASES=5 pytest -n 1, the summary is: 5 failed, 241 passed, 11 skipped, 16 errors.
All errors are related to call to deprecated create function FileDescriptor().
For the failed tests I suspected that they might be related to available memory. The unit that I use (NVIDIA Jetson AGX Xavier) has unified CPU-GPU memory. I'm not sure if it is the case for JAX, but some frameworks - e.g. TensorFlow - allocate most of GPU memory initially to speed-up processing, but because Xavier has unified memory this results in very low CPU memory. In fact, running the tests in parallel (e.g. -n auto) resulted in quick segmentation fault.
Please find below the test results:
/usr/local/lib/python3.8/dist-packages/pytest_benchmark/logger.py:44: PytestBenchmarkWarning: Benchmarks are automatically disabled because xdist plugin is active.Benchmarks cannot be performed reliably in a parallelized environment.
  warner(PytestBenchmarkWarning(text))
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /tmp/jax, configfile: pytest.ini
plugins: benchmark-3.2.3, xdist-2.2.1, forked-1.3.0, cov-2.11.1
gw0 [7201] 0 items / 4 errors
....................s.................ss.............................................ss........................................................................................s.................... [  2%]
..........s.......ss..........s................s........Fatal Python error: Segmentation fault

Thread 0x0000007fb41bb160 (most recent call first):
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 400 in read
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 432 in from_io
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 967 in _thread_receiver
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn

Thread 0x0000007fb4afb430 (most recent call first):
  File "/tmp/jax/jax/interpreters/xla.py", line 848 in _execute_compiled
  File "/tmp/jax/jax/interpreters/xla.py", line 590 in _xla_call_impl
  File "/tmp/jax/jax/core.py", line 631 in process_call
  File "/tmp/jax/jax/core.py", line 1410 in process
  File "/tmp/jax/jax/core.py", line 1398 in call_bind
  File "/tmp/jax/jax/core.py", line 1407 in bind
  File "/tmp/jax/jax/interpreters/batching.py", line 175 in process_call
  File "/tmp/jax/jax/core.py", line 1410 in process
  File "/tmp/jax/jax/core.py", line 1398 in call_bind
  File "/tmp/jax/jax/core.py", line 1407 in bind
  File "/tmp/jax/jax/api.py", line 292 in cache_miss
  File "/tmp/jax/jax/api.py", line 398 in f_jitted
  File "/tmp/jax/jax/_src/traceback_util.py", line 139 in reraise_with_filtered_traceback
  File "/tmp/jax/jax/_src/random.py", line 1082 in gamma
  File "/tmp/jax/tests/api_test.py", line 3636 in f
  File "/tmp/jax/jax/linear_util.py", line 166 in call_wrapped
  File "/tmp/jax/jax/core.py", line 636 in process_custom_jvp_call
  File "/tmp/jax/jax/custom_derivatives.py", line 279 in bind
  File "/tmp/jax/jax/custom_derivatives.py", line 213 in __call__
  File "/tmp/jax/tests/api_test.py", line 3645 in sample
  File "/tmp/jax/jax/linear_util.py", line 166 in call_wrapped
  File "/tmp/jax/jax/api.py", line 1219 in batched_fun
  File "/tmp/jax/jax/_src/traceback_util.py", line 139 in reraise_with_filtered_traceback
  File "/tmp/jax/tests/api_test.py", line 3647 in test_closure_with_vmap
  File "/usr/lib/python3.8/unittest/case.py", line 633 in _callTestMethod
  File "/usr/lib/python3.8/unittest/case.py", line 676 in run
  File "/usr/lib/python3.8/unittest/case.py", line 736 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/unittest.py", line 321 in runtest
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 162 in pytest_runtest_call
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 215 in call_and_report
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 126 in runtestprotocol
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 89 in run_one_test
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 72 in pytest_runtestloop
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 244 in <module>
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1084 in executetask
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 267 in integrate_as_primary_thread
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1060 in serve
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1554 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>
[gw0] node down: Not properly terminated
F
replacing crashed worker gw0
gw1 [7201]Fatal Python error: Segmentation fault

Thread 0x0000007f85c3a160 (most recent call first):
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 400 in read
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 432 in from_io
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 967 in _thread_receiver
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn

Current thread 0x0000007f8657a430 (most recent call first):
  File "/tmp/jax/jax/interpreters/xla.py", line 361 in _execute_compiled_primitive
  File "/tmp/jax/jax/interpreters/xla.py", line 243 in apply_primitive
  File "/tmp/jax/jax/core.py", line 628 in process_primitive
  File "/tmp/jax/jax/core.py", line 282 in bind
  File "/tmp/jax/jax/_src/lax/lax.py", line 1493 in iota
  File "/tmp/jax/jax/_src/lax/lax.py", line 4881 in _scatter_jvp
  File "/tmp/jax/jax/interpreters/ad.py", line 285 in process_primitive
  File "/tmp/jax/jax/core.py", line 282 in bind
  File "/tmp/jax/jax/_src/lax/lax.py", line 1082 in scatter
  File "/tmp/jax/tests/lax_autodiff_test.py", line 907 in <lambda>
  File "/tmp/jax/jax/linear_util.py", line 166 in call_wrapped
  File "/tmp/jax/jax/api.py", line 1694 in _jvp
  File "/tmp/jax/jax/api.py", line 1666 in jvp
  File "/tmp/jax/jax/test_util.py", line 239 in check_jvp
  File "/tmp/jax/jax/test_util.py", line 301 in _check_grads
  File "/tmp/jax/jax/test_util.py", line 314 in check_grads
  File "/tmp/jax/tests/lax_autodiff_test.py", line 910 in testScatterGrad
  File "/usr/local/lib/python3.8/dist-packages/absl/testing/parameterized.py", line 282 in bound_param_test
  File "/usr/lib/python3.8/unittest/case.py", line 633 in _callTestMethod
  File "/usr/lib/python3.8/unittest/case.py", line 676 in run
  File "/usr/lib/python3.8/unittest/case.py", line 736 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/unittest.py", line 321 in runtest
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 162 in pytest_runtest_call
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 215 in call_and_report
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 126 in runtestprotocol
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 89 in run_one_test
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 72 in pytest_runtestloop
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 244 in <module>
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1084 in executetask
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 267 in integrate_as_primary_thread
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1060 in serve
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1554 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>
[gw1] node down: Not properly terminated
F
replacing crashed worker gw1
gw2 [7201]Fatal Python error: Segmentation fault

Thread 0x0000007fab0c5160 (most recent call first):
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 400 in read
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 432 in from_io
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 967 in _thread_receiver
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn

Thread 0x0000007faba05430 (most recent call first):
  File "/tmp/jax/jax/interpreters/xla.py", line 361 in _execute_compiled_primitive
  File "/tmp/jax/jax/interpreters/xla.py", line 243 in apply_primitive
  File "/tmp/jax/jax/core.py", line 628 in process_primitive
  File "/tmp/jax/jax/core.py", line 282 in bind
  File "/tmp/jax/jax/_src/lax/lax.py", line 309 in integer_pow
  File "/tmp/jax/jax/_src/lax/lax.py", line 1908 in reciprocal
  File "/tmp/jax/jax/test_util.py", line 927 in _CheckAgainstNumpy
  File "/tmp/jax/tests/lax_test.py", line 204 in testOpAgainstNumpy
  File "/usr/local/lib/python3.8/dist-packages/absl/testing/parameterized.py", line 282 in bound_param_test
  File "/usr/lib/python3.8/unittest/case.py", line 633 in _callTestMethod
  File "/usr/lib/python3.8/unittest/case.py", line 676 in run
  File "/usr/lib/python3.8/unittest/case.py", line 736 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/unittest.py", line 321 in runtest
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 162 in pytest_runtest_call
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 215 in call_and_report
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 126 in runtestprotocol
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 89 in run_one_test
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 72 in pytest_runtestloop
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 244 in <module>
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1084 in executetask
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 267 in integrate_as_primary_thread
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1060 in serve
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1554 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>
[gw2] node down: Not properly terminated
F
replacing crashed worker gw2
gw3 [7201]Fatal Python error: Segmentation fault

Thread 0x0000007fab598160 (most recent call first):
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 400 in read
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 432 in from_io
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 967 in _thread_receiver
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn

Current thread 0x0000007fabed8430 (most recent call first):
  File "/tmp/jax/jax/interpreters/xla.py", line 361 in _execute_compiled_primitive
  File "/tmp/jax/jax/interpreters/xla.py", line 243 in apply_primitive
  File "/tmp/jax/jax/core.py", line 628 in process_primitive
  File "/tmp/jax/jax/core.py", line 282 in bind
  File "/tmp/jax/jax/_src/lax/lax.py", line 1493 in iota
  File "/tmp/jax/jax/_src/lax/lax.py", line 4881 in _scatter_jvp
  File "/tmp/jax/jax/interpreters/ad.py", line 285 in process_primitive
  File "/tmp/jax/jax/core.py", line 282 in bind
  File "/tmp/jax/jax/_src/lax/lax.py", line 1082 in scatter
  File "/tmp/jax/tests/lax_autodiff_test.py", line 907 in <lambda>
  File "/tmp/jax/jax/linear_util.py", line 166 in call_wrapped
  File "/tmp/jax/jax/api.py", line 1694 in _jvp
  File "/tmp/jax/jax/api.py", line 1666 in jvp
  File "/tmp/jax/jax/test_util.py", line 239 in check_jvp
  File "/tmp/jax/jax/test_util.py", line 301 in _check_grads
  File "/tmp/jax/jax/test_util.py", line 314 in check_grads
  File "/tmp/jax/tests/lax_autodiff_test.py", line 910 in testScatterGrad
  File "/usr/local/lib/python3.8/dist-packages/absl/testing/parameterized.py", line 282 in bound_param_test
  File "/usr/lib/python3.8/unittest/case.py", line 633 in _callTestMethod
  File "/usr/lib/python3.8/unittest/case.py", line 676 in run
  File "/usr/lib/python3.8/unittest/case.py", line 736 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/unittest.py", line 321 in runtest
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 162 in pytest_runtest_call
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 215 in call_and_report
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 126 in runtestprotocol
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 89 in run_one_test
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 72 in pytest_runtestloop
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 244 in <module>
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1084 in executetask
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 267 in integrate_as_primary_thread
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1060 in serve
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1554 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>
[gw3] node down: Not properly terminated
F
replacing crashed worker gw3
gw4 [7201]Fatal Python error: Segmentation fault

Thread 0x0000007f9342f160 (most recent call first):
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 400 in read
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 432 in from_io
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 967 in _thread_receiver
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn

Thread 0x0000007f93d6f430 (most recent call first):
  File "/tmp/jax/jax/interpreters/xla.py", line 361 in _execute_compiled_primitive
  File "/tmp/jax/jax/interpreters/xla.py", line 243 in apply_primitive
  File "/tmp/jax/jax/core.py", line 628 in process_primitive
  File "/tmp/jax/jax/core.py", line 282 in bind
  File "/tmp/jax/jax/_src/lax/lax.py", line 309 in integer_pow
  File "/tmp/jax/jax/_src/lax/lax.py", line 1908 in reciprocal
  File "/tmp/jax/jax/test_util.py", line 927 in _CheckAgainstNumpy
  File "/tmp/jax/tests/lax_test.py", line 204 in testOpAgainstNumpy
  File "/usr/local/lib/python3.8/dist-packages/absl/testing/parameterized.py", line 282 in bound_param_test
  File "/usr/lib/python3.8/unittest/case.py", line 633 in _callTestMethod
  File "/usr/lib/python3.8/unittest/case.py", line 676 in run
  File "/usr/lib/python3.8/unittest/case.py", line 736 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/unittest.py", line 321 in runtest
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 162 in pytest_runtest_call
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 255 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 311 in from_call
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 215 in call_and_report
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 126 in runtestprotocol
  File "/usr/local/lib/python3.8/dist-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 89 in run_one_test
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 72 in pytest_runtestloop
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 323 in _main
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 269 in wrap_session
  File "/usr/local/lib/python3.8/dist-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/dist-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/dist-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/dist-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/dist-packages/xdist/remote.py", line 244 in <module>
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1084 in executetask
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 285 in _perform_spawn
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 267 in integrate_as_primary_thread
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1060 in serve
  File "/usr/local/lib/python3.8/dist-packages/execnet/gateway_base.py", line 1554 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>
[gw4] node down: Not properly terminated
F
maximum crashed workers reached: 4

================================================================================================== ERRORS ==================================================================================================
____________________________________________________________________________ ERROR collecting tests/host_callback_to_tf_test.py ____________________________________________________________________________
tests/host_callback_to_tf_test.py:36: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
___________________________________________________________________________________ ERROR collecting tests/image_test.py ___________________________________________________________________________________
tests/image_test.py:34: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
_________________________________________________________________________________ ERROR collecting tests/lax_vmap_test.py __________________________________________________________________________________
ImportError while importing test module '/tmp/jax/tests/lax_vmap_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/lax_vmap_test.py:33: in <module>
    from tests.lax_test import LAX_OPS
E   ModuleNotFoundError: No module named 'tests.lax_test'
_________________________________________________________________________________ ERROR collecting tests/profiler_test.py __________________________________________________________________________________
tests/profiler_test.py:35: in <module>
    from tensorflow.python.profiler import profiler_client
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
____________________________________________________________________________ ERROR collecting tests/host_callback_to_tf_test.py ____________________________________________________________________________
tests/host_callback_to_tf_test.py:36: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
___________________________________________________________________________________ ERROR collecting tests/image_test.py ___________________________________________________________________________________
tests/image_test.py:34: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
_________________________________________________________________________________ ERROR collecting tests/profiler_test.py __________________________________________________________________________________
tests/profiler_test.py:35: in <module>
    from tensorflow.python.profiler import profiler_client
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
____________________________________________________________________________ ERROR collecting tests/host_callback_to_tf_test.py ____________________________________________________________________________
tests/host_callback_to_tf_test.py:36: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
___________________________________________________________________________________ ERROR collecting tests/image_test.py ___________________________________________________________________________________
tests/image_test.py:34: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
_________________________________________________________________________________ ERROR collecting tests/profiler_test.py __________________________________________________________________________________
tests/profiler_test.py:35: in <module>
    from tensorflow.python.profiler import profiler_client
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
____________________________________________________________________________ ERROR collecting tests/host_callback_to_tf_test.py ____________________________________________________________________________
tests/host_callback_to_tf_test.py:36: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
___________________________________________________________________________________ ERROR collecting tests/image_test.py ___________________________________________________________________________________
tests/image_test.py:34: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
_________________________________________________________________________________ ERROR collecting tests/profiler_test.py __________________________________________________________________________________
tests/profiler_test.py:35: in <module>
    from tensorflow.python.profiler import profiler_client
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
____________________________________________________________________________ ERROR collecting tests/host_callback_to_tf_test.py ____________________________________________________________________________
tests/host_callback_to_tf_test.py:36: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
___________________________________________________________________________________ ERROR collecting tests/image_test.py ___________________________________________________________________________________
tests/image_test.py:34: in <module>
    import tensorflow as tf
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
_________________________________________________________________________________ ERROR collecting tests/profiler_test.py __________________________________________________________________________________
tests/profiler_test.py:35: in <module>
    from tensorflow.python.profiler import profiler_client
/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py:41: in <module>
    from tensorflow.python.tools import module_util as _module_util
/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py:41: in <module>
    from tensorflow.python.eager import context
/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py:32: in <module>
    from tensorflow.core.framework import function_pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py:16: in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py:16: in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py:18: in <module>
    DESCRIPTOR = _descriptor.FileDescriptor(
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:952: in __init__
    _Deprecated('FileDescriptor')
/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py:98: in _Deprecated
    warnings.warn(
E   DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generated code or query the descriptor_pool.
================================================================================================= FAILURES =================================================================================================
____________________________________________________________________________________________ tests/api_test.py _____________________________________________________________________________________________
[gw0] linux -- Python 3.8.5 /usr/bin/python
worker 'gw0' crashed while running 'tests/api_test.py::CustomJVPTest::test_closure_with_vmap'
________________________________________________________________________________________ tests/lax_autodiff_test.py ________________________________________________________________________________________
[gw1] linux -- Python 3.8.5 /usr/bin/python
worker 'gw1' crashed while running 'tests/lax_autodiff_test.py::LaxAutodiffTest::testScatterGrad_shape=float32[10]_idxs=[[0]\n [0]\n [0]]_update=(3, 2)_dnums=ScatterDimensionNumbers(update_window_dims=(1,), inserted_window_dims=(), scatter_dims_to_operand_dims=(0,))'
____________________________________________________________________________________________ tests/lax_test.py _____________________________________________________________________________________________
[gw2] linux -- Python 3.8.5 /usr/bin/python
worker 'gw2' crashed while running 'tests/lax_test.py::LaxTest::testOpAgainstNumpyReciprocal_bfloat16[3]'
________________________________________________________________________________________ tests/lax_autodiff_test.py ________________________________________________________________________________________
[gw3] linux -- Python 3.8.5 /usr/bin/python
worker 'gw3' crashed while running 'tests/lax_autodiff_test.py::LaxAutodiffTest::testScatterGrad_shape=float32[10,5]_idxs=[[0]\n [2]\n [1]]_update=(3, 3)_dnums=ScatterDimensionNumbers(update_window_dims=(1,), inserted_window_dims=(0,), scatter_dims_to_operand_dims=(0,))'
____________________________________________________________________________________________ tests/lax_test.py _____________________________________________________________________________________________
[gw4] linux -- Python 3.8.5 /usr/bin/python
worker 'gw4' crashed while running 'tests/lax_test.py::LaxTest::testOpAgainstNumpyReciprocal_bfloat16[3,1]'
================================================================================ xdist: maximum crashed workers reached: 4 =================================================================================
========================================================================================= short test summary info ==========================================================================================
FAILED tests/api_test.py::CustomJVPTest::test_closure_with_vmap
FAILED tests/lax_autodiff_test.py::LaxAutodiffTest::testScatterGrad_shape=float32[10]_idxs=[[0]
 [0]
 [0]]_update=(3, 2)_dnums=ScatterDimensionNumbers(update_window_dims=(1,), inserted_window_dims=(), scatter_dims_to_operand_dims=(0,))
FAILED tests/lax_test.py::LaxTest::testOpAgainstNumpyReciprocal_bfloat16[3]
FAILED tests/lax_autodiff_test.py::LaxAutodiffTest::testScatterGrad_shape=float32[10,5]_idxs=[[0]
 [2]
 [1]]_update=(3, 3)_dnums=ScatterDimensionNumbers(update_window_dims=(1,), inserted_window_dims=(0,), scatter_dims_to_operand_dims=(0,))
FAILED tests/lax_test.py::LaxTest::testOpAgainstNumpyReciprocal_bfloat16[3,1]
ERROR tests/host_callback_to_tf_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors...
ERROR tests/image_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generate...
ERROR tests/lax_vmap_test.py
ERROR tests/profiler_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from gener...
ERROR tests/host_callback_to_tf_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors...
ERROR tests/image_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generate...
ERROR tests/profiler_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from gener...
ERROR tests/host_callback_to_tf_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors...
ERROR tests/image_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generate...
ERROR tests/profiler_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from gener...
ERROR tests/host_callback_to_tf_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors...
ERROR tests/image_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generate...
ERROR tests/profiler_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from gener...
ERROR tests/host_callback_to_tf_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors...
ERROR tests/image_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from generate...
ERROR tests/profiler_test.py - DeprecationWarning: Call to deprecated create function FileDescriptor(). Note: Create unlinked descriptors is going to go away. Please use get/find descriptors from gener...
===================================================================== 5 failed, 241 passed, 11 skipped, 16 errors in 337.44s (0:05:37) =====================================================================
root@4ff274d37bc6:/tmp/jax#

The warning about FileDescriptor()s actually comes from TensorFlow, not JAX. The tests in question are either using TensorFlow features (e.g., its profiler), testing interoperability with TensorFlow, or testing a JAX implementation returns the same output as a TensorFlow implementation.
The segmentation faults are more concerning. Are you running these tests on CPU or GPU? By default, they are probably GPU tests. You can prevent GPU usage by setting CUDA_VISIBLE_DEVICES= (i.e., empty). If you want to run GPU tests, we could test the hypothesis about unified memory by setting XLA_PYTHON_CLIENT_ALLOCATOR=platform (see https://jax.readthedocs.io/en/latest/gpu_memory_allocation.html )?

I've confirmed that jaxlib built from https://github.com/tensorflow/tensorflow/commit/8cc3ffa8d8e4dd659c1534849cf5984ef4ec3532 works perfectly on an M1.
If anyone suffering this problem (like @tigerneil) wants a temporary fix -- assuming you trust me -- then you can install my jaxlib wheel for Python 3.9 on M1's:
wget https://battle.shawwn.com/dist/jaxlib/jaxlib-0.1.74-cp39-none-macosx_11_0_arm64.whl
pip3 install jaxlib-0.1.74-cp39-none-macosx_11_0_arm64.whl --force-reinstall

At that point, you should be able to run python3 -c 'import jax.numpy as jnp; print(jnp.zeros((1,2)))' without any crashes:
$ python3 -c 'import jax.numpy as jnp; print(jnp.zeros((1,2)))'
/Users/spresser/ml/jax/jax/_src/lib/__init__.py:32: UserWarning: JAX on Mac ARM machines is experimental and minimally tested. Please see https://github.com/google/jax/issues/5501 in the event of problems.
  warnings.warn("JAX on Mac ARM machines is experimental and minimally tested. "
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
[[0. 0.]]

Note that this is actually jaxlib version 0.1.68, even though it advertises itself as 0.1.74. Meaning, you can install it, but I have no idea what might break. Things seem to work well enough for me, which is all I care about.
(I'll post about any problems I run into.)

Once I get some time, I'll try doing a git bisect between https://github.com/tensorflow/tensorflow/commit/8cc3ffa8d8e4dd659c1534849cf5984ef4ec3532 and tensorflow HEAD to track down which commit introduced the problem.

Happy to report that the latest jaxlib seems to build and run fine on M1!
I built from jax commit google/jax@d5694402bc1d9c494513e51f825591584d66157e
and tensorflow commit tensorflow/tensorflow@071a34e9ff1ed97a9eb1d323645728841fbcb7f1
My build process was:
git clone https://github.com/tensorflow/tensorflow ~/ml/tensorflow
git clone https://github.com/google/jax ~/ml/jax
ln -s $HOME/ml/tensorflow ~/ml/jax/tensorflow
cd ~/ml/jax/tensorflow
git checkout 071a34e9ff1ed97a9eb1d323645728841fbcb7f1
cd ~/ml/jax
git checkout d5694402bc1d9c494513e51f825591584d66157e

patch WORKSPACE:
diff --git a/WORKSPACE b/WORKSPACE
index 70f310f9..dc6c271c 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -1,24 +1,24 @@
 load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

-# To update TensorFlow to a new revision,
-# a) update URL and strip_prefix to the new git commit hash
-# b) get the sha256 hash of the commit by running:
-#    curl -L https://github.com/tensorflow/tensorflow/archive/<git hash>.tar.gz | sha256sum
-#    and update the sha256 with the result.
-http_archive(
-    name = "org_tensorflow",
-    sha256 = "7756b69b4a2a036ad7a4a8478f8bd7d69d0026d9c8c7fe8e8f1ae6205e978719",
-    strip_prefix = "tensorflow-968a1751ef6ccadc30ac6bd0f0be5056ac0e9288",
-    urls = [
-        "https://github.com/tensorflow/tensorflow/archive/968a1751ef6ccadc30ac6bd0f0be5056ac0e9288.tar.gz",
-    ],
-)
+# # To update TensorFlow to a new revision,
+# # a) update URL and strip_prefix to the new git commit hash
+# # b) get the sha256 hash of the commit by running:
+# #    curl -L https://github.com/tensorflow/tensorflow/archive/<git hash>.tar.gz | sha256sum
+# #    and update the sha256 with the result.
+# http_archive(
+#     name = "org_tensorflow",
+#     sha256 = "7756b69b4a2a036ad7a4a8478f8bd7d69d0026d9c8c7fe8e8f1ae6205e978719",
+#     strip_prefix = "tensorflow-968a1751ef6ccadc30ac6bd0f0be5056ac0e9288",
+#     urls = [
+#         "https://github.com/tensorflow/tensorflow/archive/968a1751ef6ccadc30ac6bd0f0be5056ac0e9288.tar.gz",
+#     ],
+# )

 # For development, one can use a local TF repository instead.
-# local_repository(
-#    name = "org_tensorflow",
-#    path = "tensorflow",
-# )
+local_repository(
+   name = "org_tensorflow",
+   path = "tensorflow",
+)

 load("//third_party/pocketfft:workspace.bzl", pocketfft = "repo")
 pocketfft()

Build jaxlib with caching:
cd ~/ml/jax/build
USE_BAZEL_VERSION=5.0.0 bazel run --action_env=PATH --remote_accept_cached=true --spawn_strategy=standalone --remote_local_fallback=false --remote_timeout=600 --verbose_failures=true --config=mkl_open_source_only :build_wheel -- --output_path=../dist --cpu=arm64

Install jaxlib:
pip3 install ~/ml/jax/dist/jaxlib-0.3.1-cp39-none-macosx_11_0_arm64.whl --no-deps --force-reinstall

Use the local jax repo:
cd ~/ml/jax
python3 setup.py develop --no-deps

Install scipy using the workaround here: https://github.com/scipy/scipy/issues/13409
brew install openblas
pip install cython pybind11 pythran numpy
OPENBLAS=$(brew --prefix openblas) CFLAGS="-falign-functions=8 ${CFLAGS}" pip3 install --no-use-pep517 scipy==1.7.0

Test it out:
$ python3
Python 3.9.10 (main, Jan 15 2022, 11:40:36)
[Clang 13.0.0 (clang-13<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import jax
~/ml/jax/jax/_src/lib/__init__.py:33: UserWarning: JAX on Mac ARM machines is experimental and minimally tested. Please see https://github.com/google/jax/issues/5501 in the event of problems.
  warnings.warn("JAX on Mac ARM machines is experimental and minimally tested. "
jax.__version__
>>> jax.__version__
'0.3.1'
>>> import jaxlib
>>> jaxlib.__version__
'0.3.1'
>>> import jax.numpy as jnp; print(jnp.zeros((1,2)))
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
[[0. 0.]]

Thanks for fixing this!


How to use Parallax Effect on my project in stickyroll

Hello, I am not finding any documentation for using Parallax Effect. I am see parallax effect in this example -https://stickyroll.github.io/react-stickyroll/#!/device-support/1
Can you give any work example or codeSandBox example-
This is my using-
const headlines = [
    <Stack direction="row">
        <Box>
            <Typography variant="h6" component="h6">
                Requirements
            </Typography>
            <Typography variant="body1" component="p">
                We are a remote developer agency based in Bangladesh,
                working with clients around the world. As passionate designers & developer, we love building website, mobile app and UI/UX that are easy to use, accessible, engaging, and delightful.
            </Typography>
        </Box>
        <Box sx={{ svg: { width: "100px" } }}>
            <ReactSVG src={Requirement} wrapper="span" />
        </Box>
    </Stack>,
    <Stack direction="row">
        <Box>
            <Typography variant="h6" component="h6">
                Hello world
            </Typography>
            <Typography variant="body1" component="p">
                We are a remote developer agency based in Bangladesh,
                working with clients around the world. As passionate designers & developer, we love building website, mobile app and UI/UX that are easy to use, accessible, engaging, and delightful.
            </Typography>
        </Box>
        <Box sx={{ svg: { width: "100px" } }}>
            <ReactSVG src={Requirement} wrapper="span" />
        </Box>
    </Stack>,
]

const Process = () => {
    return (
        <Container id="process" maxWidth={false} component="section" disableGutters>
            <Stickyroll pages={headlines} factor={0.5}>
                {({ pageIndex }) => {
                    return (

{headlines[pageIndex]}

                    );
                }}
            </Stickyroll>
        </Container>
    );
};
export default Process;

The source for the github page can be found here: https://github.com/stickyroll/react-stickyroll/blob/master/app/app.tsx


getMacCreatorAndType() is Mac-specific, legacy API

Used in t1lib.py and in ttx.py to guess filenames. Typesupply's extractor needs this API in order to guess Type1 kind of the input data. Two issues:

is Mac OS-specific
relies on Carbon APIs so not compatible with 64-bit Python (extractor uses its own AppKit API calls)

I have no idea how to fix this.  Patches are welcome.

I have no idea how to fix this. Patches are welcome.

It seems the sniffing in t1lib is only to detect LWFN (MacBinary) T1 files, and these files contain "LWFN" in their binary header, so it might be possible to just read the beginning of the file as binary and find if it contains that. Sounds good? I've been trying to find a spec for these special files to see if the header can vary in length eventually but no luck, the Adobe spec sheets don't mention anything about LWFN.

That might work.  I don't know anything about LWFN either.  Can you raise this on the fonttools Google Group?

I can't find any documentation for the LWFN file type. I can tell you that it is a Mac resource fork file, with data only in the resource fork. The file contains a standard Type1 font, chopped up into a string of POST resources. The first POST resource contains the unencrypted part, the rest contain the encrypted parts. You can find code needed to parse it in the AFDKO program 'tx', When writing and LWFN file, it first writes a temp Type1 Unix file, then calls tx.c:t1_Reformat() to re-write this as a LWFN file - but in a data fork.
Perhaps more useful is the code for sniffing and reading file formats, in tx.c:buildFontLists(). This will call 'doResMap()' if the file is Apple Resource file, or doASDFormats(), if it is an Apple Resource file which has been converted to Apple Single or Apple Double formats. You may want to copy this  code - reading and processing the POST resources is very simple, but navigating the Apple Resource fork structures to find the POST resources is complex.
My ideal approach would be to no longer support Apple Resource fork fonts!

My ideal approach would be to no longer support Apple Resource fork fonts!

That works for me.  Anyone, any objections?  Patch welcome.

After all, you can always use the Apple SplitForks tool to copy the resource fork in to a regular data file,, and then use the AFDKO tool 'tx'  to convert it to a Type 1 font file :)

Or maybe we can just update to Appkit APIs for Mac users and keep skipping the detection for other OSes which are unlikely to deal with the files anyway?


0 cover and 100% missing lines in files with 100% cover

Hi,
This is the output of my pytest (run with Tox):
  ---------- coverage: platform linux, python 3.10.2-final-0 -----------
  Name                     Stmts   Miss Branch BrPart  Cover   Missing
  --------------------------------------------------------------------
  src/schedule/header.py      50      0     16      0   100%
  src/schedule/pdf.py         16      0      2      0   100%
  src/schedule/utils.py       53      1     22      2    96%   120, 134->133
  --------------------------------------------------------------------
  TOTAL                      119      1     40      2    98%

As you can see, both src/schedule/header.py and src/schedule/pdf.py have 100% coverage and, thus, have no missing lines. However, the actions bot show 100% missing lines and 0 cover on the comment:

The pytest command that I'm running is:
pytest -v --color=yes --junitxml=pytest.xml --cov-append --cov-report=term-missing --cov-branch --cov=src tests/ | tee pytest-coverage.txt

and my GitHub step:
name: Test and Lint

on:
  workflow_dispatch:
  push:
    branches:
    - main
  pull_request:
    branches:
    - main

jobs:
  test:
    name: Test and Code Coverage Report
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.7', '3.8', '3.9', '3.10']

    steps:
    - uses: actions/checkout@v1

    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install tox tox-gh-actions coverage

    - name: Test with tox
      run: tox

    - name: Pytest coverage comment
      uses: MishaKav/pytest-coverage-comment@main
      with:
        pytest-coverage-path: pytest-coverage.txt
        junitxml-path: pytest.xml

Is this normal? Am I missing some configuration on my end?

You don't miss anything, it was a bug. Fixed that.
You can just run MishaKav/pytest-coverage-comment@main or use<EMAIL_ADDRESS>Thank you.

Thank you very much for the workflow and the quick fix.


Feature request: add "resize to fit monitor" dispatcher

First thanks for this plugin it is very interesting.
I would like to ask if there is a way to add a dispatcher that would resize the windows displayed in the current screen to make them fit in it (even if a window is only partly displayed on it. It may also have to align first border rf the most-left windows with the left border of the screen.
It could also respect the width ratio of the windows and will enable fast manipulation and alignement of multiple columns.
What do you think?

I could add something like that. The main idea of using 1/3 1/2 and 2/3 sizes is to be able to fit 2 or 3 windows in the monitor without much hassle, and still have control of the widths. Then there is resizing, which gives full flexibility, but may be slower than what would be optimal. What you are proposing is a good idea if you have an extra-wide monitor and the best ratio is not 1/3. Adding more ratios would clutter the workflow by cycling through them all the time when you want to change width, so I think you are right. I am running out of sensible key bindings, though! :-)
I will see what I can do as soon as I finish the migration to 0.37.1
Thanks for the idea!

Thanks for your quick response !
For the migration basically you have to change a few SConfigValues to CConfigValues and use the template to get the value instead of intValue.
You also have to ifdef the include of Workspace.h as it has disappeared.
I have a super ultra-wide screen (32:9 ratio) so I am able to fit 4 to 5 columns usually. Reason why if I had a dispatcher that can quickly resize the windows displayed currently visible it would be of great use.  For the size cycling I agree with you that the current setting is good. You could also think of a way to make the ratios configurable though.

The main problem I have with updating is the build system. There were several conflicts when trying to build Hyprland in a system with an older version installed system-wide, even when using Cmake prefixes. Some of these have been fixed, but I need to rewrite my build scripts, and I am in the process of doing that. The changes to the layout code are simple, like you said. Thanks!

I am on NixOS so I do not have those kind of problem for system wide stuff :)

I added scroller:fitwidth. I hope that is what you expected. It allows you to fit a combination of columns into the visible screen. Let me know if that is what you wanted, and if so, please close the issue. Thank you for the idea, I like it.

Seems to work, great implementation thanks !


Solves #298

🎯 Title and Issue Number

Closes #298

🗒 Checklist:

[x] I have mentioned the issue number in my Pull Request.
[x]  I have gone through rules of when PR will get merged.
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md
[x] I have followed proper file structure.
[x] I have gone through CONTRIBUTING GUIDELINES.md.

Thank you for contributing!❤
Don't forget to show some love by ⭐ the repository!

Hey @Praneeth-2602, Thank you for creating PR! We will review it super soon. 😇❣
As we continue to evolve and innovate, we encourage you to share your ideas, feedback, and code to help us push the boundaries of what's possible in the world of APIs🚀.
We hope you have followed CONTRIBUTING GUIDELINES.
Thank you for being an essential part of our community. Together, we'll keep building amazing things!
Don’t forget to ⭐ our repository!
Happy Coding!✨
Will See you soon❣


Remove unused function arguments from tls_int_new_record_layer

This pull request is ready to merge

Merged to master. Thank you.


Build a playground, similar to the one used by mozilla project

To make a project better integrated into Mozilla ecosystem, we should approach it the same way as Mozilla did and provide a playground in the same way

Playground is built


v0.11.0 cash on startup

Strangly this release will not start. Previews version will, there is no error message.

prompt will start adn wait for the iracing connection and after race is opened console is crashing


🐛Sir localhost not run

:warning: Make sure you have seen TROUBLESHOOTING.md before sending this issue request!

Describe the bug
[ A clear and concise description of what the bug is. ]
To Reproduce
[ Steps to reproduce the behavior. ]
Expected behavior
[ A clear and concise description of what you expected to happen. ]
Screenshots
[ If applicable, add screenshots to help explain your problem. ]
Version (please complete the following information):

OS: [e.g. Windows 10, Debian 9]
Browser [e.g. chrome, firefox]
Browser version [e.g. 22]
Docker version [docker -v]
Docker compose version [docker-compose -v]

Additional details
[ install.sh / Docker compose output, log files or any other detail about the problem here. ]

Should build fine now


[change] Pyload: Support for Group Links

Motivation for changes:
RSS plugin allows to use group_links option to generate one entry with multiple
URLs, but Pyload wasn't using it.
I was fighting with this for months before I checked the source code and have been using pyloads parsing feature instead. It never cross my mind that it wouldn't be implemented at all.
Detailed changes:

Simply load urls from entry when present.

same situation, and finally use inputs or sequence plugin to handle multi sources.

same situation, and finally use inputs or sequence plugin to handle multi sources.

I don't know anything about pyload, so I'm just going to trust you guys that this does the right thing. 😃


Standardize the .env file used by airnode-client

Description
The client container expects secrets to be provided in a .env file. If the contents of this .env file are
ENV_NAME="ENV_VALUE"

Docker sets ENV_NAME as "ENV_VALUE" (and not ENV_VALUE). See this.
Version
The api3/airnode-client:pre-alpha image built from the pre-alpha branch
Present Behaviour
The client container treats quotation marks in .env as a part of the environment variable value.
Expected Behaviour
The client container should omit the quotation marks in .env. See this.
Steps to reproduce
Launch airnode-client with a .env file with quotation marks. It will fail with invalid mnemonic because quotation marks are included in the mnemonic.

The easiest way to fix this would be to copy .env in the container along with config.json and have the node dotenv pick it up.

Docs should be updated after this: https://github.com/api3dao/airnode/blob/daf3108517386ba424335c3fcfeb57b2338a658f/Docker-client.md

Can have a look at the PR #243

This was included in 243


Added buttons while in pip

Fixes

Added buttons in pip Mode
Created HMSPipConfig class

Closes #811

PIP Event Listener for Leave room is to be added so that client apps can perform cleanup actions.


RFE: propose_downstream to work with gitlab.com/redhat RHEL dist-git

Currently the propose_downstream updates the Fedora dist-git repo. Some GitHub projects are upstream for Red Hat Enterprise Linux packages. For example convert2rhel.
To update the Red Hat dist-git before being able to create a Brew build, we do ~12 manual steps per each dist-git repo branch.

Example of the commands
$ rhpkg switch-branch converttorhel-1-rhel-6  # branch for RHEL 6 RPMs
$ rhpkg pull  # make sure we work with the latest branch content
$ rm -rf convert2rhel*  # remove all the files related to the previous release
$ wget https://raw.githubusercontent.com/oamg/convert2rhel/main/packaging/convert2rhel.spec
$ spectool --get-files --all *.spec  # download the new version tarball from GitHub
$ rhpkg new-sources *.tar.gz  # upload the tarball to dist-git
$ rhpkg srpm  # create a source RPM
$ rhpkg lint  # make sure there are no errors or warnings in the lint output
$ git add -u  # stage the changes in the dist-git repo
$ rhpkg clog  # generate a 'clog' file containing the last version changelog from the specfile
$ git commit -F clog
$ git push

As far as I know, Red Hat dist-git doesn't have a way of creating pull requests like https://src.fedoraproject.org/ does. So I guess the Packit automation could git push to the branch.
Q1 2023 planning outcome:

This is beneficial to have for teams convert2rhel, leapp, osbuild
It automates the manual steps

Thank you for opening this issue. Marking it as an "epic" since we'd need to scope this properly.
Michal, what's interesting about your use case is that you "rebase" to the latest upstream release all the time. I'm assuming you don't have any downstream patches. This is pretty special as most packages carry patches and rebase only as a last resort.

Indeed, we don't have any downstream patches at the moment. There were times when we had a downstream patch temporarily though and I can't rule out doing that in the future again.

We have concluded here that:

we will aim for RHEL 8 and 9
everything will happen in gitlab.com/redhat
once the work on this starts, we'll work with C2R to set up GL repos

After a team discussion, we didn't pick this as a top Packit team priority for the next quarter and preferred epics with bigger user benefits. (Sadly, we have limited resources...) If anyone wants to help us with this, we will be really glad. We are open to any collaboration and have already successfully implemented/started multiple affords thanks to people outside of the Packit team.
I am also not sure if this is still relevant when CentOS stream is a thing -- actually, one of the priorities we did pick is packit/packit-service#2106

I'd say this is still relevant for convert2rhel because it does not (and does not plan to) go through CentOS Stream. I don't know about leapp (@pirat89).
But I understand the lowered priority.

@bocekm Thanks Michal, for clarification about (not) going through Stream. If you know about any other package that could benefit from this, let me know and we can have a stronger motivation to take a look into this.

we are ok to handle downstream manually - currently it'S like 3x per 6months. The functionality has much better use-case under Fedora, where it can help a lot as the updates of packages are expected to be more frequent. I understand the lowered priority for this as well.


ImportError: DLL load failed while importing cpp_impl: Cannot find the specified module

Environment

System OS: Windows 10

Python Version: Python 3.8.5

Install Source:
Installed from pip
pip install spotdl
also checked pip3 install spotdlto be sure and results were the same, spotdl was already installed

Version OR Commit hash:
Version 3.3.0.

(base) C:\Users\parro>pip show spotdl
Name: spotdl
Version: 3.3.0
Summary: Download your Spotify playlists and songs along with album art and metadata
Home-page: https://github.com/spotDL/spotify-downloader
Author: spotDL Team
Author-email<EMAIL_ADDRESS>License: MIT
Location: c:\programdata\anaconda3\lib\site-packages
Requires: pytube, ytmusicapi, rapidfuzz, mutagen, spotipy, rich, tqdm
Required-by:

Expected Behaviour

I expected the download to start and have songs downloaded straight from the specified list, I have used spotdl before and after installing ffmpeg, it worked like a charm
Actual Behaviour

I got a traceback error, stating cpp_impl could not be found
Steps to Reproduce

Download Anaconda (Individual install)
pip install spotdl
```pip install ffmpeg````
spotdl https://open.spotify.com/playlist/7MnnJR4UGnAG5v0jL0yUVW

Traceback

(base) C:\Users\parro>spotdl https://open.spotify.com/playlist/7MnnJR4UGnAG5v0jL0yUVW
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\programdata\anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\ProgramData\Anaconda3\Scripts\spotdl.exe\__main__.py", line 4, in <module>
  File "c:\programdata\anaconda3\lib\site-packages\spotdl\__init__.py", line 1, in <module>
    from .__main__ import console_entry_point
  File "c:\programdata\anaconda3\lib\site-packages\spotdl\__main__.py", line 6, in <module>
    from spotdl.search.utils import get_playlist_tracks, get_album_tracks, search_for_song
  File "c:\programdata\anaconda3\lib\site-packages\spotdl\search\utils.py", line 2, in <module>
    from spotdl.search.songObj import SongObj
  File "c:\programdata\anaconda3\lib\site-packages\spotdl\search\songObj.py", line 3, in <module>
    from spotdl.search.provider import search_and_get_best_match
  File "c:\programdata\anaconda3\lib\site-packages\spotdl\search\provider.py", line 12, in <module>
    from rapidfuzz.fuzz import partial_ratio
  File "c:\programdata\anaconda3\lib\site-packages\rapidfuzz\__init__.py", line 8, in <module>
    from rapidfuzz import process, fuzz, utils, levenshtein
  File "c:\programdata\anaconda3\lib\site-packages\rapidfuzz\process.py", line 5, in <module>
    from rapidfuzz import fuzz, utils
  File "c:\programdata\anaconda3\lib\site-packages\rapidfuzz\fuzz.py", line 1, in <module>
    from rapidfuzz.cpp_impl import (
ImportError: DLL load failed while importing cpp_impl: Cannot find specified module

Any other information:
I am running python with Anaconda Prompt (which is how I had it working last time).
It is a clean install of Windows on Parallels on a Mac computer but as stated, I have had spotdl working before or another version of it, back when I needed to copy the songs to a list and then ask the software to download said list.
I may be missing something very obvious, my Python/programming skills are quite rusty.

I solved the issue by getting rid of Anaconda and installing Python 3.9.1.
Added Python to the PATH.
Did not install ffmpeg with pip but with their normal builds and then adding it to the PATH as well.
Now spotdl works like a charm.

I solved the issue by getting rid of Anaconda and installing Python 3.9.1.
Added Python to the PATH.
Did not install ffmpeg with pip but with their normal builds and then adding it to the PATH as well.
Now spotdl works like a charm.