Building on linux with ppc64le CPU

I'm trying to build jax on a cluster that uses IBM power9 processors (it's a sister cluster to Summit at ORNL).
It seems to be failing when trying to build XLA, which is strange because I've been able to install tensorflow just fine.
The full output log is here: https://gist.github.com/f0uriest/5f04e2ed9916bb750a9ea679633ac80c
Any ideas? Is there any plan to offer pre-build wheels for ppc64le architecture?

We don't support the PPC architecture ourselves and most likely don't have the engineer bandwidth to maintain such a build.
But we wouldn't object if the community wanted to supported ppc64le. There are likely two pieces:

making XLA work on PPC (probably mostly a matter of linking in the right LLVM backend in the TensorFlow/XLA tree)
maintaining a jaxlib builds.

Contributions welcome!

As to your specific question, I'd make sure that MKLDNN is disabled in the build (I believe there is an option to build.py for this.) I doubt MKLDNN works on non-Intel architectures.

I am working on the same issue. I managed to reach this point
https://gist.github.com/feifzhou/152d5c6e15e3485befa78e69cd340c32#file-gistfile1-txt
But got

Warning about 404 ERROR while downloading a .gz file
"failed: undeclared inclusion(s)" errors.

I tried both gcc 7.3.1 and 8.3.1 with same inclusion errors. Gcc 4.9.3 got me lots of syntax errors. MKLDNN was disabled.

As to your specific question, I'd make sure that MKLDNN is disabled in the build (I believe there is an option to build.py for this.) I doubt MKLDNN works on non-Intel architectures.

@feifzhou I don't know how to solve your issue, but it looks to me like Bazel isn't understanding something about the location of the standard library headers on your system. Do you have the same problem if you try to build TensorFlow? We share a lot of build infrastructure with them, so I'm wondering if this is JAX specific or a more general Bazel/TF problem.
(Ultimately we don't have cycles to work on this, but we welcome contributions!)

@f0uriest I've built v0.1.55 succsfully on an IBM power 9 but more recent version fail in the same way

@mrorro Yeah I haven't been able to build any version since 0.1.55 either. It looks like at some point they switched some of the compiler flags to ones that are only defined for x86-64 architectures. Bazel supposedly lets you override these but I haven't gotten it to work yet.

@f0uriest If you can share the output of the build, we might be able to suggest things to change.
I'd speculate there are two or three things you'd need to do :
a) update build.py to pass the correct flags, if it isn't already doing so.
b) make sure XLA links in the Power LLVM backend if targeting Power. There are already cases for x86 and ARM; I don't recall if Power is included.
c) add a Power case to build_wheel.py.

@f0uriest @mrorro
Did you all happen to make progress with this issue? We are looking to build JAX on Summit and I happened upon this issue/discussion.

Nothing so far. Would love to see if you can solve it on Summit.
On Thu, Jul 22, 2021 at 5:38 PM proutrc @.***> wrote:

@f0uriest https://github.com/f0uriest @mrorro
https://github.com/mrorro
Did you all happen to make progress with this issue? We are looking to
build JAX on Summit and I happened upon this issue/discussion.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/jax/issues/4493#issuecomment-885327371, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKT6A6WTKF7XN3XTRB2EA3TZC2ZJANCNFSM4SIJHG4A
.

I also haven't made any progress but haven't had much time to work on it either. We've been using 0.1.55 for a while, though I'm hoping to upgrade later this summer

@hawkinsp on Summit, after fixing compiler flags, we are also getting the download error:
WARNING: Download from http://mirror.tensorflow.org/github.com/tensorflow/runtime/archive/d29d1ef0a65a8f9c23e1f88067ce4205d3085e87.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found

@asedova That is a benign warning, you can ignore it.

I was able to cross-compile a ppc64le wheel on an x86-64 machine by following the instructions in #7365. I can't easily test the resulting wheel, though.
I would imagine that building natively on a ppc64le machine requires nothing other than following the standard instructions once the changes in #7365 are merged.

@asedova That is a benign warning, you can ignore it.

Thanks

I was able to cross-compile a ppc64le wheel on an x86-64 machine by following the instructions in #7365. I can't easily test the resulting wheel, though.
I would imagine that building natively on a ppc64le machine requires nothing other than following the standard instructions once the changes in #7365 are merged.

Thanks @hawkinsp we are eagerly awaiting this merge

One thing I'd like to double check: what does:
import platform
print(platform.machine())

print on your PPC machine?
And is it little endian?

On my system I get

On my system I get
>>> import platform
>>> print(platform.machine())
ppc64le

It is little-endian

yes we are LE also

@f0uriest you guys are on Sierra?

On Lassen
On Fri, Jul 23, 2021 at 8:56 AM asedova @.***> wrote:

@f0uriest you guys are on Sierra?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@asedova Traverse at PPPL

#7365 is merged. Please try building jaxlib from git head. If it doesn't work, please post logs so I can try to debug it.
I could also share the cross-compiled wheel I made for Python 3.9; but I have no idea if it actually works. So it's probably best if you make sure it builds for you.

@hawkinsp Here is what I see on initial attempt: summit_jaxlib.log
Just for record, I have tried various  versions of GCC (6.4.0, 7.4.0, 8.1.1)
Notable errors:
gcc: error: unrecognized command line option '-std=c++14'
ERROR: /tmp/_bazel_rprout/b2ebe10a0ad0f6175e81a930563cb9d3/external/com_google_protobuf/BUILD:155:11: Compiling src/google/protobuf/util/internal/datapiece.cc [for host] failed: (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command    (cd /tmp/_baz
@asedova do you see anything different?

You need a C++14 compiler to build JAX.
Something seem surprising here, though. gcc 6.1 and newer apparently support C++14: https://gcc.gnu.org/projects/cxx-status.html#cxx14
Note the documentation is quite clear that -std=c++14 is a flag gcc accepts! So this seems like something you need to figure out about your gcc installation.

@hawkinsp  Apologies, I could have goofed that one actually. I thought had GCC loaded...
Here is a run with GCC 7.4.0:summit_jaxlib-gcc7.4.0.log

@proutrc The issue here is that bazel hermeticity checking is upset that you appear to be reading header files outside what it considers to be the standard system paths.
I think your best fix here might be to write a small custom Bazel toolchain. As it happens, I show an example of how to do that in a comment in #7365. It's not that bad, you should be able to just modify my example. You would need to modify cxx_builtin_include_directories to include that header directory, and you'd need to change the other tool paths to point to the right places on your system.

@hawkinsp sorry for my ignorance.. but, is the mentioned toolchain directory from the top of the jax repp or in the build directory? My familiarity with bazel and its setup is limited, unfortunately. I am happy to work on this though, just want to make sure I am setup properly.

@proutrc In the example I gave, it's at the root of the JAX repository. (It doesn't matter a whole lot, so long as all the paths agree, and in my command line, etc. I refer to it as //toolchain, which is at the root of the repository.)

@hawkinsp
I seem to still run into similar issues. Is there anything else I am missing, besides an update to those paths for the tools and the cxx_builtin_include_directories? I am also putting the realpath in the cxx_builtin_include_directories list, but I see it has the non-realpath in the error output. Sometimes it does have the realpath though, oddly. I appreciate your help.
def _impl(ctx):
   return cc_common.create_cc_toolchain_config_info(
       ctx = ctx,
       features = features, # NEW
       cxx_builtin_include_directories = [
          "/autofs/nccs-svm1_sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/",
          "/autofs/nccs-svm1_sw/summit/gcc/7.4.0/include/",
          "/autofs/nccs-svm1_sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/",
          "/usr/include/",
       ],

Error (it does seem to get further sometimes):
[0 / 31] [Prepa] Creating source manifest for //build:build_wheel ... (5 actions, 0 running)
[68 / 529] Compiling src/google/protobuf/generated_enum_util.cc [for host]; 2s local ... (128 actions running)
[76 / 529] Compiling src/google/protobuf/generated_enum_util.cc [for host]; 5s local ... (128 actions running)
[83 / 529] Compiling src/google/protobuf/extension_set.cc [for host]; 9s local ... (128 actions running)
[89 / 529] Compiling src/google/protobuf/extension_set.cc [for host]; 13s local ... (128 actions running)
[98 / 529] Compiling src/google/protobuf/extension_set.cc [for host]; 18s local ... (128 actions, 127 running)
ERROR: /gpfs/alpine/stf007/scratch/rprout/jax/jaxlib/BUILD:352:17: Compiling jaxlib/cpu_feature_guard.c failed: undeclared inclusion(s) in rule '//jaxlib:cpu_feature_guard.so':
this rule is missing dependency declarations for the following files included by 'jaxlib/cpu_feature_guard.c':
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/limits.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/syslimits.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stddef.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stdarg.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stdint.h'
Target //build:build_wheel failed to build
INFO: Elapsed time: 40.420s, Critical Path: 22.55s
INFO: 333 processes: 241 internal, 92 local.
FAILED: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully
b''
Traceback (most recent call last):
  File "build/build.py", line 604, in <module>
    main()
  File "build/build.py", line 599, in main
    shell(command)
  File "build/build.py", line 52, in shell
    output = subprocess.check_output(cmd)
  File "/sw/summit/python/3.7/anaconda3/5.3.0/lib/python3.7/subprocess.py", line 376, in check_output
    **kwargs).stdout
  File "/sw/summit/python/3.7/anaconda3/5.3.0/lib/python3.7/subprocess.py", line 468, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/sw/.testing/belhorn/summit/bin/bazel', 'run', '--verbose_failures=true', '--host_crosstool_top=//toolchain:ppc', '--crosstool_top=//toolchain:ppc', '--config=short_logs', '--config=cuda', '--define=xla_python_enable_gpu=true', ':build_wheel', '--', '--output_path=/gpfs/alpine/stf007/scratch/rprout/jax/dist', '--cpu=ppc64le']' returned non-zero exit status 1.

@proutc Try with --bazel_options=--cpu=ppc.

@hawkinsp
INFO: Found 1 target...
[0 / 68] [Prepa] Writing file jaxlib/lapack.so-2.params
ERROR: /tmp/_bazel_rprout/b2ebe10a0ad0f6175e81a930563cb9d3/external/com_google_absl/absl/base/BUILD.bazel:596:11: Compiling absl/base/internal/exponential_biased.cc failed: undeclared inclusion(s) in rule '@com_google_absl//absl/base:exponential_biased':
this rule is missing dependency declarations for the following files included by 'absl/base/internal/exponential_biased.cc':
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stdint.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/limits.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/syslimits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstddef'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/c++config.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/os_defines.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/cpu_defines.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stddef.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ciso646'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cassert'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/algorithm'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/utility'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_relops.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_pair.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/move.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/concept_check.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/type_traits'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/initializer_list'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_algobase.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/functexcept.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/exception_defines.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/cpp_type_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/type_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/numeric_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_iterator_base_types.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_iterator_base_funcs.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/debug/assertions.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_iterator.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/ptr_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/debug/debug.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/predefined_ops.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_algo.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstdlib'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/std_abs.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/algorithmfwd.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_heap.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_tempbuf.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_construct.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/new'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/exception'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/exception.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/exception_ptr.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/cxxabi_init_exception.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/typeinfo'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/hash_bytes.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/nested_exception.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/alloc_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/alloc_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/memoryfwd.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/uniform_int_dist.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/limits'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/atomic'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/atomic_base.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/atomic_lockfree_defines.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cmath'
Target //build:build_wheel failed to build
INFO: Elapsed time: 19.438s, Critical Path: 1.74s
INFO: 245 processes: 215 internal, 30 local.
FAILED: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully
b''
Traceback (most recent call last):
  File "build/build.py", line 604, in <module>
    main()
  File "build/build.py", line 599, in main
    shell(command)
  File "build/build.py", line 52, in shell
    output = subprocess.check_output(cmd)
  File "/sw/summit/python/3.7/anaconda3/5.3.0/lib/python3.7/subprocess.py", line 376, in check_output
    **kwargs).stdout
  File "/sw/summit/python/3.7/anaconda3/5.3.0/lib/python3.7/subprocess.py", line 468, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/sw/.testing/belhorn/summit/bin/bazel', 'run', '--verbose_failures=true', '--host_crosstool_top=//toolchain:ppc', '--crosstool_top=//toolchain:ppc', '--cpu=ppc', '--config=short_logs', '--config=cuda', '--define=xla_python_enable_gpu=true', ':build_wheel', '--', '--output_path=/gpfs/alpine/stf007/scratch/rprout/jax/dist', '--cpu=ppc64le']' returned non-zero exit status 1.

Try putting the /sw/... paths in the cxx_builtin_include_directories. Or put try putting both.

Unfortunately, I have tried all as well:
Error:
[0 / 35] [Prepa] Creating source manifest for //build:build_wheel
[64 / 542] Compiling src/google/protobuf/any_lite.cc [for host]; 3s local ... (127 actions, 126 running)
[80 / 542] Compiling src/google/protobuf/extension_set.cc [for host]; 6s local ... (127 actions running)
[89 / 542] Compiling src/google/protobuf/extension_set.cc [for host]; 9s local ... (128 actions running)
[100 / 544] Compiling src/google/protobuf/extension_set.cc [for host]; 13s local ... (127 actions running)
[157 / 614] Compiling src/google/protobuf/extension_set.cc [for host]; 17s local ... (128 actions running)
[197 / 716] Compiling src/google/protobuf/extension_set.cc [for host]; 22s local ... (128 actions running)
[239 / 716] Compiling src/google/protobuf/compiler/cpp/cpp_helpers.cc [for host]; 27s local ... (128 actions running)
ERROR: /tmp/_bazel_rprout/b2ebe10a0ad0f6175e81a930563cb9d3/external/com_google_absl/absl/time/internal/cctz/BUILD.bazel:53:11: Compiling absl/time/internal/cctz/src/time_zone_posix.cc failed: undeclared inclusion(s) in rule '@com_google_absl//absl/time/internal/cctz:time_zone':
this rule is missing dependency declarations for the following files included by 'absl/time/internal/cctz/src/time_zone_posix.cc':
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstdint'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/c++config.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/os_defines.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/cpu_defines.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stdint.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/string'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stringfwd.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/memoryfwd.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/char_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_algobase.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/functexcept.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/exception_defines.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/cpp_type_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/type_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/numeric_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_pair.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/move.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/concept_check.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/type_traits'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_iterator_base_types.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_iterator_base_funcs.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/debug/assertions.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_iterator.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/ptr_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/debug/debug.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/predefined_ops.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/postypes.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cwchar'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stdarg.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include/stddef.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/allocator.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/c++allocator.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/new_allocator.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/new'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/exception'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/exception.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/exception_ptr.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/cxxabi_init_exception.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/typeinfo'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/hash_bytes.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/nested_exception.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/localefwd.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/c++locale.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/clocale'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/iosfwd'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cctype'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/ostream_insert.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/cxxabi_forced.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/stl_function.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/backward/binders.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/range_access.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/initializer_list'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/basic_string.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/atomicity.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/gthr.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/gthr-default.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/powerpc64le-none-linux-gnu/bits/atomic_word.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/alloc_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/alloc_traits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ext/string_conversions.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstdlib'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/std_abs.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstdio'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cerrno'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/functional_hash.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/bits/basic_string.tcc'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/limits.h'
  '/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed/syslimits.h'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstddef'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/ciso646'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/cstring'
  '/sw/summit/gcc/7.4.0/include/c++/7.4.0/limits'
Target //build:build_wheel failed to build
INFO: Elapsed time: 48.498s, Critical Path: 33.11s
INFO: 376 processes: 221 internal, 155 local.
FAILED: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully
b''
Traceback (most recent call last):
  File "build/build.py", line 604, in <module>
    main()
  File "build/build.py", line 599, in main
    shell(command)
  File "build/build.py", line 52, in shell
    output = subprocess.check_output(cmd)
  File "/sw/summit/python/3.7/anaconda3/5.3.0/lib/python3.7/subprocess.py", line 376, in check_output
    **kwargs).stdout
  File "/sw/summit/python/3.7/anaconda3/5.3.0/lib/python3.7/subprocess.py", line 468, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/sw/.testing/belhorn/summit/bin/bazel', 'run', '--verbose_failures=true', '--host_crosstool_top=//toolchain:ppc', '--crosstool_top=//toolchain:ppc', '--cpu=ppc', '--config=short_logs', '--config=cuda', '--define=xla_python_enable_gpu=true', ':build_wheel', '--', '--output_path=/gpfs/alpine/stf007/scratch/rprout/jax/dist', '--cpu=ppc64le']' returned non-zero exit status 1.

cxx_builtin_include_directories list:
def _impl(ctx):
   return cc_common.create_cc_toolchain_config_info(
       ctx = ctx,
       features = features, # NEW
       cxx_builtin_include_directories = [
          "/autofs/nccs-svm1_sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include",
          "/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include",
          "/autofs/nccs-svm1_sw/summit/gcc/7.4.0/include",
          "/sw/summit/gcc/7.4.0/include",
          "/autofs/nccs-svm1_sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed",
          "/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed",
          "/usr/include",
       ],

@hawkinsp perhaps relative to here?
https://github.com/bazelbuild/bazel/issues/9451

@f0uriest you said you got a previous version to build?

Not me..
On Fri, Jul 23, 2021 at 2:36 PM asedova @.***> wrote:

@f0uriest https://github.com/f0uriest you said you got a previous
version to build?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/jax/issues/4493#issuecomment-885926196, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKT6AZ6N6R4VLYQPUC7EQTTZHOG3ANCNFSM4SIJHG4A
.

@hawkinsp it looks like I was abot to get around the undeclared inclusion(s) by adding this to a CROSSTOOL file, in the toolchain/ directory. This is in addition to the cc_toolchain_config.bzl and BUILD file we altered from your example.
<EMAIL_ADDRESS>jax]$ ls toolchain/
BUILD  CROSSTOOL  cc_toolchain_config.bzl

Here we are now: summit-jaxlib.log

It actually looks like this CROSSTOOL file addition is not they key. The real key seems to be throttling Bazel wih this addition to my ~/.bazelrc file:
build --jobs 2 --local_ram_resources=HOST_RAM*0.04
test --jobs 2

It looks like you want to throttle Bazel if you use NFS at all. Summit's provided software tree (/sw) is on NFS. I had forgotten I added the above throttling in all this.  As soon as I removed it though, from my ~/.bazelrc file, the undeclared inclusion(s) came back.
@feifzhou You should try the throttling method above, by adding that to your ~/.bazelrc. Perhaps you will then also get passed the undeclared inclusion(s). Maybe you have something on NFS?
In the end, we now seem to be in a similar boat as @f0uriest. Our log now similarly points at Eigen. I will try a different GCC next (maybe some additional flags?).

@hawkinsp it looks like I was able to get around the undeclared inclusion(s) by adding this to a CROSSTOOL file, in the toolchain/ directory. This is in addition to the cc_toolchain_config.bzl and BUILD file we altered from your example.
<EMAIL_ADDRESS>jax]$ ls toolchain/
BUILD  CROSSTOOL  cc_toolchain_config.bzl

<EMAIL_ADDRESS>jax]$ cat toolchain/CROSSTOOL
compiler_flag: "-isystem"
compiler_flag: "/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed"
compiler_flag: "-isystem"
compiler_flag: "/sw/summit/gcc/7.4.0/include/c++/7.4.0"
compiler_flag: "-isystem"
compiler_flag: "/sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include"

Here we are now: summit-jaxlib.log

@proutrc I'm wondering if that's related to building with gcc 7. See this similar looking issue for PyTorch: https://github.com/pytorch/pytorch/pull/50640
You might be able to work around the problem by sending a pull request to Eigen that adds similar fallback definitions of the missing vector intrinsics.

@hawkinsp I did indeed go that route, using 8.1.1 over the weekend and this morning.
Oddly, the undeclared inclusion(s) comes back! Seemingly further though:
https://gist.github.com/proutrc/d4bc637555d3624d8aa4ccf6a65f348f

@proutrc Can you share the toolchain .bzl and BUILD files you are using in another gist?

@hawkinsp
BUILD: https://gist.github.com/proutrc/ba20b7f5d2c4b6ae7e26cfe4afdea44e
cc_toolchain: https://gist.github.com/proutrc/4cb2bd0a3a4f804243e8134295079906

I'd include both the /autofs and /sw paths just to make sure it doesn't help. (Including more paths should only help, not hurt.)

Thanks @hawkinsp, I will play with this more. I have added both /sw and /autofs paths before, but am going to try that again and will report back.
I have been setting these for cache, etc.. then clearing them every run (off NFS):
startup --output_user_root=/gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root
build --disk_cache=/gpfs/alpine/stf007/scratch/rprout/bazel-cache/
export TEST_TMPDIR=/gpfs/alpine/stf007/scratch/rprout/bazel-tmp/
Maybe I haven't found the right combo of things yet, not sure. But, Bazel definitely seems finicky about NFS.

Is it possible to use non-NFS temporary and cache directories? I don't know if it will help, but it might.

Is it possible to use non-NFS temporary and cache directories? I don't know if it will help, but it might.

I think that is what I am doing with these settings:
startup --output_user_root=/gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root
build --disk_cache=/gpfs/alpine/stf007/scratch/rprout/bazel-cache/
export TEST_TMPDIR=/gpfs/alpine/stf007/scratch/rprout/bazel-tmp/

@hawkinsp I can confirm I don't cache anything on NFS.
I also added the /sw and /autofs paths:
def _impl(ctx):
   return cc_common.create_cc_toolchain_config_info(
       ctx = ctx,
       features = features, # NEW
       cxx_builtin_include_directories = [
          #"/autofs/nccs-svm1_sw/summit/gcc/7.4.0/include",
          #"/autofs/nccs-svm1_sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include",
          #"/autofs/nccs-svm1_sw/summit/gcc/7.4.0/lib/gcc/powerpc64le-none-linux-gnu/7.4.0/include-fixed",
          "/sw/summit/gcc/8.1.1/include",
          "/autofs/nccs-svm1_sw/summit/gcc/8.1.1/include",
          "/sw/summit/gcc/8.1.1/include/c++/8.1.1",
          "/autofs/nccs-svm1_sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include",
          "/autofs/nccs-svm1_sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include-fixed",
          "/sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include",
          "/sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include-fixed/",
          "/autofs/nccs-svm1_sw/summit/gcc/8.1.1/include/c++/8.1.1",
          "/usr/include",
       ]

It is strange too, because it is obviously able to compile things leading up to the inclusion failure:
......
INFO: Found 1 target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[79 / 535] Compiling src/google/protobuf/struct.pb.cc [for host]; 0s local, remote-cache ... (3 actions, 2 running)
[124 / 535] Compiling src/google/protobuf/compiler/java/java_message_field.cc [for host]; 0s local, remote-cache ... (3 actions, 2 running)
[178 / 535] Compiling src/google/protobuf/compiler/csharp/csharp_field_base.cc [for host]; 0s local, remote-cache ... (3 actions, 2 running)
[1,702 / 2,103] Compiling internal/wait.c [for host]; 0s local, remote-cache ... (3 actions, 2 running)
[1,814 / 2,174] Compiling external/org_tensorflow/tensorflow/core/framework/cost_graph.pb.cc [for host]; 1s local, remote-cache ... (3 actions, 2 running)
ERROR: /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/external/com_github_grpc_grpc/BUILD:1883:16: Compiling src/core/ext/transport/chttp2/server/insecure/server_chttp2.cc failed: undeclared inclusion(s) in rule '@com_github_grpc_grpc//:grpc_transport_chttp2_server_insecure':
this rule is missing dependency declarations for the following files included by 'src/core/ext/transport/chttp2/server/insecure/server_chttp2.cc':
  '/sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include/stdint.h'
  '/sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include/stddef.h'
  '/sw/summit/gcc/8.1.1/lib/gcc/powerpc64le-unknown-linux-gnu/8.1.1/include/stdarg.h'
  '/sw/summit/gcc/8.1.1/include/c++/8.1.1/stdlib.h'
  '/sw/summit/gcc/8.1.1/include/c++/8.1.1/cstdlib'
.....

@hawkinsp is our toolchain not being used everywhere by chance?
[599 / 2,232] Executing genrule @local_config_cuda//cuda:cuda-include; 3s local, remote-cache ... (4 actions running)
ERROR: /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/external/org_tensorflow/tensorflow/core/platform/BUILD:453:11: Compiling tensorflow/core/platform/path.cc failed: undeclared inclusion(s) in rule '@org_tensorflow//tensorflow/core/platform:path':

Are there different "rules" or something? undeclared inclusion(s) in rule '@org_tensorflow//tensorflow/core/platform:path':

@hawkinsp I did some digging around in my build-cache, here: /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/execroot/__main__/external/
It looks like these external packages setup their own .bazelrc and possibly don't get our toolchain config. Is there a guarantee that what we set as the toolchain config in JAX makes it to these external packages?

I was able to build from main on Traverse without having to do any toolchain modifications, though I did have to manually specify the cuda/cudnn paths.
python build/build.py --enable_cuda --cuda_path /usr/local/cuda-11.3 --cuda_version=11.3 --cudnn_version=8.2.0 --cudnn_path /usr/local/cudnn/cuda-11.3/8.2.0  --noenable_mkl_dnn --cuda_compute_capabilities 7.0 --bazel_path /usr/bin/bazel --target_cpu=ppc
Thanks so much for your help with this! Hope the other ppc users can also get it working

@f0uriest few questions:
What compiler and version are you using?
Is it installed in /usr/bin?

gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
installed in /usr/bin

Thanks, it looks like we needed to add the patch referenced in this issue: https://github.com/tensorflow/tensorflow/issues/33975
Otherwise, TF does not pick up the proper inclusion paths for the compiler outside /usr/bin.
Specifically to bazel-jax/external/org_tensorflow/third_party/gpus/cuda_configure.bzl, which was in the build-cache.
That got us passed the inclusion errors.

@hawkinsp if you have any insight into this next issue we are hitting, that would be great. This one seems odd. Themlir-tblgen either fails or seems to target improper dialects. For instance, I have seen it fail targeting ArmSVE dialect and AMX dialect. Not sure what causes the different attempts.
Here is the most recent generic failure though:
[4,069 / 4,121] Compiling mlir/lib/TableGen/Constraint.cpp; 1s local, remote-cache ... (4 actions running)
ERROR: /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/external/llvm-project/mlir/BUILD:4006:18: TdGenerate external/llvm-project/mlir/include/mlir/Interfaces/ViewLikeInterface.cpp.inc failed: (Exit 1): mlir-tblgen failed: error executing command
  (cd /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/execroot/__main__ && \
  exec env - \
  bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen -gen-op-interface-defs external/llvm-project/mlir/include/mlir/Interfaces/ViewLikeInterface.td -I external/llvm-project/mlir/include -I bazel-out/ppc-opt/bin/external/llvm-project/mlir/include -I external/llvm-project/ -I bazel-out/ppc-opt/bin/external/llvm-project/ -I external/llvm-project/mlir/include/mlir/Interfaces -I bazel-out/ppc-opt/bin/external/llvm-project/mlir/include/mlir/Interfaces -o bazel-out/ppc-opt/bin/external/llvm-project/mlir/include/mlir/Interfaces/ViewLikeInterface.cpp.inc)
Execution platform: @local_execution_config_platform//:platform
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)

Separately, at the top, when I first run the jax build I see this warning (not exactly sure of the implications, or if it is related to the above, but it seems like it impacts the toolchain):
WARNING: option '--config=cuda' (source command line options) was expanded and now overrides the explicit option --crosstool_top=//toolchain:ppc with --crosstool_top=@local_config_cuda//crosstool:toolchain

It appears these are saying we need a GCC 4.9:
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)

@hawkinsp
Just to follow up more - we made it further, but we seem to be hitting similar issues with other packages that assume the location of GCC. My guess is that this will happen at various steps, the first being Tensorflow and now llvm-project.
Recent error within external\llvm-project/mlir
[4,727 / 5,554] checking cached actions
ERROR: /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/external/llvm-project/mlir/BUILD:173:18: TdGenerate external/llvm-project/mlir/include/mlir/IR/BuiltinLocationAttributes.h.inc failed: (Exit 1): mlir-tblgen failed: error executing command
  (cd /gpfs/alpine/stf007/scratch/rprout/bazel-build-cache/user-root/b2ebe10a0ad0f6175e81a930563cb9d3/execroot/__main__ && \
  exec env - \
  bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen --gen-attrdef-decls external/llvm-project/mlir/include/mlir/IR/BuiltinLocationAttributes.td -I external/llvm-project/mlir/include -I bazel-out/ppc-opt/bin/external/llvm-project/mlir/include -I external/llvm-project/ -I bazel-out/ppc-opt/bin/external/llvm-project/ -I external/llvm-project/mlir/include/mlir/IR -I bazel-out/ppc-opt/bin/external/llvm-project/mlir/include/mlir/IR -o bazel-out/ppc-opt/bin/external/llvm-project/mlir/include/mlir/IR/BuiltinLocationAttributes.h.inc)
Execution platform: @local_execution_config_platform//:platform
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by bazel-out/ppc-opt-exec-50AE0418/bin/external/llvm-project/mlir/mlir-tblgen)
Target //build:build_wheel failed to build
INFO: Elapsed time: 70.123s, Critical Path: 16.09s
INFO: 68 processes: 55 internal, 13 local.
FAILED: Build did NOT complete successfully
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully

Note, it appears our env looks empty (I don't see LD_LIBRARY_PATH or anything).  Not sure why this is happening all of a sudden.
So, it is correctly complaining about the system provided GCC (/usr/bin). But, we are trying to use the non-system GCC in /sw/summit/gcc/8.1.1/bin/gcc and /sw/summit/gcc/8.1.1/lib64.
This works up until various points with external packages. We were able to fix the similar Tensorflow issue with the patch we referenced, but are now hitting it with llvm-project.
For HPC systems that have the proper GCC in /usr/bin everything seems to go smoothly (like the PPPL folks). That seems like a bad assumption by all these packages though, from an HPC perspective. I realize this is not all within the JAX realm, but it is worth noting.
Is there a way to make our toolchain known to these other packages?And/or is there a necessary change needed in these various packages BUILD files? I could be thinking about this incorrectly, as my familiarity with Bazel is limited.
If by chance you see a way to help, please do!

I think what that means is that mlir-tblgen was built by Bazel, but the result does not run. This indicates either something misconfigured with the toolchain or a dynamic linker path problem. It's probably the former.
One thing I might try is a non-CUDA build and see if it works. CUDA requires that TF do some trickery with toolchains to support the CUDA compilers.
Another option might be to cross-compile, as I mentioned above. The main difficulty there will be that you will need to target a suitable glibc version.
Yet another option might be to compile in something like a docker container that has a more standard layout. I'm not sure if that's feasible for you.

We also got it to work without the toolchain modifications on a smaller test ppc system of ours, which has RHEL8 and GCC 8.3 in /usr/bin.
On Summit, we don't yet have RHEL8 and have GCC 4.8 in /usr/bin. We just can't seem get the GCC 8.x compiler, provided from our module system (i.e /sw/summit/gcc/8.1.1), known to all the Bazel build recipes.

@proutrc For jaxlib 0.1.70, we have a fix for the mlir-tblgen issue, see this patch.
However for jaxlib 0.1.71, I'm running into the same problem again, and I'm not sure how to fix it there because things have shifted around enough to make the patch we came up with useless... I've opened a dedicated issue on this: #7842

I'm getting this error for the most recent tag (jax-v0.2.28).
More specifically, the error is as follows,
ERROR: /home/mohammad/.cache/bazel/_bazel_mohammad/5a0cafedcffcc5a6733cc68df657e72e/external/llvm-project/mlir/BUILD.bazel:3356:11: Compiling mlir/lib/Conversion/GPUToVulkan/ConvertGPULaunchFuncToVulkanLaunchFunc.cpp failed: undeclared inclusion(s) in rule '@llvm-project//mlir:GPUToVulkanTransforms': this rule is missing dependency declarations for the following files included by 'mlir/lib/Conversion/GPUToVulkan/ConvertGPULaunchFuncToVulkanLaunchFunc.cpp':   'bazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/GPUBaseIncGen/mlir/Dialect/GPU/GPUOpsDialect.h.inc' Target //build:build_wheel failed to build INFO: Elapsed time: 464.234s, Critical Path: 90.53s INFO: 1117 processes: 176 internal, 941 local. FAILED: Build did NOT complete successfully ERROR: Build failed. Not running target FAILED: Build did NOT complete successfully

@mshafiei I don't see that error building at that tag. One thing that's worth trying: try running bazel clean using the copy of bazel being used for the build and then try the build again. The LLVM Bazel rules sometimes confuse Bazel.

@hawkinsp It worked. Thanks

I recently went through building and installing jaxlib 0.1.76 on MIT IBM Satori, a ppc64le machine, and I thought I would share my steps in case anyone happens to find them helpful:

I installed bazel using a community-created binary for ppc64le. In my case, I had to use Bazel 4.2.2, as newer versions were incompatible with the system libraries on IBM Satori (this also meant I could only build an older version of jaxlib, as the more recent ones seem to require Bazel >=5.0)
I set up and activated a dedicated Anaconda environment with cudatoolkit and loaded a system CUDA 11.1 module
I downloaded the source for the desired release (jaxlib-v0.1.76 in my case), unpacked it and run python build/build.py --enable_cuda --cuda_path /software/cuda/11.1 --cudnn_path /nobackup/users/<username>/anaconda3/envs/<environment name> in the its root. The CUDA and cuDNN paths specified in the build command need to be adapted for the specific system.
I installed the resulting .whl package using pip (found in the /dist subdirectory once the build completed successfully).

The resulting jaxlib installation seems to work well for our purposes, although it can only be used for running on a GPU. Attempting to run on a CPU triggers errors 'pwr9' is not a recognized processor for this target (ignoring processor).

@proutrc @feifzhou Have you had any success on this issue since last year? I am trying to install JAX on the Lassen PPC system and having a lot of difficulties. I am able to get around most of the undeclared dependency issues by making toolchain files as described above and installing following these commands, but end up getting this error from Eigen during compilation.
Is there perhaps a pre-configured/compiled package available for JAX on PPC/Lassen similar to the opence distributions of torch and other packages? I've been trying unsucessfully to install JAX for the last several days now

@C-J-Cundy did you try the jaxlib anaconda package for linux-ppc64le?

@mrorro Yes, when trying that I get the erorr
ImportError: /lib64/ld64.so.2: version GLIBC_2.22' not found (required by /g/g92/cundy2/.conda/envs/rljax/lib/python3.8/site-packages/jaxlib/xla_extension.so)`
It seems like the conda version is built against glibc_2.22, which isn't present on my system (and building it myself breaks everything else on the RHEL7 system)

Looks like Open-CE 1.9 should allow you to use Jax on powerpc platforms.
https://github.com/open-ce/open-ce/issues/549

@allen-adastra Do you have an example of how to use? is it as simple as doin something like conda install -c https://ftp.osuosl.org/pub/open-ce/current/ jax?

Is anyone still working on this?
I tried a build using Bazel 7.2.0 built from the dist archive.  I have cuda 11.1.1, cuDNN <IP_ADDRESS>, gcc 9.4.0.  The system is running Ubuntu 20.04 and is a Power 8 S822LC HPC (Minsky).
I'm trying to build from the git HEAD:
$ TMP=/tmp /usr/bin/python3.9 build/build.py --python_version=3.9 --noenable_mkl_dnn --enable_cuda --build_gpu_plugin --gpu_plugin_cuda_version=11

     _   _  __  __
    | | / \ \ \/ /
 _  | |/ _ \ \  /
| |_| / ___ \/  \
 \___/_/   \/_/\_\

Bazel binary path: /usr/local/bin/bazel
Bazel version: 7.2.0
Use clang: no
MKL-DNN enabled: no
Target CPU: ppc64le
Target CPU features: release
CUDA enabled: yes
NCCL enabled: yes
ROCm enabled: no

Building XLA and installing it in the jaxlib source tree...
/usr/local/bin/bazel run --verbose_failures=true //jaxlib/tools:build_wheel -- --output_path=/home/build/repos/jax/dist --jaxlib_git_hash=d3bfd32667d7742d2fc208b75d4085e923d5e03f --cpu=ppc64le --skip_gpu_kernels
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'run' from /home/build/repos/jax/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'run' from /home/build/repos/jax/.bazelrc:
  Inherited 'build' options: --nocheck_visibility --apple_platform_type=macos --macos_minimum_os=10.14 --announce_rc --define open_source_build=true --spawn_strategy=standalone --enable_platform_specific_config --experimental_cc_shared_library --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true --define=tsl_link_protobuf=true -c opt --config=short_logs --copt=-DMLIR_PYTHON_PACKAGE_PREFIX=jaxlib.mlir. --@xla//xla/python:enable_gpu=false
INFO: Reading rc options for 'run' from /home/build/repos/jax/.jax_configure.bazelrc:
  Inherited 'build' options: --strategy=Genrule=standalone --config=cuda --config=cuda_plugin --repo_env HERMETIC_PYTHON_VERSION=3.9
INFO: Found applicable config definition build:short_logs in file /home/build/repos/jax/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:cuda in file /home/build/repos/jax/.bazelrc: --repo_env TF_NEED_CUDA=1 --repo_env TF_NCCL_USE_STUB=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=sm_50,sm_60,sm_70,sm_80,compute_90 --crosstool_top=@local_config_cuda//crosstool:toolchain --@local_config_cuda//:enable_cuda --@xla//xla/python:enable_gpu=true --@xla//xla/python:jax_cuda_pip_rpaths=true --define=xla_python_enable_gpu=true --linkopt=-Wl,--disable-new-dtags
INFO: Found applicable config definition build:cuda_plugin in file /home/build/repos/jax/.bazelrc: --@xla//xla/python:enable_gpu=false --define=xla_python_enable_gpu=false
INFO: Found applicable config definition build:linux in file /home/build/repos/jax/.bazelrc: --config=posix --copt=-Wno-unknown-warning-option --copt=-Wno-stringop-truncation --copt=-Wno-array-parameter
INFO: Found applicable config definition build:posix in file /home/build/repos/jax/.bazelrc: --copt=-fvisibility=hidden --copt=-Wno-sign-compare --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
Computing main repo mapping:
Loading:
Loading: 0 packages loaded
Analyzing: target //jaxlib/tools:build_wheel (0 packages loaded, 0 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (0 packages loaded, 0 targets configured)
[0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /home/build/.cache/bazel/_bazel_build/dcc0c88d87f0eff69c7a10c9d397572b/external/local_config_python/BUILD:16:11: in py_runtime rule @@local_config_python//:py3_runtime:
Traceback (most recent call last):
	File "/virtual_builtins_bzl/common/python/py_runtime_rule.bzl", line 40, column 17, in _py_runtime_impl
Error in fail: interpreter_path must be an absolute path
ERROR: /home/build/.cache/bazel/_bazel_build/dcc0c88d87f0eff69c7a10c9d397572b/external/local_config_python/BUILD:16:11: Analysis of target '@@local_config_python//:py3_runtime' failed
ERROR: Analysis of target '//jaxlib/tools:build_wheel' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.253s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
FAILED:
ERROR: Build failed. Not running target
Traceback (most recent call last):
  File "/home/build/repos/jax/build/build.py", line 726, in <module>
    main()
  File "/home/build/repos/jax/build/build.py", line 692, in main
    shell(build_cpu_wheel_command)
  File "/home/build/repos/jax/build/build.py", line 45, in shell
    output = subprocess.check_output(cmd)
  File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/bazel', 'run', '--verbose_failures=true', '//jaxlib/tools:build_wheel', '--', '--output_path=/home/build/repos/jax/dist', '--jaxlib_git_hash=d3bfd32667d7742d2fc208b75d4085e923d5e03f', '--cpu=ppc64le', '--skip_gpu_kernels']' returned non-zero exit status 1.

Looks like the error is Error in fail: interpreter_path must be an absolute path.  Is that a Bazel problem, jax problem, or what?
Many thanks!

@bhuntsman I had a similar issue as you, and downgraded to Bazel 6.5.0. This resolved the issue of not finding the interpreter path, only to present this error:
(sum24r) python build/build.py --enable_cuda --cuda_path /nobackup/users/dsuarez/miniconda3/envs/sum24r/pkgs/cuda-toolkit --cuda_version=12.2 --cudnn_v
ersion=8.9.6 --cudnn_path /nobackup/users/dsuarez/miniconda3/envs/sum24r --noenable_mkl_dnn --cuda_compute_capabilities 7.0 --bazel_path /nobackup/user
s/dsuarez/miniconda3/envs/sum24r/bin/bazel --target_cpu=ppc --python_version=3.11
_   _  __  __
| | / \ \ / /
_  | |/ _ \ \  /
| || / ___ /
___//   /_/_
Bazel binary path: /nobackup/users/dsuarez/miniconda3/envs/sum24r/bin/bazel
Bazel version: 6.5.0
Use clang: no
MKL-DNN enabled: no
Target CPU: ppc64le
Target CPU features: release
CUDA enabled: yes
CUDA toolkit path: /nobackup/users/dsuarez/miniconda3/envs/sum24r/pkgs/cuda-toolkit
CUDNN library path: /nobackup/users/dsuarez/miniconda3/envs/sum24r
CUDA compute capabilities: 7.0
CUDA version: 12.2
CUDNN version: 8.9.6
NCCL enabled: yes
ROCm enabled: no
Building XLA and installing it in the jaxlib source tree...
/nobackup/users/dsuarez/miniconda3/envs/sum24r/bin/bazel run --verbose_failures=true //jaxlib/tools:build_wheel -- --output_path=/nobackup/users/dsuarez/jax/dist --jaxlib_git_hash=24b42eed5ea4163ac5a3d4a8f8648545572933db --cpu=ppc64le
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'run' from /nobackup/users/dsuarez/jax/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'run' from /nobackup/users/dsuarez/jax/.bazelrc:
Inherited 'build' options: --nocheck_visibility --apple_platform_type=macos --macos_minimum_os=10.14 --announce_rc --define open_source_build=true --spawn_strategy=standalone --enable_platform_specific_config --experimental_cc_shared_library --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true --define=tsl_link_protobuf=true -c opt --config=short_logs --copt=-DMLIR_PYTHON_PACKAGE_PREFIX=jaxlib.mlir. --@xla//xla/python:enable_gpu=false
INFO: Reading rc options for 'run' from /nobackup/users/dsuarez/jax/.jax_configure.bazelrc:
Inherited 'build' options: --strategy=Genrule=standalone --action_env CUDA_TOOLKIT_PATH=/nobackup/users/dsuarez/miniconda3/envs/sum24r/pkgs/cuda-toolkit --action_env CUDNN_INSTALL_PATH=/nobackup/users/dsuarez/miniconda3/envs/sum24r --action_env TF_CUDA_PATHS=/nobackup/users/dsuarez/miniconda3/envs/sum24r/pkgs/cuda-toolkit,/nobackup/users/dsuarez/miniconda3/envs/sum24r --action_env TF_CUDA_VERSION=12.2 --action_env TF_CUDNN_VERSION=8.9.6 --cpu=ppc --config=cuda --repo_env HERMETIC_PYTHON_VERSION=3.11
INFO: Found applicable config definition build:short_logs in file /nobackup/users/dsuarez/jax/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:cuda in file /nobackup/users/dsuarez/jax/.bazelrc: --repo_env TF_NEED_CUDA=1 --repo_env TF_NCCL_USE_STUB=1 --action_env TF_CUDA_COMPUTE_CAPABILITIES=sm_50,sm_60,sm_70,sm_80,compute_90 --crosstool_top=@local_config_cuda//crosstool:toolchain --@local_config_cuda//:enable_cuda --@xla//xla/python:enable_gpu=true --@xla//xla/python:jax_cuda_pip_rpaths=true --define=xla_python_enable_gpu=true --linkopt=-Wl,--disable-new-dtags
INFO: Found applicable config definition build:cuda in file /nobackup/users/dsuarez/jax/.jax_configure.bazelrc: --action_env TF_CUDA_COMPUTE_CAPABILITIES=7.0
INFO: Found applicable config definition build:linux in file /nobackup/users/dsuarez/jax/.bazelrc: --config=posix --copt=-Wno-unknown-warning-option --copt=-Wno-stringop-truncation --copt=-Wno-array-parameter
INFO: Found applicable config definition build:posix in file /nobackup/users/dsuarez/jax/.bazelrc: --copt=-fvisibility=hidden --copt=-Wno-sign-compare --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
Loading:
Loading:
Loading:
DEBUG: /home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/xla/third_party/py/python_repo.bzl:110:10: Using hermetic Python 3.11
Loading:
Loading:
DEBUG: /home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/xla/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'llvm-raw' because it already exists.
Loading:
Loading:
DEBUG: /home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/tsl/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'nvtx_archive' because it already exists.
DEBUG: /home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/xla/third_party/repo.bzl:132:14:
Warning: skipping import of repository 'jsoncpp_git' because it already exists.
Loading:
Loading:
Loading:
Loading:
Loading:
Loading:
Loading: 2 packages loaded
Analyzing: target //jaxlib/tools:build_wheel (3 packages loaded, 0 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (36 packages loaded, 10 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (36 packages loaded, 10 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (36 packages loaded, 10 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (40 packages loaded, 211 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (52 packages loaded, 216 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (53 packages loaded, 216 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (53 packages loaded, 216 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (55 packages loaded, 216 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (84 packages loaded, 3140 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (98 packages loaded, 3333 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (113 packages loaded, 3375 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (134 packages loaded, 4022 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (137 packages loaded, 4467 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (144 packages loaded, 4862 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (151 packages loaded, 4948 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (154 packages loaded, 5182 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (182 packages loaded, 7473 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (196 packages loaded, 8839 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (201 packages loaded, 12364 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (221 packages loaded, 13560 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (225 packages loaded, 14932 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (225 packages loaded, 16505 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (226 packages loaded, 16827 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (227 packages loaded, 17184 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (228 packages loaded, 17496 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (230 packages loaded, 17592 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (231 packages loaded, 19698 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (231 packages loaded, 19698 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (234 packages loaded, 20571 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (234 packages loaded, 20571 targets configured)
Analyzing: target //jaxlib/tools:build_wheel (236 packages loaded, 22264 targets configured)
INFO: Repository pypi_numpy instantiated at:
/nobackup/users/dsuarez/jax/WORKSPACE:29:13: in
/home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/pypi/requirements.bzl:49:20: in install_deps
Repository rule whl_library defined at:
/home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/rules_python/python/pip_install/pip_repository.bzl:697:30: in
ERROR: An error occurred during the fetch of repository 'pypi_numpy':
Traceback (most recent call last):
File "/home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/rules_python/python/pip_install/pip_repository.bzl", line 596, column 13, in _whl_library_impl
fail("whl_library %s failed: %s (%s) error code: '%s'" % (rctx.attr.name, result.stdout, result.stderr, result.return_code))
Error in fail: whl_library pypi_numpy failed: Collecting numpy==2.0.0 (from -r /tmp/tmp0sft14zz (line 1))
Downloading numpy-2.0.0.tar.gz (18.3 MB)
ssError: Command '['/home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/python_ppc64le-unknown-linux-gnu/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '-r', '/tmp/tmp0sft14zz']' returned non-zero exit status 1.
) error code: '1'
ERROR: /nobackup/users/dsuarez/jax/WORKSPACE:29:13: fetching whl_library rule //external:pypi_numpy: Traceback (most recent call last):
File "/home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/rules_python/python/pip_install/pip_repository.bzl", line 596, column 13, in _whl_library_impl
fail("whl_library %s failed: %s (%s) error code: '%s'" % (rctx.attr.name, result.stdout, result.stderr, result.return_code))
Error in fail: whl_library pypi_numpy failed: Collecting numpy==2.0.0 (from -r /tmp/tmp0sft14zz (line 1))
Downloading numpy-2.0.0.tar.gz (18.3 MB)
gs,
subprocess.CalledProcessError: Command '['/home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/python_ppc64le-unknown-linux-gnu/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '-r', '/tmp/tmp0sft14zz']' returned non-zero exit status 1.
) error code: '1'
Analyzing: target //jaxlib/tools:build_wheel (236 packages loaded, 22264 targets configured)
ERROR: /home/dsuarez/.cache/bazel/_bazel_dsuarez/eb751057d1cbe464324d72cbeb188b2a/external/xla/third_party/py/numpy/BUILD:5:6: @xla//third_party/py/numpy:numpy depends on @pypi_numpy//:pkg in repository @pypi_numpy which failed to fetch. no such package '@pypi_numpy//': whl_library pypi_numpy failed: Collecting numpy==2.0.0 (from -r /tmp/tmp0sft14zz (line 1))
Downloading numpy-2.0.0.tar.gz (18.3 MB)
olated', 'wheel', '--no-deps', '-r', '/tmp/tmp0sft14zz']' returned non-zero exit status 1.
) error code: '1'
ERROR: Analysis of target '//jaxlib/tools:build_wheel' failed; build aborted:
INFO: Elapsed time: 147.692s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (236 packages loaded, 22264 targets configured)
ERROR: Build failed. Not running target
Traceback (most recent call last):
File "/nobackup/users/dsuarez/jax/build/build.py", line 749, in
main()
File "/nobackup/users/dsuarez/jax/build/build.py", line 700, in main
shell(build_cpu_wheel_command)
File "/nobackup/users/dsuarez/jax/build/build.py", line 45, in shell
output = subprocess.check_output(cmd)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nobackup/users/dsuarez/miniconda3/envs/sum24r/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nobackup/users/dsuarez/miniconda3/envs/sum24r/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/nobackup/users/dsuarez/miniconda3/envs/sum24r/bin/bazel', 'run', '--verbose_failures=true', '//jaxlib/tools:build_wheel', '--', '--output_path=/nobackup/users/dsuarez/jax/dist', '--jaxlib_git_hash=24b42eed5ea4163ac5a3d4a8f8648545572933db', '--cpu=ppc64le']' r
eturned non-zero exit status 1.
Anyone know what is happening here?

@dsuarez01 I don't know that you want to spend a ton of time down that rabbit hole.  The latest Bazel ought to be able to build jax, else it's likely a Bazel bug.  That being said, two questions:

Can you tell me what you did to get bazel 6.5.0 to compile?  I got it bootstrapped but when I try to build it natively using the bootstrapped bazel, it fails with java errors and coredumps.

What version of Python do you have installed?  Looks like jax just moved the minimum requirement to 3.10 two days ago.

We are getting somewhere.  Using Bazel 6.5.0 seems to be the key, and that's also what's indicated in the .bazelversion.
The next problem I'm running into is that xla has a dependency on boringssl, which explicitly doesn't support ppc64le.  It did in the past but a review of the commit history shows that the developers stripped out all the ppc64le code prior to the bazel integrations, which makes putting it back in rather challenging.

HEY GUYS!  I got jax 0.4.25 to compile on ppc64le!
Here's what I did:

Ubuntu 20.04 on p8.
Install CUDA 11.8 using the runfile installer from here: https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Linux&target_arch=ppc64le&Distribution=RHEL&target_version=8&target_type=runfile_local
Install python 3.10 from the deadsnakes ppa repository
Create a python venv
Clone the jax repo and check out the v0.4.25 tag
Get the dist release of bazel 6.2.0 and bootstrap it
Clone the bazel repo and check out the 6.1.2 tag
Use the built bazel 6.2.0 to build the 6.1.2 distfile:

../bazel-6.2.0-dist/output/bazel build //:bazel-distfile --tool_java_runtime_version=local_jdk

Unzip the built distfile located in bazel-bin/bazel-distfile.zip
Bootstrap build bazel 6.1.2 and put the result in your /usr/local/bin
Clone my modified xla: https://github.com/bhuntsman/xla.git

The only change here is to "manually cherry-pick" commit 574a844a6172b73b03ad2d2cf89d73389fea4522, which adds ppc64le support to tsl and implib_so

Clone my modified boringssl: https://github.com/bhuntsman/boringssl.git

The changes here are to revert all the commits that removed ppc64le support, and then to regenerate all the bazel build files.

cd into your jax directory and build as follows:

TMP=/tmp python build/build.py --noenable_mkl_dnn --enable_cuda --build_gpu_plugin --gpu_plugin_cuda_version=11 --bazel_options=--override_repository=boringssl=/home/build/repos/boringssl --bazel_options=--override_repository=xla=/home/build/repos/xla

Now we have a few challenges to get this integrated.  Chiefest of which is convincing the boringssl project to bring back ppc64le support.  Originally I was looking at jax 0.4.25 as I thought perhaps it might support CUDA 11.1, but found that no it requires 11.8 as a minimum.  Updating this to the latest jax, newer xla, and newer CUDA should be possible and I'll work on it in the coming days.

Well, that's great! Certainly please upstream things.
I will note that NVIDIA has dropped ppc64 support as of CUDA 12.4, so at some point we're going to have to drop CUDA support on this architecture. We can't support it long term if NVIDIA does not.
For our own builds, we've stopped building or testing anything older than CUDA 12.1. So I'd encourage you to use CUDA 12.4.1, which seems to be the last release NVIDIA will make on that architecture, and that will keep you supported as long as possible by us.

Related question here for anyone who has access to a ppc64le system.  It looks like nvidia stopped creating installers for Ubuntu on ppc64le after CUDA 11.1, but continued to support RHEL 8 through CUDA 12.4.  However, the runfile installer doesn't specifically indicate that it's only for RHEL, and in fact installs on Ubuntu just fine.  Once installed, the kernel modules load fine and nvidia-smi indicates that it can see the cards.  nvcc works as well.  Unfortunately the resulting binaries do nothing.  I had to symlink /usr/lib/powerpc64le-linux-gnu/libmpfr.so.4 -> libmpfr.so.6.0.2 to get cuda-gdb to run, but after loading it up and running a program, I get the following as soon as it tries to hit the GPU:
fatal: CUDBG_ERROR_INITIALIZATION_FAILURE (20): The CUDA driver initialization failed.
(cuda-gdb) info cuda devices
No CUDA devices.

I'd greatly prefer to stay on Ubuntu.  Has anyone had any luck at anything like that, or is the only recourse here to be forced onto RHEL in order to have a more up to date CUDA?
Thanks!

@bhuntsman I tried to follow your guidance to install jaxlib 0.4.25 on my power9 cpu. However, I got an error as follows:
Compiling external/tsl/tsl/cuda/cuda.tramp.S failed: (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command (from target @tsl//tsl/cuda:cuda)
currently, I am using the cuda:11.5.1 and the cudnn 8.8.1.
Do you have any idea to help me overcome this error?

What OS, and also, are you trying to cross-compile?  That's not a piece of the code I had to modify earlier to get it to compile on ppc.  Maybe something else got changed later?
Also looks like you're using an older cuda than I did... maybe try the newer one?
Sorry I can't be of more assistance.  Maybe with a few more details I could recreate the problem and investigate.
Good luck!!


UrlResolutionTagHelper shoul have an asp-absolute-url attribute

What I'd like to see is as follows.
If I have a tag in razor templates like , on a web app that IIS is serving from http://whatever.domain/someroot I would like the generated link to be  instead of ```.
Basically, code changes would be around here:
https://github.com/aspnet/Mvc/blob/a67d9363e22be8ef63a1a62539991e1da3a6e30e/src/Microsoft.AspNetCore.Mvc.Razor/TagHelpers/UrlResolutionTagHelper.cs#L147
Why do I want this? Well I'm using Razor pages to write a custom login page Azure B2C. The way B2C works is it takes my custom login URL, reads that html as an Ajax call, and then injects it, with some customization into the login page served at https://login.microsoftonline.com Therefore relative URLs for the img and css don't work. Since I have a dev and production tenant, and want to be able to test out changes to the login page, I don't want to hard code the img and css urls.

Thanks for contacting us, @zippy1981.
This looks like an issue Azure Azure B2C rather than your service as the true fix here seems for the login.microsoftonline.com to respect the origin and prefix the relative paths with the domain.
As a mitigation to that issue, I do agree that the proposal would work. @danroth27, I'm not sure about the value of this type of feature. Thoughts?

@pranavkm, what will be the easiest way to generate an absolute URL in this case? Sounds like it's possible today.

MVC does not have an API that produces absolute urls for content. UrlHelper.Link is the closest bet, but it works specifically for routes. I'd go with @mkArtakMSFT's suggestion and encourage you to file an issue with the B2C folks to see if they could do something about supporting relative paths.
In the meanwhile, I'd recommend looking at https://github.com/aspnet/Mvc/blob/release/2.2/src/Microsoft.AspNetCore.Mvc.Core/Routing/UrlHelperBase.cs#L184-L191 for the code that MVC uses to produce absolute urls. I don't think this is a feature that we would baked in to the system.

I'm really not understanding why you'd be so hesitant to add this feature. There are times besides B2C when absolute URLs are necessary. It wouldn't be terribly hard to implement, or write the tests for.
I've worked around this in my application by writing my own tag helpers.  There might be something I don't know about B2C to let me solve this more elegantly. Or it might simply be done this way for security reasons.
Still, ASP.NET knows the full URL. sometimes I need to put it in as the full url. Why not at least be open to a pull request for this?

@zippy1981 we re-evaluated the decision and are willing to take this change. I've marked the issue as up-for-grabs and we'd be happy to accept a PR for this.


not able to Add sensors

Alarmo Version
v1.6.2
HA Version
2022.6
Bug description
When trying to add sensors, an error dialogue appears, saying;
Something went wrong!
Message format incorrect: extra keys not allowed @ data['name']
Response error: 400
Please report the bug.
Steps to reproduce
No response
Relevant log output
No response

Update Alarmo to 1.9.4 please.
Clear your browser cache if the version number shown in the Alarmo panel is outdated.

Thanks, it worked


Log spam while running ZooLockIT

Describe the bug
While running ZooLockIT, there is a debug log that gets logged several hundred times per second. This issue was uncovered while working on #1952. Initial discussion suggested this may be a problem rather than a case of incorrect log type(debug vs. trace). Example of the log:
2021-03-03T10:45:13,618 [zookeeper.ZooLock] DEBUG: [zlock#3db3b47d-4fc2-4b38-83cd-c733880121bf#] Renewing watch on prior node  /zltestDeleteWaiting-587915658-l1/zlock#947ce341-f843-4742-8b28-79f2de9cc7da#0000000000
2021-03-03T10:45:13,618 [zookeeper.ZooLock] DEBUG: [zlock#3db3b47d-4fc2-4b38-83cd-c733880121bf#] Renewing watch on prior node  /zltestDeleteWaiting-587915658-l1/zlock#947ce341-f843-4742-8b28-79f2de9cc7da#0000000000
2021-03-03T10:45:13,619 [zookeeper.ZooLock] DEBUG: [zlock#3db3b47d-4fc2-4b38-83cd-c733880121bf#] Renewing watch on prior node  /zltestDeleteWaiting-587915658-l1/zlock#947ce341-f843-4742-8b28-79f2de9cc7da#0000000000
2021-03-03T10:45:13,619 [zookeeper.ZooLock] DEBUG: [zlock#3db3b47d-4fc2-4b38-83cd-c733880121bf#] Renewing watch on prior node  /zltestDeleteWaiting-587915658-l1/zlock#947ce341-f843-4742-8b28-79f2de9cc7da#0000000000

To Reproduce
This can be reproduced by running ZooLockIT and viewing the logs.

The Renewing watch on prior node messages from ZooLockIT all have the path  prefix /zltestDeleteWaiting which is from the testDeleteWaiting test method. Adding @Ignore to this test suppresses these messages. Only running this test does not produce these messages. My assumption here is that the watchers being set in the testDeleteWaiting method are being triggered by the other tests.

This issue is not reproducible on every test run. It seems to be happening randomly.

So, I think I understand what's happening here in the test method testDeleteWaiting. 3 ZooLocks are created and then the first two ephemeral nodes are delete from ZK while the method continues to hold the live ZooLock objects. In a deployed system, a TabletServer would have a ZooLock and if the path was deleted (by the Master), then the TabletServer would halt. This test is somewhat similar to what would happen on a half-dead TabletServer.

@dlmarion Should we just ignore these logs from this test, then and take no action, or throttle the log message, or something else?

I was talking with @EdColeman - I'm going to look once more at the code where it renews the watcher to make sure it's not renewing on a node that does not exist.

So, the code currently logs a message, adds the watcher to the node, then if the node does not exist it removes the watcher. I'm moving the log message to the case where the node does exist. PR up soon.


feat: convert .sasnb to .flw

Summary
Implementation of a solution to convert .sasnb files to .flw (SAS Studio Flows)
The flow is then associated with SAS Studio, it appears in SAS Studio and can be executed and modified in SAS Viya GUI.
resolves #446
Testing
I have tested this change with two examples of .sasnb files: the first with one cell and the second with 3 cells (python, sas, sql)

Thank you for your comments and suggestions to enhance the code.
Studio Flows are only available for recent versions of SAS Viya, I would like to restrict this feature to newer versions of SAS Viya only. How would you suggest to proceed ?
I see there is a private API Call to get the ViyaCadence in ContentNavigator/ContentModel.ts and I would like to use that output in ContentNavigator/utils.ts ?

Thank you for your comments and suggestions to enhance the code.
Studio Flows are only available for recent versions of SAS Viya, I would like to restrict this feature to newer versions of SAS Viya only. How would you suggest to proceed ?
I see there is a private API Call to get the ViyaCadence in ContentNavigator/ContentModel.ts and I would like to use that output in ContentNavigator/utils.ts ?

My opinion is that we should use viyaCadence as minimally as possible. What happens if you try to create a flow for an instance that doesn't support it? I'm wondering if it returns an error response we could interpret and respond accordingly?
@scnwwu may have other thoughts on this

Thank you for your comments and suggestions to enhance the code.
Studio Flows are only available for recent versions of SAS Viya, I would like to restrict this feature to newer versions of SAS Viya only. How would you suggest to proceed ?
I see there is a private API Call to get the ViyaCadence in ContentNavigator/ContentModel.ts and I would like to use that output in ContentNavigator/utils.ts ?

My opinion is that we should use viyaCadence as minimally as possible. What happens if you try to create a flow for an instance that doesn't support it? I'm wondering if it returns an error response we could interpret and respond accordingly?
@scnwwu may have other thoughts on this

I guess it depends on if we can tell from the error that whether it's not supported at all or just temporarily not available.

Concerning the behavior for older versions of SAS Viya (tested on 3.5), we have an error when trying to create a new SAS Studio session: Request failed with status code 404
Right now, the user sees an error message Error converting the .sasnb file to .flw format.but a .flw file is still created. (but cannot be opened of course)
I also see an error fail to retrieve the viya cadence  in the logs when using SAS Viya 3.5

Concerning the behavior for older versions of SAS Viya (tested on 3.5), we have an error when trying to create a new SAS Studio session: Request failed with status code 404 Right now, the user sees an error message Error converting the .sasnb file to .flw format.but a .flw file is still created. (but cannot be opened of course) I also see an error fail to retrieve the viya cadence in the logs when using SAS Viya 3.5

It sounds like we could try create Studio session at the start of the conversion, and if any error, show user something like "SAS Studio service not available."?

I have added translations to German and Chinese using Google Translate.. This can be enhanced later by fluent speakers.

Looks good to me (including the Chinese translation:smile:).
Will let @scottdover finally vet and approve. Thanks!

Just a heads-up. I updated this branch to clear up the translation check failures. Nothing else should've changed here :)

Considering to generate the Flow nodes and the node name associates with the language which the user selects through VSC 'Configure Display Language'

A separate issue has been raised to track the feature updates.  Close this one to go to next stage.

@sahbic, please resolve conflicts then we're going to merge it. Thank you!

Disclaimer: While you can inspect the SAS Studio Flow JSON format (for example, by opening a .flw file in an editor), it is not a publicly documented or publicly supported format/schema. It is still internal/proprietary. We strive to keep the Flow format backwards compatible, but there is a possibility that future changes to the internal model could break this “convert SAS Notebook to SAS Studio Flow” feature in VS Code, requiring additional changes to the SAS extension to fix this feature. In other words, when you are using the Flow format/schema directly (as in this feature), you are using it as-is and at your own risk.


No loader inside a modal?

Not sure if it's a SUI core or SUI react issue...
Testcase
codepen.io

Ah, appears to be a CSS issue.  I would open an issue on SUI core, linking to here.
A Loader in a Modal is a child of a Dimmer (since the Modal has a Dimmer).  So, the Loader is white as the CSS expects it to be displayed directly on the Dimmer (black), not on something white (Modal) inside of the Dimmer.
If you wrap the Loader in an inverted Segment, this is clear:
// ...in a Modal
<Segment basic inverted>
  <Loader active inline='centered' />
</Segment>

Workarounds
Either removing the Dimmer, or using an inverted (white) Dimmer will allow the Loader to be visible.
No Dimmer
<Modal dimmer={false} />

Inverted Dimmer (white)
<Modal dimmer='inverted' />

Basic Modal
You could also use a <Modal basic /> which has no background:


MSDKUI-1900: [Android] Guidance/RP - Route overview maneuver list distances alignment off

Fixed the alignment by making it full width

Codecov Report

Merging #110 into master will increase coverage by 5.27%.
The diff coverage is 80%.

@@             Coverage Diff             @@
##             master    #110      +/-   ##
===========================================
+ Coverage     69.43%   74.7%   +5.27%
- Complexity      646     995     +349
===========================================
  Files            60      69       +9
  Lines          1724    3941    +2217
  Branches        435     438       +3
===========================================
+ Hits           1197    2944    +1747
- Misses          106     749     +643
+ Partials        421     248     -173

Impacted Files
Coverage Δ
Complexity Δ

...a/com/here/msdkui/routing/ManeuverListAdapter.java
95.23% <100%> (ø)
7 <1> (?)

...ain/java/com/here/msdkui/routing/ManeuverList.java
96.29% <50%> (ø)
10 <0> (?)

...e/msdkuiapp/routing/RoutingInstructionsFragment.kt

...java/com/here/msdkuiapp/common/PermissionsUtils.kt

.../com/here/msdkuiapp/routing/OptionPanelFragment.kt

...ere/msdkuiapp/routing/RouteManeuverListFragment.kt

...msdkuiapp/routing/RouteDescriptionListPresenter.kt

...m/here/msdkuiapp/routing/SubOptionPanelFragment.kt

.../com/here/msdkuiapp/base/BasePermissionActivity.kt

...om/here/msdkuiapp/position/LocationServiceUtils.kt

... and 121 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a850f62...5d93e90. Read the comment docs.


⚠ PrivateHD has degraded performance

In d755453, PrivateHD ($PHD) experienced degraded performance:

HTTP code: 200
Response time: 2895 ms

Resolved: PrivateHD performance has improved in 00ad9d3.


Fix formatting issue, swap string and int.

Hi @boumenot, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;


Added WP Kickstart to Build Tools

https://produkte.webtimal.ch/wp-kickstart
The script is easy to use, properly documented and actively maintained. As there is no comparable tool, it fills a niche for web developers working regularly with WordPress.

Abandoned it seems since 5 years


Microphone and loudspeaker control

Hello, it is not possible through a HomeKit app unfortunately the speaker and the microphone does not control over the app. A developer of a HomeKit app told me:
The developer of the Homebridge plugin for Fosam is to implement the services of type HMSERVICETYPESPEAKER and HMSERVICETYPEMICROPHONE, each with the characteristic HMCHARACTERISTICTYPEMUTE.
Do you understand what I say? My English is unfortunately not good, sorry.

I don't think it's possible to control the volume of the camera's microphone and speaker. Therefore, those services are not created.

I am not sure if I understand you correctly. If you are talking about the slider added after iOS 10. Mine works without a problem. The slider control the volume output on the phone. The mute/unmute toggle works as well. It's used to mute/unmute the phone's microphone, not the camera's.

That's right, in the app "Home" it works with the volume and the microphone. But I can not control the volume and the microphone in third-party apps.

plugin works with Aplle's Home but not with 3rd Party apps which rely on HMSERVICETYPESPEAKER and HMSERVICETYPEMICROPHONE.
These services should be implemented with at least HMCHARACTERISTICTYPEMUTE

Which 3rd party app are you using?

I tested it with myHome „myHome - Heim Automatisierung“ von Joerg Wissemann
https://appsto.re/de/pmoC_.i
(I'm the developer of the app)

for example "MyHome" https://itunes.apple.com/de/app/myhome-heim-automatisierung/id1064362767?mt=8

Yes, please for other apps intrigue. It only works with Apple's iOS Home App, not with third-party apps

Hey, can one still expect an implementation? Would be really great

@luisiam
When will this be implemented?


Unable to highlight text/code using a pen

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.62.0
OS Version: Windows 10 21H1

Steps to Reproduce:

Try to select any code with your pen while pushing the corresponding key on your pen
Note that only the right click menu pops up, but no text is highlighted

Let's track in https://github.com/microsoft/vscode/issues/47770


Open issues

This issue has been extracted from the issue list on:https://ies-svn.jrc.ec.europa.eu/issues/2685
Comment
Testing should only cover a test if the required metadata elements are correctly provided. This can be done automatically. Further test methods aiming the feasibility of the thematic content seem to be too advanced.
Proposed Change
Keep the testing cases on the level of correctly given elements.

Proposed resolution:

Update based on the proposed change.

Resolution depends on resolution of #27

Resolution during call on April 28 - see #27:

Implement the proposed change


Dns benchmark - Fixing dns benchmark path

/assign @wojtek-t

/lgtm
/approve

@krzysied - please fix, tests are failing

@wojtek-t The problem is it verifies that labels used by perfdash. I think we don't want to disable this check, so I created #332 to add dns config to perfdash.


gen_prime: only test odd values

Codecov Report

Merging #11 (8a39613) into main (a8aee2f) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #11   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            5         5
  Lines          229       233    +4
  Branches        72        74    +2
=========================================
+ Hits           229       233    +4

Flag
Coverage Δ

unittests
100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/pseudoprimes/pseudoprimes.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a8aee2f...8a39613. Read the comment docs.


Add Rancher CA integration

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Problem:
On the HCI mode, the rancher is exposed via the nodePort but the certificate generated by the dynamic listener is neither authorized nor trusted CA, will need to address by using private CA.
Solution:
Use the Harvester serving-ca as a custom private CA and the CA of tls-rancher-internal-ca, therefore the Rancher server accessed by NodePort on the HCI mode will be validated for x509 authorization.
Related Issue:
Related PR: https://github.com/rancher/harvester-installer/pull/64
Test plan:
validate if the user can either import a cluster or create a node using a custom node driver successfully.

updated @gitlawr


Install via https

It would be nice to revert https://github.com/sbt/website/commit/801b238f84b32764a2d0933937bdce25d6e9b434
We should investigate what the problem is and either fix the instructions or report the issue to bintray if that's where the problem is

I would suggest to sign the package and upload the key to the same server.
Then, using https link should be safe.


adds no documentation

In order to give users no more information about not using this software this adds no documentation.

I really like the project. Truly open source.


Add Options

Would act as an IOptions<T> wrapper for some T. Would make a convenient way to create an IOptions<T> from any T. We could also have a factory method (/cc @divega) that makes one of these from a T i.e. IOptions<T> Options.Create<T>(T options).

Would this would be instead of the old pattern of MyOptions implementing IOptions<MyOptions>?

Yep! It's something @PinpointTownes suggested https://github.com/aspnet/Options/issues/105#issuecomment-168866368.

I updated the sample in B to show the usage.

Ok yeah that looks much better, it was pretty gross making everything implement IOptions before...  Maybe we should name the generic Options class OptionsWrapper so people don't think its a base class...which is what the name implies...

PR out for review.

Do we want to consider an overload of Options.Create<T>(Action<T> configure) just as pure sugar?

Hmmmmmmmm

app.UseFoo(Options.Create(options =>
{
    options.Value = 10;
});

var options = Options.Create(opt =>
{
  opt.Value = 10;
});

app.UseFoo(options);

Yeah well this way you give people the option to do either one, so everyone is happy


Naming of MysqlCluster master Service

Hey,
We are using this Operator in one of our use-cases [1]. In this use-case we have a Custom Resource for Moodle and it uses the mysql cluster created using MysqlCluster Custom Resource provided by this Operator. In this setup we bind the Moodle CR with the mysql cluster created by MysqlCluster CR through its name. The naming format is "name-of-the-MysqlCluster-CR" with  the string "-mysql-master" appended to it. You can see this in the Moodle YAML [2].
I was wondering if it will be possible to update the Operator to create the underlying master Service with only the name of the MysqlCluster Custom Resource, without "-mysql-master". The reason I ask is, we want to be able to specify only the name of MysqlCluster Custom Resource as the binding target in our Moodle Custom Resource spec. This is similar to how MysqlCluster Custom Resource currently takes just the name of Secret as one of its inputs.
By providing just the name as binding target, it becomes easier to handle creation of multiple platform stacks for our Moodle Operator.
[1] https://github.com/cloud-ark/kubeplus/tree/master/examples/moodle-with-presslabs
[2] https://github.com/cloud-ark/kubeplus/blob/master/examples/moodle-with-presslabs/namespace1/moodle1.yaml

Hi @devdattakulkarni, thanks for suggestions! I don't think we can rename it because it's too alike to have name collisions. But from version 0.3.0 the service name will be <cluster-name>-mysql, the same for writes and for reads, because of using ProxySQL(#10).
What I recommend is to use the credentials secret as a link between the app and the MySQL Cluster. In that secret, fill the DB_CONNECT_URL to contains all connection details, as we do in the cluster chart.
Also, we would like to add a new field on the cluster that lets you customize this service(e.g. name, type, port..), so you can create the service with the cluster name.
Hope this helps!


Usability of Tuple constructors

Assume a Tuple of different type
using Aggregate = Tuple<A, B, C>;

The generated Tuple offers 2 relevant kind of constructors:
 b) Aggregate(C{}, B{}, A{}) // Same number of arguments, but in arbitrary order

 a) Aggregate(Tuple{B{}, A{}}) // construct from lesser Tuple, also arbitrary order

a) makes it very difficult to read from the compiler errors, because it has a complicated SFINAEd signature that doesn't allow the compiler to display a simple type conversion error.
MSVC doesn't even bother to list the failed candidates, and just gives the dreaded "Can't convert initializer list..." error.
The current resolution to this case would be Aggregate{Tuple{A, B}}, leaning on Tuple's type deduction guide, to first construct an appropriate "lesser tuple".
I also argue that reordering argument reordering, and argument omission are convenience features intended for test case construction, and belong together.
Also, only one of these conveniences (the re-ordering) is duplicated in both implementations,
and in both cases obstruct proper constructor argument related error messages.
Related to #9 and #12

@strfry I agreed that this is an issue, we should solve.
Maybe I missed it - but do you have an idea how to improve the situation? - remove a constructor? convert them to a static factory function?


SET insert union

when i try to insert a struct with a matching union value, or a union with just the one value it says it can't implicit cast, if i try to explicit cast to the union it says can't convert union to union. i don't see an actual example, how do i insert a union property without using copy from?

Hi,
We currently don't support insert an union value to a node/rel table, but we are planning to support it soon.


Feature Request: clean dust bin // Staubbehälter reinigen Notification

Notification or Sensor to Clean the dust bin like in the App
Benachrichtigung bzw Sensor zum entleeren des Staubbehälters aus der App in Homekit integrieren

Sounds good, I would also like to.
I think it is not possible because the miio api does not transmit this data.
@afharo
An occupancy sensor, if "dustbin = true" in config.json, detected if ... no data over miio, no care sensor for dust bin... 👎
Do you have any idea? Have you ever noticed anything?

I need to test it but I think it’s emitted as an error from the miio library. At the moment, we only log it as a warning.
We could set a lock based on that. And the user will need to unlock it manually when it’s fixed (because there’s no way to know it’s clean now).
Maybe even reuse the same lock to rename it based on any warning we emit? 🤔

Bumping this issue. I think that, after the latest refactor, it may be pretty easy to implement.
cc @luc-ass as he recently requested something like that

It should be solved in https://github.com/homebridge-xiaomi-roborock-vacuum/homebridge-xiaomi-roborock-vacuum/releases/tag/v0.26.0


Upgrade quarkus to v0.20.0  plsu some cleanup

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/58/


Log httpd log_error to stderr

From https://github.com/openshift/sti-perl/pull/73#issuecomment-191477670
Instead of sending stderr to a file, send it do stderr like we do in other images.
Code: 5.5/contrib/etc/httpdconf.sed#L7
See the patch applied to Perl, it was very good, including tests.

very good except it broke on docker 1.8 :)  but we have a new/better patch now.