DL4J Hanging after "Loaded [JCublasBackend] backend"

Hi,
We are running some DL4J code as part of a wider system.
This code runs fine on an Alienware development PC with CUDA 9.1 on Ubuntu, run from Eclipse.
However, when we package this application and run it on a RHEL ppc64le server with CUDA 9.1, we see that ND4J is not doing anything after the following output:
2309 [pool-8-thread-1] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
I have verified we are running the latest NVIDIA drivers and CUDA 9.1 is installed successfully.
Below is the output from running the CUDA 9.1 sample deviceQuery, which lists the GPU devices:
 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 4 CUDA Capable device(s)

Device 0: "Tesla P100-SXM2-16GB"
  CUDA Driver Version / Runtime Version          9.1 / 9.1
  CUDA Capability Major/Minor version number:    6.0
  Total amount of global memory:                 16276 MBytes<PHONE_NUMBER>0 bytes)
  (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
  GPU Max Clock rate:                            1481 MHz (1.48 GHz)
  Memory Clock rate:                             715 Mhz
  Memory Bus Width:                              4096-bit
  L2 Cache Size:                                 4194304 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
  Maximum memory pitch:<PHONE_NUMBER> bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Enabled
  Device supports Unified Addressing (UVA):      Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   2 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 1: "Tesla P100-SXM2-16GB"
  CUDA Driver Version / Runtime Version          9.1 / 9.1
  CUDA Capability Major/Minor version number:    6.0
  Total amount of global memory:                 16276 MBytes<PHONE_NUMBER>0 bytes)
  (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
  GPU Max Clock rate:                            1481 MHz (1.48 GHz)
  Memory Clock rate:                             715 Mhz
  Memory Bus Width:                              4096-bit
  L2 Cache Size:                                 4194304 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
  Maximum memory pitch:<PHONE_NUMBER> bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Enabled
  Device supports Unified Addressing (UVA):      Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   3 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 2: "Tesla P100-SXM2-16GB"
  CUDA Driver Version / Runtime Version          9.1 / 9.1
  CUDA Capability Major/Minor version number:    6.0
  Total amount of global memory:                 16276 MBytes<PHONE_NUMBER>0 bytes)
  (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
  GPU Max Clock rate:                            1481 MHz (1.48 GHz)
  Memory Clock rate:                             715 Mhz
  Memory Bus Width:                              4096-bit
  L2 Cache Size:                                 4194304 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
  Maximum memory pitch:<PHONE_NUMBER> bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Enabled
  Device supports Unified Addressing (UVA):      Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   6 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 3: "Tesla P100-SXM2-16GB"
  CUDA Driver Version / Runtime Version          9.1 / 9.1
  CUDA Capability Major/Minor version number:    6.0
  Total amount of global memory:                 16276 MBytes<PHONE_NUMBER>0 bytes)
  (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
  GPU Max Clock rate:                            1481 MHz (1.48 GHz)
  Memory Clock rate:                             715 Mhz
  Memory Bus Width:                              4096-bit
  L2 Cache Size:                                 4194304 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
  Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
  Maximum memory pitch:<PHONE_NUMBER> bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
  Run time limit on kernels:                     No
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Enabled
  Device supports Unified Addressing (UVA):      Yes
  Supports Cooperative Kernel Launch:            Yes
  Supports MultiDevice Co-op Kernel Launch:      Yes
  Device PCI Domain ID / Bus ID / location ID:   7 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
> Peer access from Tesla P100-SXM2-16GB (GPU0) -> Tesla P100-SXM2-16GB (GPU1) : Yes
> Peer access from Tesla P100-SXM2-16GB (GPU0) -> Tesla P100-SXM2-16GB (GPU2) : No
> Peer access from Tesla P100-SXM2-16GB (GPU0) -> Tesla P100-SXM2-16GB (GPU3) : No
> Peer access from Tesla P100-SXM2-16GB (GPU1) -> Tesla P100-SXM2-16GB (GPU0) : Yes
> Peer access from Tesla P100-SXM2-16GB (GPU1) -> Tesla P100-SXM2-16GB (GPU2) : No
> Peer access from Tesla P100-SXM2-16GB (GPU1) -> Tesla P100-SXM2-16GB (GPU3) : No
> Peer access from Tesla P100-SXM2-16GB (GPU2) -> Tesla P100-SXM2-16GB (GPU0) : No
> Peer access from Tesla P100-SXM2-16GB (GPU2) -> Tesla P100-SXM2-16GB (GPU1) : No
> Peer access from Tesla P100-SXM2-16GB (GPU2) -> Tesla P100-SXM2-16GB (GPU3) : Yes
> Peer access from Tesla P100-SXM2-16GB (GPU3) -> Tesla P100-SXM2-16GB (GPU0) : No
> Peer access from Tesla P100-SXM2-16GB (GPU3) -> Tesla P100-SXM2-16GB (GPU1) : No
> Peer access from Tesla P100-SXM2-16GB (GPU3) -> Tesla P100-SXM2-16GB (GPU2) : Yes

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 9.1, CUDA Runtime Version = 9.1, NumDevs = 4
Result = PASS

Can someone please help us with diagnosing this issue?
It seems CUDA is installed correctly but DL4J is not producing any output and the following Java code is just hanging when calling Nd4j.create() for the first time:
...
Nd4j.create()
...

Note that this same code works fine on the AlienWare on Ubuntu 64 bit.
Aha! Link: https://skymindai.aha.io/features/ND4J-143

Can you include the version and make sure you're using the latest? Cuda 9.1 isn't even supported by nvidia now.

Yeah, this looks a lot like the issue we were seeing with V100s a few releases back (before we had the CC 7.0 support in the build).
And given it's CUDA 9.1, this must be an old version of DL4J/ND4J.
@madhs53 Upgrade to 1.0.0-beta3 and CUDA 9.2 or 10.0 and try again. That will very likely fix your problem.

Thanks to you both, I'll try with CUDA 10.0 and let you know how it goes.

It looks like I'm getting the same issue, even with CUDA 10.0 and running DL4J 1.0.0-beta3.
Here's my POM:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>IA_DL_DL4J</groupId>
  <artifactId>IA_DL_DL4J</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <build>
    <sourceDirectory>src</sourceDirectory>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
       <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-shade-plugin</artifactId>
          <version>3.0.0</version>
          <configuration>
            <filters>
                <filter>
                    <artifact>*:*</artifact>
                    <excludes>
                        <exclude>META-INF/*.SF</exclude>
                        <exclude>META-INF/*.DSA</exclude>
                        <exclude>META-INF/*.RSA</exclude>
                    </excludes>
                </filter>
            </filters>
          </configuration>
          <executions>
            <execution>
              <phase>package</phase>
              <goals>
                <goal>shade</goal>
              </goals>
              <configuration>
                <transformers>
                    <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                        <resource>reference.conf</resource>
                    </transformer>
                    <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                </transformers>
              </configuration>
            </execution>
          </executions>
        </plugin>
    </plugins>
  </build>
    <dependencies>
          <dependency>
              <groupId>org.deeplearning4j</groupId>
              <artifactId>deeplearning4j-core</artifactId>
              <version>1.0.0-beta3</version>
          </dependency>
           <dependency>
              <groupId>org.deeplearning4j</groupId>
              <artifactId>deeplearning4j-nlp</artifactId>
              <version>1.0.0-beta3</version>
          </dependency>
          <dependency>
              <groupId>org.deeplearning4j</groupId>
              <artifactId>deeplearning4j-cuda-10.0</artifactId>
              <version>1.0.0-beta3</version>
          </dependency>
          <dependency>
             <groupId>org.nd4j</groupId>
             <artifactId>nd4j-cuda-10.0-platform</artifactId>
             <version>1.0.0-beta3</version>
          </dependency>
          <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.6.1</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-core</artifactId>
            <version>1.2.3</version>
        </dependency>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-ui_2.11</artifactId>
            <version>1.0.0-beta3</version>
        </dependency>
        <dependency>
            <groupId>edu.stanford.nlp</groupId>
            <artifactId>stanford-corenlp</artifactId>
            <version>3.9.1</version>
        </dependency>
    </dependencies>
</project>

Here's the output from CUDA deviceQuery on the command line:
CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 4 CUDA Capable device(s)

Device 0: "Tesla P100-SXM2-16GB"
 CUDA Driver Version / Runtime Version          10.0 / 10.0
 CUDA Capability Major/Minor version number:    6.0
 Total amount of global memory:                 16281 MBytes<PHONE_NUMBER>4 bytes)
 (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
 GPU Max Clock rate:                            1481 MHz (1.48 GHz)
 Memory Clock rate:                             715 Mhz
 Memory Bus Width:                              4096-bit
 L2 Cache Size:                                 4194304 bytes
 Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
 Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
 Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
 Total amount of constant memory:               65536 bytes
 Total amount of shared memory per block:       49152 bytes
 Total number of registers available per block: 65536
 Warp size:                                     32
 Maximum number of threads per multiprocessor:  2048
 Maximum number of threads per block:           1024
 Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
 Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
 Maximum memory pitch:<PHONE_NUMBER> bytes
 Texture alignment:                             512 bytes
 Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
 Run time limit on kernels:                     No
 Integrated GPU sharing Host Memory:            No
 Support host page-locked memory mapping:       Yes
 Alignment requirement for Surfaces:            Yes
 Device has ECC support:                        Enabled
 Device supports Unified Addressing (UVA):      Yes
 Device supports Compute Preemption:            Yes
 Supports Cooperative Kernel Launch:            Yes
 Supports MultiDevice Co-op Kernel Launch:      Yes
 Device PCI Domain ID / Bus ID / location ID:   2 / 1 / 0
 Compute Mode:
    < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 1: "Tesla P100-SXM2-16GB"
 CUDA Driver Version / Runtime Version          10.0 / 10.0
 CUDA Capability Major/Minor version number:    6.0
 Total amount of global memory:                 16281 MBytes<PHONE_NUMBER>4 bytes)
 (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
 GPU Max Clock rate:                            1481 MHz (1.48 GHz)
 Memory Clock rate:                             715 Mhz
 Memory Bus Width:                              4096-bit
 L2 Cache Size:                                 4194304 bytes
 Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
 Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
 Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
 Total amount of constant memory:               65536 bytes
 Total amount of shared memory per block:       49152 bytes
 Total number of registers available per block: 65536
 Warp size:                                     32
 Maximum number of threads per multiprocessor:  2048
 Maximum number of threads per block:           1024
 Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
 Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
 Maximum memory pitch:<PHONE_NUMBER> bytes
 Texture alignment:                             512 bytes
 Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
 Run time limit on kernels:                     No
 Integrated GPU sharing Host Memory:            No
 Support host page-locked memory mapping:       Yes
 Alignment requirement for Surfaces:            Yes
 Device has ECC support:                        Enabled
 Device supports Unified Addressing (UVA):      Yes
 Device supports Compute Preemption:            Yes
 Supports Cooperative Kernel Launch:            Yes
 Supports MultiDevice Co-op Kernel Launch:      Yes
 Device PCI Domain ID / Bus ID / location ID:   3 / 1 / 0
 Compute Mode:
    < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 2: "Tesla P100-SXM2-16GB"
 CUDA Driver Version / Runtime Version          10.0 / 10.0
 CUDA Capability Major/Minor version number:    6.0
 Total amount of global memory:                 16281 MBytes<PHONE_NUMBER>4 bytes)
 (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
 GPU Max Clock rate:                            1481 MHz (1.48 GHz)
 Memory Clock rate:                             715 Mhz
 Memory Bus Width:                              4096-bit
 L2 Cache Size:                                 4194304 bytes
 Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
 Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
 Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
 Total amount of constant memory:               65536 bytes
 Total amount of shared memory per block:       49152 bytes
 Total number of registers available per block: 65536
 Warp size:                                     32
 Maximum number of threads per multiprocessor:  2048
 Maximum number of threads per block:           1024
 Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
 Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
 Maximum memory pitch:<PHONE_NUMBER> bytes
 Texture alignment:                             512 bytes
 Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
 Run time limit on kernels:                     No
 Integrated GPU sharing Host Memory:            No
 Support host page-locked memory mapping:       Yes
 Alignment requirement for Surfaces:            Yes
 Device has ECC support:                        Enabled
 Device supports Unified Addressing (UVA):      Yes
 Device supports Compute Preemption:            Yes
 Supports Cooperative Kernel Launch:            Yes
 Supports MultiDevice Co-op Kernel Launch:      Yes
 Device PCI Domain ID / Bus ID / location ID:   6 / 1 / 0
 Compute Mode:
    < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 3: "Tesla P100-SXM2-16GB"
 CUDA Driver Version / Runtime Version          10.0 / 10.0
 CUDA Capability Major/Minor version number:    6.0
 Total amount of global memory:                 16281 MBytes<PHONE_NUMBER>4 bytes)
 (56) Multiprocessors, ( 64) CUDA Cores/MP:     3584 CUDA Cores
 GPU Max Clock rate:                            1481 MHz (1.48 GHz)
 Memory Clock rate:                             715 Mhz
 Memory Bus Width:                              4096-bit
 L2 Cache Size:                                 4194304 bytes
 Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
 Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
 Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
 Total amount of constant memory:               65536 bytes
 Total amount of shared memory per block:       49152 bytes
 Total number of registers available per block: 65536
 Warp size:                                     32
 Maximum number of threads per multiprocessor:  2048
 Maximum number of threads per block:           1024
 Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
 Max dimension size of a grid size    (x,y,z):<PHONE_NUMBER>, 65535, 65535)
 Maximum memory pitch:<PHONE_NUMBER> bytes
 Texture alignment:                             512 bytes
 Concurrent copy and kernel execution:          Yes with 3 copy engine(s)
 Run time limit on kernels:                     No
 Integrated GPU sharing Host Memory:            No
 Support host page-locked memory mapping:       Yes
 Alignment requirement for Surfaces:            Yes
 Device has ECC support:                        Enabled
 Device supports Unified Addressing (UVA):      Yes
 Device supports Compute Preemption:            Yes
 Supports Cooperative Kernel Launch:            Yes
 Supports MultiDevice Co-op Kernel Launch:      Yes
 Device PCI Domain ID / Bus ID / location ID:   7 / 1 / 0
 Compute Mode:
    < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
> Peer access from Tesla P100-SXM2-16GB (GPU0) -> Tesla P100-SXM2-16GB (GPU1) : Yes
> Peer access from Tesla P100-SXM2-16GB (GPU0) -> Tesla P100-SXM2-16GB (GPU2) : No
> Peer access from Tesla P100-SXM2-16GB (GPU0) -> Tesla P100-SXM2-16GB (GPU3) : No
> Peer access from Tesla P100-SXM2-16GB (GPU1) -> Tesla P100-SXM2-16GB (GPU0) : Yes
> Peer access from Tesla P100-SXM2-16GB (GPU1) -> Tesla P100-SXM2-16GB (GPU2) : No
> Peer access from Tesla P100-SXM2-16GB (GPU1) -> Tesla P100-SXM2-16GB (GPU3) : No
> Peer access from Tesla P100-SXM2-16GB (GPU2) -> Tesla P100-SXM2-16GB (GPU0) : No
> Peer access from Tesla P100-SXM2-16GB (GPU2) -> Tesla P100-SXM2-16GB (GPU1) : No
> Peer access from Tesla P100-SXM2-16GB (GPU2) -> Tesla P100-SXM2-16GB (GPU3) : Yes
> Peer access from Tesla P100-SXM2-16GB (GPU3) -> Tesla P100-SXM2-16GB (GPU0) : No
> Peer access from Tesla P100-SXM2-16GB (GPU3) -> Tesla P100-SXM2-16GB (GPU1) : No
> Peer access from Tesla P100-SXM2-16GB (GPU3) -> Tesla P100-SXM2-16GB (GPU2) : Yes

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 10.0, CUDA Runtime Version = 10.0, NumDevs = 4
Result = PASS

It's still getting stuck at the same stage:
[05-12-2018 19:41:36] UI Server starting...
790 [pool-8-thread-1] INFO play.api.Play - Application started (Prod)
995 [pool-8-thread-1] INFO play.core.server.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
996 [pool-8-thread-1] INFO org.deeplearning4j.ui.play.PlayUIServer - DL4J UI Server started at http://localhost:9000
1240 [pool-8-thread-1] INFO org.deeplearning4j.ui.play.PlayUIServer - StatsStorage instance attached to UI: FileStatsStorage(temp/dl4j_training_stats.tmp)
[05-12-2018 19:41:38] Creating arraylist with INDArray pairs...
3254 [pool-8-thread-1] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend

run from Eclipse.

Can you make sure you run it from the command line via Maven, and not via Eclipse? Eclipse is notorious for not updating things properly when you update versions in your pom.xml.
If it runs from Maven, then it's just Eclipse not updating things. If it still fails via Maven, we'll look into this further.

Sorry for the confusion.
I only run it from Eclipse when running on the Ubuntu AlienWare PC which is where it works.
When I run it on the server (RHEL ppc64le) it fails.
Here's how I run it on the server:

Export JAR using "mvn package" on the directory containing my pom.xml (project root in Eclipse) and transfer to server to /lib/ (the JAR is about 1.3 GB in size).
Run java -cp -classpath /lib/*

All works fine until the code runs Nd4j.create() and it stops doing anything after we receive this output:
3254 [pool-8-thread-1] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend

Is there some kind of extra tracing I can enable to see what's going on? or will a thread dump be of help?

A thread dump would help, yes (if only to confirm what I suspect it is).
@raver119 / @sshepel Did 1.0.0-beta3 go out with CC 6.0 support (which is what the P100 is)? I know we have CC 6.1 and 7.0, not sure about 6.0.

threaddump.txt
Attached thread dump.

Double checked with some extra debugging output.
Looks like pool-8-thread-1 is the thread calling Nd4j.create().

Hmm, thread dump looks like there might be an uncaught exception slipping through on my end, checking now.

Ok, it definitely was a lost Exception.
Here it is:
3247 [pool-8-thread-1] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
java.lang.ExceptionInInitializerError
        at org.nd4j.jita.concurrency.CudaAffinityManager.getNumberOfDevices(CudaAffinityManager.java:201)
        at org.nd4j.jita.constant.ConstantProtector.purgeProtector(ConstantProtector.java:56)
        at org.nd4j.jita.constant.ConstantProtector.<init>(ConstantProtector.java:49)
        at org.nd4j.jita.constant.ConstantProtector.<clinit>(ConstantProtector.java:37)
        at org.nd4j.jita.constant.ProtectedCudaConstantHandler.<clinit>(ProtectedCudaConstantHandler.java:64)
        at org.nd4j.jita.constant.CudaConstantHandler.<clinit>(CudaConstantHandler.java:33)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:5549)
        at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5482)
        at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:215)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.model.type.text.tfidf.TFIDFVectorSpaceInstance.setup(TFIDFVectorSpaceInstance.java:214)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.model.type.text.tfidf.TFIDFVectorSpaceModelType.executeTraining(TFIDFVectorSpaceModelType.java:49)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.model.type.text.tfidf.TFIDFVectorSpaceModelType.executeTraining(TFIDFVectorSpaceModelType.java:1)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.DL4JDeepLearningFramework.executeTraining(DL4JDeepLearningFramework.java:190)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.DL4JDeepLearningFramework.access$2(DL4JDeepLearningFramework.java:187)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.DL4JDeepLearningFramework$1.run(DL4JDeepLearningFramework.java:112)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: ND4J is probably missing dependencies. For more information, please refer to: http://nd4j.org/getstarted.html
        at org.nd4j.nativeblas.NativeOpsHolder.<init>(NativeOpsHolder.java:68)
        at org.nd4j.nativeblas.NativeOpsHolder.<clinit>(NativeOpsHolder.java:36)
        ... 22 more
Caused by: java.lang.UnsatisfiedLinkError: no jnind4jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        at java.lang.System.loadLibrary(System.java:1122)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1258)
        at org.bytedeco.javacpp.Loader.load(Loader.java:999)
        at org.bytedeco.javacpp.Loader.load(Loader.java:891)
        at org.nd4j.nativeblas.Nd4jCuda.<clinit>(Nd4jCuda.java:10)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.bytedeco.javacpp.Loader.load(Loader.java:950)
        at org.bytedeco.javacpp.Loader.load(Loader.java:891)
        at org.nd4j.nativeblas.Nd4jCuda$NativeOps.<clinit>(Nd4jCuda.java:409)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.nd4j.nativeblas.NativeOpsHolder.<init>(NativeOpsHolder.java:46)
        ... 23 more
Caused by: java.lang.UnsatisfiedLinkError: /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so)
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
        at java.lang.Runtime.load0(Runtime.java:809)
        at java.lang.System.load(System.java:1086)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1238)
        ... 34 more

Ok, that helps (cc @saudet )
What version of RHEL are you running?

uname -r
3.10.0-693.17.1.el7.ppc64le

cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.5 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.5"
PRETTY_NAME="Red Hat Enterprise Linux"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.5"

cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)

Hope that helps.

And last one in case it helps to clarify:
Operating System: Red Hat Enterprise Linux
CPE OS Name: cpe:/o:redhat:enterprise_linux:7.5:GA:server
Kernel: Linux 3.10.0-693.17.1.el7.ppc64le
Architecture: ppc64-le

We're building with Ubuntu 16.04, not sure if we'd get a recent enough version of GCC with CentOS or RHEL.

What's the most recent version of GCC we can use with that version of RHEL, and how?

FYI I can confirm that /lib64/libstdc++.so.6 exists on the system and is readable. So not sure why I'm getting this error.
https://access.redhat.com/solutions/19458
GCC 4.8 apparently for RHEL 7.5.x.

ouch :(

devtoolset-7 is apparently available for ppc64le:
http://mirror.centos.org/altarch/7/sclo/ppc64le/rh/devtoolset-7/
Not sure we can make this work with CUDA, but if you have some time to test this out, could you try to build from source with this @madhs53?

I can confirm GCC is already available on the java library path:
    java.library.path = /usr/local/cuda/lib64
        /usr/local/lib64
        /usr/java/packages/lib/ppc64le
        /usr/lib64
        /lib64
        /lib
        /usr/lib

ls -la /usr/lib64/ | grep gcc
-rwxr-xr-x.   1 root root   133696 Apr 10  2018 libgcc_s-4.8.5-20150702.so.1
lrwxrwxrwx.   1 root root       28 Jul 17 15:53 libgcc_s.so.1 -> libgcc_s-4.8.5-20150702.so.1

/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found
Just so I can explain to our RHEL admins, is this a version compatibility issue? or is there still some problem with the path? What am I building with the above? Sorry I don't really understand the underlying issue.

Recent enough versions of GCC for Ubuntu uses a version of libstdc++ that is more recent than available for RHEL 7. We need to create a build that works in RHEL 7, with a version of GCC that isn't too old for libnd4j, but that isn't too recent either for CUDA. devtoolset-7 looks like a good solution, but it needs to be tested, so please give it a try if you can, and report the results here.

So to confirm, I download and run the RPMs from here:
http://mirror.centos.org/altarch/7/sclo/ppc64le/rh/devtoolset-7/
Do I then do anything else?

I don't know exactly, but I suppose they work similarly to the ones for x86_64 and the instructions here:
https://www.softwarecollections.org/en/scls/rhscl/devtoolset-7/
Then try to rebuild Deeplearning4j from scratch as per the instructions here:
https://github.com/deeplearning4j/deeplearning4j

Thanks.
What's the easiest way to build dl4j from source without maven installed?

Not possible, you'll need Maven.

We just need to unzip it somewhere, it's really easy: https://maven.apache.org/install.html

Thanks.
To confirm for this test, am I:

Installing devtoolset-7 on both Ubuntu and RHEL, building on Ubuntu and attempting to execute again on RHEL?

Or 2: installing devtoolset-7 on RHEL and building from there?

devtoolset-7 is for CentOS or RHEL, it won't work on Ubuntu.

Hi saudet.
We installed devtoolset-7 onto RHEL and did the following rebuild:
mvn clean install -Dlibnd4j.cuda=10.0
We then re-attempted the test with the new JAR and received the same error message as before:
Caused by: java.lang.UnsatisfiedLinkError: /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so)
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
        at java.lang.Runtime.load0(Runtime.java:809)
        at java.lang.System.load(System.java:1086)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1238)
        ... 34 more

That means it didn't actually get installed, for some reason. /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so is not a file that you built. You'll need to get all artifacts built and installed correctly on the same machine.

Ok sorry about that, now I'm executing the correct build.
We are getting this error when building from source:
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
  CMake 3.6 or higher is required.  You are running version <IP_ADDRESS>

I'll see if we can get that updated.

It should be available as cmake3.

Ok, cmake3 installed.
Now getting a new error during rebuild:
-- Building minifier...
CMake Error at blas/CMakeLists.txt:281 (message):
  You need at least GCC 4.9

gcc --version
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.

Minifier should be skipped by default, but even then i think 4.9 was required...

Have you enabled it? It's selectable

Yep that was the problem, thanks heaps for that.
The build is now progressing, I'll report back with the results.

FYI the build seems to be sitting at this position for quite some time, I'll let it keep running.
[  6%] Building NVCC (Device) object blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o
-- Removing /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/./nd4jcuda_generated_transform_strict.cu.o
/usr/local/bin/cmake -E remove /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/./nd4jcuda_generated_transform_strict.cu.o
-- Generating dependency file: /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.NVCC-depend
/usr/local/cuda-10.0/bin/nvcc -M -D__CUDACC__ /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blas/../include/loops/cuda/transform/transform_strict.cu -o /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.NVCC-depend -m64 --std c++11 -D__CUDABLAS__=true -Dnd4jcuda_EXPORTS -Xcompiler ,\"-mcmodel=medium\",\"-DLIBND4J_ALL_OPS=true\",\"-mcpu=power8\",\"-mtune=power8\",\"-D__POWER\",\"-fopenmp\",\"-fPIC\",\"-O3\",\"-fPIC\",\"-fassociative-math\",\"-funsafe-math-optimizations\",\"-fmax-errors=2\",\"-D_RELEASE=true\" -DCUDA_10 -w --cudart=static -O3 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -DNVCC -I/usr/local/cuda-10.0/include -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blas -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/helpers -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/loops -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/graph -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/ops -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/types -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/array -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/cnpy -I/usr/include -I/usr/local/include -I/usr/local/cuda-10.0/include
-- Generating temporary cmake readable file: /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend.tmp
/usr/local/bin/cmake -D input_file:FILEPATH=/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.NVCC-depend -D output_file:FILEPATH=/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend.tmp -P /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/cmake/FindCUDA/make2cmake.cmake
-- Copy if different /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend.tmp to /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend
/usr/local/bin/cmake -E copy_if_different /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend.tmp /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend
-- Removing /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend.tmp and /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.NVCC-depend
/usr/local/bin/cmake -E remove /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.depend.tmp /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/nd4jcuda_generated_transform_strict.cu.o.NVCC-depend
-- Generating /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/./nd4jcuda_generated_transform_strict.cu.o
/usr/local/cuda-10.0/bin/nvcc /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blas/../include/loops/cuda/transform/transform_strict.cu -c -o /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/./nd4jcuda_generated_transform_strict.cu.o -m64 --std c++11 -D__CUDABLAS__=true -Dnd4jcuda_EXPORTS -Xcompiler ,\"-mcmodel=medium\",\"-DLIBND4J_ALL_OPS=true\",\"-mcpu=power8\",\"-mtune=power8\",\"-D__POWER\",\"-fopenmp\",\"-fPIC\",\"-O3\",\"-fPIC\",\"-fassociative-math\",\"-funsafe-math-optimizations\",\"-fmax-errors=2\",\"-D_RELEASE=true\" -DCUDA_10 -w --cudart=static -O3 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -DNVCC -I/usr/local/cuda-10.0/include -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blas -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/helpers -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/loops -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/graph -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/ops -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/types -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/array -I/var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/include/cnpy -I/usr/include -I/usr/local/include -I/usr/local/cuda-10.0/include
Generated /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/./nd4jcuda_generated_summarystatsreduce.cu.o successfully.
Generated /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/./nd4jcuda_generated_transform_same.cu.o successfully.
Generated /var/tmp/mvn/ws/dl4j/deeplearning4j-master/libnd4j/blasbuild/cuda/blas/CMakeFiles/nd4jcuda.dir/__/include/loops/cuda/transform/./nd4jcuda_generated_transform_strict.cu.o successfully.

The build eventually failed with this message:
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:add-resource (add-resource) @ nd4j ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-test-resources) @ nd4j ---
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireActiveProfile failed with message:
Profile "testresources" is not activated.

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:38 h
[INFO] Finished at: 2018-12-19T11:31:38+11:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-test-resources) on project nd4j: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :nd4j

Run the build without tests as per the instructions here:
https://github.com/deeplearning4j/deeplearning4j

It says it right there, you're missing the .git directory. Try to run with  -Dmaven.gitcommitid.skip=true:
https://github.com/git-commit-id/maven-git-commit-id-plugin

Yeah sorry, I don't know a lot about maven/git/etc. Where is it expecting that .git directory to be and what is it for? Does it need anything in it?
I'm now rerunning with -Dmaven.gitcommitid.skip=true

Didn't seem to help:
mvn clean install -Dmaven.test.skip -Dlibnd4j.cuda=10.0 -Dmaven.gitcommitid.skip=true -rf :jackson
[ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:2.2.2:revision (default) on project jackson: .git directory is not found! Please specify a valid [dotGitDirectory] in your pom.xml -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Ok, either remove that plugin from the pom.xml files or clone the repository using git to get the .git directory.

Is there a way I can build without tensorflow support? I don't really care about that, and pretty sure tensorflow doesn't run on ppc 64.

Looks like I can't just remove those executions from the poms as I then get this:
[ERROR] /rstudio/working/mvn/ws/dl4j/repo/deeplearning4j/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/DynamicCustomOp.java:[36,32] package org.tensorflow.framework does not exist
Any ideas?

Thanks, getting further now.
New error:
[ERROR] Failed to execute goal on project nd4j-native: Could not resolve dependencies for project org.nd4j:nd4j-native:jar:1.0.0-SNAPSHOT: The following artifacts could not be resolved: org.bytedeco.javacpp-presets:mkl:jar:linux-ppc64le:2019.0-1.4.3, org.bytedeco.javacpp-presets:mkl-dnn:jar:linux-ppc64le:0.16-1.4.3: Could not find artifact org.bytedeco.javacpp-presets:mkl:jar:linux-ppc64le:2019.0-1.4.3 in maven-restlet (http://maven.restlet.org) -> [Help 1]

Which version of Maven is that? This shouldn't happen, but maybe it's caused by a bug in an old version of Maven...

Looks like 3.6:
mvn --version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-25T05:41:47+11:00)
Maven home: /rstudio/working/mvn/apache-maven-3.6.0
Java version: 1.8.0_171, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk-<IP_ADDRESS>-7.b10.el7.ppc64le/jre
Default locale: en_AU, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-693.17.1.el7.ppc64le", arch: "ppc64le", family: "unix"

I was editing poms before to try and work around the other issues.
I'm going to re-clone the dl4j project and rebuild from scratch.

Same error:
[ERROR] Failed to execute goal on project nd4j-native: Could not resolve dependencies for project org.nd4j:nd4j-native:jar:1.0.0-SNAPSHOT: The following artifacts could not be resolved: org.bytedeco.javacpp-presets:mkl:jar:linux-ppc64le:2019.0-1.4.3, org.bytedeco.javacpp-presets:mkl-dnn:jar:linux-ppc64le:0.16-1.4.3: Could not find artifact org.bytedeco.javacpp-presets:mkl:jar:linux-ppc64le:2019.0-1.4.3 in maven-restlet (http://maven.restlet.org) -> [Help 1]

As we'll be using cuda, is there a way we can skip the native mkl stuff? Or is this still required?

Note a couple of issues in the past (closed) with similar errors:
#2126
#6549

I don't know how this stuff works but I notice that for this version there are linux ppc64 jars:
http://repo2.maven.org/maven2/org/bytedeco/javacpp-presets/mkl/2017.3-1.3/
But for later versions there aren't. Is this an issue?

I see, there appears to be a little issue at build time for that. It's working fine for me on Ubuntu with -Djavacpp.platform=linux-ppc64le though, so a command like this builds everything correctly:
mvn clean install -Dmaven.test.skip -Dlibnd4j.cuda=10.0 -Djavacpp.platform=linux-ppc64le

Plus the bits for git and protoc if needed.

You'll have to remove -Dlibnd4j.cpu.compile.skip=true. I never said you should be adding this.

Ah. Sorry about that. It was just something I was trying earlier, forgot to remove it in this case.
Doing a full rebuild now without that flag.

Ok new error message now:
[ERROR] Failed to execute goal org.bytedeco:javacpp:1.4.3:build (javacpp.compiler) on project nd4j-native: Failed to execute JavaCPP Builder: Cannot run program "powerpc64le-linux-gnu-g++" (in directory "/rstudio/working/mvn/ws/dl4j/repo2/deeplearning4j/nd4j/nd4j-backends/nd4j-backend-impls/nd4j-native/target/classes/org/nd4j/nativeblas/linux-ppc64le"): error=2, No such file or directory -> [Help 1]
Command used:
mvn clean install -Dmaven.test.skip -DskipTests=true -Dlibnd4j.cuda=10.0 -DprotocCommand=/rstudio/working/mvn/ws/dl4j/repo2/deeplearning4j/protobuf-3.5.1/src/protoc -Djavacpp.platform=linux-ppc64le

Ok, try with -Djavacpp.platform.compiler=/usr/bin/g++

Thanks that worked.
Almost at the end now, this is the error during the 'jumpy' stage:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (python-install-cython) on project jumpy: Command execution failed.: Cannot run program "pip" (in directory "/rstudio/working/mvn/ws/dl4j/repo2/deeplearning4j/jumpy"): error=2, No such file or directory -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :jumpy

Yeah, you can ignore that. You should have usable artifacts in your local repository cache now! Try them out.

FYI only 3 to go:
[INFO] jumpy 1.0.0-SNAPSHOT ............................... FAILURE [ 10.354 s]
[INFO] pydatavec 1.0.0-SNAPSHOT ........................... SKIPPED
[INFO] pydl4j 1.0.0-SNAPSHOT .............................. SKIPPED

Thanks, I'll try it out now.

Think I worked it out. I have to use 1.0.0-SNAPSHOT as the version for the dependencies in my pom.
Will report back later once we've tried again.

Great news, it's working!
The only problem now is not enough memory on the GPU. Going to use the parallel wrapper to distribute across all 4 to see if it helps.
I'm glad we got this working as DL4J is easily our framework of choice. Thanks so much for all your help.

Great to hear that! I don't think it's a high priority for us, but at least it's good to know that we can consider switching our linux-ppc64le builds to CentOS as well. I think the main blocker would be CUDA 9.0 not working with devtoolset-7, but we'll probably be dropping support for that version of CUDA before long. @sshepel Do you see any other potential issues?

Ok, it definitely was a lost Exception. Here it is:
3247 [pool-8-thread-1] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
java.lang.ExceptionInInitializerError
        at org.nd4j.jita.concurrency.CudaAffinityManager.getNumberOfDevices(CudaAffinityManager.java:201)
        at org.nd4j.jita.constant.ConstantProtector.purgeProtector(ConstantProtector.java:56)
        at org.nd4j.jita.constant.ConstantProtector.<init>(ConstantProtector.java:49)
        at org.nd4j.jita.constant.ConstantProtector.<clinit>(ConstantProtector.java:37)
        at org.nd4j.jita.constant.ProtectedCudaConstantHandler.<clinit>(ProtectedCudaConstantHandler.java:64)
        at org.nd4j.jita.constant.CudaConstantHandler.<clinit>(CudaConstantHandler.java:33)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:5549)
        at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5482)
        at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:215)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.model.type.text.tfidf.TFIDFVectorSpaceInstance.setup(TFIDFVectorSpaceInstance.java:214)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.model.type.text.tfidf.TFIDFVectorSpaceModelType.executeTraining(TFIDFVectorSpaceModelType.java:49)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.model.type.text.tfidf.TFIDFVectorSpaceModelType.executeTraining(TFIDFVectorSpaceModelType.java:1)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.DL4JDeepLearningFramework.executeTraining(DL4JDeepLearningFramework.java:190)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.DL4JDeepLearningFramework.access$2(DL4JDeepLearningFramework.java:187)
        at au.gov.dhs.ice.server.pipeline.deeplearning.framework.dl4j.DL4JDeepLearningFramework$1.run(DL4JDeepLearningFramework.java:112)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: ND4J is probably missing dependencies. For more information, please refer to: http://nd4j.org/getstarted.html
        at org.nd4j.nativeblas.NativeOpsHolder.<init>(NativeOpsHolder.java:68)
        at org.nd4j.nativeblas.NativeOpsHolder.<clinit>(NativeOpsHolder.java:36)
        ... 22 more
Caused by: java.lang.UnsatisfiedLinkError: no jnind4jcuda in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
        at java.lang.Runtime.loadLibrary0(Runtime.java:870)
        at java.lang.System.loadLibrary(System.java:1122)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1258)
        at org.bytedeco.javacpp.Loader.load(Loader.java:999)
        at org.bytedeco.javacpp.Loader.load(Loader.java:891)
        at org.nd4j.nativeblas.Nd4jCuda.<clinit>(Nd4jCuda.java:10)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.bytedeco.javacpp.Loader.load(Loader.java:950)
        at org.bytedeco.javacpp.Loader.load(Loader.java:891)
        at org.nd4j.nativeblas.Nd4jCuda$NativeOps.<clinit>(Nd4jCuda.java:409)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.nd4j.nativeblas.NativeOpsHolder.<init>(NativeOpsHolder.java:46)
        ... 23 more
Caused by: java.lang.UnsatisfiedLinkError: /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/removed_user_id/.javacpp/cache/ice_server_pipeline_dl4j_gpu.jar/org/nd4j/nativeblas/linux-ppc64le/libjnind4jcuda.so)
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
        at java.lang.Runtime.load0(Runtime.java:809)
        at java.lang.System.load(System.java:1086)
        at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1238)
        ... 34 more

Hello, I have the same problem as this. How can I solve it please?

@Nour-Rekik please don't comment on old issues. Post over on the forums: https://community.konduit.ai/ in summary this is a general error message you would find with any c/c++ code, it tells you right there please ensure you google these things:
 version `CXXABI_1.3.8' not found

You are running on an absurdly old system. Please post more details of what you're doing on the forums or use something newer.


chore(requirements): update to Django 1.10.1

https://docs.djangoproject.com/en/1.10/releases/1.10.1/

Current coverage is 87.20% (diff: 100%)

No coverage report found for master at 63fff81.

Powered by Codecov. Last update 63fff81...e16fb97


reported fan speed not always correct when powered on

problem process

turn on fan
set to high
turn off
turn on
issue: the fan is now on high but its reported in the interface as low.

There isn't an async_update method in the integration which is probably the issue.

this might fix it. https://github.com/bbrendon/homeassistant-quietcool/commit/07e961f44e75378447ae77f43b7672d997d9f15b


⚠ Processors has degraded performance

In b166b22, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 44 ms

Resolved: Processors performance has improved in 140ccf2.


Add props for Dialog to disable Escape to close and click overlay to close

When we need user to confirm from two dangerous options, he/she must select from these options, how about add two props:

closeOnEsc default true
closeOnOverlayClick default true

Just like https://chakra-ui.com/docs/components/modal/props

Willing to help if this proposal is approved

I can understand the motivation behind this, but this library is a port of headlessui and there are no plans to add more functionality beyond what exists in that library.  Sorry!


Better error messaging/handling

I have run into this issue a few times, and hacked around it before, but would like to discover the root cause: TypeError: Cannot read property 'test' of undefined.  Ideally there would be a better error message, saying what rule it was trying to test when it found this error, or a way to show the stack trace.  This shows up in both grunt-htmllint and htmllint-cli, using htmllint version 0.5.0.
HTML file:

{{title}}

.htmllintrc
{
  "attr-no-dup": true,
  "attr-name-ignore-regex": "^[\\*|\\[].*",
  "class-no-dup": true,
  "doctype-first": "smart",
  "doctype-html5": true,
  "focusable-tabindex-style": true,
  "head-req-title": true,
  "id-no-dup": true,
  "id-class-style": "none",
  "id-class-ignore-regex": "",
  "class-style": "dash",
  "img-req-src": true,
  "indent-style": "spaces",
  "indent-width": 2,
  "indent-width-cont": true,
  "title-no-dup": true,
  "attr-name-style": "camelCase"
}

The fact that a Javascript error happens at all is a bug, but I agree that it would also be good to have better error reporting for these kinds of errors when they come up. I should have time to look at this over the weekend.

The problem in this case is the line "id-class-style": "none". There's no "none" format; use false (or any falsy value) to avoid any linting.
htmllint should check for invalid option values. I suppose it would be best to do this when the options object is defined or modified. I'll close this issue once I figure that out...

Okay, htmllint now has an awesome option value parsing and option/rule dependency management system, which will probably be released in version 1.0.0. It generates an error whenever you pass an undefined value to an option, and sets that option's value to false. So that pretty much closes this issue.
The docs do say that "none" can be used for class-style. I've added it only as a valid value for class-style and will correct the wiki.


Build a FOSS block explorer site

 Issue by Peppernrino
Thursday Jan 26, 2017 at 17:17 GMT
Originally opened as https://github.com/Erkan-Yilmaz/Gridcoin-tasks/issues/73

This way, everyone can contribute/fix bugs, and no one would have an unfair advantage or insight into numbers or trends that aren't readily apparent to most daily users. Also, any adshare gained could be redistributed to other Gridcoin projects, not retained by a single person.
If one was feeling up to the task, this could also be used as an opportunity to start the previously discussed Gridcoin/CureCoin/Foldingcoin "Science Coin Consortium" joint block explorer site.

 Comment by grctest
Friday Jan 27, 2017 at 09:43 GMT

There's open source block explorer libraries we can use, however these don't cover any of the custom elements within the GRC network.
It'd be really cool if either gridresearchcorp or gridcoinstats were to open source in return for a fat bounty (which would offset potentially lost ad revenue).

no one would have an unfair advantage or insight into numbers or trends that aren't readily apparent to most daily users.

Most of the gathered data by startail is publicly displayed, thus there is an equal level of access to transparent data.  In terms of having an unfair advantage, there's nothing stopping users from gathering BOINC statistics to compare against the NN to aid in competitive crunching.

 Comment by Peppernrino
Saturday Jan 28, 2017 at 07:13 GMT

except maybe skill level, or computer savvy. i'm talking about AVERAGE people. normies. 😛 lol

 Comment by grcjamezz
Monday Apr 03, 2017 at 07:06 GMT

Awesome idea @Peppernrino.
Yes as CU said " Most of the gathered data by startail is publicly displayed, thus there is an equal level of access to transparent data."
But the other 99.2% of us have no idea how to access that data let alone read/use or display the NN data I guess in the same fasion moving from ASCII to binary blocks and gridcoinresearchcorp not being able to read the blocks now , startails site does and RTmoney_ is well versed in gridcoin.

 Comment by Erelas
Sunday Jul 09, 2017 at 21:49 GMT

This "normie" would like to help, and has very little idea where to start.  I do however, understand the concept of not re-inventing the wheel and would hope that offering a bounty to gridresearchcorp or gridcoinstats would be the way to go.  Once the source was in hand, then if for nothing else, in the interest of learning more about it, I would personally be willing to host an explorer.

 Comment by bryhardt
Sunday Jul 09, 2017 at 21:59 GMT

i would be more than willing to host an explorer with the pool. just didnt want to duplicate either or step on toes. i have some experience running explorer for another not so normal coin, so another one would be cool.

 Comment by Erelas
Sunday Jul 09, 2017 at 22:16 GMT

Educate me!  No seriously, without being too much of a bother, I am happy to learn to do new things.

 Comment by frank0051
Wednesday Aug 02, 2017 at 20:46 GMT

Just curious, given we have at least 3 block explorers, how come none of our community members have open-sourced theirs???

 Comment by frank0051
Monday Aug 07, 2017 at 14:24 GMT

I've added a suggestion to one of the Steem It posts about open-source block. Up-vote it so we can get on CM's radar, haha. https://steemit.com/gridcoinstats/@sc-steemit/brainstorming-gridcoinstats-ideas-thread-25-sbd-in-prices#@frank0051/re-sc-steemit-brainstorming-gridcoinstats-ideas-thread-25-sbd-in-prices-20170807t142205918z

 Comment by Erelas
Monday Aug 07, 2017 at 18:10 GMT

AH, glad you sent another e-mail because I totally missed your first one.
HOnestly, not to avoid your question, I really don't have an answer.  For
all I know it might be part of a thesis or something scholastic and he
wishes to keep it under wraps until published?  Honestly no clue and I
don't have the technical knowledge to answer adequately.  Thanks for the
info though, I will take a look at the FOSS stuff you sent the link to,
might be able to figure it out eventually :)
On Mon, Aug 7, 2017 at 10:24 AM, frank0051<EMAIL_ADDRESS>wrote:

I've added a suggestion to one of the Steem It posts about open-source
block. Up-vote it so we can get on CM's radar, haha. https://steemit.com/
gridcoinstats/@sc-steemit/brainstorming-gridcoinstats-
ideas-thread-25-sbd-in-prices#@frank0051/re-sc-steemit-
brainstorming-gridcoinstats-ideas-thread-25-sbd-in-prices-
20170807t142205918z
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Erkan-Yilmaz/Gridcoin-tasks/issues/73#issuecomment-320677954,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AceBdrfqPYtu42oqwdyHxFTaoGmj61hvks5sVx4SgaJpZM4Lu5VZ
.

Completed here: https://github.com/gridcoin-community/BlockExplorer


The date input box is in English

Describe the bug
The date input box is in English
example p1
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
Other browsers are in Chinese
example p2
Screenshots

Environment (please complete the following information):

OS: windows
Architecture: amd64
Version: jcefmaven<IP_ADDRESS>

Additional context
May I ask if there is a solution
Source code
Add sources to reproduce the problem.

Im by no means an expert, but I would assume that you need to set the browser locale from code in jcef. Have you tried this? You should be able to set it using a command line switch: https://magpcss.org/ceforum/viewtopic.php?f=6&t=15470


[SUPPORT] Hudi Write Performance

Hello,
I want to start using Hudi on my datalake, so I'm running some performance tests comparing current processing time with and without Hudi. We have a lot of tables in our datalake so we are processing these tables in groups in the same spark context with different threads.
I made a test processing all table sources again, with regular parquet it took 15 minutes, with Hudi bulk insert 29 minutes, Hudi has some operations that regular parquet doesn't have, for example sorting but the big performance difference was in writing parquet operation, is there any difference writing parquet with Hudi and regular parquet? I used gzip codec in both.
In Hudi I configured bulk parallelism to 20 and regular parquet I made a coalesce 20.
Hudi Version: 0.8.0-SNAPSHOT
Spark Version: 3.0.1
11 Executors with 5 cores each and 35g of memory
spark submit:
spark-submit --deploy-mode cluster --conf spark.executor.cores=5 --conf spark.executor.memoryOverhead=3000 --conf spark.yarn.maxAppAttempts=1 --conf spark.executor.memory=35g --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --packages org.apache.spark:spark-avro_2.12:2.4.4 --jars s3://dl/lib/spark-daria_2.12-0.38.2.jar,s3://dl/lib/hudi-spark-bundle_2.12-0.8.0-SNAPSHOT.jar --class TableProcessorWrapper s3://dl/code/projects/data_projects/batch_processor_engine/batch-processor-engine_2.12-3.0.1_0.5.jar courier_api_group01
val hudiOptions = Map[String, String](
      "hoodie.table.name"                        -> tableName,
      "hoodie.datasource.write.operation"        -> "bulk_insert",
      "hoodie.bulkinsert.shuffle.parallelism"    -> "20",
      "hoodie.parquet.small.file.limit"          -> "536870912",
      "hoodie.parquet.max.file.size"             -> "1073741824",
      "hoodie.parquet.block.size"                -> "536870912",
      "hoodie.copyonwrite.record.size.estimate"  -> "1024",
      "hoodie.datasource.write.precombine.field" -> deduplicationColumn,
      "hoodie.datasource.write.recordkey.field"  -> primaryKey.mkString(","),
      "hoodie.datasource.write.keygenerator.class" -> (if (primaryKey.size == 1) {
                                                         "org.apache.hudi.keygen.SimpleKeyGenerator"
                                                       } else { "org.apache.hudi.keygen.ComplexKeyGenerator" }),
      "hoodie.datasource.write.partitionpath.field"           -> partitionColumn,
      "hoodie.datasource.write.hive_style_partitioning"       -> "true",
      "hoodie.datasource.write.table.name"                    -> tableName,
      "hoodie.datasource.hive_sync.table"                     -> tableName,
      "hoodie.datasource.hive_sync.database"                  -> databaseName,
      "hoodie.datasource.hive_sync.enable"                    -> "true",
      "hoodie.datasource.hive_sync.partition_fields"          -> partitionColumn,
      "hoodie.datasource.hive_sync.partition_extractor_class" -> "org.apache.hudi.hive.MultiPartKeysValueExtractor",
      "hoodie.datasource.hive_sync.jdbcurl"                   -> "jdbc:hive2://ip-10-0-19-157.us-west-2.compute.internal:10000"    )

Regular Parquet

Hudi has a Rdd conversion Part

Hudi Write, took double time

It was one real world processing that I tried, but I notice this slow writing on every processing that I use Hudi.
Is it normal? Is there any way to tunning it? Am i doing something wrong?
Thank you so much!!!!!

Hello,
I changed the option hoodie.datasource.write.row.writer.enable and took only 21 minutes, 30% faster, great!!!!

Hello,
I changed the option hoodie.datasource.write.row.writer.enable and took only 21 minutes, 30% faster, great!!!!

@rubenssoto yes. row writer is the difference. the df.rdd conversion in Spark takes that hit. I recommend sorting the file initially, since it gives you lots of returns for query performance

@rubenssoto yes. row writer is the difference. the df.rdd conversion in Spark takes that hit. I recommend sorting the file initially, since it gives you lots of returns for query performance

Do you mean an Order By before df.write.format('hudi').save()  ?

Do you mean an Order By before df.write.format('hudi').save()  ?

No, I mean the sorting Hudi internally does that you mentioned before. So this is not even configurable for row writing. So all good. That should explain the extra time (21-15)
    return colOrderedDataset
        .sort(functions.col(HoodieRecord.PARTITION_PATH_METADATA_FIELD), functions.col(HoodieRecord.RECORD_KEY_METADATA_FIELD))
        .coalesce(config.getBulkInsertShuffleParallelism());

No, I mean the sorting Hudi internally does that you mentioned before. So this is not even configurable for row writing. So all good. That should explain the extra time (21-15)
    return colOrderedDataset
        .sort(functions.col(HoodieRecord.PARTITION_PATH_METADATA_FIELD), functions.col(HoodieRecord.RECORD_KEY_METADATA_FIELD))
        .coalesce(config.getBulkInsertShuffleParallelism());

Great, thank you for the explanation....its makes sense.
If I understand this code right, Hudi will order by partition key and record key, so if I have an unpartitioned table the data on my files should be ordered by record key(primary key), is it?
I check one table of mine, and it is my min and max of my primary key of each file

Using that code as a reference, I imagined the files like:
file01   1      1000
file02  1001 2000
file03  2001 3000
....... and so on

Great, thank you for the explanation....its makes sense.
If I understand this code right, Hudi will order by partition key and record key, so if I have an unpartitioned table the data on my files should be ordered by record key(primary key), is it?
I check one table of mine, and it is my min and max of my primary key of each file

Using that code as a reference, I imagined the files like:
file01   1      1000
file02  1001 2000
file03  2001 3000
....... and so on

yes thats correct. they are lexicographically sorted if you notice.
This is a trick we used at Uber even before Hudi. It helps layout data initially sorted, so range pruning is faster, and also when dealing with partitions with unequal size, the sort based on partitionpath ensures we are writing the smallest number of files in total. otherwise, if you hash partition 1000 times across 1000 partition paths, you ll end up with 1M files. In this approach, you will end up with atmost 2000 files. huge benefit. and from there on, when doing upserts/inserts, Hudi will maintain the file sizes.

yes thats correct. they are lexicographically sorted if you notice.
This is a trick we used at Uber even before Hudi. It helps layout data initially sorted, so range pruning is faster, and also when dealing with partitions with unequal size, the sort based on partitionpath ensures we are writing the smallest number of files in total. otherwise, if you hash partition 1000 times across 1000 partition paths, you ll end up with 1M files. In this approach, you will end up with atmost 2000 files. huge benefit. and from there on, when doing upserts/inserts, Hudi will maintain the file sizes.


game getting initialized 2x in demo

when i put an alert in GameController's constructor i see it twice

in the demo under the /demo folder

likely need a game loop?

oddly might be caused by Object.getOwnPropertyNames in generateDocumentation.js


Conditional rendering Entity event listeners are added if the el is null

I am getting an error when using conditional rendering where event listeners are trying to be added to a null element.
my fix is a change to the file lib/Index.js line 94. Wrap the event listeners for 'click' and 'loaded' in an if(el)

Thanks for reporting, will take a look at it.


Fix the "a tag or digest is required" check

Description
Context:
If you use name.ParseReference(imageUri) with where only the image is set, no tag or digest, then the default tag will be latest.
latest is not allowed/used everywhere so we want to disallow providing just the image, a tag or a digest must be provided.
The Bug:
if taggedRef.TagStr() == name.DefaultTag would disallow setting latest as a tag, which is not the intended behaviour
How can this be tested?
Run the unit tests, provide in the dynakube an custom image that has no tag or digest set
Checklist

~[] Unit tests have been updated/added~
[x] PR is labeled accordingly with a single label
[x] I have read and understood the contribution guidelines

Codecov Report

Merging #2210 (6a8275e) into main (644c071) will increase coverage by 0.01%.
The diff coverage is 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##             main    #2210      +/-   ##
==========================================
+ Coverage   64.34%   64.36%   +0.01%
==========================================
  Files         242      242
  Lines       15326    15326
==========================================
+ Hits         9862     9865       +3
+ Misses       4734     4732       -2
+ Partials      730      729       -1

Files
Coverage Δ

pkg/controllers/dynakube/version/updater.go
81.02% <100.00%> (ø)

... and 1 file with indirect coverage changes


Specify is_parametric in axis metadata

Describe the issue
The pipeline is prepared to accept axis metadata with is_parametric.
optional bool is_parametric

We should add the corresponding field to the upstream proto definition and populate the field for all parametric axes. There are currently 9 as I understand it: x_opaque, x_transparent, x_transparent_figures, y_opaque, y_transparent_ascender, y_transparent_descender, y_transparent_figures, y_transparent_lowercase, and y_transparent_uppercase.

Hi, @nathan-williams. Could you help us gain more context about this new field? For example, what is the purpose and effect of adding it?

The only known use today is to inform axis ordering. For example, in a list of axes, we would want to show GRAD before XOPQ. This property has existed on our side for quite a while, but we realized just recently that the property was not being populated.
Let me know if I can support in identifying the parametric axes. They stand out pretty well among the existing axes if you know what to look for (i.e. axes with tags that match /(X|Y).../ more or less).
@davelab6 may be able to provide more insight into the definition of a parametric axis.


Thanos sidecar makes too much 404 Not Found Error to s3 object storage

I'm using thanos-sidecar with prometheus.
In prometheus pod, there are 3 containers (prometheus, config-reloader, thanos-sidecar)
Also there are several thanos components. (thanos-store, thanos-compactor, thanos-query, thanos-ruler)
The thanos-store,compactor, sidecar watch the same s3 object bucket.
For 2 months, I used a managed s3 object storage.
According to the csp, there are too many 404 Not Found Errors from the bucket which the thanos-sidecar uploads the blocks to.
To check the reason, I made a minio s3 storage in my k8s cluster and change the objstore.config of thanos-sidecar and store from managed to minio object storage.  (thanos-compactor watches the old s3 object bucket now)
And then I traced the calls at the minio console.

For every 3 minutes, the thanos-sidecar or store causes 404 not found error.
How can I fix it?
prometheus container

args:

--web.console.templates=/etc/prometheus/consoles
--web.console.libraries=/etc/prometheus/console_libraries
--config.file=/etc/prometheus/config_out/prometheus.env.yaml
--storage.tsdb.path=/prometheus
--storage.tsdb.retention.time=10d
--web.enable-lifecycle
--web.route-prefix=/
--web.config.file=/etc/prometheus/web_config/web-config.yaml
--storage.tsdb.max-block-duration=2h
--storage.tsdb.min-block-duration=2h

config-reloader

args:

--listen-address=:8080
--reload-url=http://<IP_ADDRESS>:9090/-/reload
--config-file=/etc/prometheus/config/prometheus.yaml.gz
--config-envsubst-file=/etc/prometheus/config_out/prometheus.env.yaml
--watched-dir=/etc/prometheus/rules/prometheus-monitoring-prometheus-rulefiles-0

thanos-sidecar

args:

sidecar
--prometheus.url=http://<IP_ADDRESS>:9090/
--grpc-address=:10901
--http-address=:10902
--objstore.config-file=/etc/objstore-config/bucketconfig.yaml
--tsdb.path=/prometheus
--log.level=info
--log.format=logfmt
--min-time=-10d

thanos-store

args:

store
--log.level=debug
--objstore.config-file=/etc/objstore-config/bucketconfig.yaml
--ignore-deletion-marks-delay=1d
--grpc-address=<IP_ADDRESS>:10901
--http-address=<IP_ADDRESS>:10902
--data-dir=/var/thanos/store

@wotkddl21 Can you share what objects are being requested by the sidecar and get 404? Is it meta.json file?
Ideally this shouldn't happen as the sidecar has cache for uploaded metadata. So the 404 will happen once for each block, so should be 2h interval for each prometheus deployment.

@wotkddl21 Can you share what objects are being requested by the sidecar and get 404? Is it meta.json file?
Ideally this shouldn't happen as the sidecar has cache for uploaded metadata. So the 404 will happen once for each block, so should be 2h interval for each prometheus deployment.

@yeya24 How can I get the request that the sidecar made?
The fact that the interval of 404 request should be 2hours is reasonable
But in my case the sidecar makes the 404 error with the 3 minutes of interval
Though I uploaded 2 pictures, the 404 errors from the thanos sidecar were more than 1000

Since you are viewing the Minio console does it have that information? Or maybe you can enable some kind of server side access logs to see if we can get that data?

@yeya24 At the Minio console, the information on the screenshot is all (just error code and method without any specific information)
I think I cant get the server side access log

I got some server side access log!!!
After changing the object storage from the minio to the ECS, I can get the access log below.
Full target : thanos/01GHENB4SBWDRGZM6R6ZYSAMPJ/deletion-mark.json
protocol: s3
request type: GET
That bucket 01GHENB4SBWDRGZM6R6ZYSAMPJ was created at 2022-11-10 02:00.
The request sent by the thanos store was created at 2022-11-15:16:23
Now my main question is that is there any way to stop thanos store requesting the deletion-mark.json?
( p.s. As far as I know, the amazon s3 always generates the deletion-mark.json but the ECS does not.)

In my case, I had stopped the compactor because of the server bug about the ECS s3.
So it leads the block uploaded in the s3 doesn't have deletion-mark.json and no-compaction.json at all.
After I started the compactor, the amount of 404 error per seconds decreased.
In conclusion, that was my bad.
If you want to use thanos architecture (especially the store), don't forget running the compactor


eigsh documentation parameters

Adapted the documentation of scipy/sparse/linalg/eigsh to represent the function more accurately. As stated in the first line of the documentation, A can be real symmetric or complex hermitian. However, in the parameter list itself it is only mentioned to be real symmetric.

Ok, I changed the PR to the by @ilayn proposed content (though the faulty line actually has nothing to do with my PR, of course it is to be corrected).
Consequently, this PR shouldn't actually be mine anymore...

Why not yours ? 😃 It's perfectly fine to do extra cosmetics while we are at it when touching the files. All is rightfully yours.

Last comments addressed and all green except for an unrelated CI issue, so merging. Thanks @puenka, all.


Cannot use withDevTools from @angular-architects/ngrx-toolkit with @ngrx/store and @ngrx/effects

Hello guys,
I was trying to use the withDevtools from the ngrx-toolkit, to view my state in Redux Dev Tools like this:
export const ThemeStore = signalStore(
  { providedIn: 'root' },
  withDevtools('theme'),
  withState(initialState)
);

And inside my app.config.ts file I was also using the provideStore() and provideEffects() from ngrx, and I noticed when I used the store and effects from ngrx, the signalStore cannot appear in the redux devtools:

But when I removed the provideStore() and the provideEffects(), from my app.config.ts file it worked as expected:

When I read the docs(the README file), there was no explicit statement stating that the withDevtools feature from the @angular-architects/ngrx-toolkit will not work with the provideStore() and the provideEffects() feature from ngrx.
I may be wrong in my assessment, and I don't also know if this is the intended behaviour.
All in all, please let me know if this must be explicitly added in the README file, because maybe it is  the intended behaviour, and I'll be happy to add it. Or it is a mistake on my part.

Hello, so I use both Signal Store and the Global Store with enabled extensions. DevTools show my my global store but also the Signal Store. That must work.
Your Redux DevTools should give you an option to switch between them. What happens if you click on the select box top right?

Thanks a lot @rainerhahnekamp , after I switched it, I saw that it worked. Thanks a lot, for the response. It is now working as expected.
This is with ngrx/store:

And this is with ngrx/signals:

Thanks for clarifying.
I don't know must it be explicitly stated in the docs, or it'll be obvious to others?

Great and thanks for letting us know. I think a note in the docs would not hurt.

Sure, please go ahead!

Hi, @Konadu-Akwasi-Akuoko.  Do you mind sharing the exact configuration and setup you did for this to work for you? I'm quite stuck on this myself. I installed the @angular-architects/ngrx-toolkit package and configured withDevtools in my store (just that), but I'm unable to view my signal store state in Redux Devtools. I also have a global store for ngrx/store and ngrx/effects.

Hi, @Konadu-Akwasi-Akuoko. Do you mind sharing the exact configuration and setup you did for this to work for you? I'm quite stuck on this myself. I installed the @angular-architects/ngrx-toolkit package and configured withDevtools in my store (just that), but I'm unable to view my signal store state in Redux Devtools. I also have a global store for ngrx/store and ngrx/effects.

Sure sorry for not replying earlier

@Konadu-Akwasi-Akuoko just to add: The store might not show up immediately in the devtools, only when the component is rendered the first time that uses it.


Add Flarum as Forum alternative

Adds Flarum as an alternative forum software

Hey, thanks for your PR!
We are welcome to all projects, but since there are lists like awesome-self-hosted, we decided to add only startups (for-profit companies founded less than 10 years ago).
Thanks for understanding.


rejections from controller are not specific enough

The rejection handler in the controller favors the first rejection vs a potentially more specific one later in the route matching. This is attributed to this method in BasicHttpService:
/** Rejection handler to terminate connection on a bad request. Delegates to Spray handler. */
    protected def customRejectionHandler(implicit transid: TransactionId) = RejectionHandler {
        case rejections =>
            logging.info(this, s"[REJECT] $rejections")
            BasicHttpService.customRejectionHandler.apply(rejections)
    }

The rejections list should be scanned and prioritized to take either the last rejection from the list, or a CustomRejection if present.

another odd variant of this:
if i nodejs request with json: true, body: data to a web-exported route /my-action.text, then i get back the "requires authentication" error. whereas if i use headers: {content-type: 'application/json'}, body: JSON.stringify(data) then things are happy.
this is i assume due to json:true also setting up the accept headers? i'm not sure, but this one was pretty opaque.

@starpit, this will happen as the content type must be JSON or Form Data.

A JSON:true option will set the content type to JSON. The puzzling thing is that whisk still rejects. I am guessing that this due to a mismatch of Accept. Because we are invoking with a .Text web exported suffix.

Can you post here the HTTP header being sent when you do a request with json: true?

@starpit that's consistent with this issue.

@dubeejw i just verified: the difference seems to boil down to an Accept: application/json' header, when invoking as .text`

@l2fprod, you're not alone :)

@dubeejw want to take this one?

@rabbah the weird thing to me is that text is valid JSON. so an Accept: application/json should not be incompatible with a .text invocation?

there is another related issue: when the entity is empty, this will run twice:
entity(as[Option[JsObject]]) {
    body => process(body)
} ~ entity(as[FormData]) {
     form => process(Some(form.fields.toMap.toJson.asJsObject))
}

@starpit, it'd be very helpful if you posted the exact options you are using to send the request. Below are some options I experimented with and the result.
Works as expected.
var options = {
  method: 'POST',
  url: "https://<IP_ADDRESS>/api/v1/experimental/web/.../default/headers.json",
  body: JSON.stringify({"key": "value"}),
  headers: {
    Content-Type': 'application/json'
  }
};

POST /api/v1/experimental/web/.../default/headers.json HTTP/1.1
Content-Type: application/json
host: <IP_ADDRESS>
accept: application/json
content-length: 15
Connection: close

Works as expected.
var options2 = {
  method: 'POST',
  json: true,
  url: "https://<IP_ADDRESS>/api/v1/experimental/web/.../default/headers.json",
  body: {"key": "value"},
};

POST /api/v1/experimental/web/.../default/headers.json HTTP/1.1
host: <IP_ADDRESS>
accept: application/json
content-type: application/json
content-length: 21
Connection: close

Does not work as a content-type is not sent. Maybe we should default to application/json if content-type isn't sent?
var options3 = {
  method: 'POST',
  url: "https://<IP_ADDRESS>/api/v1/experimental/web/lime%40us.ibm.com/default/headers.json",
  body: JSON.stringify({"key": "value"}),
};

POST /api/v1/experimental/web/.../default/headers.json HTTP/1.1
host: <IP_ADDRESS>
content-length: 15
Connection: close

Does not work as expected because the body is a string not JSON
var options4 = {
  method: 'POST',
   json: true,
  url: "https://<IP_ADDRESS>/api/v1/experimental/web/lime%40us.ibm.com/default/headers.json",
  **body: '{"key": "value"}',**
};

POST /api/v1/experimental/web/lime%40us.ibm.com/default/headers.json HTTP/1.1
host: <IP_ADDRESS>
accept: application/json
content-type: application/json
content-length: 22
Connection: close

$ cat t.js
function main() {
  return {
    headers: {
      'Content-Type': 'text/html'
    },
    body: 'hi there'
  };
}

$ curl -k https://guest.dockerhost/default/t.json
{
  "headers": {
    "Content-Type": "text/html"
  },
  "body": "hi there"
}

$ curl -k https://guest.dockerhost/default/t.http
hi there

$ curl -k https://guest.dockerhost/default/t.http -H 'Accept: application/json'
{
  "error": "The resource requires authentication, which was not supplied with the request",
  "code": 2644
}

There are actually two different defects in this issue. The defect discussed by @starpit is fixed by https://github.com/openwhisk/openwhisk/pull/2010.


WIP - CAMEL-11108: camel-infinispan : change the uri syntax from infinispan:hostName to infinispan:cacheName

@tristantarrant may you check if I missed something on infinispan side

Looks good to me :-)

Closed via 49f2bb51710a17e1506262d9d58ca1543a747156


images within a public page

if you place an image inside a public page, that image is not populated (in the output folder)
e.g: http://notes.avelino.run/#/page/livro%2Fmarketing 5.0

Hi. This sounds like a duplicate of https://github.com/logseq/logseq/issues/7150. If it's not, would recommend opening an issue on the logseq repo with reproducible steps. This repo is for behavior on the github action or CLI. I've updated the issue template on this repo to make it clear going forward


Handle Wikipedia API errors and warnings in the Wikipedia Cog

Relevant Issues
Closes #793
Description
I read the Wikipedia API docs, and noticed that when the Wikipedia API encounters some kind of error or warning, it'll send back the response with the proper error message, but regardless of that the HTTP Reponse Code still remains 200, so I added an additional check to avoid such issues in the future. Docs (the Wikipedia API uses the same API system)
Did you:

[x] Join the Python Discord Community?
[x] Read all the comments in this template?
[x] Ensure there is an issue open, or link relevant discord discussions?
[x] Read and agree to the contributing guidelines?

Thank you for you review! I should've paid more attention to the style guide! 👍
Learned my lesson!

We should handle APIErrors and ResponseCode errors (if the response code is not 200). APIError could mean there is an error with the feed of the response which this PR is solving. We might have unexpected errors where the response code is not 200, do we have any instances of this situation of non 200 responses?

We should handle APIErrors and ResponseCode errors (if the response code is not 200). APIError could mean there is an error with the feed of the response which this PR is solving. We might have unexpected errors where the response code is not 200, do we have any instances of this situation of non 200 responses?
This issue can be delt separately from the issues being solved by this PR but I would like to know what others think about this.

The Wikipedia API will always send back a response code of 200 according to the MediaWiki API specification docs (Wikipedia uses the same). So I don't think there's a case, the only reason I added the check is for the sake of consistency, but there shouldn't be any case when it is 200, and if it's not 200 then it must be an APIError.


Bash completion: -bash: complete: nosort: invalid option name

The command
eval "$(turm completion bash)"

Fails on older Bashs, e.g. 4.2.46(2)-release on CentOS, see also https://github.com/redcursor/bash-tab-completion/issues/1.

Just for completeness, this and similar issues are ultimately related to old Slurm versions (and their respective old systems such as CentOS 7 in this example). So just as an idea until compatibility issues such as https://github.com/kabouzeid/turm/issues/17 are solved: Put a sentence in the README about supported Slurm versions, e.g. Slurm>=22.


Crash on Samsung S6 / Android 7

Hello.
I noticed one crash in Google play store that happens on Samsung Galaxy S6:
net.sqlcipher.database.SQLiteException:
at net.sqlcipher.database.SQLiteDatabase.dbopen (SQLiteDatabase.java)
at net.sqlcipher.database.SQLiteDatabase.openDatabaseInternal (SQLiteDatabase.java:2499)
at net.sqlcipher.database.SQLiteDatabase.openDatabase (SQLiteDatabase.java:1204)
at net.sqlcipher.database.SQLiteDatabase.openOrCreateDatabase (SQLiteDatabase.java:1267)
at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase (SQLiteOpenHelper.java:162)
at com.commonsware.cwac.saferoom.Helper$OpenHelper.getWritableSupportDatabase (Helper.java:142)
at com.commonsware.cwac.saferoom.Helper.getWritableDatabase (Helper.java:77)
at androidx.room.RoomDatabase.query (RoomDatabase.java:256)
at androidx.room.util.DBUtil.query (DBUtil.java:54)
at xxx.xxxxxxxx.xxxxxxxxxxx.room.models.DataDao_Impl.getByKey (DataDao_Impl.java:158)
at xxx.xxxxxxxx.xxxxxxxxxxx.SecureStorage$1.run (SecureStorage.java:130)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1133)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:607)
at java.lang.Thread.run (Thread.java:762)

I couldn't reproduce it on Samsung devices I have (don't have S6, but have few other).
Any idea why it might happen and how to fix it?
Thanks.

Hi! The project documentation has a few support options. This issue tracker is for reproducible bug reports.
SafeRoom is just a thin wrapper around SQLCipher for Android. It is fairly likely that your problems lie in something tied to SQLCipher for Android, more so than SafeRoom.
That being said, there is nothing in that stack trace that would indicate the exact cause of the problem, so I am afraid that I do not have any particular advice for you.

@commonsguy ok, thank you!

please @iskugor how did you import " androidx.room.util.DBUtil"
i tried many versions of:
implementation 'android.arch.persistence.room:runtime:1.0.0-beta2'     annotationProcessor 'android.arch.persistence.room:compiler:1.0.0-beta2'
Thank you


zoom fit all markers

I try zoomToBoundingBox() but not working for me

can you explain what you are looking for or provide an example?

thx ,i have fit it


Eiso's "Our Roadmap" on Landing Homepage typo

@dpordomingo

complex set of dependencies it really is. we envision every organization running

it really is. We

done