CUDNN_STATUS_NOT_INITIALIZED error with tensorflow-gpu 2.4.0-rc2 RTX3070 CUDA11.0 cudnn8.0.2 windows10 and pip

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): windows10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: --
TensorFlow installed from (source or binary): binary
TensorFlow version: 2.4.0rc2
Python version :3.8.5
Installed using virtualenv? pip? conda?: pip
Bazel version (if compiling from source): --
GCC/Compiler version (if compiling from source): --
CUDA/cuDNN version: CUDA11.0 cudnn-11.0-windows-x64-v<IP_ADDRESS>  graphics driver version: 457.30
GPU model and memory: RTX 3070 8G

Describe the problem
I install cuda cudnn following the requirement for TensorFlow-gpu 2.4.0-rc2 as shown in https://github.com/tensorflow/tensorflow/releases, but get cudnn initialization error. I have uninstalled and reinstalled for several times, and I also tried TensorFlow-gpu 2.4.0-rc1 several days ago, the same errer was raised.
Provide the exact sequence of commands / steps that you executed before running into the problem
The following is the code to re produce the error:
import tensorflow as tf
import numpy as np
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential

x = np.random.normal(size=(100, 28, 28, 1)).astype(np.float32)
y = np.zeros([100, 10], dtype=np.float32)
y[:, 1] = 1.

train_ds = tf.data.Dataset.from_tensor_slices((x, y)).shuffle(buffer_size=100).batch(32)
num_classes = 10

model = Sequential([
  layers.Conv2D(16, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(32, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation='relu'),
  layers.MaxPooling2D(),
  layers.Flatten(),
  layers.Dense(128, activation='relu'),
  layers.Dense(num_classes)
])
model.compile(optimizer='adam',
              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
epochs=10
history = model.fit(
  train_ds,
  epochs=epochs
)

Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
logs:
2020-11-20 11:38:38.097262: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-20 11:38:40.566842: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-20 11:38:40.568068: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2020-11-20 11:38:40.604223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-20 11:38:40.604439: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-20 11:38:40.618831: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-20 11:38:40.618938: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-20 11:38:40.622412: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-20 11:38:40.623498: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-20 11:38:40.631377: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-20 11:38:40.633911: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-20 11:38:40.634568: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-20 11:38:40.634734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-20 11:38:40.635316: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-20 11:38:40.636515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-20 11:38:40.637060: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-20 11:38:40.637713: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-20 11:38:40.638072: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-20 11:38:40.638277: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-20 11:38:40.638448: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-20 11:38:40.638724: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-20 11:38:40.638921: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-20 11:38:40.639189: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-20 11:38:40.639443: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-20 11:38:41.262522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-20 11:38:41.262635: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2020-11-20 11:38:41.262694: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2020-11-20 11:38:41.262900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6177 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:65:00.0, compute capability: 8.6)
2020-11-20 11:38:41.263873: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
Epoch 1/10
2020-11-20 11:38:42.052106: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2020-11-20 11:38:42.135350: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-20 11:38:42.994775: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-20 11:38:43.000109: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-20 11:38:43.875475: E tensorflow/stream_executor/cuda/cuda_dnn.cc:336] Could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED
2020-11-20 11:38:43.875620: E tensorflow/stream_executor/cuda/cuda_dnn.cc:340] Error retrieving driver version: Unimplemented: kernel reported driver version not implemented on Windows
2020-11-20 11:38:43.877456: E tensorflow/stream_executor/cuda/cuda_dnn.cc:336] Could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED
2020-11-20 11:38:43.877755: E tensorflow/stream_executor/cuda/cuda_dnn.cc:340] Error retrieving driver version: Unimplemented: kernel reported driver version not implemented on Windows
Traceback (most recent call last):
  File "F:/python_ws/helloworld/main.py", line 29, in <module>
    history = model.fit(
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1100, in fit
    tmp_logs = self.train_function(iterator)
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\eager\def_function.py", line 828, in __call__
    result = self._call(*args, **kwds)
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\eager\def_function.py", line 888, in _call
    return self._stateless_fn(*args, **kwds)
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\eager\function.py", line 2942, in __call__
    return graph_function._call_flat(
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\eager\function.py", line 1918, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\eager\function.py", line 555, in call
    outputs = execute.execute(
  File "C:\Users\dell\Anaconda3\envs\tf24rcpy38\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
	 [[node sequential/conv2d/Conv2D (defined at F:/python_ws/helloworld/main.py:29) ]] [Op:__inference_train_function_830]

Function call stack:
train_function

Process finished with exit code 1

Same issue here on linux
System information
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04 LTS
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: --
TensorFlow installed from (source or binary): binary
TensorFlow version: 2.4.0rc2
Python version :3.8.5
Installed using virtualenv? pip? conda?: pip
Bazel version (if compiling from source): --
GCC/Compiler version (if compiling from source): --
CUDA/cuDNN version: CUDA 11.1 - cudnn 8.0.5
GPU model and memory: RTX 3080

/usr/bin/python3.8 /home/noobzik/Documents/deep_learning_day2/test.py
2020-11-20 09:08:11.149682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-20 09:08:12.068627: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-20 09:08:12.069149: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2020-11-20 09:08:12.098515: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.099146: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1727] Found device 0 with properties:
pciBusID: 0000:08:00.0 name: GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 9.78GiB deviceMemoryBandwidth: 707.88GiB/s
2020-11-20 09:08:12.099167: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-20 09:08:12.101469: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-20 09:08:12.101524: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2020-11-20 09:08:12.102397: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-20 09:08:12.102612: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-20 09:08:12.105025: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2020-11-20 09:08:12.105664: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-20 09:08:12.105792: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-20 09:08:12.105901: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.106646: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.107247: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1869] Adding visible gpu devices: 0
2020-11-20 09:08:12.107522: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-20 09:08:12.108178: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-20 09:08:12.108256: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.108699: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1727] Found device 0 with properties:
pciBusID: 0000:08:00.0 name: GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 9.78GiB deviceMemoryBandwidth: 707.88GiB/s
2020-11-20 09:08:12.108716: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-20 09:08:12.108728: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-20 09:08:12.108736: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2020-11-20 09:08:12.108744: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2020-11-20 09:08:12.108752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2020-11-20 09:08:12.108760: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2020-11-20 09:08:12.108768: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2020-11-20 09:08:12.108785: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-20 09:08:12.108846: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.109331: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.109761: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1869] Adding visible gpu devices: 0
2020-11-20 09:08:12.109787: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2020-11-20 09:08:12.519303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-20 09:08:12.519333: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1273]      0
2020-11-20 09:08:12.519339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1286] 0:   N
2020-11-20 09:08:12.519512: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.520184: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.520627: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:941] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-20 09:08:12.521050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1413] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8306 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3080, pci bus id: 0000:08:00.0, compute capability: 8.6)
Epoch 1/10
2020-11-20 09:08:12.960040: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:127] None of the MLIR optimization passes are enabled (registered 2)
2020-11-20 09:08:12.962440: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency:<PHONE_NUMBER> Hz
2020-11-20 09:08:13.021345: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-20 09:08:13.564688: I tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Loaded cuDNN version 8005
2020-11-20 09:08:14.155647: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2020-11-20 09:08:14.198183: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output:
Relying on driver to perform ptx compilation.
Modify $PATH to customize ptxas location.
This message will be only logged once.
2020-11-20 09:08:15.458184: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-20 09:08:15.543012: E tensorflow/stream_executor/cuda/cuda_blas.cc:226] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2020-11-20 09:08:16.853135: E tensorflow/stream_executor/cuda/cuda_blas.cc:226] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2020-11-20 09:08:18.129453: E tensorflow/stream_executor/cuda/cuda_blas.cc:226] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2020-11-20 09:08:18.131488: E tensorflow/stream_executor/cuda/cuda_blas.cc:226] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2020-11-20 09:08:18.131506: W tensorflow/stream_executor/stream.cc:1455] attempting to perform BLAS operation using StreamExecutor without BLAS support
Traceback (most recent call last):
  File "/home/noobzik/Documents/deep_learning_day2/test.py", line 28, in <module>
    history = model.fit(
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1103, in fit
    tmp_logs = self.train_function(iterator)
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 784, in __call__
    result = self._call(*args, **kwds)
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 844, in _call
    return self._stateless_fn(*args, **kwds)
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2971, in __call__
    return graph_function._call_flat(
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1947, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 556, in call
    outputs = execute.execute(
  File "/home/noobzik/.local/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InternalError:  Blas xGEMM launch failed : a.shape=[1,32,576], b.shape=[1,576,128], m=32, n=128, k=576
	 [[node sequential/dense/MatMul (defined at /Documents/deep_learning_day2/test.py:28) ]] [Op:__inference_train_function_830]

Function call stack:
train_function

Process finished with exit code 1

I'm also experiencing similar issue for the following minimum example, with an RTX 3070.
import tensorflow as tf
import tensorflow.keras.layers as layers
import numpy as np

x = np.random.normal(size=(100, 28, 28, 1)).astype(np.float32)
y = np.zeros([100, 10], dtype=np.float32)
y[:, 1] = 1.

train_ds = tf.data.Dataset.from_tensor_slices((x, y)).shuffle(buffer_size=100).batch(32)
num_classes = 10

model = tf.keras.Sequential([
    layers.Conv2D(16, 3, padding='same', activation='relu'),
    layers.MaxPooling2D(),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dense(num_classes)
])
model.compile(optimizer='adam',
    loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
    metrics=['accuracy'])
epochs=10
history = model.fit(
    train_ds,
    epochs=epochs
)

Running the script above produces this error.
2020-11-23 16:25:25.510389: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7208 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:01:00.0, compute capability: 8.6)
Epoch 1/10
2020-11-23 16:25:25.887983: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2020-11-23 16:25:25.901960: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency:<PHONE_NUMBER> Hz
2020-11-23 16:25:25.948792: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2020-11-23 16:25:26.513272: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2020-11-23 16:25:26.515165: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2020-11-23 16:25:26.969194: E tensorflow/stream_executor/cuda/cuda_dnn.cc:336] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
2020-11-23 16:25:26.971093: E tensorflow/stream_executor/cuda/cuda_dnn.cc:336] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Traceback (most recent call last):
  File "problem.py", line 31, in <module>
    epochs=epochs
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1100, in fit
    tmp_logs = self.train_function(iterator)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 828, in __call__
    result = self._call(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 888, in _call
    return self._stateless_fn(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2943, in __call__
    filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1919, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 560, in call
    ctx=ctx)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.UnknownError:  Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
         [[node sequential/conv2d/Conv2D (defined at problem.py:31) ]] [Op:__inference_train_function_654]

Function call stack:
train_function

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04 in docker image nvidia/cuda:11.0-cudnn8-runtime-ubuntu18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: --
TensorFlow installed from (source or binary): binary
TensorFlow version: 2.4.0rc2
Python version: 3.6.9
Installed using virtualenv? pip? conda?: pip
Bazel version (if compiling from source): --
GCC/Compiler version (if compiling from source): --
CUDA/cuDNN version: CUDA 11.0
GPU model and memory: RTX 3070 8G

Add this to the start of your code
physical_devices = tf.config.list_physical_devices('GPU') tf.config.experimental.set_memory_growth(physical_devices[0], True). I tried the code on my system and without it this does not work. After adding the above code it works.

I am having the same issues, even with king398's fix.
Windows 10, RTX 3070

@Bchi1994 Please give your console log

console log below. Notebook stops at Epoch 1/10. First epoch doesn't even run.
(gpuenv) A:>jupyter notebook
[I 08:13:01.424 NotebookApp] Serving notebooks from local directory: A:
[I 08:13:01.425 NotebookApp] Jupyter Notebook 6.1.4 is running at:
[I 08:13:01.425 NotebookApp] http://localhost:8888/?token=adddde1ae77b99e0982784028d7e31c267f3d41d6354f105
[I 08:13:01.425 NotebookApp]  or http://<IP_ADDRESS>:8888/?token=adddde1ae77b99e0982784028d7e31c267f3d41d6354f105
[I 08:13:01.425 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 08:13:01.462 NotebookApp]
To access the notebook, open this file in a browser:
    file:///C:/Users/Marshall%20Proehl/AppData/Roaming/jupyter/runtime/nbserver-10076-open.html
Or copy and paste one of these URLs:
    http://localhost:8888/?token=adddde1ae77b99e0982784028d7e31c267f3d41d6354f105
 or http://<IP_ADDRESS>:8888/?token=adddde1ae77b99e0982784028d7e31c267f3d41d6354f105

[I 08:13:06.566 NotebookApp] Kernel started: d201c589-4410-4bfc-ad66-3bac93cc8f2f, name: gpuenv
[W 08:13:10.994 NotebookApp] 404 GET /api/kernels/b4f92c73-f90f-4443-ac78-0ab5136bbbb1/channels?session_id=d8c0cd6407c841a8aa3c1cf9936d9361 (::1): Kernel does not exist: b4f92c73-f90f-4443-ac78-0ab5136bbbb1
[W 08:13:11.000 NotebookApp] 404 GET /api/kernels/b4f92c73-f90f-4443-ac78-0ab5136bbbb1/channels?session_id=d8c0cd6407c841a8aa3c1cf9936d9361 (::1) 7.01ms referer=None
[I 08:13:11.972 NotebookApp] Starting buffering for d201c589-4410-4bfc-ad66-3bac93cc8f2f:b9933f06e0904b1584e397219767e459
[I 08:13:12.194 NotebookApp] Kernel restarted: d201c589-4410-4bfc-ad66-3bac93cc8f2f
[I 08:13:12.199 NotebookApp] Restoring connection for d201c589-4410-4bfc-ad66-3bac93cc8f2f:b9933f06e0904b1584e397219767e459
[I 08:13:12.199 NotebookApp] Replaying 3 buffered messages
2020-11-23 08:13:12.799496: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-23 08:13:14.586771: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-23 08:13:14.589935: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2020-11-23 08:13:14.620388: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-23 08:13:14.620488: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-23 08:13:14.632991: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-23 08:13:14.633056: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-23 08:13:14.636453: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-23 08:13:14.638195: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-23 08:13:14.644747: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-23 08:13:14.647026: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-23 08:13:14.647834: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-23 08:13:14.647933: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-23 08:13:15.067588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-23 08:13:15.067679: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2020-11-23 08:13:15.068727: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2020-11-23 08:13:15.069412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/device:GPU:0 with 6589 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:2d:00.0, compute capability: 8.6)
2020-11-23 08:13:15.070354: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-23 08:13:15.071065: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-23 08:13:15.071130: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-23 08:13:15.071531: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-23 08:13:15.071909: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-23 08:13:15.072277: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-23 08:13:15.072649: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-23 08:13:15.073014: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-23 08:13:15.073380: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-23 08:13:15.073750: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-23 08:13:15.074182: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-23 08:13:15.074481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-23 08:13:15.074705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2020-11-23 08:13:15.074946: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2020-11-23 08:13:15.075236: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/device:GPU:0 with 6589 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:2d:00.0, compute capability: 8.6)
2020-11-23 08:13:15.075489: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-23 08:13:15.079841: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-23 08:13:15.079965: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-23 08:13:15.080293: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-23 08:13:15.080605: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-23 08:13:15.080943: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-23 08:13:15.081528: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-23 08:13:15.081811: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-23 08:13:15.082098: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-23 08:13:15.082402: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-23 08:13:15.082682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-23 08:13:15.083073: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-23 08:13:17.409141: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:2d:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-23 08:13:17.409218: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-23 08:13:17.410830: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-23 08:13:17.410857: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-23 08:13:17.410880: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-23 08:13:17.411172: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-23 08:13:17.411400: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-23 08:13:17.411632: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-23 08:13:17.411867: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-23 08:13:17.412144: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-23 08:13:17.412365: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-23 08:13:17.412571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2020-11-23 08:13:17.412802: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2020-11-23 08:13:17.413107: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6589 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:2d:00.0, compute capability: 8.6)
2020-11-23 08:13:17.413283: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-23 08:13:18.066778: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2020-11-23 08:13:20.325576: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-23 08:13:20.838041: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-23 08:13:20.864096: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll

@Bchi1994 That is weird because  i also have a rtx 3070 and it works fine and the console log have nothing wrong. Can your try something other than jupyter notebook and something like vscode or pycharm.

Try this code:
`
import tensorflow as tf
import tensorflow.keras.layers as layers
import numpy as np
physical_devices = tf.config.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(physical_devices[0], True)
x = np.random.normal(size=(100, 28, 28, 1)).astype(np.float32)
y = np.zeros([100, 10], dtype=np.float32)
y[:, 1] = 1.
train_ds = tf.data.Dataset.from_tensor_slices((x, y)).shuffle(buffer_size=100).batch(32)
num_classes = 10
model = tf.keras.Sequential([
layers.Conv2D(16, 3, padding='same', activation='relu'),
layers.MaxPooling2D(),
layers.Flatten(),
layers.Dense(128, activation='relu'),
layers.Dense(num_classes)
])
model.compile(optimizer='adam',
loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
epochs=10
history = model.fit(
train_ds,
epochs=epochs
)
`

Please give me your code you are using
.

Just tried that code. Same issue.

Tensorflow version- 2.4.0rc2
CUDA - 11.1
Cuddn - 8.0.4

Try using tf.keras instead of keras

Keras - 2.3.1
Numpy - 1.19.3

Ok. Thank. I am going to try a fresh install of CUDA, Cudnn files and a new kernel

Wait. It just did something funky. Check this out. It is running on my GPU. Epochs continue to run, but slowly.

What funky  stuff you did

Also try reinstalling tensorflow with this
pip install tensorflow-gpu==2.4.0rc2

Ah found what you did wrong. You are using cuda 11.0 use cuda 11.1 instead

So it runs. First epoch takes forever. I will try CUDA 11.1

Still better than no training at all. Please share your results  with cuda 11.1 I think it will solve the issue.Alos if any error arise tell me because I also had many during my installation,

King, thanks for all of the help. I will report back with CUDA 11.1

King - can you please double check your keras version? It says Keras 2.3.1 is not compatible with python 3.8. It makes me default to 2.4.3

Add this to the start of your code
physical_devices = tf.config.list_physical_devices('GPU') tf.config.experimental.set_memory_growth(physical_devices[0], True). I tried the code on my system and without it this does not work. After adding the above code it works.

Yep, It works, But slowly...like with CPU...
Maybe we need to wait for TensorFlow 2.4.0 stable?..

@iMacroC What is your cuda version

@Bchi1994 the code works after adding my fix. Try this
`import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Embedding
from tensorflow.keras.layers import Flatten
from tensorflow.keras.callbacks import ModelCheckpoint
from keras.utils import np_utils
physical_devices = tf.config.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(physical_devices[0], True)
reshape X to be [samples, time steps, features]
#X = numpy.reshape(dataX, (n_patterns, seq_length, 1))
X = numpy.reshape(dataX, (n_patterns, seq_length, 1))
normalize
X = X / float(n_vocab)
one hot encode the output variable
y = np_utils.to_categorical(dataY)
define the LSTM model
model = Sequential()
model.add(LSTM(256, input_shape=(X.shape[1], X.shape[2]),return_sequences=True))
model.add(Dropout(0.25))
model.add(LSTM(128))
model.add(Dropout(0.25))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
define the checkpoint
filepath="weights\weights-improvement-{epoch:02d}-{loss:.4f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
history = model.fit(X, y, epochs=10, batch_size=128, callbacks=callbacks_list)`

@iMacroC What is your cuda version Because my 3070 is much faster than my cpu

thanks for your reply, my cuda version is 11.1 and cudnn 8.0.5(i also tried 8.0.4), and I tried tf-nightly latest.

@iMacroC update your driver version

@iMacroC update your driver version and please give your cuda version
my graphcard version is the latest. cuda version:11.1

driver version:457.30

King, the added code did no resolve the issue.

@iMacroC update your driver version and please give your cuda version

and the GPU always take up less than 10%

@iMacroC Well because  the tensor core usage does not show up in task manager. And please give your code.

Please open up a new issue where you can report this as bug.

@iMacroC Please downgrade your cudnn version 8.0.4

@iMacroC Well because the tensor core usage does not show up in task manager. And please give your code.

I'm sure that bug is nothing to do with any code Because It still likes that when I run another code.

@iMacroC Please downgrade your cudnn version 8.0.4

I'm trying.

@iMacroC Well because the tensor core usage does not show up in task manager. And please give your code.

Well, I think it works, I saw It took up 3.9GB of GPU memory, and I ran it with CPU that really much faster than CPU.
Maybe It works a few days ago but I didn't notice it, I'm dumb😂.
Thank you a lot, king have a nice day~

Add this to the start of your code
physical_devices = tf.config.list_physical_devices('GPU') tf.config.experimental.set_memory_growth(physical_devices[0], True). I tried the code on my system and without it this does not work. After adding the above code it works.

Thanks. After doing this, it works, but with a lot of " SubProcess ended with return code:<PHONE_NUMBER>" logs.
Here are the logs:
`C:\Users\dell\Anaconda3\envs\tf24rcpy38\python.exe F:/python_ws/helloworld/main.py
2020-11-24 11:00:38.194273: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 11:00:42.903949: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-24 11:00:42.913020: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2020-11-24 11:00:43.020016: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-24 11:00:43.020975: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 11:00:43.704812: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 11:00:43.705024: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 11:00:43.819171: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-24 11:00:43.882861: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-24 11:00:44.261355: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-24 11:00:44.553173: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-24 11:00:44.561022: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 11:00:44.561879: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-24 11:00:44.573103: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-24 11:00:44.575050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-24 11:00:44.575246: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 11:00:44.575382: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 11:00:44.575508: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 11:00:44.575607: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-24 11:00:44.575703: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-24 11:00:44.575798: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-24 11:00:44.575892: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-24 11:00:44.575989: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 11:00:44.576126: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-24 11:00:45.884370: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-24 11:00:45.884634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2020-11-24 11:00:45.884828: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2020-11-24 11:00:45.886725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6177 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:65:00.0, compute capability: 8.6)
2020-11-24 11:00:45.890140: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
Epoch 1/10
2020-11-24 11:00:46.609881: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2020-11-24 11:00:46.727182: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 11:00:48.641407: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 11:00:48.664931: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 11:00:55.465955: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0
2020-11-24 11:00:55.476935: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.477355: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code -1, output:
Relying on driver to perform ptx compilation.
Modify $PATH to customize ptxas location.
This message will be only logged once.
2020-11-24 11:00:55.495264: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.505659: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.513483: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.522493: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.529765: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.537654: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.544917: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.555125: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.562459: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.570028: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.578117: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.588121: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.596128: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.604730: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.612640: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.621622: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.629292: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.637466: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.646300: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.655732: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.663745: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.672191: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.681755: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.691945: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.700897: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.710924: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.719762: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.735948: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.744464: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.754029: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.762616: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.771835: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.779568: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.789210: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.796810: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.805953: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.813376: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.821732: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.829126: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.836992: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.845043: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.853761: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.862608: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.870928: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.880380: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.889693: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.898458: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.908249: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.916724: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.925245: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.933417: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.942290: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.952000: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.961089: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.969386: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.979448: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.986976: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:55.995521: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.003094: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.011507: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.018985: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.027062: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.034241: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.042440: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.050911: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.058736: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.066726: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.074963: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.082080: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.090430: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.099547: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.108371: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.116563: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.124680: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.133001: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.145863: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.153865: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.163631: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.171954: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.181588: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.190385: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.198608: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.206919: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:56.210145: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
2020-11-24 11:00:58.489630: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.499115: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.507912: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.515182: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.522723: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.530861: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.538688: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.545983: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.553683: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.560845: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.569419: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.576978: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.585068: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.593196: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.600828: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.608429: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.616191: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.623257: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.631061: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.638267: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.646361: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.653124: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.660391: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:00:58.667087: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.497692: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.506843: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.516525: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.525251: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.534312: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.543491: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.553514: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.562711: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.572227: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.580275: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.588705: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.596683: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.605453: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.613505: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.623202: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.631250: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.640166: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.649097: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.657448: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.664910: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.674268: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.682554: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.692150: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.701405: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.711509: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.720593: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.730733: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.739644: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.749101: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.757748: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.766046: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.774079: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.782077: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.790603: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.798497: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.806652: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.814703: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.822530: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.830704: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.839036: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.847247: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.854780: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.863119: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.871690: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.882237: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.890302: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.899966: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.909215: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.918679: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.926946: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.935564: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.944474: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.952801: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.960992: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.969896: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.977376: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.984752: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.991827: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:01.999606: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.007061: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.015008: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.022309: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.029951: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.036768: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.047201: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.054776: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.063287: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.070961: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.079560: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.087030: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.095929: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.103320: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.110892: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.118649: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.127237: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.134678: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.142608: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.149630: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.157365: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.164635: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.172751: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.179793: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.187403: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.194324: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
1/4 [======>.......................] - ETA: 48s - loss: 2.4265 - accuracy: 0.0000e+002020-11-24 11:01:02.215221: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.222204: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.230079: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.237134: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.245388: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.252375: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.259599: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.266760: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.274516: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.281775: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.290926: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.298924: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.308027: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.316889: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.327089: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.336705: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.346930: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.354811: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.363928: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.371999: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.380944: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.389260: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.397524: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.406995: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.415328: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.424967: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.433409: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.441608: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.450791: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.458621: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.467210: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.474306: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.482627: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.490891: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.499159: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.506927: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.515376: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.523803: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.531917: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.541327: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.551271: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.559517: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.567779: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.576012: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.584057: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.592555: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.600303: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.607562: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.615161: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.622574: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.629965: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.637044: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.644850: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.651838: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.659355: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.666934: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.675380: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.682316: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.690735: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.697730: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.704928: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.711762: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.719146: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.726072: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.734054: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.742744: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.751064: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.759569: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.768144: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.777125: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.786024: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.794497: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.803136: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.811530: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.820017: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.828733: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.836724: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.846180: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.854388: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.862594: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.871313: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.879708: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.888192: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.896674: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.908469: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.915937: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.924363: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.931642: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.940305: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.949159: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.958799: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.966439: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.975817: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.983740: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:02.992306: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.000855: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.009752: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.018124: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.026080: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.034827: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.043344: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.051412: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.059549: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.067828: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.075873: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.083383: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.091320: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.099204: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.108319: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.115734: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.124677: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.132423: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.140749: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.149201: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.158124: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.165651: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.173691: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.182812: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.192249: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.200852: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.209406: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.217653: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.226868: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.235147: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.243960: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.251897: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.260091: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.269050: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.278059: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.285698: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.295152: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.302871: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.311629: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.319470: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.328460: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.337835: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.346620: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.357059: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.367298: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.376655: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.385568: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.395059: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.404620: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.413070: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.421923: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.430290: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.438610: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.446763: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.454767: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.462421: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.470958: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.479367: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.488132: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.496728: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.506079: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.513635: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.522656: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.530445: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.537999: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.546228: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.554638: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.564040: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.572561: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.580943: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.589490: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.597075: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.605639: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.613388: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.621423: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.629280: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.637207: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.645053: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.654938: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.663969: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.673375: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.681404: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.689680: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.757484: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.843839: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.879757: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.895875: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.909203: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.919332: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.928785: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.937421: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.945068: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.953275: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.960997: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.969469: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.980766: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.988775: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 11:01:03.996054: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
4/4 [==============================] - 18s 601ms/step - loss: 1.8508 - accuracy: 0.5053
Epoch 2/10
4/4 [==============================] - 0s 5ms/step - loss: 0.0721 - accuracy: 1.0000
Epoch 3/10
4/4 [==============================] - 0s 5ms/step - loss: 4.0893e-05 - accuracy: 1.0000
Epoch 4/10
4/4 [==============================] - 0s 6ms/step - loss: 3.6856e-09 - accuracy: 1.0000
Epoch 5/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 6/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 7/10
4/4 [==============================] - 0s 5ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 8/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 9/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 10/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Process finished with exit code 0
`

@SilenceEagle Well at least it works nonetheless

@iMacroC Well because the tensor core usage does not show up in task manager. And please give your code.

Well, I think it works, I saw It took up 3.9GB of GPU memory, and I ran it with CPU that really much faster than CPU.
Maybe It works a few days ago but I didn't notice it, I'm dumb😂.
Thank you a lot, king have a nice day~

Haha glad it works. installing tensor flow is harder than using it

@iMacroC Well because the tensor core usage does not show up in task manager. And please give your code.

Well, I think it works, I saw It took up 3.9GB of GPU memory, and I ran it with CPU that really much faster than CPU.
Maybe It works a few days ago but I didn't notice it, I'm dumb😂.
Thank you a lot, king have a nice day~

Haha glad it works. installing tensor flow is harder than using it

ty~

@SilenceEagle Well at least it works nonetheless. Also the reason for it is that you are using cuda 11.0 use 11.1 instead.

thanks again

@king398, what cudatoolkit version do you have when you conda list?

@Bchi1994 i am using pip

@king398, what cudatoolkit version do you have when you pip list?

@Bchi1994 cuda 11.1 and cudnn 8.0.4

@king398, thanks! I created a new env and installed everything using pip. Now getting this error on your simple code. Numpy versuib = 1.19.2

@Bchi1994 hope I helped you

@king398  I update to cuda11.1 and cudnn 8.0.4 on win10, then re run the code , then get ERROR: could not load dynamic library 'cusolver64_10.dll'; dlerror: cusolver64_10.dll not found, here are the logs:
C:\Users\dell\Anaconda3\envs\tf24rc1py36\python.exe F:/python_ws/helloworld/main.py
2020-11-24 15:31:41.471855: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 15:31:43.575381: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-24 15:31:43.575947: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2020-11-24 15:31:43.604386: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-24 15:31:43.604578: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 15:31:43.619126: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 15:31:43.619306: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 15:31:43.622735: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-24 15:31:43.623815: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-24 15:31:43.624574: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cusolver64_10.dll'; dlerror: cusolver64_10.dll not found
2020-11-24 15:31:43.627545: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-24 15:31:43.628190: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 15:31:43.628767: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1757] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
Traceback (most recent call last):
  File "F:/python_ws/helloworld/main.py", line 7, in <module>
    tf.config.experimental.set_memory_growth(physical_devices[0], True)
IndexError: list index out of range

Process finished with exit code 1

Then I follow the suggest from https://github.com/tensorflow/tensorflow/issues/44291#issuecomment-716165602 to copy the cusolver64_10.dll from 11.0\bin to 11.1\bin. It works, but also raises a lot of SubProcess ended with return code:<PHONE_NUMBER> with cuda11.1:
C:\Users\dell\Anaconda3\envs\tf24rc1py36\python.exe F:/python_ws/helloworld/main.py
2020-11-24 15:40:49.251398: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 15:40:51.479650: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-24 15:40:51.480425: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2020-11-24 15:40:51.517736: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-24 15:40:51.517929: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 15:40:51.539764: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 15:40:51.540060: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 15:40:51.544531: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-24 15:40:51.545737: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-24 15:40:51.550261: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-24 15:40:51.553451: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-24 15:40:51.554099: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 15:40:51.554297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-24 15:40:51.558503: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-24 15:40:51.559314: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties:
pciBusID: 0000:65:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.815GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-24 15:40:51.559535: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-24 15:40:51.559632: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 15:40:51.559726: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 15:40:51.559827: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-24 15:40:51.559920: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-24 15:40:51.560022: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-24 15:40:51.560117: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-24 15:40:51.560214: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 15:40:51.560372: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2020-11-24 15:40:52.181967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-24 15:40:52.182085: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0
2020-11-24 15:40:52.182148: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N
2020-11-24 15:40:52.182387: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6177 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:65:00.0, compute capability: 8.6)
2020-11-24 15:40:52.183284: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
Epoch 1/10
2020-11-24 15:40:52.605971: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2020-11-24 15:40:52.674317: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-24 15:40:53.621868: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-24 15:40:53.626698: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-24 15:40:55.528792: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0

2020-11-24 15:40:55.540214: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>

2020-11-24 15:40:55.540714: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code -1, output:
Relying on driver to perform ptx compilation.
Modify $PATH to customize ptxas location.
This message will be only logged once.
2020-11-24 15:40:55.553723: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
2020-11-24 15:40:55.565792: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
...(lots of )
2020-11-24 15:40:56.419765: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
Epoch 1/10
1/4 [======>.......................] - ETA: 16s - loss: 2.4413 - accuracy: 0.0000e+002020-11-24 15:40:57.710428: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code:<PHONE_NUMBER>
4/4 [==============================] - 7s 603ms/step - loss: 1.7298 - accuracy: 0.5053
Epoch 2/10
4/4 [==============================] - 0s 7ms/step - loss: 0.0242 - accuracy: 1.0000
Epoch 3/10
4/4 [==============================] - 0s 8ms/step - loss: 2.5926e-05 - accuracy: 1.0000
Epoch 4/10
4/4 [==============================] - 0s 8ms/step - loss: 1.4742e-08 - accuracy: 1.0000
Epoch 5/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 6/10
4/4 [==============================] - 0s 5ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 7/10
4/4 [==============================] - 0s 5ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 8/10
4/4 [==============================] - 0s 6ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 9/10
4/4 [==============================] - 0s 7ms/step - loss: 0.0000e+00 - accuracy: 1.0000
Epoch 10/10
4/4 [==============================] - 0s 8ms/step - loss: 0.0000e+00 - accuracy: 1.0000

Process finished with exit code 0

It seems the SubProcess ended with return code:<PHONE_NUMBER> couldn't be solved by updating to cuda11.1.
However, the code works, so my primary issue has been solved. Thanks again!

@SilenceEagle No problem. You can raise a issue for that

No problem. You can raise a issue for that
hey king need help same issue 3070 not picking up  by the tensorflow

@rahat14  what problems does it give

cusolver64-10.dll not found but I have Cuda 11.1 and cudnn lastest one

@rahat14 see this https://github.com/tensorflow/tensorflow/issues/44159

@king398 hi king, i met the same issue. my environment is: cuda 11.1, nvidia-driver: 457.51, cudnn: 8.0.4 , os:win10. tensorflow-gpu:2.4.0.

thanks sooo much if you could help.
i have tried cudnn8.0.5, the save issue.

Add this to the start of your code
physical_devices = tf.config.list_physical_devices('GPU') tf.config.experimental.set_memory_growth(physical_devices[0], True). I tried the code on my system and without it this does not work. After adding the above code it works.

@king398 Hi King, the two lines above solved for me too! Thanks!!
My setup:
tensorflow 2.4.0
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 452.56       Driver Version: 452.56       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.relgpu_drvr445TC445_37.28845127_0

@king398 Hi King, the two lines above solved for me too! Thanks!!
My setup:
tensorflow 2.4.0
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 452.56       Driver Version: 452.56       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.relgpu_drvr445TC445_37.28845127_0

@mschio No problem

@mschio No problem