RuntimeError: CUDA environment is not correctly set up

I'm trying to run 'lda2vec' which calls 'chainer' on my AWS g2.2xlarge Ubuntu 14.04 LTS instance.
I'm getting  this error:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-4-bccd002b2788> in <module>()
     20
     21 gpu_id = int(os.getenv('CUDA_GPU', 0))
---> 22 cuda.get_device(gpu_id).use()
     23 print "Using GPU " + str(gpu_id)
     24

/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/chainer/cuda.pyc in get_device(*args)
    220     for arg in args:
    221         if type(arg) in _integer_types:
--> 222             check_cuda_available()
    223             return Device(arg)
    224         if isinstance(arg, ndarray):

/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/chainer/cuda.pyc in check_cuda_available()
     85                '(see https://github.com/pfnet/chainer#installation).')
     86         msg += str(_resolution_error)
---> 87         raise RuntimeError(msg)
     88     if (not cudnn_enabled and
     89             not _cudnn_disabled_by_user and

RuntimeError: CUDA environment is not correctly set up
(see https://github.com/pfnet/chainer#installation).CuPy is not correctly installed. Please check your environment, uninstall Chainer and reinstall it with `pip install chainer --no-cache-dir -vvvv`.

original error: libcublas.so.7.0: cannot open shared object file: No such file or directory

I installed 'chainer' from source:
In [9]: import chainer

In [10]: chainer.__version__
Out[10]: '1.22.0'

Tensorflow runs and utilizes the GPU (so we know CUDA 7.5 is working):
In [1]: import tensorflow
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.7.5 locally

NVIDIA graphics driver: nvidia-graphics-drivers-367_367.57.orig.tar
cuda 7.5: cudnn-7.5-linux-x64-v5.1.tgz
ubuntu@ip-10-0-1-127:~$ ls -l /usr/local/cuda-7.0/lib64/libcublas.so.7.0*
lrwxrwxrwx 1 root root       19 Mar 27 16:01 /usr/local/cuda-7.0/lib64/libcublas.so.7.0 -> libcublas.so.7.0.28
-rwxr-xr-x 1 root root 31160168 Mar 27 16:02 /usr/local/cuda-7.0/lib64/libcublas.so.7.0.28

ubuntu@ip-10-0-1-127:~$ ls -l /usr/local/cuda-7.0/lib64/
total 723244
-rw-r--r-- 1 root root 26032916 Mar 27 16:01 libcublas_device.a
lrwxrwxrwx 1 root root       16 Mar 27 16:01 libcublas.so -> libcublas.so.7.0
lrwxrwxrwx 1 root root       19 Mar 27 16:01 libcublas.so.7.0 -> libcublas.so.7.0.28
-rwxr-xr-x 1 root root 31160168 Mar 27 16:02 libcublas.so.7.0.28
-rw-r--r-- 1 root root 35269768 Mar 27 16:02 libcublas_static.a
-rw-r--r-- 1 root root   310328 Mar 27 16:01 libcudadevrt.a
lrwxrwxrwx 1 root root       16 Mar 27 16:02 libcudart.so -> libcudart.so.7.0
lrwxrwxrwx 1 root root       19 Mar 27 16:01 libcudart.so.7.0 -> libcudart.so.7.0.28
-rwxr-xr-x 1 root root   377896 Mar 27 16:01 libcudart.so.7.0.28
-rw-r--r-- 1 root root   708938 Mar 27 16:02 libcudart_static.a
lrwxrwxrwx 1 root root       15 Mar 27 16:01 libcufft.so -> libcufft.so.7.0
lrwxrwxrwx 1 root root       18 Mar 27 16:01 libcufft.so.7.0 -> libcufft.so.7.0.28
-rwxr-xr-x 1 root root 62554648 Mar 27 16:01 libcufft.so.7.0.28
-rw-r--r-- 1 root root 91554528 Mar 27 16:02 libcufft_static.a
lrwxrwxrwx 1 root root       16 Mar 27 16:02 libcufftw.so -> libcufftw.so.7.0
lrwxrwxrwx 1 root root       19 Mar 27 16:02 libcufftw.so.7.0 -> libcufftw.so.7.0.28
-rwxr-xr-x 1 root root   443648 Mar 27 16:01 libcufftw.so.7.0.28
-rw-r--r-- 1 root root    44288 Mar 27 16:01 libcufftw_static.a
lrwxrwxrwx 1 root root       17 Mar 27 16:01 libcuinj64.so -> libcuinj64.so.7.0
lrwxrwxrwx 1 root root       20 Mar 27 16:01 libcuinj64.so.7.0 -> libcuinj64.so.7.0.28
-rwxr-xr-x 1 root root  5276064 Mar 27 16:01 libcuinj64.so.7.0.28
-rw-r--r-- 1 root root  1649726 Mar 27 16:01 libculibos.a
lrwxrwxrwx 1 root root       16 Mar 27 16:01 libcurand.so -> libcurand.so.7.0
lrwxrwxrwx 1 root root       19 Mar 27 16:02 libcurand.so.7.0 -> libcurand.so.7.0.28
-rwxr-xr-x 1 root root 51746464 Mar 27 16:01 libcurand.so.7.0.28
-rw-r--r-- 1 root root 51981092 Mar 27 16:02 libcurand_static.a
lrwxrwxrwx 1 root root       18 Mar 27 16:01 libcusolver.so -> libcusolver.so.7.0
lrwxrwxrwx 1 root root       21 Mar 27 16:01 libcusolver.so.7.0 -> libcusolver.so.7.0.28
-rwxr-xr-x 1 root root 41170544 Mar 27 16:01 libcusolver.so.7.0.28
-rw-r--r-- 1 root root 17900430 Mar 27 16:01 libcusolver_static.a
lrwxrwxrwx 1 root root       18 Mar 27 16:01 libcusparse.so -> libcusparse.so.7.0
lrwxrwxrwx 1 root root       21 Mar 27 16:01 libcusparse.so.7.0 -> libcusparse.so.7.0.28
-rwxr-xr-x 1 root root 43486576 Mar 27 16:01 libcusparse.so.7.0.28
-rw-r--r-- 1 root root 51094980 Mar 27 16:01 libcusparse_static.a
lrwxrwxrwx 1 root root       14 Mar 27 16:01 libnppc.so -> libnppc.so.7.0
lrwxrwxrwx 1 root root       17 Mar 27 16:02 libnppc.so.7.0 -> libnppc.so.7.0.28
-rwxr-xr-x 1 root root   418944 Mar 27 16:02 libnppc.so.7.0.28
-rw-r--r-- 1 root root    20568 Mar 27 16:01 libnppc_static.a
lrwxrwxrwx 1 root root       14 Mar 27 16:02 libnppi.so -> libnppi.so.7.0
lrwxrwxrwx 1 root root       17 Mar 27 16:02 libnppi.so.7.0 -> libnppi.so.7.0.28
-rwxr-xr-x 1 root root 70595696 Mar 27 16:01 libnppi.so.7.0.28
-rw-r--r-- 1 root root 99608960 Mar 27 16:01 libnppi_static.a
lrwxrwxrwx 1 root root       14 Mar 27 16:02 libnpps.so -> libnpps.so.7.0
lrwxrwxrwx 1 root root       17 Mar 27 16:01 libnpps.so.7.0 -> libnpps.so.7.0.28
-rwxr-xr-x 1 root root  5851016 Mar 27 16:01 libnpps.so.7.0.28
-rw-r--r-- 1 root root  8461132 Mar 27 16:02 libnpps_static.a
lrwxrwxrwx 1 root root       16 Mar 27 16:02 libnvblas.so -> libnvblas.so.7.0
lrwxrwxrwx 1 root root       19 Mar 27 16:02 libnvblas.so.7.0 -> libnvblas.so.7.0.28
-rwxr-xr-x 1 root root   451984 Mar 27 16:01 libnvblas.so.7.0.28
lrwxrwxrwx 1 root root       24 Mar 27 16:01 libnvrtc-builtins.so -> libnvrtc-builtins.so.7.0
lrwxrwxrwx 1 root root       27 Mar 27 16:01 libnvrtc-builtins.so.7.0 -> libnvrtc-builtins.so.7.0.28
-rwxr-xr-x 1 root root 24079328 Mar 27 16:01 libnvrtc-builtins.so.7.0.28
lrwxrwxrwx 1 root root       15 Mar 27 16:01 libnvrtc.so -> libnvrtc.so.7.0
lrwxrwxrwx 1 root root       18 Mar 27 16:01 libnvrtc.so.7.0 -> libnvrtc.so.7.0.27
-rwxr-xr-x 1 root root 18179472 Mar 27 16:01 libnvrtc.so.7.0.27
lrwxrwxrwx 1 root root       18 Mar 27 16:02 libnvToolsExt.so -> libnvToolsExt.so.1
lrwxrwxrwx 1 root root       22 Mar 27 16:01 libnvToolsExt.so.1 -> libnvToolsExt.so.1.0.0
-rwxr-xr-x 1 root root    37568 Mar 27 16:01 libnvToolsExt.so.1.0.0
-rw-r--r-- 1 root root    25840 Mar 27 16:02 libOpenCL.so
drwxr-xr-x 2 root root     4096 Mar 27 16:02 stubs
ubuntu@ip-10-0-1-127:~$

Progress!  I ran:
$ sudo ldconfig /usr/local/cuda-7.0/lib64

Using GPU 0

Is this going to be a problem?

ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check your nvcc installation and try again.
OSError                                   Traceback (most recent call last)
 in ()
102         t0 = time.time()
103         optimizer.zero_grads()
--> 104         l = model.fit_partial(s.copy(), a.copy(), f.copy())
105         prior = model.prior()
106         loss = prior * fraction
/hdfs/lda2vec/examples/hacker_news/lda2vec/lda2vec_model.pyc in fit_partial(self, rsty_ids, raut_ids, rwrd_ids, window)
40         sty_ids, aut_ids, wrd_ids = move(self.xp, rsty_ids, raut_ids, rwrd_ids)
41         pivot_idx = next(move(self.xp, rwrd_ids[window: -window]))
---> 42         pivot = F.embed_id(pivot_idx, self.sampler.W)
43         sty_at_pivot = rsty_ids[window: -window]
44         aut_at_pivot = raut_ids[window: -window]
/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/chainer/functions/connection/embed_id.pyc in embed_id(x, W, ignore_label)
109
110     """
--> 111     return EmbedIDFunction(ignore_label=ignore_label)(x, W)
/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/chainer/function.pyc in call(self, *inputs)
197         # Forward prop
198         with cuda.get_device(*in_data):
--> 199             outputs = self.forward(in_data)
200             assert type(outputs) == tuple
201         for hook in six.itervalues(hooks):
/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/chainer/functions/connection/embed_id.pyc in forward(self, inputs)
47                 mask[..., None], 0, W.take(xp.where(mask, 0, x), axis=0)),
48
---> 49         return W.take(x, axis=0),
50
51     def backward(self, inputs, grad_outputs):
cupy/core/core.pyx in cupy.core.core.ndarray.take (cupy/core/core.cpp:13176)()
cupy/core/core.pyx in cupy.core.core.ndarray.take (cupy/core/core.cpp:13065)()
cupy/core/core.pyx in cupy.core.core._take (cupy/core/core.cpp:67356)()
cupy/core/elementwise.pxi in cupy.core.core.ElementwiseKernel.call (cupy/core/core.cpp:42995)()
cupy/util.pyx in cupy.util.memoize.decorator.ret (cupy/util.cpp:1471)()
cupy/core/elementwise.pxi in cupy.core.core._get_elementwise_kernel (cupy/core/core.cpp:41341)()
cupy/core/elementwise.pxi in cupy.core.core._get_simple_elementwise_kernel (cupy/core/core.cpp:33972)()
cupy/core/elementwise.pxi in cupy.core.core._get_simple_elementwise_kernel (cupy/core/core.cpp:33794)()
cupy/core/carray.pxi in cupy.core.core.compile_with_cache (cupy/core/core.cpp:33449)()
/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/cupy/cuda/compiler.pyc in compile_with_cache(source, options, arch, cache_dir)
123             options += '-m32',
124
--> 125     env = (arch, options, _get_nvcc_version())
126     if '#include' in source:
127         pp_src = '%s %s' % (env, preprocess(source, options))
/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/cupy/cuda/compiler.pyc in _get_nvcc_version()
20     if _nvcc_version is None:
21         cmd = ['nvcc', '--version']
---> 22         _nvcc_version = _run_nvcc(cmd, '.')
23
24     return _nvcc_version
/home/ubuntu/anaconda/lib/python2.7/site-packages/chainer-1.22.0-py2.7-linux-x86_64.egg/cupy/cuda/compiler.pyc in _run_nvcc(cmd, cwd)
59               'Check PATH environment variable: '
60               + str(e)
---> 61         raise OSError(msg)
62
63
OSError: Failed to run nvcc command. Check PATH environment variable: [Errno 2] No such file or directory
In [5]:

I tried installing the cuda-toolkit to to get nvcc.
$ sudo apt-get install nvidia-cuda-toolkit
It installed the 375 version of the driver and broke CUDA.  I need the 376 version.
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: Unable to get the number of gpus available: unknown error)

How did you install CUDA? Chainer uses nvcc command, that is a compiler for CUDA environment and is provided by CUDA. You need to set PATH environment correctly.
If you are using amazon linux AMI provided by NVIDIA, nvcc is already in PATH. And, all you need to do is pip install chainer.
If you don't use it and install CUDA manually, I recommend you to install CUDA from deb package. And, you need to add /usr/local/cuda/bin to PATH environment variable. My fabfile may helps you: https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py

I’m using my own AMI.  I’ve been having trouble getting the GPU on the g2.2xlarge instance to work.
I had to use the 367 version of the NVIDIA driver to get TensorFlow to work.
nvidia-graphics-drivers-367_367.57.orig.tar
Unfortunately, this package did not install ‘nvcc’.  Other attempts to install CUDA installed the 375 version of the driver which doesn’t work
on the g2.2xlarge instance.
I’m trying this now:
cuda_7.5.18_linux.run
from:
https://developer.nvidia.com/cuda-75-downloads-archive

On Apr 9, 2017, at 12:13 AM, Yuya Unno<EMAIL_ADDRESS>wrote:
How did you install CUDA? Chainer uses nvcc command, that is a compiler for CUDA environment and is provided by CUDA. You need to set PATH environment correctly.
If you are using amazon linux AMI provided by NVIDIA, nvcc is already in PATH. And, all you need to do is pip install chainer.
If you don't use it and install CUDA manually, I recommend you to install CUDA from deb package. And, you need to add /usr/local/cuda/bin to PATH environment variable. My fabfile may helps you: https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-292768993, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2zplWQK-tP2vhBtafTWSJk03Ckouks5ruIUvgaJpZM4M3z8a.

Installation failed:
The NVIDIA CUDA Toolkit provides command-line and graphical
Do you accept the previously read EULA? (accept/decline/quit): accept
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): y
Do you want to install the OpenGL libraries? ((y)es/(n)o/(q)uit) [ default is yes ]: n
Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]: y
Toolkit location must be an absolute path.
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]:
Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/ubuntu ]:
Installing the NVIDIA display driver...
The driver installation has failed due to an unknown error. Please consult the driver installation log located at /var/log/nvidia-installer.log.
===========
= Summary =
Driver:   Installation Failed
Toolkit:  Installation skipped
Samples:  Installation skipped
Logfile is /tmp/cuda_install_14884.log
ubuntu@ip-10-0-1-189:~$
ERROR: An NVIDIA kernel module 'nvidia' appears to already be loaded in your kernel.  This may be because it is in use (for example, by an X server, a CUDA program, or the NVIDIA Persistence Daemon), but this may also happen if your kernel was configured without support for module unloading.  Please be sure to exit any programs that may be using the GPU(s) before attempting to upgrade your driver.  If no GPU-based programs are running, you know that your kernel supports module unloading, and you still receive this message, then an error may have occured that has corrupted an NVIDIA kernel module's usage count, for which the simplest remedy is to reboot your computer.
ERROR: Installation has failed.  Please see the file '/var/log/nvidia-installer.log' for details.  You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com.
18,1          Bot
ubuntu@ip-10-0-1-189:~$ lsmod | grep nvidia
nvidia_drm             14412  0
nvidia_modeset        764387  1 nvidia_drm
nvidia              11488748  1 nvidia_modeset
drm                   303102  4 ttm,drm_kms_helper,cirrus,nvidia_drm
ubuntu@ip-10-0-1-189:~$ sudo apt-get --purge remove nvidia*
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nvidia-graphics-drivers
E: Unable to locate package nvidia-graphics-drivers-367_367.57.orig.tar
E: Couldn't find any package by regex 'nvidia-graphics-drivers-367_367.57.orig.tar'

On Apr 9, 2017, at 8:39 AM, David Laxer<EMAIL_ADDRESS>wrote:
I’m using my own AMI.  I’ve been having trouble getting the GPU on the g2.2xlarge instance to work.
I had to use the 367 version of the NVIDIA driver to get TensorFlow to work.
nvidia-graphics-drivers-367_367.57.orig.tar
Unfortunately, this package did not install ‘nvcc’.  Other attempts to install CUDA installed the 375 version of the driver which doesn’t work
on the g2.2xlarge instance.
I’m trying this now:
cuda_7.5.18_linux.run
from:
https://developer.nvidia.com/cuda-75-downloads-archive https://developer.nvidia.com/cuda-75-downloads-archive

On Apr 9, 2017, at 12:13 AM, Yuya Unno<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
How did you install CUDA? Chainer uses nvcc command, that is a compiler for CUDA environment and is provided by CUDA. You need to set PATH environment correctly.
If you are using amazon linux AMI provided by NVIDIA, nvcc is already in PATH. And, all you need to do is pip install chainer.
If you don't use it and install CUDA manually, I recommend you to install CUDA from deb package. And, you need to add /usr/local/cuda/bin to PATH environment variable. My fabfile may helps you: https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-292768993, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2zplWQK-tP2vhBtafTWSJk03Ckouks5ruIUvgaJpZM4M3z8a.

This is the history of the issue:
http://stackoverflow.com/questions/42984743/nvidia-smi-has-failed-because-it-couldnt-communicate-with-the-nvidia-driver

On Apr 9, 2017, at 9:04 AM, David Laxer<EMAIL_ADDRESS>wrote:
Installation failed:
The NVIDIA CUDA Toolkit provides command-line and graphical
Do you accept the previously read EULA? (accept/decline/quit): accept
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): y
Do you want to install the OpenGL libraries? ((y)es/(n)o/(q)uit) [ default is yes ]: n
Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]: y
Toolkit location must be an absolute path.
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]:
Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/ubuntu ]:
Installing the NVIDIA display driver...
The driver installation has failed due to an unknown error. Please consult the driver installation log located at /var/log/nvidia-installer.log.
===========
= Summary =
Driver:   Installation Failed
Toolkit:  Installation skipped
Samples:  Installation skipped
Logfile is /tmp/cuda_install_14884.log
ubuntu@ip-10-0-1-189:~$
ERROR: An NVIDIA kernel module 'nvidia' appears to already be loaded in your kernel.  This may be because it is in use (for example, by an X server, a CUDA program, or the NVIDIA Persistence Daemon), but this may also happen if your kernel was configured without support for module unloading.  Please be sure to exit any programs that may be using the GPU(s) before attempting to upgrade your driver.  If no GPU-based programs are running, you know that your kernel supports module unloading, and you still receive this message, then an error may have occured that has corrupted an NVIDIA kernel module's usage count, for which the simplest remedy is to reboot your computer.
ERROR: Installation has failed.  Please see the file '/var/log/nvidia-installer.log' for details.  You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com http://www.nvidia.com/.
18,1          Bot
ubuntu@ip-10-0-1-189:~$ lsmod | grep nvidia
nvidia_drm             14412  0
nvidia_modeset        764387  1 nvidia_drm
nvidia              11488748  1 nvidia_modeset
drm                   303102  4 ttm,drm_kms_helper,cirrus,nvidia_drm
ubuntu@ip-10-0-1-189:~$ sudo apt-get --purge remove nvidia*
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nvidia-graphics-drivers
E: Unable to locate package nvidia-graphics-drivers-367_367.57.orig.tar
E: Couldn't find any package by regex 'nvidia-graphics-drivers-367_367.57.orig.tar'

On Apr 9, 2017, at 8:39 AM, David Laxer<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
I’m using my own AMI.  I’ve been having trouble getting the GPU on the g2.2xlarge instance to work.
I had to use the 367 version of the NVIDIA driver to get TensorFlow to work.
nvidia-graphics-drivers-367_367.57.orig.tar
Unfortunately, this package did not install ‘nvcc’.  Other attempts to install CUDA installed the 375 version of the driver which doesn’t work
on the g2.2xlarge instance.
I’m trying this now:
cuda_7.5.18_linux.run
from:
https://developer.nvidia.com/cuda-75-downloads-archive https://developer.nvidia.com/cuda-75-downloads-archive

On Apr 9, 2017, at 12:13 AM, Yuya Unno<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
How did you install CUDA? Chainer uses nvcc command, that is a compiler for CUDA environment and is provided by CUDA. You need to set PATH environment correctly.
If you are using amazon linux AMI provided by NVIDIA, nvcc is already in PATH. And, all you need to do is pip install chainer.
If you don't use it and install CUDA manually, I recommend you to install CUDA from deb package. And, you need to add /usr/local/cuda/bin to PATH environment variable. My fabfile may helps you: https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-292768993, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2zplWQK-tP2vhBtafTWSJk03Ckouks5ruIUvgaJpZM4M3z8a.

Tensorflow still works with the GPU.  Any suggestions on how to get only the toolkit with ‘nvcc’?
E.g.
ubuntu@ip-10-0-1-189:~$ nvidia-smi
Sun Apr  9 16:23:03 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.57                 Driver Version: 367.57                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID K520           Off  | 0000:00:03.0     Off |                  N/A |
| N/A   33C    P0    41W / 125W |   3742MiB /  4036MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      3222    C   python                                        3740MiB |
+-----------------------------------------------------------------------------+
ubuntu@ip-10-0-1-189:~$
ubuntu@ip-10-0-1-189:~/models/neural_gpu$ python neural_gpu_trainer.py --problem=bmul
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.7.5 locally
Error appending to /tmp/neural_gpu/log
Creating checkpoint directory /tmp/neural_gpu.
Generating data for bmul.
cut 1.20 lr 0.100 iw 0.80 cr 0.30 nm 64 d0.1000 gn 4.00 layers 2 kw 3 h 4 kh 3 batch 32 noise 0.00
Creating model.
Creating backward pass for the model.
WARNING:tensorflow:Tried to colocate gpu0/gradients/gpu0/Gather_2_grad/Shape with an op target_embedding/read that had a different device: /device:GPU:0 vs /device:CPU:0. Ignoring colocation property.
Created model for gpu 0 in 6.29 s.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GRID K520
major: 3 minor: 0 memoryClockRate (GHz) 0.797
pciBusID 0000:00:03.0
Total memory: 3.94GiB
Free memory: 3.91GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GRID K520, pci bus id: 0000:00:03.0)
Created model. Checkpoint dir /tmp/neural_gpu
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:247] PoolAllocator: After 2495 get requests, put_count=2461 evicted_count=1000 eviction_rate=0.406339 and unsatisfied allocation rate=0.454509
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:259] Raising pool_size_limit_ from 100 to 110
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:247] PoolAllocator: After 6023 get requests, put_count=4617 evicted_count=1000 eviction_rate=0.216591 and unsatisfied allocation rate=0.403287
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:259] Raising pool_size_limit_ from 256 to 281

On Apr 9, 2017, at 9:04 AM, David Laxer<EMAIL_ADDRESS>wrote:
Installation failed:
The NVIDIA CUDA Toolkit provides command-line and graphical
Do you accept the previously read EULA? (accept/decline/quit): accept
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): y
Do you want to install the OpenGL libraries? ((y)es/(n)o/(q)uit) [ default is yes ]: n
Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]: y
Toolkit location must be an absolute path.
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]:
Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/ubuntu ]:
Installing the NVIDIA display driver...
The driver installation has failed due to an unknown error. Please consult the driver installation log located at /var/log/nvidia-installer.log.
===========
= Summary =
Driver:   Installation Failed
Toolkit:  Installation skipped
Samples:  Installation skipped
Logfile is /tmp/cuda_install_14884.log
ubuntu@ip-10-0-1-189:~$
ERROR: An NVIDIA kernel module 'nvidia' appears to already be loaded in your kernel.  This may be because it is in use (for example, by an X server, a CUDA program, or the NVIDIA Persistence Daemon), but this may also happen if your kernel was configured without support for module unloading.  Please be sure to exit any programs that may be using the GPU(s) before attempting to upgrade your driver.  If no GPU-based programs are running, you know that your kernel supports module unloading, and you still receive this message, then an error may have occured that has corrupted an NVIDIA kernel module's usage count, for which the simplest remedy is to reboot your computer.
ERROR: Installation has failed.  Please see the file '/var/log/nvidia-installer.log' for details.  You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com http://www.nvidia.com/.
18,1          Bot
ubuntu@ip-10-0-1-189:~$ lsmod | grep nvidia
nvidia_drm             14412  0
nvidia_modeset        764387  1 nvidia_drm
nvidia              11488748  1 nvidia_modeset
drm                   303102  4 ttm,drm_kms_helper,cirrus,nvidia_drm
ubuntu@ip-10-0-1-189:~$ sudo apt-get --purge remove nvidia*
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nvidia-graphics-drivers
E: Unable to locate package nvidia-graphics-drivers-367_367.57.orig.tar
E: Couldn't find any package by regex 'nvidia-graphics-drivers-367_367.57.orig.tar'

On Apr 9, 2017, at 8:39 AM, David Laxer<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
I’m using my own AMI.  I’ve been having trouble getting the GPU on the g2.2xlarge instance to work.
I had to use the 367 version of the NVIDIA driver to get TensorFlow to work.
nvidia-graphics-drivers-367_367.57.orig.tar
Unfortunately, this package did not install ‘nvcc’.  Other attempts to install CUDA installed the 375 version of the driver which doesn’t work
on the g2.2xlarge instance.
I’m trying this now:
cuda_7.5.18_linux.run
from:
https://developer.nvidia.com/cuda-75-downloads-archive https://developer.nvidia.com/cuda-75-downloads-archive

On Apr 9, 2017, at 12:13 AM, Yuya Unno<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
How did you install CUDA? Chainer uses nvcc command, that is a compiler for CUDA environment and is provided by CUDA. You need to set PATH environment correctly.
If you are using amazon linux AMI provided by NVIDIA, nvcc is already in PATH. And, all you need to do is pip install chainer.
If you don't use it and install CUDA manually, I recommend you to install CUDA from deb package. And, you need to add /usr/local/cuda/bin to PATH environment variable. My fabfile may helps you: https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-292768993, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2zplWQK-tP2vhBtafTWSJk03Ckouks5ruIUvgaJpZM4M3z8a.

I got further … by copying ‘nvcc’ from an NVIDIA AMI into /usr/local/cuda-7.0/bin.
Any ideas?
===============================
nvcc fatal   : Path to libdevice library not specified
['nvcc', '-shared', '-O3', '-use_fast_math', '-m64', '-Xcompiler', '-DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden', '-Xlinker', '-rpath,/home/ubuntu/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.11-64/cuda_ndarray', '-I/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/sandbox/cuda', '-I/home/ubuntu/anaconda/lib/python2.7/site-packages/numpy/core/include', '-I/home/ubuntu/anaconda/include/python2.7', '-I/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/gof', '-o', '/home/ubuntu/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.11-64/cuda_ndarray/cuda_ndarray.so', 'mod.cu', '-L/home/ubuntu/anaconda/lib', '-lcublas', '-lpython2.7', '-lcudart']
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status', 1, 'for cmd', 'nvcc -shared -O3 -use_fast_math -m64 -Xcompiler -DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/home/ubuntu/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.11-64/cuda_ndarray -I/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/sandbox/cuda -I/home/ubuntu/anaconda/lib/python2.7/site-packages/numpy/core/include -I/home/ubuntu/anaconda/include/python2.7 -I/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/gof -o /home/ubuntu/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.11-64/cuda_ndarray/cuda_ndarray.so mod.cu -L/home/ubuntu/anaconda/lib -lcublas -lpython2.7 -lcudart')
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: cuda unavailable)
In [3]: quit
ubuntu@ip-10-0-1-189:/hdfs/lda2vec/examples/hacker_news/lda2vec$ nvcc -shared -O3 -use_fast_math -m64 -Xcompiler -DCUDA_NDARRAY_CUH=c72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/home/ubuntu/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.11-64/cuda_ndarray -I/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/sandbox/cuda -I/home/ubuntu/anaconda/lib/python2.7/site-packages/numpy/core/include -I/home/ubuntu/anaconda/include/python2.7 -I/home/ubuntu/anaconda/lib/python2.7/site-packages/theano/gof -o /home/ubuntu/.theano/compiledir_Linux-3.13--generic-x86_64-with-debian-jessie-sid-x86_64-2.7.11-64/cuda_ndarray/cuda_ndarray.so mod.cu -L/home/ubuntu/anaconda/lib -lcublas -lpython2.7 -lcudart
nvcc fatal   : Path to libdevice library not specified

On Apr 9, 2017, at 9:24 AM, David Laxer<EMAIL_ADDRESS>wrote:
Tensorflow still works with the GPU.  Any suggestions on how to get only the toolkit with ‘nvcc’?
E.g.
ubuntu@ip-10-0-1-189:~$ nvidia-smi
Sun Apr  9 16:23:03 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.57                 Driver Version: 367.57                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID K520           Off  | 0000:00:03.0     Off |                  N/A |
| N/A   33C    P0    41W / 125W |   3742MiB /  4036MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      3222    C   python                                        3740MiB |
+-----------------------------------------------------------------------------+
ubuntu@ip-10-0-1-189:~$
ubuntu@ip-10-0-1-189:~/models/neural_gpu$ python neural_gpu_trainer.py --problem=bmul
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.7.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.7.5 locally
Error appending to /tmp/neural_gpu/log
Creating checkpoint directory /tmp/neural_gpu.
Generating data for bmul.
cut 1.20 lr 0.100 iw 0.80 cr 0.30 nm 64 d0.1000 gn 4.00 layers 2 kw 3 h 4 kh 3 batch 32 noise 0.00
Creating model.
Creating backward pass for the model.
WARNING:tensorflow:Tried to colocate gpu0/gradients/gpu0/Gather_2_grad/Shape with an op target_embedding/read that had a different device: /device:GPU:0 vs /device:CPU:0. Ignoring colocation property.
Created model for gpu 0 in 6.29 s.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:910] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: GRID K520
major: 3 minor: 0 memoryClockRate (GHz) 0.797
pciBusID 0000:00:03.0
Total memory: 3.94GiB
Free memory: 3.91GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0:   Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GRID K520, pci bus id: 0000:00:03.0)
Created model. Checkpoint dir /tmp/neural_gpu
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:247] PoolAllocator: After 2495 get requests, put_count=2461 evicted_count=1000 eviction_rate=0.406339 and unsatisfied allocation rate=0.454509
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:259] Raising pool_size_limit_ from 100 to 110
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:247] PoolAllocator: After 6023 get requests, put_count=4617 evicted_count=1000 eviction_rate=0.216591 and unsatisfied allocation rate=0.403287
I tensorflow/core/common_runtime/gpu/pool_allocator.cc:259] Raising pool_size_limit_ from 256 to 281

On Apr 9, 2017, at 9:04 AM, David Laxer<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Installation failed:
The NVIDIA CUDA Toolkit provides command-line and graphical
Do you accept the previously read EULA? (accept/decline/quit): accept
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): y
Do you want to install the OpenGL libraries? ((y)es/(n)o/(q)uit) [ default is yes ]: n
Install the CUDA 7.5 Toolkit? ((y)es/(n)o/(q)uit): y
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]: y
Toolkit location must be an absolute path.
Enter Toolkit Location [ default is /usr/local/cuda-7.5 ]:
Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
Install the CUDA 7.5 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/ubuntu ]:
Installing the NVIDIA display driver...
The driver installation has failed due to an unknown error. Please consult the driver installation log located at /var/log/nvidia-installer.log.
===========
= Summary =
Driver:   Installation Failed
Toolkit:  Installation skipped
Samples:  Installation skipped
Logfile is /tmp/cuda_install_14884.log
ubuntu@ip-10-0-1-189:~$
ERROR: An NVIDIA kernel module 'nvidia' appears to already be loaded in your kernel.  This may be because it is in use (for example, by an X server, a CUDA program, or the NVIDIA Persistence Daemon), but this may also happen if your kernel was configured without support for module unloading.  Please be sure to exit any programs that may be using the GPU(s) before attempting to upgrade your driver.  If no GPU-based programs are running, you know that your kernel supports module unloading, and you still receive this message, then an error may have occured that has corrupted an NVIDIA kernel module's usage count, for which the simplest remedy is to reboot your computer.
ERROR: Installation has failed.  Please see the file '/var/log/nvidia-installer.log' for details.  You may find suggestions on fixing installation problems in the README available on the Linux driver download page at www.nvidia.com http://www.nvidia.com/.
18,1          Bot
ubuntu@ip-10-0-1-189:~$ lsmod | grep nvidia
nvidia_drm             14412  0
nvidia_modeset        764387  1 nvidia_drm
nvidia              11488748  1 nvidia_modeset
drm                   303102  4 ttm,drm_kms_helper,cirrus,nvidia_drm
ubuntu@ip-10-0-1-189:~$ sudo apt-get --purge remove nvidia*
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nvidia-graphics-drivers
E: Unable to locate package nvidia-graphics-drivers-367_367.57.orig.tar
E: Couldn't find any package by regex 'nvidia-graphics-drivers-367_367.57.orig.tar'

On Apr 9, 2017, at 8:39 AM, David Laxer<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
I’m using my own AMI.  I’ve been having trouble getting the GPU on the g2.2xlarge instance to work.
I had to use the 367 version of the NVIDIA driver to get TensorFlow to work.
nvidia-graphics-drivers-367_367.57.orig.tar
Unfortunately, this package did not install ‘nvcc’.  Other attempts to install CUDA installed the 375 version of the driver which doesn’t work
on the g2.2xlarge instance.
I’m trying this now:
cuda_7.5.18_linux.run
from:
https://developer.nvidia.com/cuda-75-downloads-archive https://developer.nvidia.com/cuda-75-downloads-archive

On Apr 9, 2017, at 12:13 AM, Yuya Unno<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
How did you install CUDA? Chainer uses nvcc command, that is a compiler for CUDA environment and is provided by CUDA. You need to set PATH environment correctly.
If you are using amazon linux AMI provided by NVIDIA, nvcc is already in PATH. And, all you need to do is pip install chainer.
If you don't use it and install CUDA manually, I recommend you to install CUDA from deb package. And, you need to add /usr/local/cuda/bin to PATH environment variable. My fabfile may helps you: https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py https://github.com/unnonouno/chainer-vagrant-ec2/blob/master/fabfile.py
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-292768993, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2zplWQK-tP2vhBtafTWSJk03Ckouks5ruIUvgaJpZM4M3z8a.

How about using latest CUDA 8.0? I recommend to use deb file instead of run file. I feel run is unstable.

I was told CUDA 8.0 doesn’t work with the g2.2xlarge AWS machine.
I tried running ‘lda2vec’ & chainer on the public AMI:
NVIDIA DIGITS 4.0 on Ubuntu 14.04-6250ff83-f573-4050-80fc-413ac84d044d-ami-94054383.3
ami-d5ce80b5
This works. Chainer can’t detect cuDNN 6.5 - which I installed.  cudnn-6.5-linux-x64-v2.tar
But, lda2vec appears to runs properly.
In [3]: %load lda2vec_run.py
In [4]:     for s, a, f in utils.chunks(batchsize, story_id, author_id, flattene
...: d):
...:         t0 = time.time()
...:         optimizer.zero_grads()
...:         l = model.fit_partial(s.copy(), a.copy(), f.copy())
...:         prior = model.prior()
...:         loss = prior * fraction
...:         loss.backward()
...:         optimizer.update()
...:         msg = ("J:{j:05d} E:{epoch:05d} L:{loss:1.3e} "
...:                "P:{prior:1.3e} R:{rate:1.3e}")
...:         prior.to_cpu()
...:         loss.to_cpu()
...:         t1 = time.time()
...:         dt = t1 - t0
...:         rate = batchsize / dt
...:         logs = dict(loss=float(l), epoch=epoch, j=j,
...:                     prior=float(prior.data), rate=rate)
...:         print msg.format(**logs)
...:         j += 1
...:     serializers.save_hdf5("lda2vec.hdf5", model)
...:
In [4]:     for s, a, f in utils.chunks(batchsize, story_id, author_id, flattene
...: d):
...:         t0 = time.time()
...:         optimizer.zero_grads()
...:         l = model.fit_partial(s.copy(), a.copy(), f.copy())
...:         prior = model.prior()
...:         loss = prior * fraction
...:         loss.backward()
...:         optimizer.update()
...:         msg = ("J:{j:05d} E:{epoch:05d} L:{loss:1.3e} "
...:                "P:{prior:1.3e} R:{rate:1.3e}")
...:         prior.to_cpu()
...:         loss.to_cpu()
...:         t1 = time.time()
...:         dt = t1 - t0
...:         rate = batchsize / dt
...:         logs = dict(loss=float(l), epoch=epoch, j=j,
...:                     prior=float(prior.data), rate=rate)
...:         print msg.format(**logs)
...:         j += 1
...:     serializers.save_hdf5("lda2vec.hdf5", model)
...:
RuntimeError                              Traceback (most recent call last)
 in ()
20
21 gpu_id = int(os.getenv('CUDA_GPU', 0))
---> 22 cuda.get_device(gpu_id).use()
23 print "Using GPU " + str(gpu_id)
24
/home/ubuntu/anaconda2/lib/python2.7/site-packages/chainer-2.0.0b1-py2.7.egg/chainer/cuda.pyc in get_device(*args)
168     for arg in args:
169         if type(arg) in _integer_types:
--> 170             check_cuda_available()
171             return Device(arg)
172         if isinstance(arg, ndarray):
/home/ubuntu/anaconda2/lib/python2.7/site-packages/chainer-2.0.0b1-py2.7.egg/chainer/cuda.pyc in check_cuda_available()
73                '(see https://github.com/pfnet/chainer#installation).')
74         msg += str(_resolution_error)
---> 75         raise RuntimeError(msg)
76     if (not cudnn_enabled and
77             not _cudnn_disabled_by_user and
RuntimeError: CUDA environment is not correctly set up
(see https://github.com/pfnet/chainer#installation).No module named cupy
Still not sure how to configure my AMI.

On Apr 11, 2017, at 1:46 AM, Yuya Unno<EMAIL_ADDRESS>wrote:
How about using latest CUDA 8.0? I recommend to use deb file instead of run file. I feel run is unstable.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-293192377, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2y9e4btK1QjdfJ1Zcib6vtty8MhMks5ruz33gaJpZM4M3z8a.

I’m running on AWS EC2 g2.x2large spot instances.
Does chainer support checkpointing, so that I can continue training my models at the point that the spot instance was preempted?

On Apr 11, 2017, at 3:35 AM, David Laxer<EMAIL_ADDRESS>wrote:
I was told CUDA 8.0 doesn’t work with the g2.2xlarge AWS machine.
I tried running ‘lda2vec’ & chainer on the public AMI:
NVIDIA DIGITS 4.0 on Ubuntu 14.04-6250ff83-f573-4050-80fc-413ac84d044d-ami-94054383.3
ami-d5ce80b5
This works. Chainer can’t detect cuDNN 6.5 - which I installed.  cudnn-6.5-linux-x64-v2.tar
But, lda2vec appears to runs properly.
In [3]: %load lda2vec_run.py
In [4]:     for s, a, f in utils.chunks(batchsize, story_id, author_id, flattene
...: d):
...:         t0 = time.time()
...:         optimizer.zero_grads()
...:         l = model.fit_partial(s.copy(), a.copy(), f.copy())
...:         prior = model.prior()
...:         loss = prior * fraction
...:         loss.backward()
...:         optimizer.update()
...:         msg = ("J:{j:05d} E:{epoch:05d} L:{loss:1.3e} "
...:                "P:{prior:1.3e} R:{rate:1.3e}")
...:         prior.to_cpu()
...:         loss.to_cpu()
...:         t1 = time.time()
...:         dt = t1 - t0
...:         rate = batchsize / dt
...:         logs = dict(loss=float(l), epoch=epoch, j=j,
...:                     prior=float(prior.data), rate=rate)
...:         print msg.format(**logs)
...:         j += 1
...:     serializers.save_hdf5("lda2vec.hdf5", model)
...:
In [4]:     for s, a, f in utils.chunks(batchsize, story_id, author_id, flattene
...: d):
...:         t0 = time.time()
...:         optimizer.zero_grads()
...:         l = model.fit_partial(s.copy(), a.copy(), f.copy())
...:         prior = model.prior()
...:         loss = prior * fraction
...:         loss.backward()
...:         optimizer.update()
...:         msg = ("J:{j:05d} E:{epoch:05d} L:{loss:1.3e} "
...:                "P:{prior:1.3e} R:{rate:1.3e}")
...:         prior.to_cpu()
...:         loss.to_cpu()
...:         t1 = time.time()
...:         dt = t1 - t0
...:         rate = batchsize / dt
...:         logs = dict(loss=float(l), epoch=epoch, j=j,
...:                     prior=float(prior.data), rate=rate)
...:         print msg.format(**logs)
...:         j += 1
...:     serializers.save_hdf5("lda2vec.hdf5", model)
...:
RuntimeError                              Traceback (most recent call last)
 in ()
20
21 gpu_id = int(os.getenv('CUDA_GPU', 0))
---> 22 cuda.get_device(gpu_id).use()
23 print "Using GPU " + str(gpu_id)
24
/home/ubuntu/anaconda2/lib/python2.7/site-packages/chainer-2.0.0b1-py2.7.egg/chainer/cuda.pyc in get_device(*args)
168     for arg in args:
169         if type(arg) in _integer_types:
--> 170             check_cuda_available()
171             return Device(arg)
172         if isinstance(arg, ndarray):
/home/ubuntu/anaconda2/lib/python2.7/site-packages/chainer-2.0.0b1-py2.7.egg/chainer/cuda.pyc in check_cuda_available()
73                '(see https://github.com/pfnet/chainer#installation https://github.com/pfnet/chainer#installation).')
74         msg += str(_resolution_error)
---> 75         raise RuntimeError(msg)
76     if (not cudnn_enabled and
77             not _cudnn_disabled_by_user and
RuntimeError: CUDA environment is not correctly set up
(see https://github.com/pfnet/chainer#installation https://github.com/pfnet/chainer#installation).No module named cupy
Still not sure how to configure my AMI.

On Apr 11, 2017, at 1:46 AM, Yuya Unno<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
How about using latest CUDA 8.0? I recommend to use deb file instead of run file. I feel run is unstable.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/pfnet/chainer/issues/2516#issuecomment-293192377, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2y9e4btK1QjdfJ1Zcib6vtty8MhMks5ruz33gaJpZM4M3z8a.


Didn't we decide that using filter as a param name was bad?

Referring to: http://gcloud-python-bigtable.readthedocs.org/en/data-api-complete/data-api.html#conditional-mutations
Pretty sure in the resource-manager API for gcloud-python we said that using method(filter=filter) was a bad way to go.

It makes me uncomfortable using filter. I think the standard recommendation is to use filter_. (In general when foo is a builtin keyword then foo_ should be used.)
/cc @tseaver

For actual keywords, we wouldn't have a choice:  for builtins, adding the underscore seems right, if we can't see a better spelling.  We got away without the underscore in datastore because the query held /passed filters, rather than a single filter.

Should we add filter, open and others to bad-names in our PyLint config (which this project shares)?

Similar question, is filter acceptable as a @property. As far as I'm concerned the answer is yes since foo.filter will never be in the global namespace. But, I can be persuaded.

@tseaver I mostly resolved this with 86bb23bf5f1b6f475be2bf050bde91fc3dce55f3.
However, there is still the issue of Row.filter. I forget, did we decide to keep it or ditch it?

filter as an attribute can never shadow the built-in, so I'm fine with it.

Cool. Closing this.


term::TermDecorator could be faster

term::TermDecorator could be faster on Linux by line-buffering its output instead of outputting many parts of the line with their own syscall.
Diff I applied to the term crate (not slog-term!):
diff --git a/src/lib.rs b/src/lib.rs
index fd02172..e217055 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -69,7 +69,7 @@ pub use terminfo::TerminfoTerminal;
 #[cfg(windows)]
 pub use win::WinConsole;

-use std::io::{self, Stdout, Stderr};
+use std::io::{self, LineWriter, Stdout, Stderr};

 pub mod terminfo;

@@ -77,15 +77,15 @@ pub mod terminfo;
 mod win;

 /// Alias for stdout terminals.
-pub type StdoutTerminal = Terminal<Output = Stdout> + Send;
+pub type StdoutTerminal = Terminal<Output = LineWriter<Stdout>> + Send;
 /// Alias for stderr terminals.
-pub type StderrTerminal = Terminal<Output = Stderr> + Send;
+pub type StderrTerminal = Terminal<Output = LineWriter<Stderr>> + Send;

 #[cfg(not(windows))]
 /// Return a Terminal wrapping stdout, or None if a terminal couldn't be
 /// opened.
 pub fn stdout() -> Option<Box<StdoutTerminal>> {
-    TerminfoTerminal::new(io::stdout()).map(|t| Box::new(t) as Box<StdoutTerminal>)
+    TerminfoTerminal::new(LineWriter::new(io::stdout())).map(|t| Box::new(t) as Box<StdoutTerminal>)
 }

 #[cfg(windows)]
@@ -101,7 +101,7 @@ pub fn stdout() -> Option<Box<StdoutTerminal>> {
 /// Return a Terminal wrapping stderr, or None if a terminal couldn't be
 /// opened.
 pub fn stderr() -> Option<Box<StderrTerminal>> {
-    TerminfoTerminal::new(io::stderr()).map(|t| Box::new(t) as Box<StderrTerminal>)
+    TerminfoTerminal::new(LineWriter::new(io::stderr())).map(|t| Box::new(t) as Box<StderrTerminal>)
 }

 #[cfg(windows)]

I used the following program to check how fast a logging statement is:
#[macro_use]
extern crate slog;
extern crate slog_term;

use slog::BorrowedKV;
use slog::Drain;
use slog::Logger;
use slog_term::*;
use std::sync::Mutex;

fn main() {
    let compact = FullFormat::new(TermDecorator::new().build()).build().fuse();
    let log: Logger = Logger::root(Mutex::new(compact).fuse(), o!("version" => "1.0", "foobar" => "barfoo"));

    for _ in 0..10_000 {
        trace!(log, "logging a trace message");
        debug!(log.new(o!("foo" => "bar")), "debug values"; "x" => 1, "y" => -1);
        info!(log, "some interesting info"; "where" => "right here");
        warn!(log, "be cautious!"; "why" => "you never know...");
        error!(log, "wrong {}", "foobar"; "type" => "unknown");
        crit!(log, "abandoning test");
    }
}

[dependencies]
slog = "2.0.0-3.1"
slog-term = "2.0.0-3.0"

strace without optimization:
[...]
rite(2, "0", 1)                        = 1
write(2, "7", 1)                        = 1
write(2, " ", 1)                        = 1
write(2, "16", 2)                       = 2
write(2, ":", 1)                        = 1
write(2, "42", 2)                       = 2
write(2, ":", 1)                        = 1
write(2, "28", 2)                       = 2
write(2, ".", 1)                        = 1
write(2, "904", 3)                      = 3
write(2, "\33(B\33[m", 6)               = 6
write(2, " ", 1)                        = 1
write(2, "\33[33m", 5)                  = 5
write(2, "WARN", 4)                     = 4
write(2, "\33(B\33[m", 6)               = 6
write(2, " ", 1)                        = 1
write(2, "\33[1m", 4)                   = 4
write(2, "be cautious!", 12)            = 12
write(2, "\33(B\33[m", 6)               = 6
write(2, ", ", 2)                       = 2
write(2, "\33[1m", 4)                   = 4
write(2, "why", 3)                      = 3
write(2, "\33(B\33[m", 6)               = 6
write(2, ":", 1)                        = 1
write(2, "\33(B\33[m", 6)               = 6
write(2, " ", 1)                        = 1
write(2, "\33(B\33[m", 6)               = 6
write(2, "you never know...", 17)       = 17
write(2, "\33(B\33[m", 6)               = 6
write(2, ", ", 2)                       = 2
write(2, "\33[1m", 4)                   = 4
write(2, "foobar", 6)                   = 6
write(2, "\33(B\33[m", 6)               = 6
write(2, ":", 1)                        = 1
write(2, "\33(B\33[m", 6)               = 6
write(2, " ", 1)                        = 1
write(2, "\33(B\33[m", 6)               = 6
write(2, "barfoo", 6)                   = 6
write(2, "\33(B\33[m", 6)               = 6
write(2, ", ", 2)                       = 2
write(2, "\33[1m", 4)                   = 4
write(2, "version", 7)                  = 7
write(2, "\33(B\33[m", 6)               = 6
[...]

strace with optimization:
write(2, "\33(B\33[mApr 07 16:33:00.256\33(B\33[m "..., 168) = 168
write(2, "\33(B\33[mApr 07 16:33:00.257\33(B\33[m "..., 264) = 264
write(2, "\33(B\33[mApr 07 16:33:00.257\33(B\33[m "..., 213) = 213
write(2, "\33(B\33[mApr 07 16:33:00.257\33(B\33[m "..., 209) = 209
write(2, "\33(B\33[mApr 07 16:33:00.257\33(B\33[m "..., 200) = 200
write(2, "\33(B\33[mApr 07 16:33:00.257\33(B\33[m "..., 160) = 160

Without the optimization, it ran in 11.6 seconds, with the optimization, it ran it 4.5 seconds.

Agreed. Though I was wondering if this should be decorator's job, or the IO given to decorator should do it. One can just wrap the io in https://doc.rust-lang.org/std/io/struct.BufWriter.html.

Oh, you are already doing it but though it's tricky with term...

Please not however that on Windows colors and other decoration are separate calls and the data is not being sent in-line as escape codes, so it can't be buffered, AFAIK.

It should be possible to buffer the evaluation of the format string on Windows though, removing this kind of call:
write(2, "16", 2)                       = 2
write(2, ":", 1)                        = 1
write(2, "42", 2)                       = 2
write(2, ":", 1)                        = 1
write(2, "28", 2)                       = 2
write(2, ".", 1)                        = 1
write(2, "904", 3)                      = 3

(prints the current time).

Oh dear. This is https://github.com/slog-rs/term/blob/12f4e905af008a666c5e0dda1cd1b362ae7061e9/src/lib.rs#L773 . Very inefficient.
Do you think you can come up with a PR? Since you have all the instrumentation there, maybe you can come up with best layer to stuff buffering there.

I made an issue in the term repository, see above.

Another argument for line buffering is that when interleaved with other logging systems (GStreamer with lots of threads in my case) I often see line-buffered output of those interleaved with parts of slog output. Makes it really hard to work with when it happens.

I absolutely agree that slog should do line buffering.
However, I would like to note that any buffering beyond that should be configurable and off by default. I use slog-term for debugging crates that use unsafe code. It's important that I see each line as soon as it happens in case my app crashes.
I remember being bitten by a nasty bug because I buffered over multiple lines when logging to a file. Flushing after every newline fixed it. I just want to make sure that doesn't happen to new users 😉

Seems like it's open, so no one produced a PR. At this point I'm only merging things that other people are willing to put an effort into. :)


Add Examples

Example  of zlip tokio write ecnoder and decoder.
Example reading zstd from stdin and write gzip to stdout, using tokio
Related to Nemo157/async-compression#101

Codecov Report

Merging #102 into prīmum will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           prīmum     #102   +/-   ##
=======================================
  Coverage   79.33%   79.33%
=======================================
  Files          62       62
  Lines        2139     2139
=======================================
  Hits         1697     1697
  Misses        442      442

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 74125e6...be6d01a. Read the comment docs.

Hmm, my biggest issue with this is that it's showing something you really should not do (memory -> memory (de)compression using async).
What do you think of instead having something that reads from stdin to memory (using tokio_02::bufread::ZstdDecoder wrapped around tokio::io::Stdin), logs the total length to stderr, then writes it back to stdout (using tokio_02::write::GzipEncoder wrapped around tokio::io::Stdout). That would show usage of both the bufread and write adaptors, and would be a much more realistic example.
Having a short comment showing how to run it would also be useful then (I chose zstd + gzip above to make it easy to test with cli tools):
echo 'hello' | zstd | cargo run --example ... | gunzip -c

I've also left some inline comments on things I'd change to make this follow "best practices".

I have added an example as you suggested.
But I also think that when it comes to examples sometime simplicity should overcome it's "not good practice".
And it's very common to see async examples that use memory vector.
That's why I think you should consider keeping both examples, (with a comment that warn about it).
As a newcomer, the first example helps me a lot more to understand how to work with aync-compression.
Of course, if you want me to I will remove it.

Sorry about the delay, rebased to fix a conflict with the Cargo.toml
bors r+

Codecov Report

Merging #102 (f013f87) into prīmum (796abb2) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           prīmum     #102   +/-   ##
=======================================
  Coverage   78.63%   78.63%
=======================================
  Files          78       78
  Lines        2518     2518
=======================================
  Hits         1980     1980
  Misses        538      538

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 796abb2...f013f87. Read the comment docs.


Initialize Lattices with networkx.Graphs

What should we add?
It would be nice to allow the initialization on Lattices not only with retworkx graphs but also with networks graphs. Networkx, to date, still has a lot more features and pre-defined graphs that users have access to, and is probably the standard graph library in Python.

I agree with everything in this issue, I would appreciate issues being opened on retworkx when you encounter a feature gap. Mainly because we'd like retworkx to be usable everywhere and we can prioritize features needed for a real use case. It's hard to know which functionality to prioritize without requests from users.
As an aside the current dev branch has a bunch of new graph generators, you can see the list here: https://qiskit.org/documentation/retworkx/dev/api.html#generators although I don't think any are really applicable to lattices. @k-tamuraphys has already requested some additional functionality here: https://github.com/Qiskit/retworkx/issues/150#issuecomment-945284818 which might not make it into 0.11 but definitely will be a priority for 0.12.

This issue is not mainly because I was encountering a particular feature gap, but I was more generally concerned about supporting networkx since it's more standardized 🙂


Integrate NIST Profile Validation to UP

User Story
As a report stream sender,
I would like ReportStream to validate my message against the profile I specify in MSH 21,
so I can ensure receivers receive valid RadxMars messages
Description/Use Case
Implement the RadxMars software requirements document  sections:

III. Routing support - essentially just create the following topic: mars-otc-elr.
V. Validation support - logic on how, where, and when we apply validation

Risks/Impacts/Considerations
Dev Notes
Acceptance Criteria

[ ] All requirements in sections III, and V of SRD implemented
[ ] NIST validation library integrated to ReportStream
[ ] Sender validation occurs on HL7v2 messages in Convert Function after de-batching

Messages we got were not valid


WorkItems Completed: The column 'Column1; of the table wasn't found error

Hello Nicholas
I have installed the FLowViz(Jira).pbit  and running into issue. I am getting the following error message.
I have entered the url and project information. Any ideas …

Hi there,
Thanks for sharing. I will take a look today and let you know if I am having similar issues 👍🏼

I have tested this on 5-6 different areas and I am not seeing this issue.
Can you tell me more about your setup? For example is it Jira Cloud/On-Prem? Please email me directly if there is anything sensitive you can't share publicly...

Thanks Nicholas for your prompt response. Today, I tried accessing a different project and the reports pulled up fine as expected. Not sure, Why I am getting the error message I shared above. We are on Jira Cloud.  Any ideas..

Sorry there's nothing further I can think of. I've tried all different
project variations I have and not having any users see the same issue...
On Thu, Feb 9, 2023 at 2:45 AM pmtester-tech @.***>
wrote:

Thanks Nicholas for your prompt response. Today, I tried accessing a
different project and the reports pulled up fine as expected. Not sure, Why
I am getting the error message I shared above. We are on Jira Cloud. Any
ideas..
—
Reply to this email directly, view it on GitHub
https://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1423538368,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHACM2XKVS7LMZK3NMVYQJ3WWRK5FANCNFSM6AAAAAAUTODY7Y
.
You are receiving this because you commented.Message ID:
@.***>

Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?

The values should be pagination - the only way I could foresee it not getting populated is if nothing is “Done” or “Resolved”. What’s the state of “done” for this team? And does it also have a “reason” as that could be how they’re getting omitted…Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 18:28, pmtester-tech @.***> wrote:
Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>

Thanks for the quick response. Really appreciate it. There are several automations linked and the status of Done is called  ‘DEV Done’’ for this project. Are you able to guide me as to where I could add the additional status of ‘DEV Done’ along with Done and resolved?
Sent from my iPhone
On Feb 10, 2023, at 12:55 PM, Nick Brown @.***> wrote:

The values should be pagination - the only way I could foresee it not getting populated is if nothing is “Done” or “Resolved”. What’s the state of “done” for this team? And does it also have a “reason” as that could be how they’re getting omitted…Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 18:28, pmtester-tech @.***> wrote:
Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426209998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHPCIJMH4J7KHOW74YLWW2FKTANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>

You should be able to find them in a JQL. It’s just really odd if items are appearing as completed but not coming through in the dataset.It might be better to have a call/screen share? Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 23:04, pmtester-tech @.***> wrote:
Thanks for the quick response. Really appreciate it. There are several automations linked and the status of Done is called  ‘DEV Done’’ for this project. Are you able to guide me as to where I could add the additional status of ‘DEV Done’ along with Done and resolved?
Sent from my iPhone
On Feb 10, 2023, at 12:55 PM, Nick Brown @.***> wrote:

The values should be pagination - the only way I could foresee it not getting populated is if nothing is “Done” or “Resolved”. What’s the state of “done” for this team? And does it also have a “reason” as that could be how they’re getting omitted…Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 18:28, pmtester-tech @.***> wrote:
Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426209998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHPCIJMH4J7KHOW74YLWW2FKTANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>

That’s a great idea. Let me know if you are available tommorow or what might work for you for a screen share I am in Central time zone.
Thanks so much for all your assistance.
Sent from my iPhone
On Feb 10, 2023, at 5:47 PM, Nick Brown @.***> wrote:

You should be able to find them in a JQL. It’s just really odd if items are appearing as completed but not coming through in the dataset.It might be better to have a call/screen share? Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 23:04, pmtester-tech @.***> wrote:
Thanks for the quick response. Really appreciate it. There are several automations linked and the status of Done is called ‘DEV Done’’ for this project. Are you able to guide me as to where I could add the additional status of ‘DEV Done’ along with Done and resolved?
Sent from my iPhone
On Feb 10, 2023, at 12:55 PM, Nick Brown @.***> wrote:

The values should be pagination - the only way I could foresee it not getting populated is if nothing is “Done” or “Resolved”. What’s the state of “done” for this team? And does it also have a “reason” as that could be how they’re getting omitted…Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 18:28, pmtester-tech @.***> wrote:
Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426209998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHPCIJMH4J7KHOW74YLWW2FKTANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426501086, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHP6LBJ6N23765NK4DDWW3HSBANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>

Next week is better for me - I’m in the UK so I think 6 hours ahead of you? Would 9am (3pm my time) your time on Wednesday or 10am (4pm my time) on Friday work?Thanks,NickSent from my iPhoneOn 11 Feb 2023, at 00:24, pmtester-tech @.***> wrote:
That’s a great idea. Let me know if you are available tommorow or what might work for you for a screen share I am in Central time zone.
Thanks so much for all your assistance.
Sent from my iPhone
On Feb 10, 2023, at 5:47 PM, Nick Brown @.***> wrote:

You should be able to find them in a JQL. It’s just really odd if items are appearing as completed but not coming through in the dataset.It might be better to have a call/screen share? Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 23:04, pmtester-tech @.***> wrote:
Thanks for the quick response. Really appreciate it. There are several automations linked and the status of Done is called ‘DEV Done’’ for this project. Are you able to guide me as to where I could add the additional status of ‘DEV Done’ along with Done and resolved?
Sent from my iPhone
On Feb 10, 2023, at 12:55 PM, Nick Brown @.***> wrote:

The values should be pagination - the only way I could foresee it not getting populated is if nothing is “Done” or “Resolved”. What’s the state of “done” for this team? And does it also have a “reason” as that could be how they’re getting omitted…Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 18:28, pmtester-tech @.***> wrote:
Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426209998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHPCIJMH4J7KHOW74YLWW2FKTANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426501086, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHP6LBJ6N23765NK4DDWW3HSBANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>

Hello Nicolas,
hope you are having a great weekend. Friday at 10 am would work for me.
Looking forward to speaking with you!!
I will send you teams meeting invite tomorrow morning.
Thanks
Anju Ahluwalia
Sent from my iPhone
On Feb 11, 2023, at 3:22 AM, Nick Brown @.***> wrote:

Next week is better for me - I’m in the UK so I think 6 hours ahead of you? Would 9am (3pm my time) your time on Wednesday or 10am (4pm my time) on Friday work?Thanks,NickSent from my iPhoneOn 11 Feb 2023, at 00:24, pmtester-tech @.***> wrote:
That’s a great idea. Let me know if you are available tommorow or what might work for you for a screen share I am in Central time zone.
Thanks so much for all your assistance.
Sent from my iPhone
On Feb 10, 2023, at 5:47 PM, Nick Brown @.***> wrote:

You should be able to find them in a JQL. It’s just really odd if items are appearing as completed but not coming through in the dataset.It might be better to have a call/screen share? Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 23:04, pmtester-tech @.***> wrote:
Thanks for the quick response. Really appreciate it. There are several automations linked and the status of Done is called ‘DEV Done’’ for this project. Are you able to guide me as to where I could add the additional status of ‘DEV Done’ along with Done and resolved?
Sent from my iPhone
On Feb 10, 2023, at 12:55 PM, Nick Brown @.***> wrote:

The values should be pagination - the only way I could foresee it not getting populated is if nothing is “Done” or “Resolved”. What’s the state of “done” for this team? And does it also have a “reason” as that could be how they’re getting omitted…Thanks,NickSent from my iPhoneOn 10 Feb 2023, at 18:28, pmtester-tech @.***> wrote:
Hello Nicholas,
Are you able to tell me what these values mean during the creation of Workitems Completed table creation? In what scenario would the table WorkItems Completed not get populated?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426209998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHPCIJMH4J7KHOW74YLWW2FKTANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426501086, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHP6LBJ6N23765NK4DDWW3HSBANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: @.***>
—
Reply to this email directly, view it on GitHubhttps://github.com/nbrown02/FlowViz-Jira/issues/21#issuecomment-1426671290, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARGVRHIOM4L76VSMVSSMC7DWW5K4FANCNFSM6AAAAAAUTODY7Y.
You are receiving this because you authored the thread.Message ID: @.***>


Bug: Lichess/Chess.com can still detect it

I played the chessbot for a week and got banned, can u fix it??

Of course we detect it. Just play chess without it.


Undefined index: fillable when using fields_sample.json

When I'm running php artisan infyom:api Post --fieldsFile=fields_sample.json , using the sample file in the vendor directory, i get

Any ideas?

What does your fields_sample.json look like?

Its the one in the vendors folder, attaching it here
fields_sample.json.txt


Balance monitoring retries

We want to use a retry mechanism for single executions of the balance check.
The retries will be performed during the period of retry timeout, logging a warning on each error. When the timeout is hit an error will be logged and retries stopped. The next balance check will be triggered at the next tick.
Depends on https://github.com/keep-network/keep-common/pull/82

I am good to go but please update keep-common dependency first to point to main and not a feature branch given that keep-network/keep-common#82 has been merged.

✅ 4139244

Are there plans to port this behavior to keep-core as well?

✅ https://github.com/keep-network/keep-core/pull/2592


"circularly references itself" errors after upgrade to 13

Command: swagger-typescript-api --extract-enums --no-client --path https://raw.githubusercontent.com/europace/baufismart-kundenangaben-api/master/kundenangaben-openapi.yaml --name kundenangaben-api.model.ts
Worked with 12, but now typescript throws "circularly references itself" errors with 13.

Same here. Upgrading from 12 to 13 introduced circular references similar to below.
The previous output:
export interface Config {
  type?:
    | 'anExampleConfig'
    | 'anotherExampleConfig';
}

export interface AnExampleConfig extends Config  {
  field?:
    | 'one'
    | 'two'
    | 'three';
  anotherField?: boolean;
}

The new, circular dependency style:
type BaseConfigConfigTypeMapping<Key, Type> = {
  type: Key;
} & Type;

interface BaseConfig {
  configType?:
    | 'anExampleConfig'
    | 'anotherExampleConfig';
}

export type Config = BaseConfig &
  (
    | BaseConfigConfigTypeMapping<'anExampleConfig', ExampleConfig>
    | BaseConfigConfigTypeMapping<'anotherExampleConfig', AnotherExampleConfig>
  );

export interface ExampleConfig extends Config  {
  field?:
    | 'one'
    | 'two'
    | 'three';
  anotherField?: boolean;
}

@js2me Not sure if you have an idea how to fix this properly or if this is the proper fix for this?

I am getting similar error, can get this fixed?

@NerijusD Are you able to provide another test case?

I can confirm https://github.com/acacode/swagger-typescript-api/issues/564#issuecomment-1849846996 significantly reduced my errors from dozens to just 2. Perhaps someone here can make a PR with a test case and patch?

can confirm that this indeed  fixes the issue

Hi!
I am facing the same issue. Does anyone have any information on this?
Thanks in advance

@pablo-olmedo-poex Does this fix your issue? https://github.com/acacode/swagger-typescript-api/issues/564#issuecomment-1849846996
If not, can you provide some examples?

@pablo-olmedo-poex Does this fix your issue? #564 (comment)
If not, can you provide some examples?

This fix just reduced the amount of errors from 10 to 4 😢

@pablo-olmedo-poex Does this fix your issue? #564 (comment)
If not, can you provide some examples?

Hey, would like to add i also have this issue, i've tried the latest versions of mayor version 11-13. Probably because of a change of how the type is defined in the API.
This is the command used to generate: npx swagger-typescript-api -p <The API path> -o ./libs/search/src/lib/apiClient --modular
This is what it used to generate (the relevant parts):
type UtilRequiredKeys<T, K extends keyof T> = Omit<T, K> & Required<Pick<T, K>>;

export type Geometry = (
	| UtilRequiredKeys<Point, 'type'>
	| UtilRequiredKeys<LineString, 'type'>
	| UtilRequiredKeys<Polygon, 'type'>
	| UtilRequiredKeys<MultiPoint, 'type'>
	| UtilRequiredKeys<MultiLineString, 'type'>
	| UtilRequiredKeys<MultiPolygon, 'type'>
) & {
	type: string;
};

export interface LineString {
	/** LineString */
	type: string;
	/** Coordinates in WGS 84 format */
	coordinates: number[][];
}

// Including LineString is: Point, Polygon, MultiPoint etc

This is what its generating now with typescript errors in the comments:
type UtilRequiredKeys<T, K extends keyof T> = Omit<T, K> & Required<Pick<T, K>>;

// Type alias 'Geometry' circularly references itself.ts(2456)
export type Geometry = (Point | LineString | Polygon | MultiPoint | MultiLineString | MultiPolygon) & {
	type: string;
};

// Type alias 'LineString' circularly references itself.ts(2456)
export type LineString = UtilRequiredKeys<Geometry, 'type'> & {
	/** LineString */
	type: string;
	/** Coordinates in WGS 84 format */
	coordinates: number[][];
};

// Including LineString is: Point, Polygon, MultiPoint etc

This fix just reduced the amount of errors from 10 to 4 😢

It seems like implementing this fix in a PR is something anyone in this thread could do to get us 60% closer to a solution. I know it significantly reduced errors in my use-case as well.

const componentsToParse = _.sortBy(this.schemaComponentsMap.filter(
      _.compact(['schemas', this.config.extractResponses && 'responses']),
    ), (value) => {
      return value.rawTypeData.hasOwnProperty("discriminator") ? 0 : 1;
    });

Looks like this is not works properly. This fix produces code like this (from "oneOf" + "ref"):
export type MyDefinitionsDefinitionsKwargFilter =
    | ({
          type: 'KwargFilterValue';
      } & any)
    | ({
          type: 'KwargFilterProduct';
      } & any)

Which is literally useless (works same as just "any").


FAILED TO FIND CHESSBOARD, exiting..

Hi,
Not sure if the project is still active or not, but I did get it to compile on Arch Linux and am projecting/generating/capturing image files.  I cannot seem to get the calibration done though.  Never seems to find the chessboard.  I had to comment out the exif data collection part of slcrunch because it would seg fault.  I should be calibrating using slcrunch, right?
Any assistance would be appreciated.
Thanks

NM, I wasn't using the -chessidx option -- it calibrated when I did.  Thanks


Improve logging

Currently, we log errors from running the command (exec.Command) in a tempfile.
These files are difficult to access on GitHub, in case, a CI step fails.
Should we instead log everything to console?
The problem with that is that the logs may get very messy and difficult to use.

I think this is a good idea. If messy logs an issue we can consider adding a "verbose" flag?

Yes. I think a verbose flag would be a good idea :) Not sure how to implement it nicely though.


link to Papineni et al. (2002)

Sacrebleu is being used by users not familiar with the definition of BLEU.
We should at least give a link (and credit) to the original BLEU paper.

Hi,
This is already done in my 2.0.0 branch for which I'll send a PR soon.

OK.


The example code cannot be reproduced

Hello, Tim Sainburg, When I ran the example code you provided, I encountered the following issues. I hope you can help me resolve them.
The example code as follow
from umap.parametric_umap import ParametricUMAP
import tensorflow as tf
from tensorflow.keras.datasets import mnist
import matplotlib.pyplot as plt
import pandas as pd
(train_images, Y_train), (test_images, Y_test) = mnist.load_data()
train_images = train_images.reshape((train_images.shape[0], -1))/255
test_images = test_images.reshape((test_images.shape[0], -1))/255
dims = (28, 28, 1)
n_components = 2
encoder = tf.keras.Sequential([
tf.keras.layers.InputLayer(input_shape=dims),
tf.keras.layers.Conv2D(
filters=32, kernel_size=3, strides=(2, 2), activation="relu", padding="same"
),
tf.keras.layers.Conv2D(
filters=64, kernel_size=3, strides=(2, 2), activation="relu", padding="same"
),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(units=256, activation="relu"),
tf.keras.layers.Dense(units=256, activation="relu"),
tf.keras.layers.Dense(units=n_components),
])
embedder = ParametricUMAP(encoder=encoder, dims=dims, n_epochs=50, verbose=True)
embedding = embedder.fit_transform(train_images)
print(embedder._history)
This is my output
ParametricUMAP(dims=(28, 28, 1),
encoder=<tensorflow.python.keras.engine.sequential.Sequential object at 0x0000000012CE5588>,
optimizer=<tensorflow.python.keras.optimizer_v2.adam.Adam object at 0x0000000012C8AA08>)
Construct fuzzy simplicial set
Mon Jun 24 16:28:23 2024 Finding Nearest Neighbors
Mon Jun 24 16:28:23 2024 Building RP forest with 17 trees
Mon Jun 24 16:28:25 2024 NN descent for 16 iterations
1  /  16
2  /  16
3  /  16
4  /  16
5  /  16
Stopping threshold met -- exiting after 5 iterations
Mon Jun 24 16:28:46 2024 Finished Nearest Neighbor Search
Mon Jun 24 16:28:50 2024 Construct embedding
completed  0  /  50 epochs
completed  5  /  50 epochs
completed  10  /  50 epochs
completed  15  /  50 epochs
completed  20  /  50 epochs
completed  25  /  50 epochs
completed  30  /  50 epochs
completed  35  /  50 epochs
completed  40  /  50 epochs
completed  45  /  50 epochs
Mon Jun 24 16:29:03 2024 Finished embedding
Traceback (most recent call last):
print(embedder._history)
AttributeError: 'ParametricUMAP' object has no attribute '_history'

I believe _history has been moved in a recent update to tensorflow / keras, try inspecting the ParametricUMAP object or downgrading keras to the version from around 2020.
Alternatively you can the version of parametric umap that we are actively maintaining: https://github.com/lmcinnes/umap

Thanks，Can the current version of UMAP rank feature importance?

I suppose it could if you wanted it to


Import localizer module on demand

Do you want to request a feature or report a bug?
maybe this is a feature or just a question, since many packages support "import on demand".
What's the current behavior?
I am using moment with big calendar, but now in the bundle file, I can still see globalize.js and oldGlobalize.js.

What's the expected behavior?
Since I just need to use moment, I hope only import moment related code.

The reason why I ask this is because our security team reports there is XSS issue in oldGlobalize.js. If I could exclude this, I could continue using react-big-calendar, otherwise I have to choose other alternatives.  I am still not sure if this XSS issue is reported by mistake or not since snyk does not say there is any issues -  https://github.ibm.com/IBM-IOTM/PO/issues/url   . No matter what "import on demand" is an good improvement, :), agree?

@ilovepumpkin I am not sure how you are getting globalize into your bundle other than if you are using it somewhere. Or, you probably need to remove it from your node_modules. I will close this for now since I don't think this is an issue with rbc but more an issue with how you are managing your dependencies.


Add a page for 'Women in Tech'

FOSS Community is also supporting the role of Women in Technology. To achieve this, a separate page needs to be created for 'Women in Tech'.
The page will be created and managed by females in the community.

Hi I am interested in working on this, I would like to know the example design layout and the data you would like to add and display out

thanks @devanshu0605, I have added the page.