ImportError: Please build Cython components with: pip install --editable . or python setup.py build_ext --inplace

2021-06-08 23:33:42 | INFO | fairseq_cli.train | model guided_bart_large, criterion LabelSmoothedCrossEntropyCriterion
2021-06-08 23:33:42 | INFO | fairseq_cli.train | num. model params: 469292032 (num. trained: 469292032)
2021-06-08 23:33:45 | INFO | fairseq_cli.train | training on 1 GPUs
2021-06-08 23:33:45 | INFO | fairseq_cli.train | max tokens per GPU = 2048 and max sentences per GPU = None
2021-06-08 23:33:45 | INFO | fairseq.trainer | no existing checkpoint found /projects/tir5/users/pliu3/zdou/fairseq/bart.large/model.pt
2021-06-08 23:33:45 | INFO | fairseq.trainer | loading train data for epoch 0
2021-06-08 23:33:46 | INFO | fairseq.data.data_utils | loaded 287227 examples from: /data/songpeng/project_my/guided_summarization/bart/data-bin/cnn_dm-bin/train.source-target.source
2021-06-08 23:33:47 | INFO | fairseq.data.data_utils | loaded 287227 examples from: /data/songpeng/project_my/guided_summarization/bart/data-bin/cnn_dm-bin/train.source-target.target
2021-06-08 23:33:47 | INFO | fairseq.data.data_utils | loaded 287227 examples from: /data/songpeng/project_my/guided_summarization/bart/data-bin/cnn_dm-bin/train.source-target.z
2021-06-08 23:33:47 | INFO | fairseq.tasks.guided_translation | /data/songpeng/project_my/guided_summarization/bart/data-bin/cnn_dm-bin/ train source-target 287227 examples
2021-06-08 23:33:49 | WARNING | fairseq.data.data_utils | 4 samples have invalid sizes and will be skipped, max_positions=(1024, 1024, 1024), first few sample ids=[189447, 112053, 286032, 172051]
Traceback (most recent call last):
File "/data/songpeng/project_my/guided_summarization/bart/fairseq/data/data_utils.py", line 221, in batch_by_size
from fairseq.data.data_utils_fast import batch_by_size_fast
ModuleNotFoundError: No module named 'fairseq.data.data_utils_fast'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train.py", line 11, in
cli_main()
File "/data/songpeng/project_my/guided_summarization/bart/fairseq_cli/train.py", line 318, in cli_main
main(args)
File "/data/songpeng/project_my/guided_summarization/bart/fairseq_cli/train.py", line 81, in main
extra_state, epoch_itr = checkpoint_utils.load_checkpoint(args, trainer)
File "/data/songpeng/project_my/guided_summarization/bart/fairseq/checkpoint_utils.py", line 152, in load_checkpoint
epoch_itr = trainer.get_train_iterator(
File "/data/songpeng/project_my/guided_summarization/bart/fairseq/trainer.py", line 275, in get_train_iterator
return self.task.get_batch_iterator(
File "/data/songpeng/project_my/guided_summarization/bart/fairseq/tasks/fairseq_task.py", line 176, in get_batch_iterator
batch_sampler = data_utils.batch_by_size(
File "/data/songpeng/project_my/guided_summarization/bart/fairseq/data/data_utils.py", line 223, in batch_by_size
raise ImportError(
ImportError: Please build Cython components with: pip install --editable . or python setup.py build_ext --inplace
srun: error: pgpu10: task 0: Exited with exit code 1

Are there some errors in my fairseq?

And then I use pip install --editable . , I got
Obtaining file:///data/songpeng/project_my/guided_summarization/bart
Requirement already satisfied: cffi in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.14.5)
Requirement already satisfied: cython in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (0.29.23)
Requirement already satisfied: numpy in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.20.2)
Requirement already satisfied: regex in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (2021.4.4)
Requirement already satisfied: sacrebleu in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.5.1)
Requirement already satisfied: torch in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.8.1)
Requirement already satisfied: tqdm in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (4.61.0)
Requirement already satisfied: pycparser in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from cffi->fairseq==0.9.0) (2.20)
Requirement already satisfied: portalocker==2.0.0 in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (2.0.0)
Requirement already satisfied: typing_extensions in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from torch->fairseq==0.9.0) (<IP_ADDRESS>)
Installing collected packages: fairseq
Running setup.py develop for fairseq
ERROR: Command errored out with exit status 1:
command: /data/songpeng/.conda/envs/feqsem/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project_my/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project_my/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /data/songpeng/project_my/guided_summarization/bart/
Complete output (106 lines):
No CUDA runtime is found, using CUDA_HOME='/cm/shared/apps/cuda11.1/toolkit/11.1.0'
running develop
running egg_info
writing fairseq.egg-info/PKG-INFO
writing dependency_links to fairseq.egg-info/dependency_links.txt
writing entry points to fairseq.egg-info/entry_points.txt
writing requirements to fairseq.egg-info/requires.txt
writing top-level names to fairseq.egg-info/top_level.txt
reading manifest file 'fairseq.egg-info/SOURCES.txt'
writing manifest file 'fairseq.egg-info/SOURCES.txt'
running build_ext
cythoning fairseq/data/data_utils_fast.pyx to fairseq/data/data_utils_fast.cpp
cythoning fairseq/data/token_block_utils_fast.pyx to fairseq/data/token_block_utils_fast.cpp
building 'fairseq.libbleu' extension
Emitting ninja build file /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/libbleu.o /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/module.o -o build/lib.linux-x86_64-3.8/fairseq/libbleu.cpython-38-x86_64-linux-gnu.so
building 'fairseq.data.data_utils_fast' extension
Emitting ninja build file /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_my/guided_summarization/bart/fairseq/data/data_utils_fast.cpp -o /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=data_utils_fast -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /data/songpeng/project_my/guided_summarization/bart/fairseq/data/data_utils_fast.cpp:626:
/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
|  ^~~~~~~
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o -o build/lib.linux-x86_64-3.8/fairseq/data/data_utils_fast.cpython-38-x86_64-linux-gnu.so
building 'fairseq.data.token_block_utils_fast' extension
Emitting ninja build file /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_my/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp -o /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=token_block_utils_fast -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /data/songpeng/project_my/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:627:
/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
|  ^~~~~~~
/data/songpeng/project_my/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp: In function ‘PyArrayObject* __pyx_f_7fairseq_4data_22token_block_utils_fast__get_slice_indices_fast(PyArrayObject*, PyObject*, int, int, int)’:
/data/songpeng/project_my/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:3306:36: warning: comparison of integer expressions of different signedness: ‘__pyx_t_7fairseq_4data_22token_block_utils_fast_DTYPE_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
3306 |       __pyx_t_4 = ((__pyx_v_sz_idx < __pyx_t_10) != 0);
|                     ~^~
/data/songpeng/project_my/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:3501:36: warning: comparison of integer expressions of different signedness: ‘__pyx_t_7fairseq_4data_22token_block_utils_fast_DTYPE_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
3501 |       __pyx_t_3 = ((__pyx_v_sz_idx < __pyx_t_10) != 0);
|                     ~^~
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o -o build/lib.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.cpython-38-x86_64-linux-gnu.so
building 'fairseq.libnat' extension
Emitting ninja build file /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_my/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat/edit_dist.o -L/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.8/fairseq/libnat.cpython-38-x86_64-linux-gnu.so
building 'fairseq.libnat_cuda' extension
Traceback (most recent call last):
File "", line 1, in
File "/data/songpeng/project_my/guided_summarization/bart/setup.py", line 118, in
setup(
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/develop.py", line 136, in install_for_development
self.run_command('build_ext')
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 708, in build_extensions
build_ext.build_extensions(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
objects = self.compiler.compile(sources,
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 524, in unix_wrap_ninja_compile
cuda_post_cflags = unix_cuda_flags(cuda_post_cflags)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 423, in unix_cuda_flags
cflags + _get_cuda_arch_flags(cflags))
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1561, in _get_cuda_arch_flags
arch_list[-1] += '+PTX'
IndexError: list index out of range
----------------------------------------
ERROR: Command errored out with exit status 1: /data/songpeng/.conda/envs/feqsem/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project_my/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project_my/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
(feqsem) [songpeng@pbcmlg01 bart]$ cd /data/songpeng/project_my/guided_summarization/bart/
(feqsem) [songpeng@pbcmlg01 bart]$ cp -r data-bin /data/songpeng/project_formal/guided_summarization/bart/
(feqsem) [songpeng@pbcmlg01 bart]$ cd /data/songpeng/project_formal/guided_summarization/bart/
(feqsem) [songpeng@pbcmlg01 bart]$  pip install --editable .
Obtaining file:///data/songpeng/project_formal/guided_summarization/bart
Requirement already satisfied: cffi in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.14.5)
Requirement already satisfied: cython in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (0.29.23)
Requirement already satisfied: numpy in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.20.2)
Requirement already satisfied: regex in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (2021.4.4)
Requirement already satisfied: sacrebleu in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.5.1)
Requirement already satisfied: torch in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (1.8.1)
Requirement already satisfied: tqdm in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from fairseq==0.9.0) (4.61.0)
Requirement already satisfied: pycparser in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from cffi->fairseq==0.9.0) (2.20)
Requirement already satisfied: portalocker==2.0.0 in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (2.0.0)
Requirement already satisfied: typing_extensions in /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages (from torch->fairseq==0.9.0) (<IP_ADDRESS>)
Installing collected packages: fairseq
Running setup.py develop for fairseq
ERROR: Command errored out with exit status 1:
command: /data/songpeng/.conda/envs/feqsem/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project_formal/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project_formal/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /data/songpeng/project_formal/guided_summarization/bart/
Complete output (133 lines):
No CUDA runtime is found, using CUDA_HOME='/cm/shared/apps/cuda11.1/toolkit/11.1.0'
running develop
running egg_info
creating fairseq.egg-info
writing fairseq.egg-info/PKG-INFO
writing dependency_links to fairseq.egg-info/dependency_links.txt
writing entry points to fairseq.egg-info/entry_points.txt
writing requirements to fairseq.egg-info/requires.txt
writing top-level names to fairseq.egg-info/top_level.txt
writing manifest file 'fairseq.egg-info/SOURCES.txt'
reading manifest file 'fairseq.egg-info/SOURCES.txt'
writing manifest file 'fairseq.egg-info/SOURCES.txt'
running build_ext
cythoning fairseq/data/data_utils_fast.pyx to fairseq/data/data_utils_fast.cpp
cythoning fairseq/data/token_block_utils_fast.pyx to fairseq/data/token_block_utils_fast.cpp
building 'fairseq.libbleu' extension
creating /data/songpeng/project_formal/guided_summarization/bart/build
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu
Emitting ninja build file /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] c++ -MMD -MF /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/module.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_formal/guided_summarization/bart/fairseq/clib/libbleu/module.cpp -o /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/module.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=libbleu -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
[2/2] c++ -MMD -MF /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/libbleu.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_formal/guided_summarization/bart/fairseq/clib/libbleu/libbleu.cpp -o /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/libbleu.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=libbleu -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/fairseq
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/libbleu.o /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/module.o -o build/lib.linux-x86_64-3.8/fairseq/libbleu.cpython-38-x86_64-linux-gnu.so
building 'fairseq.data.data_utils_fast' extension
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data
Emitting ninja build file /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_formal/guided_summarization/bart/fairseq/data/data_utils_fast.cpp -o /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=data_utils_fast -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /data/songpeng/project_formal/guided_summarization/bart/fairseq/data/data_utils_fast.cpp:626:
/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
|  ^
creating build/lib.linux-x86_64-3.8/fairseq/data
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o -o build/lib.linux-x86_64-3.8/fairseq/data/data_utils_fast.cpython-38-x86_64-linux-gnu.so
building 'fairseq.data.token_block_utils_fast' extension
Emitting ninja build file /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_formal/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp -o /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=token_block_utils_fast -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /data/songpeng/project_formal/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:627:
/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
|  ^~~~~
/data/songpeng/project_formal/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp: In function ‘PyArrayObject* __pyx_f_7fairseq_4data_22token_block_utils_fast__get_slice_indices_fast(PyArrayObject*, PyObject*, int, int, int)’:
/data/songpeng/project_formal/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:3306:36: warning: comparison of integer expressions of different signedness: ‘__pyx_t_7fairseq_4data_22token_block_utils_fast_DTYPE_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
3306 |       __pyx_t_4 = ((__pyx_v_sz_idx < __pyx_t_10) != 0);
|                     ~~~~~^~
/data/songpeng/project_formal/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:3501:36: warning: comparison of integer expressions of different signedness: ‘__pyx_t_7fairseq_4data_22token_block_utils_fast_DTYPE_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
3501 |       __pyx_t_3 = ((__pyx_v_sz_idx < __pyx_t_10) != 0);
|                     ~~~~~^~
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o -o build/lib.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.cpython-38-x86_64-linux-gnu.so
building 'fairseq.libnat' extension
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat
Emitting ninja build file /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat/edit_dist.o.d -pthread -B /data/songpeng/.conda/envs/feqsem/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/TH -I/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/THC -I/data/songpeng/.conda/envs/feqsem/include/python3.8 -c -c /data/songpeng/project_formal/guided_summarization/bart/fairseq/clib/libnat/edit_dist.cpp -o /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat/edit_dist.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=libnat -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/ATen/Parallel.h:140,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/utils.h:3,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/cloneable.h:5,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:3,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/all.h:13,
from /data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/torch.h:3,
from /data/songpeng/project_formal/guided_summarization/bart/fairseq/clib/libnat/edit_dist.cpp:9:
/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/include/ATen/ParallelOpenMP.h:83: warning: ignoring #pragma omp parallel [-Wunknown-pragmas]
83 | #pragma omp parallel for if ((end - begin) >= grain_size)
|
g++ -pthread -shared -B /data/songpeng/.conda/envs/feqsem/compiler_compat -L/data/songpeng/.conda/envs/feqsem/lib -Wl,-rpath=/data/songpeng/.conda/envs/feqsem/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat/edit_dist.o -L/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.8/fairseq/libnat.cpython-38-x86_64-linux-gnu.so
building 'fairseq.libnat_cuda' extension
creating /data/songpeng/project_formal/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat_cuda
Traceback (most recent call last):
File "", line 1, in
File "/data/songpeng/project_formal/guided_summarization/bart/setup.py", line 118, in
setup(
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/develop.py", line 136, in install_for_development
self.run_command('build_ext')
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 708, in build_extensions
build_ext.build_extensions(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
objects = self.compiler.compile(sources,
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 524, in unix_wrap_ninja_compile
cuda_post_cflags = unix_cuda_flags(cuda_post_cflags)
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 423, in unix_cuda_flags
cflags + _get_cuda_arch_flags(cflags))
File "/data/songpeng/.conda/envs/feqsem/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1561, in _get_cuda_arch_flags
arch_list[-1] += '+PTX'
IndexError: list index out of range
----------------------------------------
ERROR: Command errored out with exit status 1: /data/songpeng/.conda/envs/feqsem/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project_formal/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project_formal/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
(feqsem) [songpeng@pbcmlg01 bart]$ tmux a -t gpu0
self.run_command('build_ext')
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 708, in build_extensions
build_ext.build_extensions(self)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
objects = self.compiler.compile(sources,
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 524, in unix_wrap_ninja_compile
cuda_post_cflags = unix_cuda_flags(cuda_post_cflags)
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 423, in unix_cuda_flags
cflags + _get_cuda_arch_flags(cflags))
File "/home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1561, in _get_cuda_arch_flags
arch_list[-1] += '+PTX'
IndexError: list index out of range
----------------------------------------
Rolling back uninstall of fairseq
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-eval-lm
from /tmp/pip-uninstall-xk2ohr_a/fairseq-eval-lm
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-generate
from /tmp/pip-uninstall-xk2ohr_a/fairseq-generate
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-hydra-train
from /tmp/pip-uninstall-xk2ohr_a/fairseq-hydra-train
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-interactive
from /tmp/pip-uninstall-xk2ohr_a/fairseq-interactive
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-preprocess
from /tmp/pip-uninstall-xk2ohr_a/fairseq-preprocess
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-score
from /tmp/pip-uninstall-xk2ohr_a/fairseq-score
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-train
from /tmp/pip-uninstall-xk2ohr_a/fairseq-train
Moving to /home/songpeng/.conda/envs/fairseq/bin/fairseq-validate
from /tmp/pip-uninstall-xk2ohr_a/fairseq-validate
Moving to /home/songpeng/.conda/envs/fairseq/lib/python3.8/site-packages/fairseq.egg-link
from /tmp/pip-uninstall-io6x89th/fairseq.egg-link
ERROR: Command errored out with exit status 1: /home/songpeng/.conda/envs/fairseq/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project_my/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project_my/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

It does not work for me....
I can run the fairseq sample you provided (https://github.com/pytorch/fairseq/blob/master/examples/bart/README.summarization.md)
Does it suggest my fairseq is correct?
Can you give more information?

After git clone the project, should I  run pip install --editable . for the project ?

I used this version of fairseq. If you can install this version, it means that your environment can support our code.
Yes. Also, you shouldn't install fairseq first, just install our repo is enough.

Thank you for your patience, but I still have a problem....
I prepared two same enviroments conda create A --clone B.
One can run pip install --editable ./ successfully for fairseq .
But for this repo, I still receive the error.
The steps are

PYTHON: conda create -n A python=3.8
PYTORCH: conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia
APEX: follow this
REPO: git clone this repo
cd guided_summarization/bart/ and pip install --editable ./

The error is:
(pytorch) [songpeng@pbcmlg01 bart]$ pip install --editable ./
Obtaining file:///data/songpeng/project/guided_summarization/bart
Requirement already satisfied: cffi in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (1.14.5)
Requirement already satisfied: cython in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (0.29.23)
Requirement already satisfied: numpy in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (1.20.2)
Requirement already satisfied: regex in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (2021.4.4)
Requirement already satisfied: sacrebleu in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (1.5.1)
Requirement already satisfied: torch in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (1.8.1)
Requirement already satisfied: tqdm in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from fairseq==0.9.0) (4.61.0)
Requirement already satisfied: pycparser in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from cffi->fairseq==0.9.0) (2.20)
Requirement already satisfied: portalocker==2.0.0 in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from sacrebleu->fairseq==0.9.0) (2.0.0)
Requirement already satisfied: typing_extensions in /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages (from torch->fairseq==0.9.0) (<IP_ADDRESS>)
Installing collected packages: fairseq
Attempting uninstall: fairseq
Found existing installation: fairseq 1.0.0a0+2fd9d8a
Uninstalling fairseq-1.0.0a0+2fd9d8a:
Successfully uninstalled fairseq-1.0.0a0+2fd9d8a
Running setup.py develop for fairseq
ERROR: Command errored out with exit status 1:
command: /data/songpeng/.conda/envs/pytorch/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /data/songpeng/project/guided_summarization/bart/
Complete output (106 lines):
No CUDA runtime is found, using CUDA_HOME='/cm/shared/apps/cuda11.1/toolkit/11.1.0'
running develop
running egg_info
writing fairseq.egg-info/PKG-INFO
writing dependency_links to fairseq.egg-info/dependency_links.txt
writing entry points to fairseq.egg-info/entry_points.txt
writing requirements to fairseq.egg-info/requires.txt
writing top-level names to fairseq.egg-info/top_level.txt
reading manifest file 'fairseq.egg-info/SOURCES.txt'
writing manifest file 'fairseq.egg-info/SOURCES.txt'
running build_ext
cythoning fairseq/data/data_utils_fast.pyx to fairseq/data/data_utils_fast.cpp
cythoning fairseq/data/token_block_utils_fast.pyx to fairseq/data/token_block_utils_fast.cpp
building 'fairseq.libbleu' extension
Emitting ninja build file /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
g++ -pthread -shared -B /data/songpeng/.conda/envs/pytorch/compiler_compat -L/data/songpeng/.conda/envs/pytorch/lib -Wl,-rpath=/data/songpeng/.conda/envs/pytorch/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/libbleu.o /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libbleu/module.o -o build/lib.linux-x86_64-3.8/fairseq/libbleu.cpython-38-x86_64-linux-gnu.so
building 'fairseq.data.data_utils_fast' extension
Emitting ninja build file /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o.d -pthread -B /data/songpeng/.conda/envs/pytorch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/pytorch/include/python3.8 -c -c /data/songpeng/project/guided_summarization/bart/fairseq/data/data_utils_fast.cpp -o /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=data_utils_fast -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
from /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /data/songpeng/project/guided_summarization/bart/fairseq/data/data_utils_fast.cpp:626:
/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
|  ^~~~~~~
g++ -pthread -shared -B /data/songpeng/.conda/envs/pytorch/compiler_compat -L/data/songpeng/.conda/envs/pytorch/lib -Wl,-rpath=/data/songpeng/.conda/envs/pytorch/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/data_utils_fast.o -o build/lib.linux-x86_64-3.8/fairseq/data/data_utils_fast.cpython-38-x86_64-linux-gnu.so
building 'fairseq.data.token_block_utils_fast' extension
Emitting ninja build file /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/1] c++ -MMD -MF /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o.d -pthread -B /data/songpeng/.conda/envs/pytorch/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include -I/data/songpeng/.conda/envs/pytorch/include/python3.8 -c -c /data/songpeng/project/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp -o /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o -std=c++11 -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=token_block_utils_fast -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
from /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /data/songpeng/project/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:627:
/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with "
|  ^~~~~~~
/data/songpeng/project/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp: In function ‘PyArrayObject* __pyx_f_7fairseq_4data_22token_block_utils_fast__get_slice_indices_fast(PyArrayObject*, PyObject*, int, int, int)’:
/data/songpeng/project/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:3306:36: warning: comparison of integer expressions of different signedness: ‘__pyx_t_7fairseq_4data_22token_block_utils_fast_DTYPE_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
3306 |       __pyx_t_4 = ((__pyx_v_sz_idx < __pyx_t_10) != 0);
|                     ~~~~~^~
/data/songpeng/project/guided_summarization/bart/fairseq/data/token_block_utils_fast.cpp:3501:36: warning: comparison of integer expressions of different signedness: ‘__pyx_t_7fairseq_4data_22token_block_utils_fast_DTYPE_t’ {aka ‘long int’} and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
3501 |       __pyx_t_3 = ((__pyx_v_sz_idx < __pyx_t_10) != 0);
|                     ~~~~~^~
g++ -pthread -shared -B /data/songpeng/.conda/envs/pytorch/compiler_compat -L/data/songpeng/.conda/envs/pytorch/lib -Wl,-rpath=/data/songpeng/.conda/envs/pytorch/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.o -o build/lib.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.cpython-38-x86_64-linux-gnu.so
building 'fairseq.libnat' extension
Emitting ninja build file /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
g++ -pthread -shared -B /data/songpeng/.conda/envs/pytorch/compiler_compat -L/data/songpeng/.conda/envs/pytorch/lib -Wl,-rpath=/data/songpeng/.conda/envs/pytorch/lib -Wl,--no-as-needed -Wl,--sysroot=/ /data/songpeng/project/guided_summarization/bart/build/temp.linux-x86_64-3.8/fairseq/clib/libnat/edit_dist.o -L/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/torch/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -o build/lib.linux-x86_64-3.8/fairseq/libnat.cpython-38-x86_64-linux-gnu.so
building 'fairseq.libnat_cuda' extension
Traceback (most recent call last):
File "", line 1, in
File "/data/songpeng/project/guided_summarization/bart/setup.py", line 118, in
setup(
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run
self.install_for_development()
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/setuptools/command/develop.py", line 136, in install_for_development
self.run_command('build_ext')
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 708, in build_extensions
build_ext.build_extensions(self)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
_build_ext.build_ext.build_extensions(self)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
objects = self.compiler.compile(sources,
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 524, in unix_wrap_ninja_compile
cuda_post_cflags = unix_cuda_flags(cuda_post_cflags)
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 423, in unix_cuda_flags
cflags + _get_cuda_arch_flags(cflags))
File "/data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1561, in _get_cuda_arch_flags
arch_list[-1] += '+PTX'
IndexError: list index out of range
----------------------------------------
Rolling back uninstall of fairseq
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-eval-lm
from /tmp/pip-uninstall-etrt6wr1/fairseq-eval-lm
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-generate
from /tmp/pip-uninstall-etrt6wr1/fairseq-generate
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-hydra-train
from /tmp/pip-uninstall-etrt6wr1/fairseq-hydra-train
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-interactive
from /tmp/pip-uninstall-etrt6wr1/fairseq-interactive
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-preprocess
from /tmp/pip-uninstall-etrt6wr1/fairseq-preprocess
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-score
from /tmp/pip-uninstall-etrt6wr1/fairseq-score
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-train
from /tmp/pip-uninstall-etrt6wr1/fairseq-train
Moving to /data/songpeng/.conda/envs/pytorch/bin/fairseq-validate
from /tmp/pip-uninstall-etrt6wr1/fairseq-validate
Moving to /data/songpeng/.conda/envs/pytorch/lib/python3.8/site-packages/fairseq.egg-link
from /tmp/pip-uninstall-un3pd2ki/fairseq.egg-link
ERROR: Command errored out with exit status 1: /data/songpeng/.conda/envs/pytorch/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/data/songpeng/project/guided_summarization/bart/setup.py'"'"'; file='"'"'/data/songpeng/project/guided_summarization/bart/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Hi, sorry that I'm not familiar with the fairseq installing procedure either and may just be able to give you some general suggestions:
I saw the error message No CUDA runtime is found, using CUDA_HOME='/cm/shared/apps/cuda11.1/toolkit/11.1.0'. If you import torch and run torch.cuda.is_available() inside container, does it detect the GPU?
Maybe you can try not to install apex?

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I'm running into the same issue. May I ask what is "migrate the guidxxx.py files to the corresponding directory of an available fairseq"? I guess you first install a new fairseq, then move Gsum-related code to the installed fairseq? Thanks!

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I'm running into the same issue. May I ask what is "migrate the guidxxx.py files to the corresponding directory of an available fairseq"? I guess you first install a new fairseq, then move Gsum-related code to the installed fairseq? Thanks!

Hi, that's exactly what I did.
And it may help you https://github.com/neulab/guided_summarization/issues/14#issuecomment-858437170

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I meet the same issue. And Following #14, I firstly install fairseq and try to copy Gsum-releated code to location where fairseq installed. Though I copy data/guided_language_pair_dataset.py, I still meet the error below

Do you have any idea about how to fix it? Thanks a lot!

Perhaps you should edit the file fairseq\fairseq\data_init_.py

add code :
from .guided_language_pair_dataset import GuidedLanguagePairDataset

and add  'GuidedLanguagePairDataset'  into list all
like below

all = [
'AppendTokenDataset',
'BacktranslationDataset',
'BaseWrapperDataset',
'ColorizeDataset',
'ConcatDataset',
'ConcatSentencesDataset',
'CountingIterator',
'DenoisingDataset',
'Dictionary',
'EpochBatchIterator',
'FairseqDataset',
'FairseqIterableDataset',
'GroupedIterator',
'IdDataset',
'IndexedCachedDataset',
'IndexedDataset',
'IndexedRawTextDataset',
'LanguagePairDataset',
'LeftPadDataset',
'ListDataset',
'LMContextWindowDataset',
'LRUCacheDataset',
'MaskTokensDataset',
'MMapIndexedDataset',
'MonolingualDataset',
'MultiCorpusSampledDataset',
'NestedDictionaryDataset',
'NoisingDataset',
'NumelDataset',
'NumSamplesDataset',
'OffsetTokensDataset',
'PadDataset',
'PrependDataset',
'PrependTokenDataset',
'ReplaceDataset',
'RollDataset',
'FileAudioDataset',
'RawLabelDataset',
'ResamplingDataset',
'RightPadDataset',
'RoundRobinZipDatasets',
'ShardedDataset',
'ShardedIterator',
'SortDataset',
'StripTokenDataset',
'SubsampleDataset',
'TokenBlockDataset',
'TransformEosDataset',
'TransformEosLangPairDataset',
'TruncateDataset',
'TruncatedDictionary',
'GuidedLanguagePairDataset'
]

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I meet the same issue. And Following #14, I firstly install fairseq and try to copy Gsum-releated code to location where fairseq installed. Though I copy data/guided_language_pair_dataset.py, I still meet the error below

Do you have any idea about how to fix it? Thanks a lot!
I copy:

models/bart/guided_hub_interface.py
models/bart/guided_model.py
tasks/guided_translation.py
data/guided_language_pair_dataset.py

Perhaps you should edit the file fairseq\fairseq\data_init_.py
add code :
from .guided_language_pair_dataset import GuidedLanguagePairDataset
and add 'GuidedLanguagePairDataset' into list all
like below
all = [
'AppendTokenDataset',
'BacktranslationDataset',
'BaseWrapperDataset',
'ColorizeDataset',
'ConcatDataset',
'ConcatSentencesDataset',
'CountingIterator',
'DenoisingDataset',
'Dictionary',
'EpochBatchIterator',
'FairseqDataset',
'FairseqIterableDataset',
'GroupedIterator',
'IdDataset',
'IndexedCachedDataset',
'IndexedDataset',
'IndexedRawTextDataset',
'LanguagePairDataset',
'LeftPadDataset',
'ListDataset',
'LMContextWindowDataset',
'LRUCacheDataset',
'MaskTokensDataset',
'MMapIndexedDataset',
'MonolingualDataset',
'MultiCorpusSampledDataset',
'NestedDictionaryDataset',
'NoisingDataset',
'NumelDataset',
'NumSamplesDataset',
'OffsetTokensDataset',
'PadDataset',
'PrependDataset',
'PrependTokenDataset',
'ReplaceDataset',
'RollDataset',
'FileAudioDataset',
'RawLabelDataset',
'ResamplingDataset',
'RightPadDataset',
'RoundRobinZipDatasets',
'ShardedDataset',
'ShardedIterator',
'SortDataset',
'StripTokenDataset',
'SubsampleDataset',
'TokenBlockDataset',
'TransformEosDataset',
'TransformEosLangPairDataset',
'TruncateDataset',
'TruncatedDictionary',
'GuidedLanguagePairDataset'
]

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I meet the same issue. And Following #14, I firstly install fairseq and try to copy Gsum-releated code to location where fairseq installed. Though I copy data/guided_language_pair_dataset.py, I still meet the error below

Do you have any idea about how to fix it? Thanks a lot!
I copy:

models/bart/guided_hub_interface.py
models/bart/guided_model.py
tasks/guided_translation.py
data/guided_language_pair_dataset.py

Perhaps you should edit the file fairseq\fairseq\data_init_.py
add code :
from .guided_language_pair_dataset import GuidedLanguagePairDataset
and add 'GuidedLanguagePairDataset' into list all
like below
all = [
'AppendTokenDataset',
'BacktranslationDataset',
'BaseWrapperDataset',
'ColorizeDataset',
'ConcatDataset',
'ConcatSentencesDataset',
'CountingIterator',
'DenoisingDataset',
'Dictionary',
'EpochBatchIterator',
'FairseqDataset',
'FairseqIterableDataset',
'GroupedIterator',
'IdDataset',
'IndexedCachedDataset',
'IndexedDataset',
'IndexedRawTextDataset',
'LanguagePairDataset',
'LeftPadDataset',
'ListDataset',
'LMContextWindowDataset',
'LRUCacheDataset',
'MaskTokensDataset',
'MMapIndexedDataset',
'MonolingualDataset',
'MultiCorpusSampledDataset',
'NestedDictionaryDataset',
'NoisingDataset',
'NumelDataset',
'NumSamplesDataset',
'OffsetTokensDataset',
'PadDataset',
'PrependDataset',
'PrependTokenDataset',
'ReplaceDataset',
'RollDataset',
'FileAudioDataset',
'RawLabelDataset',
'ResamplingDataset',
'RightPadDataset',
'RoundRobinZipDatasets',
'ShardedDataset',
'ShardedIterator',
'SortDataset',
'StripTokenDataset',
'SubsampleDataset',
'TokenBlockDataset',
'TransformEosDataset',
'TransformEosLangPairDataset',
'TruncateDataset',
'TruncatedDictionary',
'GuidedLanguagePairDataset'
]

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I meet the same issue. And Following #14, I firstly install fairseq and try to copy Gsum-releated code to location where fairseq installed. Though I copy data/guided_language_pair_dataset.py, I still meet the error below

Do you have any idea about how to fix it? Thanks a lot!
I copy:

models/bart/guided_hub_interface.py
models/bart/guided_model.py
tasks/guided_translation.py
data/guided_language_pair_dataset.py

Perhaps you should edit the file fairseq\fairseq\data_init_.py
add code :
from .guided_language_pair_dataset import GuidedLanguagePairDataset
and add 'GuidedLanguagePairDataset' into list all
like below
all = [
'AppendTokenDataset',
'BacktranslationDataset',
'BaseWrapperDataset',
'ColorizeDataset',
'ConcatDataset',
'ConcatSentencesDataset',
'CountingIterator',
'DenoisingDataset',
'Dictionary',
'EpochBatchIterator',
'FairseqDataset',
'FairseqIterableDataset',
'GroupedIterator',
'IdDataset',
'IndexedCachedDataset',
'IndexedDataset',
'IndexedRawTextDataset',
'LanguagePairDataset',
'LeftPadDataset',
'ListDataset',
'LMContextWindowDataset',
'LRUCacheDataset',
'MaskTokensDataset',
'MMapIndexedDataset',
'MonolingualDataset',
'MultiCorpusSampledDataset',
'NestedDictionaryDataset',
'NoisingDataset',
'NumelDataset',
'NumSamplesDataset',
'OffsetTokensDataset',
'PadDataset',
'PrependDataset',
'PrependTokenDataset',
'ReplaceDataset',
'RollDataset',
'FileAudioDataset',
'RawLabelDataset',
'ResamplingDataset',
'RightPadDataset',
'RoundRobinZipDatasets',
'ShardedDataset',
'ShardedIterator',
'SortDataset',
'StripTokenDataset',
'SubsampleDataset',
'TokenBlockDataset',
'TransformEosDataset',
'TransformEosLangPairDataset',
'TruncateDataset',
'TruncatedDictionary',
'GuidedLanguagePairDataset'
]

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I meet the same issue. And Following #14, I firstly install fairseq and try to copy Gsum-releated code to location where fairseq installed. Though I copy data/guided_language_pair_dataset.py, I still meet the error below

Do you have any idea about how to fix it? Thanks a lot!
I copy:

models/bart/guided_hub_interface.py
models/bart/guided_model.py
tasks/guided_translation.py
data/guided_language_pair_dataset.py

Perhaps you should edit the file fairseq\fairseq\data_init_.py
add code :
from .guided_language_pair_dataset import GuidedLanguagePairDataset
and add 'GuidedLanguagePairDataset' into list all
like below
all = [
'AppendTokenDataset',
'BacktranslationDataset',
'BaseWrapperDataset',
'ColorizeDataset',
'ConcatDataset',
'ConcatSentencesDataset',
'CountingIterator',
'DenoisingDataset',
'Dictionary',
'EpochBatchIterator',
'FairseqDataset',
'FairseqIterableDataset',
'GroupedIterator',
'IdDataset',
'IndexedCachedDataset',
'IndexedDataset',
'IndexedRawTextDataset',
'LanguagePairDataset',
'LeftPadDataset',
'ListDataset',
'LMContextWindowDataset',
'LRUCacheDataset',
'MaskTokensDataset',
'MMapIndexedDataset',
'MonolingualDataset',
'MultiCorpusSampledDataset',
'NestedDictionaryDataset',
'NoisingDataset',
'NumelDataset',
'NumSamplesDataset',
'OffsetTokensDataset',
'PadDataset',
'PrependDataset',
'PrependTokenDataset',
'ReplaceDataset',
'RollDataset',
'FileAudioDataset',
'RawLabelDataset',
'ResamplingDataset',
'RightPadDataset',
'RoundRobinZipDatasets',
'ShardedDataset',
'ShardedIterator',
'SortDataset',
'StripTokenDataset',
'SubsampleDataset',
'TokenBlockDataset',
'TransformEosDataset',
'TransformEosLangPairDataset',
'TruncateDataset',
'TruncatedDictionary',
'GuidedLanguagePairDataset'
]

wow, Thanks, it works! @SONGPENG  But I meet anothor OOM error when loading pretrained model and finetuning on my own dataset.

Many issues in original fairseq show changing UPDATE_FREQ and TOTAL_NUM_UPDATES can reduce the batch size to solve this problem. Did you change the settings in z_train.sh?

I migrate the guidxxx.py files to the corresponding directory of an available fairseq.
It seems to work temporarily.

Hi, I meet the same issue. And Following #14, I firstly install fairseq and try to copy Gsum-releated code to location where fairseq installed. Though I copy data/guided_language_pair_dataset.py, I still meet the error below

Do you have any idea about how to fix it? Thanks a lot!
I copy:

models/bart/guided_hub_interface.py
models/bart/guided_model.py
tasks/guided_translation.py
data/guided_language_pair_dataset.py

Perhaps you should edit the file fairseq\fairseq\data_init_.py
add code :
from .guided_language_pair_dataset import GuidedLanguagePairDataset
and add 'GuidedLanguagePairDataset' into list all
like below
all = [
'AppendTokenDataset',
'BacktranslationDataset',
'BaseWrapperDataset',
'ColorizeDataset',
'ConcatDataset',
'ConcatSentencesDataset',
'CountingIterator',
'DenoisingDataset',
'Dictionary',
'EpochBatchIterator',
'FairseqDataset',
'FairseqIterableDataset',
'GroupedIterator',
'IdDataset',
'IndexedCachedDataset',
'IndexedDataset',
'IndexedRawTextDataset',
'LanguagePairDataset',
'LeftPadDataset',
'ListDataset',
'LMContextWindowDataset',
'LRUCacheDataset',
'MaskTokensDataset',
'MMapIndexedDataset',
'MonolingualDataset',
'MultiCorpusSampledDataset',
'NestedDictionaryDataset',
'NoisingDataset',
'NumelDataset',
'NumSamplesDataset',
'OffsetTokensDataset',
'PadDataset',
'PrependDataset',
'PrependTokenDataset',
'ReplaceDataset',
'RollDataset',
'FileAudioDataset',
'RawLabelDataset',
'ResamplingDataset',
'RightPadDataset',
'RoundRobinZipDatasets',
'ShardedDataset',
'ShardedIterator',
'SortDataset',
'StripTokenDataset',
'SubsampleDataset',
'TokenBlockDataset',
'TransformEosDataset',
'TransformEosLangPairDataset',
'TruncateDataset',
'TruncatedDictionary',
'GuidedLanguagePairDataset'
]

wow, Thanks, it works! @SONGPENG But I meet anothor OOM error when loading pretrained model and finetuning on my own dataset.

Many issues in original fairseq show changing UPDATE_FREQ and TOTAL_NUM_UPDATES can reduce the batch size to solve this problem. Did you change the settings in z_train.sh?

Sorry, that's not what happened to me. But I think changing the settings in z_train.sh is workable and you can give it a try. : D


updated the config.js to serve js file on react-foundation-docs/

Fixes # Fix routing / base of React app at GitHub Pages #29
Changes proposed in this pull request:

updated output object in the webpack/config.js file  to serve js files on the path react-foundation-docs/

Thanks for the PR!
Testing this with: rm -rf dist && npm run dist && npm run serve
I can then visit http://localhost:5000/ and the site loads, but I only see a directory listing of the app.js at http://localhost:5000/react-foundation-docs
Will that mean the deployed site also won't load?

Let's close this as https://github.com/digiaonline/react-foundation-docs/pull/31 got it working first. Thanks again!


What is the role of Client Application in this example? Is it redundant? Azure APIM, AAD

What is the role of the Client application in this example below?
https://docs.microsoft.com/en-us/azure/api-management/api-management-howto-protect-backend-with-aad
An access token can be requested via Developer Portal without the steps related to the Client application on the link above.
Any idea?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 823eb96f-e5f1-0832-b175-0b31009be92f
Version Independent ID: 330085ae-dcf6-e930-86a8-0e4a60ab1314
Content: Protect API backend in API Management using OAuth 2.0 and Azure AD - Azure API Management
Content Source: articles/api-management/api-management-howto-protect-backend-with-aad.md
Service: api-management
GitHub Login: @miaojiang
Microsoft Alias: apimpm

@PingPongSet The Client Application here is the Developer Portal and is the one that fetches the token for making requests to the backend (APIM). The Client ID and Client Secret for the Client Application is required to be configured as shown in the docs to enable OAuth 2.0 user authorization in the developer portal.

@PingPongSet I can get it to work using both endpoints and using both the client/backend app but using a separate client is the better approach like I've mentioned before.
Please do note that this repository is primarily for addressing doc related feedback. For other queries, Microsoft Q&A is a better forum to engage the wider community. Since there is no doc update required here, we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Improve error messages from LocalTaskNodeExecutor

Make use of the new problem reporting system and document the error messages.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test QFL

OK, I've already triggered QuickFeedbackLinux build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test RFM

OK, I've already triggered ReadyForMerge build for you.


Uncaught ReferenceError: exports is not defined

I get the following error message as soon as BrowserSync reloads the page after I make some changes in any file under src:
Uncaught ReferenceError: exports is not defined
Failed to load resource: the server responded with a status of 404 (Not Found)
(SystemJS) XHR error (404 Not Found) loading http://localhost:5555/main.web
Does anybody have an idea on how to solve this?
Thank you!

This is likely related to your APP_BASE. See this: https://github.com/NathanWalker/angular-seed-advanced/issues/275


Support a "labels" field in AxisInfo if present.

ezmsg-lsl has a SpaceAxis that I patch into AxisArray, and SpaceAxis has a labels field. This solves an issues that we discussed in #43
Units that transform the AxisArray in a way that modifies the length of the dimension with the "labels" field, should probably also modify the labels, if present! This PR does that for Units that I'm aware of that modify the shape of the array along axes other than "time".
For affine_transform, it's difficult to know what the new labels should be so in most cases they are just dropped. However, if the affine_transform has rows or columns of zeros then we can get a good guess at thew new labels.

I didn't know about an lsl connector; this is really cool!  When you're ready, let's add it into the README as another extension :D
AxisArray is VERY inspired by XArray (my absolutely favorite data structure for offline analyses) that unfortunately has a little too much going on under the hood to be performant enough for low latency, high-message rate stream processing.  An earlier version of the ezmsg-sigproc lib actually used DataArrays for the messages and that was a lovely developer experience even if my CPUs pegged processing 8 channels of 250 Hz EEG data -- but I digress.
I feel like the general solution to this particular problem is adding a coords implementation into AxisArray that captures some core set of functionality that we find in XArray's DataArray coords.  I wrote the majority of the AxisArray implementation in an afternoon when I was pressured by an incoming deadline.  I spent about an hour thinking about how to get coords in without destroying performance or my guiding mantra that "AxisArray is simply a metadata wrapper around a numpy array".  I came to the conclusion at that time that a limited coords functionality in AxisArray would 1. be incredibly useful, 2. could be performant if we removed all the integrity checks/safety rails that XArray's implementation has.  I simply didn't have the bandwidth (at that time) to even do a first-pass implementation.
In summary, AxisArray needs some TLC for

Better Typehinting (even simple fixes like sel/isel returning subclasses of AxisArray, not just AxisArrays)
Better dev experience/consistency (why is concatenate a staticmethod?)
A lightweight coords implementation

Coming back to today and this PR: These operations on coords/labels probably eventually belong in AxisArray helper functions/implementation rather than spread throughout the signal processing modules.  Collecting common operations on AxisArrays together in one place would make this whole thing a lot easier to maintain, but if this gets you where you're going today I support a merge after review (feel free to merge it in yourself once my review posts)

Moved to https://github.com/ezmsg-org/ezmsg-sigproc/pull/1


bpf: Shift ingress ipcache source lookup to netdev

Ingress paths may be one of the following:

From a local endpoint. In this case the source is known.
From the overlay. In this case the source is taken from the overlay.
From a local netdev. In this case, we don't know the source and should
perform a lookup to determine what we can about the source.

This final case was being implemented in the ingress policy lookup
stage, which adds complexity to the ingress BPF program and also makes
monitor output less consistent, as the actual source is not known until
that point and may not be propagated to the correct parts of the program
so that the real identity is reported in monitor output.
Solve these problems by shifting this ipcache lookup all the way back
into the netdev program.
Changes in instruction complexity on Linux 4.9.17 with GCC 7.2.0:
bpf_lxc ingress:        49458 -> 26197 (-47%)
bpf_netdev handle_ipv4: 3471 -> 4682   (+35%)
bpf_netdev handle_ipv6: 8847 -> 13326  (+50%)
Note that the complexity limit in this case is 64K on Linux 4.9, so
although we increase the complexity in each of the netdev programs by
fair margin, the decrease in complexity in the ingress lxc program
provides us significantly more margin for feature development.
Furthermore, depending on where the ipcache is called from, it clearly
has a different impact on complexity limit; In the lxc program from the
ingress policy function, it represents ~23K instructions of complexity
due to the nesting of where it is called from. In the netdev programs,
this is only 1.2K or 4.5K.

This change is

test-me-please

Hit #4850:
https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/5031/testReport/junit/k8s-1/10/K8sValidatedDemosTest_Tests_Star_Wars_Demo/
f0c5341e_K8sValidatedDemosTest_Tests_Star_Wars_Demo.zip
Found a "NACK received for version" in Cilium logs

2018-07-18T02:21:00.340554727Z level=warning msg="[external/envoy/source/common/config/grpc_mux_impl.cc:218] gRPC config for type.googleapis.com/cilium.NetworkPolicyHosts update rejected: NetworkPolicyHosts: Duplicate host entry '<IP_ADDRESS>' for policy 11696, already mapped to 3" subsys=envoy-upstream threadID=1104
    2018-07-18T02:21:00.34056298Z level=warning msg="[bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_mux_subscription_lib/common/config/grpc_mux_subscription_impl.h:70] gRPC config for type.googleapis.com/cilium.NetworkPolicyHosts rejected: NetworkPolicyHosts: Duplicate host entry '<IP_ADDRESS>' for policy 11696, already mapped to 3" subsys=envoy-config threadID=1104

test-me-please

Green build. Rebasing to fix up bpf.sha.


Bring the task reference field to the frontend.

On some days our team creates more kanboard tasks as we can work on.
This leaves us with a lot of tasks and the overview gets lost more and more.
The filter and searchbar already help a lot, but some kind of tag would even increase the productivity.
How we work with kanboard: (possibly unimportant)

We use swimlanes for our products (desktop / web-version / documentation etc)
We use columns for the progress (new / in progress / finished / etc) - simplified
We use categories for a rough classification (bug / feature request / etc)

It would be nice to be able to add searchable texts / tags to each task, so we would not have to overload our Descriptions/Titles.
for example:

customer names
external supplier issue-ids
inhouse tags to find similar tasks

The reference might be what we are looking for:

It's already integrated in the search
It would be able to store the tags we need

Reference #1223
Best regards!
Tomas

You could write your own plugin and use the  metadata feature
You could also take a look at https://github.com/fguillot/kanboard/issues/646 or https://github.com/fguillot/kanboard/issues/1411#issuecomment-158904040

Thanks!
I will try the plugin path.


Country Request: Cambodia

Hi, I would like to ask if Cambodia could too be added to the list. Thanks in advance! :-)

added!

Cambodia is now available


https proxy example please

I'm trying to learn to use goproxy through the example https://github.com/elazarl/goproxy/tree/master/examples/goproxy-httpdump.
I found that it can only handle http requests, not https'. Could you provide an example that can handle https requests please? (As almost all web servers are moving away from http to https now, e.g., github, wikipedia, wordpress, etc).
PS, by "handle https requests" I meant that none of the https requests are dumped by the current goproxy-httpdump program.
Thanks

Suppose tracking https requests is impossible (or too difficult), then my next question is how to output those "Tunnel to" requests from goproxy-httpdump?
I.e., each https request is lead by a normal HTTP request of type "Tunnel to", which has the following headers:
CONNECT csi.gstatic.com:443 HTTP/1.1
Host: csi.gstatic.com:443
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36

A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.

Version: 3.3 (TLS/1.2)
Random: ...

I need to dump those requests as well from goproxy-httpdump.
Thanks

1
https://github.com/elazarl/goproxy/blob/master/examples/goproxy-transparent/transparent.go
This example is not very well...

What about this example:
https://github.com/elazarl/goproxy/blob/master/examples/goproxy-eavesdropper/main.go

The problem is that I found goproxy can only handle http requests, not https'.
That was the question, and I am afraid that I wasn't able to find the answer from the goproxy-eavesdropper example. Maybe I'm wrong about that.

I've built a simple HTTPS logging proxy on top of goproxy, at https://github.com/fredcy/proxy

I guess if you add this to your main function, it should work. It will do man-in-the-middle attacks against HTTPS.
GoProxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*:443$"))).
	HandleConnectFunc(func(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
		return goproxy.MitmConnect, host
	})

That's pretty much what I do in my logging proxy.
https://github.com/fredcy/proxy/blob/73914b909caf97d0f92a0d9b075c3fda33197b0f/main.go#L109-L110

Thanks a lot @fredcy & @rahulwa, I'll look into it this weekend and get back to you...

This, in deed, is doing man-in-the-middle attacks against HTTPS, which is not allowed by default.
The fiddler circumvents it by installing its own Root Certificate into the system (on Windows), @fredcy, how did you solve the problem? Thx.

I imported the CA certificate embedded in goproxy into the Keystore on my MacOS client and marked it as trusted. With that done my Mac client uses the proxy without any complaints or warnings about cert problems.
The needed certificate is contained in the certs.go file.  At the time of writing it's here: https://github.com/elazarl/goproxy/blob/4327d5f85a6da046b9b4818382a3e51f795a249b/certs.go#L24-L57
I think the ca.pem file is supposed to be exactly that cert but it's out of date. I opened https://github.com/elazarl/goproxy/issues/216 about that.

You are correct. CA.pem out of date.
Mind sending a PR to fix that?
On Thu, Apr 13, 2017, 5:15 PM Fred Yankowski<EMAIL_ADDRESS>wrote:

I imported the CA certificate embedded in goproxy into the Keystore on my
MacOS client and marked it as trusted. With that done my Mac client uses
the proxy without any complaints or warnings about cert problems.
The needed certificate is contained in the certs.go file. At the time of
writing it's here:
https://github.com/elazarl/goproxy/blob/4327d5f85a6da046b9b4818382a3e51f795a249b/certs.go#L24-L57
I think the ca.pem file is supposed to be exactly that cert but it's out
of date. I opened #216 https://github.com/elazarl/goproxy/issues/216
about that.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/elazarl/goproxy/issues/192#issuecomment-293907698,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAP4ogwo-heMiAv0svNPntYOOOb9dAPcks5rvi3mgaJpZM4KUFEf
.

OK. https://github.com/elazarl/goproxy/pull/218

I spent time working on something else, but
looks good. :+1:


Where are your released checkpoints trained

Thanks for your excellent work. I would like to ask, where are your released checkpoints trained?  my evaluation results (fid, fvd) on the tiktok dataset are quite different from the results you published.
I wanted to inquire if the released checkpoint is not the checkpoint you used in the tiktok results on your paper.

Hi, we do release a different checkpoint using different hyperparameters which generalizes better in this repo.

we find that is not "better" but worse

Hi, we do release a different checkpoint using different hyperparameters which generalizes better in this repo.

we find that is not "better" but worse

I mean the ckpt we released can work better for ood reference image but its numerical results on TikTok benchmark could be worse.

Hi, we do release a different checkpoint using different hyperparameters which generalizes better in this repo.

we find that is not "better" but worse

I mean the ckpt we released can work better for ood reference image but its numerical results on TikTok benchmark could be worse.

Thanks for your excellent work. Will you realse the Sup. Mat. recently？

Thanks for your excellent work. I would like to ask, where are your released checkpoints trained? my evaluation results (fid, fvd) on the tiktok dataset are quite different from the results you published.
I wanted to inquire if the released checkpoint is not the checkpoint you used in the tiktok results on your paper.

Hi, I am working on reproducing the FVD too. How do you process the TikTok Dataset? I find only about 15 videos are used as the test split in other Paper, for example DISCO. Could you tell me about the details about you test pipeline? Looking forward you reply!

@Worromots Hi, I believe this issue #133 is also related. We used evaluation code released by DisCo and followed their train/test split for TikTok dataset.

@Worromots Hi, I believe this issue #133 is also related. We used evaluation code released by DisCo and followed their train/test split for TikTok dataset.

Thank you for your answer; it has resolved a question that has puzzled me for many days. I have another question: for the test set of the TED dataset, did you generate only the first n frames during testing, or did you generate all the frames? For example, for a clip with 200 frames, do you need to generate all 200 frames during testing, or just the first 32 frames?

@Worromots Hi, I believe this issue #133 is also related. We used evaluation code released by DisCo and followed their train/test split for TikTok dataset.

Thank you for your answer; it has resolved a question that has puzzled me for many days. I have another question: for the test set of the TED dataset, did you generate only the first n frames during testing, or did you generate all the frames? For example, for a clip with 200 frames, do you need to generate all 200 frames during testing, or just the first 32 frames?

We generate all the frames for testing.

I mean the ckpt we released can work better for ood reference image but its numerical results on TikTok benchmark could be worse.

Sorry, I found that the numerical results on the Tiktok benchmark are far away from the results you mentioned in your paper and this results have puzzled me for many days, can you release the results on Tiktok or Ted-talk achieved by your released checkpoint.

I mean the ckpt we released can work better for ood reference image but its numerical results on TikTok benchmark could be worse.

Sorry, I found that the numerical results on the Tiktok benchmark are far away from the results you mentioned in your paper and this results have puzzled me for many days, can you release the results on Tiktok or Ted-talk achieved by your released checkpoint.

Hi, I'm not sure whether your evaluation codebase is the same as ours or not since we haven't released this part yet. We will release all the ckpts and codebase in the future, please stay tuned, thx.

I mean the ckpt we released can work better for ood reference image but its numerical results on TikTok benchmark could be worse.

Sorry, I found that the numerical results on the Tiktok benchmark are far away from the results you mentioned in your paper and this results have puzzled me for many days, can you release the results on Tiktok or Ted-talk achieved by your released checkpoint.

Hi, I'm not sure whether your evaluation codebase is the same as ours or not since we haven't released this part yet. We will release all the ckpts and codebase in the future, please stay tuned, thx.

Hi, what is the resolution of TED/TiKTOK dataset you evaluate?
512px or 256px or 384px?

We use 512x512 for evaluation.

We use 512x512 for evaluation.

Thanks for your reply. Do you only use the first 100 frames of each video in TED/TiKTOK test dataset to extract poses and perform inference?

{'FVD-3DRN50': 44.1599514373319, 'FVD-3DInception':<PHONE_NUMBER>223029}
FID:  48.077198307792685
This is the result I reproduced using the gen_val.sh script from the DISCO codebase, where I generated all frames for evaluation.


Unable to develop on M1 mac

I am trying to run the build command in tree but it's saying it does not exist in the package.json
So I tried to run  npx father build After that I got the 2 folders lib and es in the tree folder. and after that I run
yarn dev in vite folder

I want to contribute to this project.

Hi @sanketkheni01
I'm happy that you want to contribute to the project. Which part do you want to contribute?
I believe you need to run yarn again on vite folder to make sure everything is good. Here are the complete steps to set up development https://github.com/thanhlmm/refi-app/blob/main/DEVELOPMENT.md.
Basically, we will need to build 2 libraries: rc-tree and firestore-serialzierbefore installvite` dependencies

Hey, I'm sorry for the misleading command. You should run
yarn compile

In tree folder to build

Can you please provide which node version, python version, and os you are using? I am getting errors while running yarn command in server it's something related to node-gyp

Node 17 was the problem. switched to 14 and now everything just worked.


7400 Parts

I have about 20 7400 parts ready to go in a library.
You may want to create a new library for digital parts since there will eventually be a lot of them.
Which library should I add them to?
John Eaton

@ubruhin did you want to create a "logic" library, or should they go into a base library?
@ouabache I wanted to take a look at your open PRs, but so far haven't found the time to do so, sorry.

I am worried that if we put everything into the base that it will grow to be unmanageable. Splitting off connectors was a good idea and I think that digital is big enough that it could use its own library. We can use the IC library for analog parts and create a Logic lib for digital.
I have a complete set of symbols for the first 20 7400 parts. These include full chip plus gates plus demorgan gates. I also have 2 different sizes of gates. If you could create a LibrePCB_Logic,lplib then I will check them in.
I did follow the industry practice of putting power and ground on the top and bottom. This violates your guidelines but I feel that it will be better accepted by users. Let me know if I should change this.
I will also create a set of generic devices for both thru hole and surface mount parts.

I did follow the industry practice of putting power and ground on the top and bottom. This violates your guidelines

This is more of a rough guideline (e.g. for ICs), I think for well-established symbols it might make sense to put them on the top and bottom.
I also think creating a separate LibrePCB_Logic library for digital logic symbols / components might make sense. The concrete devices would then go into manufacturer-specific libs I guess. Do you agree @ubruhin?

I would say LibrePCB_ICs.lplib is fine for the manufacturer independent elements (mainly symbols+components).

The concrete devices would then go into manufacturer-specific libs I guess.

:+1:

OK, I will add them them into that repo
On Mon, Mar 30, 2020 at 11:39 AM U. Bruhin<EMAIL_ADDRESS>wrote:

I would say LibrePCB_ICs.lplib
https://github.com/LibrePCB-Libraries/LibrePCB_ICs.lplib is fine for
the manufacturer independent elements (mainly symbols+components).
The concrete devices would then go into manufacturer-specific libs I guess.
👍
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LibrePCB-Libraries/LibrePCB_Base.lplib/issues/61#issuecomment-606171361,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKX3XF4BXSV374FFK2L2CLRKDRNXANCNFSM4LWG6EQQ
.

Everything is checked in

OK, So two of the symbols failed with a "non-canonical file" error. How can I debug this?

OK, So two of the symbols failed with a "non-canonical file" error. How can I debug this?

You can open the symbols in LibrePCB, append an "x" (or any letter) to the name, then save the part and run "git diff" in the library. Then you'll see how the generated part should look like.

Thanks, That worked.
I had been editing the uuid_cache files so that my generated files would
match my originals and some spaces slipped in. That did it.
On Mon, Mar 30, 2020 at 11:53 PM Danilo Bargen<EMAIL_ADDRESS>wrote:

OK, So two of the symbols failed with a "non-canonical file" error. How
can I debug this?
You can open the symbols in LibrePCB, append an "x" (or any letter) to the
name, then save the part and run "git diff" in the library. Then you'll see
how the generated part should look like.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LibrePCB-Libraries/LibrePCB_Base.lplib/issues/61#issuecomment-606436146,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKX3XFFSHTWRN2BZXYGFWLRKGHNHANCNFSM4LWG6EQQ
.

As discussed, this issue is not related to the base library so I'll close it.


Add cmd+k support for search

Add support for cmd+k for search both on landing page and navigation bar.

Thanks for your contribution 🫡


Modified the time from ISO format to unix timestamp.ALso correct spel…

Modified the time from ISO format to unix timestamp.ALso correct spel ling mistake in SourceMessage
Issue #41

Coverage decreased (-0.03%) to 74.956% when pulling e9b5d0d3c42d462dd6b3255ddb0c73091f72a2d3 on feature/guble-#41-change-time-from-ISO-8601-to-unix-timestamp into 325ff33b87cd69a21d894dcb19ba80e579196976 on master.

Please also change the README.md

Ok..I will also change the Readme.md..I forgot about it


Remove HEAD from limit_except directive in nginx conf files.

Removing unnecessary http method. HEAD is implicit when GET is present.
For further reading about this from the nginx docs: http://nginx.org/en/docs/http/ngx_http_core_module.html#limit_except

Thanks for the fix @jdkelley


exits with status code 1

I am running on Ubuntu 16.04 LTS and using dochat.sh
+ main
++ id -u
+ '[' 0 -ne 0 ']'
+ setupUserGroup
+ '[' -n 29 ']'
+ groupmod -o -g 29 audio
+ '[' -n 44 ']'
+ groupmod -o -g 44 video
++ id -g user
+ '[' 1001 '!=' 1000 ']'
+ groupmod -o -g 1001 group
++ id -u user
+ '[' 1001 '!=' 1000 ']'
+ usermod -o -u 1001 user
+ chown user:group '/home/user/.wine/drive_c/users/user/Application Data' '/home/user/WeChat Files'
+ setupHostname
+ export HOSTNAME=DoChat
+ HOSTNAME=DoChat
+ echo DoChat
+ hostname DoChat
+ exec gosu user /entrypoint.sh
+ main
++ id -u
+ '[' 1001 -ne 0 ']'
+ startWechat
+ hello
++ cat /VERSION
[DoChat] 盒装微信 v0.5.2
+ VERSION=0.5.2
+ echo '[DoChat] 盒装微信 v0.5.2'
+ disableUpgrade
+ wine REG ADD 'HKEY_CURRENT_USER\Software\Tencent\WeChat' /v NeedUpdateType /t REG_DWORD /d 0 /f

Interesting... It seems that the reg add command failed in your docker container, which works without any problem for me (and many other users).
I have added an error message and ignore the failure as a workaround for this problem just now.
Please wait for the docker image published (need about 15 mins) as v0.5.3 and try again.