Pytorch 1.6-1.8 compatability - CUDA11/3090 ready

Modified from pull request from @half-potato for compatibility with torch 1.6. Replaced THBlas functions with aten tensor functions.
Tested for torch 1.7 and 1.8 with cuda 10 and 11.
Worked with RTX2080 and RTX3090.
@xdynames

#90 #89 #88 #74

Hi! Matthew,
I have a RTX3090, and cloned your project that you modified 14 hours ago.
While ./make.sh still get the error about:
nvcc fatal   : Unsupported gpu architecture 'compute_86'

I got a Ubuntu 18.04, CUDA 11.1 pytorch 1.7, and gcc 7.5.0 / g++ 7.5.0
I guess it's probably the CUDA caused error, ANY HELP WOULD BE APPRECIATED!!

Jerry do you install you use a nightly binary for your Pytorch? https://discuss.pytorch.org/t/rtx-3000-support/98158
I have built this in a docker container using Nvidia's base image of CUDA11.1 then using the pip command in the link to install pytorch compiled with the RTX3000 support and it seems to work well

Thanks, @XDynames .
I used to got a pytorch from
pip install torch==1.7.0+cu110 torchvision==0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
And under your suggestion about I should use nightly binary, so I use the pip command:
pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html
And my pytorch version looks like now:

And now I got a ninja related error like this:

result in the RuntimeError: Error compiling objects for extension
I will double check all  the NVIDIA packages, and find a way to solve the ninja problem.
Thanks AGAIN!

I used this [docker image](docker pull nvidia/cuda:11.1-devel-ubuntu18.04) - installed conda then torch-nightly.
I then cloned and compiled DCNv2. This could be an issue with Cuda11.0 or some other conflicting packages.
When DCN doesn't compile usually the error from the cause is above your screen cap - if you run the ./make again the compiled parts will not run and it will make it clearer what is causing the issue.

Hi, @MatthewHowe Thanks for your great abvice!
I double checked my CUDA installation, and nvcc settings. After proper set those environment variables. It won't cause the correspond errors like ['nvcc', '-v'].
While on the  contrary, ninja still have report an error about the FAIL in 'THCudaBlas_SgemmBatched'.
It seems to be a new problem.
The log is like this:
FAILED: /home/liurui/DCNv2/build/temp.linux-x86_64-3.7/home/liurui/DCNv2/DCN/src/cuda/dcn_v2_cuda.o
/usr/local/cuda-11.1/bin/nvcc -DWITH_CUDA -I/home/liurui/DCNv2/DCN/src -I/home/liurui/anaconda3/envs/FairMOT/lib/python3.7/site-packages/torch/include -I/home/liurui/anaconda3/envs/FairMOT/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/liurui/anaconda3/envs/FairMOT/lib/python3.7/site-packages/torch/include/TH -I/home/liurui/anaconda3/envs/FairMOT/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.1/include -I/home/liurui/anaconda3/envs/FairMOT/include/python3.7m -c -c /home/liurui/DCNv2/DCN/src/cuda/dcn_v2_cuda.cu -o /home/liurui/DCNv2/build/temp.linux-x86_64-3.7/home/liurui/DCNv2/DCN/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_80,code=sm_80 -ccbin g++ -std=c++14
/home/liurui/DCNv2/DCN/src/cuda/dcn_v2_cuda.cu(126): error: identifier "THCudaBlas_SgemmBatched" is undefined

Maybe we should look into replacing SgemmBatched with a non deprecated version for 1.8 support?

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html

the same error

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html

the same error

you can try downgrade pytorch version to 1.7 stable, it work fine with me.

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html

the same error

Current fix is to pick an older binary of torch pre- 13NOV

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html

the same error

you can try downgrade pytorch version to 1.7 stable, it work fine with me.
Thank you. I will try it again.

I have compiled successfully using pytorch1.7. Thanks. @jerryhitit @MatthewHowe

I successfully compiled on Windows 10, CUDA 11.1 (RTX3090), and PyTorch 1.7. Thank you so much!

@MatthewHowe Hi Matthew, I failed to compile using pytorch1.7 with RuntimeError: Error compiling objects for extension.
I used the latest version of you which supports pytorch1.7
My environment (using anaconda virtual env):

gcc 7.5.0
ninja 1.10.2
ubuntu 18.04
python 3.7
pytorch 1.7
cudatoolkit 10.2
torch.cuda.is_available return True and CUDA home is not None
Error Message:
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1522, in _run_ninja_build
env=env)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/subprocess.py", line 481, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "setup.py", line 69, in
cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 653, in build_extensions
build_ext.build_extensions(self)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 482, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1238, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1538, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
Could you help me?

Double check that your versions all line up - if you want to use CUDA 10.2 make sure CUDNN is the correct version and the pytorch binary you are using is compiled with CUDA 10.2

Double check that your versions all line up - if you want to use CUDA 10.2 make sure CUDNN is the correct version and the pytorch binary you are using is compiled with CUDA 10.2

Hi @XDynames , I solved this by modifying my python interrupter file "anaconda3/envs/py37/lib/python3.7/site-packages/torch/utils/cpp_extension.py"
But I met another gcc compile problem.
running install
running bdist_egg
running egg_info
writing DCNv2.egg-info/PKG-INFO
writing dependency_links to DCNv2.egg-info/dependency_links.txt
writing top-level names to DCNv2.egg-info/top_level.txt
reading manifest file 'DCNv2.egg-info/SOURCES.txt'
writing manifest file 'DCNv2.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
building '_ext' extension
Emitting ninja build file /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.10.2
g++ -pthread -shared -B /NAS/home01/tanzhenwei/anaconda3/envs/py37/compiler_compat -L/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib -Wl,-rpath=/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib -Wl,--no-as-needed -Wl,--sysroot=/ /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/vision.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_im2col_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_psroi_pooling_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_cuda.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_psroi_pooling_cuda.o -L/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/_ext.cpython-37m-x86_64-linux-gnu.so
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_cuda.o: No such file or directory
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.o: No such file or directory
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_psroi_pooling_cuda.o: No such file or directory
error: command 'g++' failed with exit status 1
Do you have any advice? The environment is the same.

@MatthewHowe     Hi MatthewHowe.  Thanks for your great job, I successfully compiled on Ubuntu18.04.5, CUDA 11.1 (RTX3090), and PyTorch 1.7. 0 .
For there still some packages need to be compiled manually. I wonder if there are some guidelines , principles or rules to modify the source code from CUDA10(even earlier versions) version to CUDA 11 version so that I can compiled it  with current environment. Though I browsed the files changed, i still have no idea about how to do it properly.
Would you mind provide some guidance? Looking forward for your reply.

@ConnerWK Not to put a fine point on it but the code for DCN has become a bit messy - what we have done was to repla low level BLAS & CUDABLAS function calls with a higher level ATEN equivalent
This is viewed by us as a band-aid, so we've started working on a pure pytorch NN.module based solution that will not require building. Currently we have deformable convolution  V1/2 passing all the unit tests from this code but have yet to break ground on ROI pooling
Let me know if this is something you'd be interested in

@XDynames Thanks for your quick reply. Your project sounds great! I look forward to seeing your new achievements soon.
Be honest,  I'm a novice in this field with poor knowledge and experience. Recently, I meet troubles in compiling two packages listed below(extreme_utils and ROI align layer which is a util part of a modified CenterNet).
https://github.com/zju3dv/snake/tree/master/lib/csrc/extreme_utils
https://github.com/zju3dv/snake/tree/master/lib/csrc/roi_align_layer
It works well in my old platform( UBUNTU18.04, RTX2080Ti,  pytorch1.2.0, cuda10.0, nvidia-410 driver,  if my memory serves me right). However, I can't compile it properly in the RTX3090  env(UBUNTU18.04.5, pytorch1.7.0, cuda11.1, nvidia-455 driver).
Luckily, I find your work here and compiled DCN successfully. For error messages are limited, I asked you if there are some rules or congruent relationship to modify source code. It seems I have a lot more to learn.
Thanks for your reply again!  I look forward to seeing your new achievements soon.

Can you solve this problem, I have compiled successfully in cuda11,pytorch1.7(RTX 3090), thank u very   @MatthewHowe
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1607370156314/work/aten/src/THC/THCCachingHostAllocator.cpp line=278 error=700 : an illegal memory access was encountered
terminate called after throwing an instance of 'std::runtime_error'
what():  NCCL error in: /opt/conda/conda-bld/pytorch_1607370156314/work/torch/lib/c10d/../c10d/NCCLUtils.hpp:136, unhandled cuda error, NCCL version 2.7.8
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
/opt/conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 33 leaked semaphores to clean up at shutdown
len(cache))
Traceback (most recent call last):
File "C_ddp.py", line 349, in
main()
File "C_ddp.py", line 109, in main
mp.spawn(main_worker, nprocs=ngpus_per_node, args=(ngpus_per_node, args))
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
while not context.join():
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 118, in join
raise Exception(msg)
Exception:
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
fn(i, *args)
File "/home/wj/Detection/CenterNetV2/C_ddp.py", line 277, in main_worker
center_loss, center_fuse_loss, scale_loss, offset_loss = model({'img':img , 'label':label , 'heatmap_t':heatmap_t , 'hm_FuseClass_t':hm_FuseClass_t})
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 619, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/wj/Detection/CenterNetV2/nets/resnet_dcn_model.py", line 35, in forward
out=self.backbone(x)[0]
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/wj/Detection/CenterNetV2/networks/resnet_dcn.py", line 261, in forward
x = self.deconv_layers(x)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 929, in forward
output_padding, self.groups, self.dilation)
RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR
You can try to repro this exception using the following code snippet. If that doesn't trigger the error, please include your original repro script when reporting this issue.
import torch
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.allow_tf32 = True
data = torch.randn([32, 256, 40, 40], dtype=torch.float, device='cuda', requires_grad=True)
net = torch.nn.Conv2d(256, 256, kernel_size=[4, 4], padding=[1, 1], stride=[2, 2], dilation=[1, 1], groups=1)
net = net.cuda().float()
out = net(data)
out.backward(torch.randn_like(out))
torch.cuda.synchronize()
ConvolutionParams
data_type = CUDNN_DATA_FLOAT
padding = [1, 1, 0]
stride = [2, 2, 0]
dilation = [1, 1, 0]
groups = 1
deterministic = true
allow_tf32 = true
input: TensorDescriptor 0x55923cf1b4e0
type = CUDNN_DATA_FLOAT
nbDims = 4
dimA = 32, 256, 40, 40,
strideA = 409600, 1600, 40, 1,
output: TensorDescriptor 0x55923cf1d1b0
type = CUDNN_DATA_FLOAT
nbDims = 4
dimA = 32, 256, 20, 20,
strideA = 102400, 400, 20, 1,
weight: FilterDescriptor 0x55923cf4cec0
type = CUDNN_DATA_FLOAT
tensor_format = CUDNN_TENSOR_NCHW
nbDims = 4
dimA = 256, 256, 4, 4,
Pointer addresses:
input: 0x7fcf70a80000
output: 0x7fcf6fe00000
weight: 0x7fd1d1700000

Can you solve this problem, I have compiled successfully in cuda11,pytorch1.7(RTX 3090), thank u very   @MatthewHowe
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1607370156314/work/aten/src/THC/THCCachingHostAllocator.cpp line=278 error=700 : an illegal memory access was encountered
terminate called after throwing an instance of 'std::runtime_error'
what():  NCCL error in: /opt/conda/conda-bld/pytorch_1607370156314/work/torch/lib/c10d/../c10d/NCCLUtils.hpp:136, unhandled cuda error, NCCL version 2.7.8
error in modulated_deformable_im2col_cuda: no kernel image is available for execution on the device
/opt/conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 33 leaked semaphores to clean up at shutdown
len(cache))
Traceback (most recent call last):
File "C_ddp.py", line 349, in
main()
File "C_ddp.py", line 109, in main
mp.spawn(main_worker, nprocs=ngpus_per_node, args=(ngpus_per_node, args))
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
while not context.join():
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 118, in join
raise Exception(msg)
Exception:
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
fn(i, *args)
File "/home/wj/Detection/CenterNetV2/C_ddp.py", line 277, in main_worker
center_loss, center_fuse_loss, scale_loss, offset_loss = model({'img':img , 'label':label , 'heatmap_t':heatmap_t , 'hm_FuseClass_t':hm_FuseClass_t})
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 619, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/wj/Detection/CenterNetV2/nets/resnet_dcn_model.py", line 35, in forward
out=self.backbone(x)[0]
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/wj/Detection/CenterNetV2/networks/resnet_dcn.py", line 261, in forward
x = self.deconv_layers(x)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 929, in forward
output_padding, self.groups, self.dilation)
RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR
You can try to repro this exception using the following code snippet. If that doesn't trigger the error, please include your original repro script when reporting this issue.
import torch
torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.allow_tf32 = True
data = torch.randn([32, 256, 40, 40], dtype=torch.float, device='cuda', requires_grad=True)
net = torch.nn.Conv2d(256, 256, kernel_size=[4, 4], padding=[1, 1], stride=[2, 2], dilation=[1, 1], groups=1)
net = net.cuda().float()
out = net(data)
out.backward(torch.randn_like(out))
torch.cuda.synchronize()
ConvolutionParams
data_type = CUDNN_DATA_FLOAT
padding = [1, 1, 0]
stride = [2, 2, 0]
dilation = [1, 1, 0]
groups = 1
deterministic = true
allow_tf32 = true
input: TensorDescriptor 0x55923cf1b4e0
type = CUDNN_DATA_FLOAT
nbDims = 4
dimA = 32, 256, 40, 40,
strideA = 409600, 1600, 40, 1,
output: TensorDescriptor 0x55923cf1d1b0
type = CUDNN_DATA_FLOAT
nbDims = 4
dimA = 32, 256, 20, 20,
strideA = 102400, 400, 20, 1,
weight: FilterDescriptor 0x55923cf4cec0
type = CUDNN_DATA_FLOAT
tensor_format = CUDNN_TENSOR_NCHW
nbDims = 4
dimA = 256, 256, 4, 4,
Pointer addresses:
input: 0x7fcf70a80000
output: 0x7fcf6fe00000
weight: 0x7fd1d1700000

Double check that your versions all line up - if you want to use CUDA 10.2 make sure CUDNN is the correct version and the pytorch binary you are using is compiled with CUDA 10.2

Hi @XDynames , I solved this by modifying my python interrupter file "anaconda3/envs/py37/lib/python3.7/site-packages/torch/utils/cpp_extension.py"
But I met another gcc compile problem.
running install
running bdist_egg
running egg_info
writing DCNv2.egg-info/PKG-INFO
writing dependency_links to DCNv2.egg-info/dependency_links.txt
writing top-level names to DCNv2.egg-info/top_level.txt
reading manifest file 'DCNv2.egg-info/SOURCES.txt'
writing manifest file 'DCNv2.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
building '_ext' extension
Emitting ninja build file /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.10.2
g++ -pthread -shared -B /NAS/home01/tanzhenwei/anaconda3/envs/py37/compiler_compat -L/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib -Wl,-rpath=/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib -Wl,--no-as-needed -Wl,--sysroot=/ /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/vision.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_im2col_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_psroi_pooling_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_cuda.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_psroi_pooling_cuda.o -L/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/_ext.cpython-37m-x86_64-linux-gnu.so
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_cuda.o: No such file or directory
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.o: No such file or directory
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_psroi_pooling_cuda.o: No such file or directory
error: command 'g++' failed with exit status 1
Do you have any advice? The environment is the same.

do you solve this problem? I find the same issue too .

Double check that your versions all line up - if you want to use CUDA 10.2 make sure CUDNN is the correct version and the pytorch binary you are using is compiled with CUDA 10.2

Hi @XDynames , I solved this by modifying my python interrupter file "anaconda3/envs/py37/lib/python3.7/site-packages/torch/utils/cpp_extension.py"
But I met another gcc compile problem.
running install
running bdist_egg
running egg_info
writing DCNv2.egg-info/PKG-INFO
writing dependency_links to DCNv2.egg-info/dependency_links.txt
writing top-level names to DCNv2.egg-info/top_level.txt
reading manifest file 'DCNv2.egg-info/SOURCES.txt'
writing manifest file 'DCNv2.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
running build_ext
building '_ext' extension
Emitting ninja build file /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
1.10.2
g++ -pthread -shared -B /NAS/home01/tanzhenwei/anaconda3/envs/py37/compiler_compat -L/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib -Wl,-rpath=/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib -Wl,--no-as-needed -Wl,--sysroot=/ /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/vision.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_im2col_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cpu/dcn_v2_psroi_pooling_cpu.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_cuda.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.o /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_psroi_pooling_cuda.o -L/NAS/home01/tanzhenwei/anaconda3/envs/py37/lib/python3.7/site-packages/torch/lib -L/usr/local/cuda/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.7/_ext.cpython-37m-x86_64-linux-gnu.so
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_cuda.o: No such file or directory
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.o: No such file or directory
g++: error: /NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/build/temp.linux-x86_64-3.7/NAS/project01/rzimmerm_substitles/FairMot_compressing/src/lib/models/networks/DCNv2/DCN/src/cuda/dcn_v2_psroi_pooling_cuda.o: No such file or directory
error: command 'g++' failed with exit status 1
Do you have any advice? The environment is the same.

do you solve this problem? I find the same issue too .

pytorch version1.7 stable
gcc  8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
CUDA Version: 11.0
I can run pytorch on other project, so pytorch and cuda version should match.
make return error as follow:
Emitting ninja build file /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
FAILED: /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/home/opt/mot/DCNv2/src -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/TH -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/root/anaconda3/envs/FairMOT/include/python3.8 -c -c /home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu -o /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -ccbin g++ -std=c++14
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(324): error: identifier "THCudaBlas_Sgemv" is undefined
3 errors detected in the compilation of "/tmp/tmpxft_0011fd41_00000000-6_dcn_v2_cuda.cpp1.ii".

pytorch version1.7 stable
gcc  8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
CUDA Version: 11.0
I can run pytorch on other project, so pytorch and cuda version should match.
make return error as follow:
Emitting ninja build file /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
FAILED: /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/home/opt/mot/DCNv2/src -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/TH -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/root/anaconda3/envs/FairMOT/include/python3.8 -c -c /home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu -o /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -ccbin g++ -std=c++14
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(324): error: identifier "THCudaBlas_Sgemv" is undefined
3 errors detected in the compilation of "/tmp/tmpxft_0011fd41_00000000-6_dcn_v2_cuda.cpp1.ii".

pytorch version1.7 stable
gcc 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
CUDA Version: 11.0
I can run pytorch on other project, so pytorch and cuda version should match.
make return error as follow:
Emitting ninja build file /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
FAILED: /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/home/opt/mot/DCNv2/src -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/TH -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/root/anaconda3/envs/FairMOT/include/python3.8 -c -c /home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu -o /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -ccbin g++ -std=c++14
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(324): error: identifier "THCudaBlas_Sgemv" is undefined
3 errors detected in the compilation of "/tmp/tmpxft_0011fd41_00000000-6_dcn_v2_cuda.cpp1.ii".

I use this version https://github.com/lbin/DCNv2, THCState_getCurrentStream" is undefined solved.

pytorch version1.7 stable
gcc 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)
CUDA Version: 11.0
I can run pytorch on other project, so pytorch and cuda version should match.
make return error as follow:
Emitting ninja build file /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
FAILED: /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/home/opt/mot/DCNv2/src -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/TH -I/root/anaconda3/envs/FairMOT/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda/include -I/root/anaconda3/envs/FairMOT/include/python3.8 -c -c /home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu -o /home/opt/mot/DCNv2/build/temp.linux-x86_64-3.8/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=sm_75 -ccbin g++ -std=c++14
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined
/home/opt/mot/DCNv2/src/cuda/dcn_v2_cuda.cu(324): error: identifier "THCudaBlas_Sgemv" is undefined
3 errors detected in the compilation of "/tmp/tmpxft_0011fd41_00000000-6_dcn_v2_cuda.cpp1.ii".

I use this version https://github.com/lbin/DCNv2, THCState_getCurrentStream" is undefined solved.

Is there a solution for compliling this branch for PyTorch = 1.8 and CUDA = 11.1 (from torch.version.cuda)?

@hhcs9527 Not yet, we have a version of deformable convolution - not ROI pooling that does work with those versions but it is currently not working well in multi GPU training (very slow)
You might be able to patch what is here again by working out a suitable ATEN function to replace the depreciated BLAS calls used - we felt like we'd be doing this for ever after literally having some of the functions we used as replacements deprecated in the next version of pytorch (which dropped a day after we submitted this pull request)

@XDynames Thanks for your insight into this repository.
Since the current version of PyTorch is unfriendly to RTX3090, I thought it might get great performance after updating to PyTorch1.8, but I can use PyTorch1.7.1 instead.
Thanks for your great work.

@MatthewHowe嗨，Matthew，我无法使用pytorch1.7和RuntimeError进行编译：编译扩展对象时出错。
我使用了支持pytorch1.7
我的环境的最新版本（使用anaconda虚拟环境）：

gcc 7.5.0
忍者1.10.2
Ubuntu 18.04
python 3.7
pytorch 1.7
cudatoolkit 10.2
torch.cuda.is_available返回True，并且CUDA主页不是None
错误消息：
ninja：构建停止：子命令失败。
追溯（最近一次通话）：
文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py”，第1522行，位于_run_ninja_build
env = env ）
文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/subprocess.py”，行481，在运行
output = stdout，stderr = stderr）
子进程中。 ，'-v']'返回非零退出状态1。
上面的异常是以下异常的直接原因：
追溯（最近一次通话）：
文件“ setup.py”，第69行，在
cmdclass = {“ build_ext”中：torch.utils.cpp_extension.BuildExtension}，
文件“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / site-packages / setuptools / init.py“，第153行，在安装程序中
返回distutils.core.setup（** attrs）
文件” / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / distutils / core.py”，第148行，在安装程序
dist.run_commands（）中，
文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py”，行966，在run_commands
self.run_command（cmd）
文件中，“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / distutils / dist.py”行985中，在run_command
cmd_obj.run（）中
运行
self.run_command（cmd_name）的文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build.py”第135行，
文件“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / distutils / cmd.py”，第313行，位于run_command
self.distribution.run_command（命令）
文件中，“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3 .7 / distutils / dist.py”，第985行，位于run_command
cmd_obj.run（）
文件中的“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/command/build_ext “ .py”，运行中的第79行
_build_ext.run（self）
文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py”，运行中的339行
自己。build_extensions（）
在build_extensions
build_ext.build_extensions（self）
文件的第653行中，文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py”，文件“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / distutils / command / build_ext.py“，行448，在build_extensions
self._build_extensions_serial（）
文件中，” / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / distutils / command / build_ext.py”，第473行，位于_build_extensions_serial
self.build_extension（ext）
文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages”中/setuptools/command/build_ext.py“，第196行，位于build_extension
_build_ext.build_extension（self，ext）中
文件“ /NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py”，第533行，位于build_extensionDepend
= ext.depends中）
文件“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / python3.7 / site-packages / torch / utils / cpp_extension.py“，第482行，在unix_wrap_ninja_compile
with_cuda = with_cuda中）
文件“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib /python3.7/site-packages/torch/utils/cpp_extension.py“，第1238行，在_write_ninja_file_and_compile_objects中
error_prefix ='为扩展名编译对象时出错'）
文件“ / NAS / home01 / tanzhenwei / anaconda3 / envs / torch17 / lib / _run_ninja_build中的python3.7 / site-packages / torch / utils / cpp_extension.py“行1538
从e引发RuntimeError（message）
RuntimeError：编译扩展对象时出错
你可以帮帮我吗？

@MatthewHowe Hi Matthew, I failed to compile using pytorch1.7 with RuntimeError: Error compiling objects for extension.
I used the latest version of you which supports pytorch1.7
My environment (using anaconda virtual env):

gcc 7.5.0
ninja 1.10.2
ubuntu 18.04
python 3.7
pytorch 1.7
cudatoolkit 10.2
torch.cuda.is_available return True and CUDA home is not None
Error Message:
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1522, in _run_ninja_build
env=env)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/subprocess.py", line 481, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "setup.py", line 69, in
cmdclass={"build_ext": torch.utils.cpp_extension.BuildExtension},
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
_build_ext.run(self)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 339, in run
self.build_extensions()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 653, in build_extensions
build_ext.build_extensions(self)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 448, in build_extensions
self._build_extensions_serial()
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 473, in _build_extensions_serial
self.build_extension(ext)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/distutils/command/build_ext.py", line 533, in build_extension
depends=ext.depends)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 482, in unix_wrap_ninja_compile
with_cuda=with_cuda)
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1238, in _write_ninja_file_and_compile_objects
error_prefix='Error compiling objects for extension')
File "/NAS/home01/tanzhenwei/anaconda3/envs/torch17/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1538, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
Could you help me?

May I ask how you solve this problem

In this case I think the users cuda toolkit packages is version 10.2 and their cuda version is 11.1 causing a conflict

In this case I think the users cuda toolkit packages is version 10.2 and their cuda version is 11.1 causing a conflict
Hi, I checked the versions of my cuda toolkit packages and cuda version, they are all 11.0... and I have the same problem...

I still had many error on Windows. Does this work on Windows?

I still had many errors on Windows. Does this work on Windows?
Edit: Windows 10, torch==1.7.1, cuda 11.0
Edit 2: Add errors
DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(28): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_im2col_bilinear_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(28): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(29): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_im2col_bilinear_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(29): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(65): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_gradient_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(65): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(66): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_gradient_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(66): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(92): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_coordinate_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(92): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(93): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_coordinate_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(93): error: identifier "__floorf" is undefined in device code

meet the same error like u
Windows 10, python 3.8, torch 1.7.0, cuda 10.2

I still had many errors on Windows. Does this work on Windows?
Edit: Windows 10, torch==1.7.1, cuda 11.0
Edit 2: Add errors
DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(28): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_im2col_bilinear_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(28): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(29): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_im2col_bilinear_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(29): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(65): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_gradient_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(65): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(66): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_gradient_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(66): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(92): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_coordinate_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(92): error: identifier "__floorf" is undefined in device code

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(93): error: calling a __host__ function("__floorf") from a __device__ function("dmcn_get_coordinate_weight_cuda") is not allowed

DCNv2/DCN/src/cuda/dcn_v2_im2col_cuda.cu(93): error: identifier "__floorf" is undefined in device code

Finally build successfully!!!
clone the code from this version https://github.com/lbin/DCNv2/tree/pytorch_1.7
and replace all the floor(...) to floorf(...),
ceil(...) to ceilf(...),
round(...) to roundf(...)

try this one https://github.com/jinfagang/DCNv2_latest
------------------ 原始邮件 ------------------
发件人:                                                                                "CharlesShang/DCNv2"                                                                                @.>;
发送时间: 2021年3月24日(星期三) 晚上6:54
@.>;
<EMAIL_ADDRESS>主题: Re: [CharlesShang/DCNv2] Pytorch 1.6-1.8 compatability - CUDA11/3090 ready (#92)
@rathaROG commented on this pull request.
In DCN/src/cpu/dcn_v2_cpu.cpp:
> @@ -1,5 +1,6 @@  #include <vector>
@haruishi43 Thanks for your reply! Can you help verify this? This is what I did:
git clone https://github.com/CharlesShang/DCNv2.git cd DCNv2 git remote add tteepe https://github.com/tteepe/DCNv2.git git fetch tteepe git checkout origin/master python setup.py build develop
I also made some changes in dcn_v2_im2col_cuda.cu and dcn_v2_psroi_pooling_cuda.cu:
ceil() to ceilf()
floor() to floorf()
round() to roundf()
My system: windows 10, cuda 11.1.1, cudnn <IP_ADDRESS>, anaconda python 3.6.12 with these packages:
Cython @ file:///C:/ci/cython_1614014892888/work cython-bbox==0.1.3 torch==1.8.0 torchaudio==0.8.0 torchvision==0.9.0
All errors:
cpu\dcn_v2_cpu.cpp(82): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(101): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(176): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(216): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(224): error C3861: 'THFloatBlas_gemv': identifier not found cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined cuda/dcn_v2_cuda.cu(126): error: identifier "THCudaBlas_SgemmBatched" is undefined cuda/dcn_v2_cuda.cu(273): error: identifier "THCudaBlas_Sgemm" is undefined cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined cuda/dcn_v2_cuda.cu(324): error: identifier "THCudaBlas_Sgemv" is undefined
What did I miss?
Please help me. I really want to make it work.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

try this one https://github.com/jinfagang/DCNv2_latest
…
------------------ 原始邮件 ------------------ 发件人: "CharlesShang/DCNv2" @.>; 发送时间: 2021年3月24日(星期三) 晚上6:54 @.><EMAIL_ADDRESS>主题: Re: [CharlesShang/DCNv2] Pytorch 1.6-1.8 compatability - CUDA11/3090 ready (#92) @rathaROG commented on this pull request. In DCN/src/cpu/dcn_v2_cpu.cpp: > @@ -1,5 +1,6 @@ #include <vector> @haruishi43 Thanks for your reply! Can you help verify this? This is what I did: git clone https://github.com/CharlesShang/DCNv2.git cd DCNv2 git remote add tteepe https://github.com/tteepe/DCNv2.git git fetch tteepe git checkout origin/master python setup.py build develop I also made some changes in dcn_v2_im2col_cuda.cu and dcn_v2_psroi_pooling_cuda.cu: ceil() to ceilf() floor() to floorf() round() to roundf() My system: windows 10, cuda 11.1.1, cudnn <IP_ADDRESS>, anaconda python 3.6.12 with these packages: Cython @ file:///C:/ci/cython_1614014892888/work cython-bbox==0.1.3 torch==1.8.0 torchaudio==0.8.0 torchvision==0.9.0 All errors: cpu\dcn_v2_cpu.cpp(82): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(101): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(176): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(216): error C3861: 'THFloatBlas_gemm': identifier not found cpu\dcn_v2_cpu.cpp(224): error C3861: 'THFloatBlas_gemv': identifier not found cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined cuda/dcn_v2_cuda.cu(126): error: identifier "THCudaBlas_SgemmBatched" is undefined cuda/dcn_v2_cuda.cu(273): error: identifier "THCudaBlas_Sgemm" is undefined cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined cuda/dcn_v2_cuda.cu(324): error: identifier "THCudaBlas_Sgemv" is undefined What did I miss? Please help me. I really want to make it work. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Thank you very much for your suggestion!
I also had similar errors: https://github.com/jinfagang/DCNv2_latest/issues/17
Since I can't find what caused errors on my side, could you please share the details how you managed to make it work? In including your python version and all packages and how you install pytorch?

upgrade ur vs to 2019
------------------ 原始邮件 ------------------
发件人:                                                                                "CharlesShang/DCNv2"                                                                                @.>;
发送时间: 2021年3月25日(星期四) 晚上7:31
@.>;
<EMAIL_ADDRESS>主题: Re: [CharlesShang/DCNv2] Pytorch 1.6-1.8 compatability - CUDA11/3090 ready (#92)
@rathaROG commented on this pull request.
In DCN/src/cpu/dcn_v2_cpu.cpp:
> @@ -1,5 +1,6 @@  #include <vector>
Hi @haruishi43, I still had one more problem:
[4/4] "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64/link.exe" dcn_v2_cuda.o dcn_v2_cpu.o dcn_v2_im2col_cpu.o dcn_v2_psroi_pooling_cpu.o dcn_v2_cuda.cuda.o dcn_v2_im2col_cuda.cuda.o dcn_v2_psroi_pooling_cuda.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda_cu.lib @.@at@@@.@@._N1@Z torch_cuda_cpp.lib @.@at@@YAHXZ torch.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\lib\site-packages\torch\lib torch_python.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib/x64" cudart.lib /out:DCNv2_gpu.pyd FAILED: DCNv2_gpu.pyd "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64/link.exe" dcn_v2_cuda.o dcn_v2_cpu.o dcn_v2_im2col_cpu.o dcn_v2_psroi_pooling_cpu.o dcn_v2_cuda.cuda.o dcn_v2_im2col_cuda.cuda.o dcn_v2_psroi_pooling_cuda.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda_cu.lib @.@at@@@.@@._N1@Z torch_cuda_cpp.lib @.@at@@YAHXZ torch.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\lib\site-packages\torch\lib torch_python.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib/x64" cudart.lib /out:DCNv2_gpu.pyd    Creating library DCNv2_gpu.lib and object DCNv2_gpu.exp MSVCRT.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config DCNv2_gpu.pyd : fatal error LNK1120: 1 unresolved externals ninja: build stopped: subcommand failed.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

upgrade ur vs to 2019
…
------------------ 原始邮件 ------------------ 发件人: "CharlesShang/DCNv2" @.>; 发送时间: 2021年3月25日(星期四) 晚上7:31 @.><EMAIL_ADDRESS>主题: Re: [CharlesShang/DCNv2] Pytorch 1.6-1.8 compatability - CUDA11/3090 ready (#92) @rathaROG commented on this pull request. In DCN/src/cpu/dcn_v2_cpu.cpp: > @@ -1,5 +1,6 @@ #include <vector> Hi @haruishi43, I still had one more problem: [4/4] "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64/link.exe" dcn_v2_cuda.o dcn_v2_cpu.o dcn_v2_im2col_cpu.o dcn_v2_psroi_pooling_cpu.o dcn_v2_cuda.cuda.o dcn_v2_im2col_cuda.cuda.o dcn_v2_psroi_pooling_cuda.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda_cu.lib @.@at@@@.@@._N1@Z torch_cuda_cpp.lib @.@at@@YAHXZ torch.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\lib\site-packages\torch\lib torch_python.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib/x64" cudart.lib /out:DCNv2_gpu.pyd FAILED: DCNv2_gpu.pyd "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64/link.exe" dcn_v2_cuda.o dcn_v2_cpu.o dcn_v2_im2col_cpu.o dcn_v2_psroi_pooling_cpu.o dcn_v2_cuda.cuda.o dcn_v2_im2col_cuda.cuda.o dcn_v2_psroi_pooling_cuda.cuda.o /nologo /DLL c10.lib c10_cuda.lib torch_cpu.lib torch_cuda_cu.lib @.@at@@@.@@._N1@Z torch_cuda_cpp.lib @.@at@@YAHXZ torch.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\lib\site-packages\torch\lib torch_python.lib /LIBPATH:C:\dev\exc\Anaconda3\envs\DEFT\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\lib/x64" cudart.lib /out:DCNv2_gpu.pyd Creating library DCNv2_gpu.lib and object DCNv2_gpu.exp MSVCRT.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config DCNv2_gpu.pyd : fatal error LNK1120: 1 unresolved externals ninja: build stopped: subcommand failed. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

Thanks for the clue! I already had the latest version of VS2019, and I realized that I didn't add the path of cl.exe of VS2019 in the system path variable. In case you're interested, I also made a windows-ready repo here:
https://github.com/rathaROG/DCNv2_Windows

Thanks for the repo 👍------------------ 原始邮件 ------------------
@.>
发送时间: 2021年3月26日(星期五) 上午6:46
@.>;
<EMAIL_ADDRESS>主题: Re: [CharlesShang/DCNv2] Pytorch 1.6-1.8 compatability - CUDA11/3090 ready (#92)

Hi! Matthew,
I have a RTX3090, and cloned your project that you modified 14 hours ago.
While ./make.sh still get the error about:
nvcc fatal : Unsupported gpu architecture 'compute_86'

I got a Ubuntu 18.04, CUDA 11.1 pytorch 1.7, and gcc 7.5.0 / g++ 7.5.0
I guess it's probably the CUDA caused error, ANY HELP WOULD BE APPRECIATED!!

I have the same issue and it was fixed by the following steps:
My computer: NVIDIA-SMI 460.39       Driver Version: 460.39       CUDA Version: 11.2, nvcc - V ==> Build cuda_11.0_bu.TC445_37.28540450_0

Install Pytorch 1.6.0  py3.8_cuda10.2.89_cudnn7.6.5_0 conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch
Clone the latest source from DCNv2_latest
Add the following line in setup.py '--gpu-architecture=compute_75','--gpu-code=sm_75'

extra_compile_args["nvcc"] = [             "-DCUDA_HAS_FP16=1",             "-D__CUDA_NO_HALF_OPERATORS__",             "-D__CUDA_NO_HALF_CONVERSIONS__",             "-D__CUDA_NO_HALF2_OPERATORS__",             '--gpu-architecture=compute_75','--gpu-code=sm_75'         ]

./make.sh

@JohnPekl have you tried running export TORCH_CUDA_ARCH_LIST='8.0+PTX' before running make.sh? It's only a temporary workaround but it should allow it to compile.

@JohnPekl have you tried running export TORCH_CUDA_ARCH_LIST='8.0+PTX' before running make.sh? It's only a temporary workaround but it should allow it to compile.

@haruishi43 , I haven't tried running export TORCH_CUDA_ARCH_LIST='8.0+PTX. The four mentioned steps are all that I have done.

I have compiled successfully using pytorch1.7. Thanks. @jerryhitit @MatthewHowe

I have downgrade pytorch version to 1.7 stable,but it doesn't work for me.
Do you have any good suggestions？Thank you in advance.

I used this [docker image]docker pull nvidia/cuda:11.1-devel-ubuntu18.04 - installed conda then torch-nightly.
I then cloned and compiled DCNv2. This could be an issue with Cuda11.0 or some other conflicting packages.
When DCN doesn't compile usually the error from the cause is above your screen cap - if you run the ./make again the compiled parts will not run and it will make it clearer what is causing the issue.

Hi @MatthewHowe I appreciate your work! Do you have specific commands to compile DCNv2? Thanks!

@Xpangz this is a different error
In your case the g++ linker is failing to find compiled objects it expects to be created by the first build stage
Double check all your versions, if you used the above pip install to get the pytorch binary compiled with cuda 11 it will not be compatible with your install version of cuda 10.2
Your cuda version, cuda toolkit binary and pytorch (what cuda/cudnn it was compiled with) all have to agree for this to build

@XDynames thank you for your reply, I used conda install pytorch environment again, and it get solved.

hello，I met the same error as yours ,could you explain how to use conda install for more details? I recreate the envs and reinstall the pytorch : conda install pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=11.0 -c pytorch,
but it still doesn't work .I'm creay

seems ok after using https://github.com/tteepe/DCNv2

Your cuda version, cuda toolkit binary and pytorch (what cuda/cudnn it was compiled with) all have to agree for this to build

Hello， Could you please share your version of pytorch cudatoolkit   gcc-v   nvcc -v information?I'm suffering this problem for command 'g++' failed with exit status 1 and it drives me crazy @ @XDynames

can't fix when i downgrade my pytorch to 1.7.0 stable. sad....

您好，已经收到您的邮件，我会尽快给您回复。

您好，已经收到您的邮件，我会尽快给您回复。

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html
I made the following error according to the above command:
ERROR: torch has an invalid wheel, .dist-info directory not found

您好，已经收到您的邮件，我会尽快给您回复。

pip install --pre torch torchvision -f https://download.pytorch.org/whl/nightly/cu110/torch_nightly.html

the same error

How did you modify it
please help me

I successfully compiled on Windows 10, CUDA 11.1 (RTX3090), and PyTorch 1.7. Thank you so much!

你能解决这个问题吗，我在cuda11，pytorch1.7（RTX 3090）编译成功，非常感谢 @MatthewHowe
modulated_deformable_im2col_cuda 中的错误：没有可在设备上执行的内核映像 THCudaCheck FAIL 文件=/opt/conda/conda-bld/pytorch_1607370156314/work/aten/src/THC/THCCachingHostAllocator.cpp 行=278 错误=700：非法内存 在抛出 'std::runtime_error' what() 实例后遇到访问终止调用 ：/opt/conda/conda-bld/pytorch_1607370156314/work/torch/lib/c10d/../c10d/NCCLUtils 中的 NCCL 错误。 hpp:136，未处理的 cuda 错误，modulated_deformable_im2col_cuda 中的 NCCL 版本 2.7.8 错误：没有可在设备上执行的内核映像 /opt/conda/lib/python3.7/multiprocessing/semaphore_tracker.py:144：用户警告：semaphore_tracker：关闭时似乎有 33 个泄漏的信号量需要清理 len(cache)) Traceback（最近一次调用最后）： 文件“C_ddp.py”，第 349 行，在 main() 文件“C_ddp.py”，第 109 行，在 main mp.spawn(main_worker, nprocs=ngpus_per_node, args =(ngpus_per_node, args)) 文件“/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py”，第 199 行，在 spawn return start_processes(fn, args, nprocs, join, daemon , start_method='spawn') 文件“/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py”，第 157 行，在 start_processes 而不是 context.join()： 文件“/opt /conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py”，第 118 行，加入 引发异常（msg） 异常：
-- 进程 0 因以下错误而终止： Traceback（最近一次调用最后一次）： 文件“/opt/conda/lib/python3.7/site-packages/torch/multiprocessing/spawn.py”，第 19 行，在 _wrap fn (i, *args) 文件“/home/wj/Detection/CenterNetV2/C_ddp.py”，第 277 行，在 main_worker center_loss, center_fuse_loss, scale_loss, offset_loss = model({'img':img, 'label':label, 'heatmap_t':heatmap_t, 'hm_FuseClass_t':hm_FuseClass_t}) 文件“/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py”，第 727 行，在 _call_impl 结果 = self.前向（*输入，**kwargs） 文件“/opt/conda/lib/python3.7/site-packages/torch/nn/parallel/distributed.py”，第 619 行，前向 输出 = self.模块（*输入[0]，**kwargs[0]） 文件“/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py”，第 727 行，在 _call_impl 结果 = self.forward(*input, **kwargs) 文件“/home /wj/Detection/CenterNetV2/nets/resnet_dcn_model.py”，第 35 行，in forward out=self.backbone(x)[0] 文件“/opt/conda/lib/python3.7/site-packages/torch/nn /modules/module.py”，第 727 行，在 _call_impl 结果 = self.forward(*input, **kwargs) 文件“/home/wj/Detection/CenterNetV2/networks/resnet_dcn.py”，第 261 行，向前 x = self.deconv_layers(x) 文件“/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py”，第 727 行，在 _call_impl 结果中 = self.forward(*input, * *夸格斯） 文件“/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py”，第 117 行，前向 输入 = 模块（输入） 文件“/opt/conda/lib/python3. 7/site-packages/torch/nn/modules/module.py”，第 727 行，在 _call_impl 结果 = self.forward(*input, **kwargs) 文件“/opt/conda/lib/python3.7/site- packages/torch/nn/modules/conv.py"，第 929 行，在前向 output_padding、self.groups、self.dilation) RuntimeError：cuDNN 错误：CUDNN_STATUS_INTERNAL_ERROR 您可以尝试使用以下代码片段重现此异常。如果这不会触发错误，请在报告此问题时包含您的原始复制脚本。
import torch.backends.cuda.matmul.allow_tf32 = True torch.backends.cudnn.benchmark = True torch.backends.cudnn.deterministic = True torch.backends.cudnn.allow_tf32 = True data = torch.randn([32, 256 , 40, 40], dtype=torch.float, device='cuda', requires_grad=True) net = torch.nn.Conv2d(256, 256, kernel_size=[4, 4], padding=[1, 1], stride=[2, 2], dilation=[1, 1], groups=1) net = net.cuda().float() out = net(data) out.backward(torch.randn_like(out)) 火炬。 cuda.synchronize()
ConvolutionParams data_type = CUDNN_DATA_FLOAT padding = [1, 1, 0] stride = [2, 2, 0] dilation = [1, 1, 0] groups = 1 确定性 = true allow_tf32 = true 输入：TensorDescriptor 0x55923cf1b4e0 type = CUDNN_DATA_FLOAT nbDims = 4 dimA = 32, 256, 40, 40, strideA = 409600, 1600, 40, 1, 输出：TensorDescriptor 0x55923cf1d1b0 type = CUDNN_DATA_FLOAT nbDims = 4 dimA = 32, 256, 20, 20, strideA = 102400, 400, 20, 1 权重：FilterDescriptor 0x55923cf4cec0 type = CUDNN_DATA_FLOAT tensor_format = CUDNN_TENSOR_NCHW nbDims = 4 dimA = 256, 256, 4, 4, 指针地址： 输入：0x7fcf70a80000 输出：0x7fcf6fe00000 权重：0x7fd1d1700000

How did you solve this problem？
After I successfully compiled dcnv2, this problem also occurred when compiling the whole program. Please help me
大佬救救我

我在 Windows 10、CUDA 11.1 (RTX3090) 和 PyTorch 1.7 上成功编译。太感谢了！

Excuse me, how did you compile it successfully? Did you limit the computing power of the graphics card to 75? Can you help me?

I forked from https://github.com/MatthewHowe/DCNv2 which fixes to be compatible with [torch 1.7 / 1.8 with cuda 10 / 11]
This folder structure is slightly different from the original, which can cause errors in several projects using DCNv2.
So, I made the folder structure the same as the original: https://github.com/3846chs/DCNv2.git
My environment:
torch 1.7.1+cu110
cuda 11
RTX 3090

您好，已经收到您的邮件，我会尽快给您回复。

Hi, I met same error(compute_86) "torch 1.7 / 1.8 with cuda 10 / 11" .
Did you used Anaconda environment? @3846chs

Hi, can someone help me to see why I get the following error here?
error: command '/usr/local/cuda-11.0/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/bin/nvcc' failed: No such file or directory: '/usr/local/cuda-11.0/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/bin/nvcc'
The full error message is as follows:
./make.sh
...
/home/sda/lijun/Moving-object-detection-DSFNet/lib/models/DCNv2-master/DCN/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:398:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES’   398 |   AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cpu_backward", [&] {                                                                                |   ^~~~~~~~~~~~~~~~~~~~~~~~~~                                                    In file included from /home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/ATen/Tensor.h:3,                                                                               from /home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/ATen/Context.h:4,                                                                              from /home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/ATen/ATen.h:9,                                                                                from /home/sda/lijun/Moving-object-detection-DSFNet/lib/models/DCNv2-master/DCN/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/ATen/core/TensorBody.h:363:7: note: declared here                                                     363 |   T * data() const {
|       ^~~~
/usr/local/cuda-11.0/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/bin/nvcc -DWITH_CUDA -I/home/sda/lijun/Moving-object-detection-DSFNet/lib/models/DCNv2-master/DCN/src -I/home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include -I/home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/TH -I/home/junmuzi/anaconda3/envs/mod6/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-11.0/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/include -I/home/junmuzi/anaconda3/envs/mod6/include/python3.7m -c /home/sda/lijun/Moving-object-detection-DSFNet/lib/models/DCNv2-master/DCN/src/cuda/dcn_v2_cuda.cu -o build/temp.linux-x86_64-cpython-37/home/sda/lijun/Moving-object-detection-DSFNet/lib/models/DCNv2-master/DCN/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="_gcc" -DPYBIND11_STDLIB="_libstdcpp" -DPYBIND11_BUILD_ABI="_cxxabi1011" -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -ccbin g++ -std=c++14
error: command '/usr/local/cuda-11.0/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/bin/nvcc' failed: No such file or directory: '/usr/local/cuda-11.0/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/usr/local/cuda-11.7/:/bin/nvcc'

您好，已经收到您的邮件，我会尽快给您回复。

I forked from https://github.com/MatthewHowe/DCNv2 which fixes to be compatible with [torch 1.7 / 1.8 with cuda 10 / 11] This folder structure is slightly different from the original, which can cause errors in several projects using DCNv2. So, I made the folder structure the same as the original: https://github.com/3846chs/DCNv2.git
My environment: torch 1.7.1+cu110 cuda 11 RTX 3090

3Q, wonderful work

您好，已经收到您的邮件，我会尽快给您回复。


bug: CRD ApisixRoute with an ApisixPluginConfig & ApisixUpstream not configured correctly

Current Behavior
I followed this official apisix tutorial but with Consul discovery type. Everything works correctly if configured via ApiSix API but the reference to the plugin is lost if configured via CRD ( apisix ingress controller).
I have tried different configurations with CRD. The following doesn't work. plugin_config_id field is missing from the configuration applied by the CRDs (example --> "plugin_config_id": "246f7509")
apiVersion: apisix.apache.org/v2
kind: ApisixPluginConfig
metadata:
  name: json-server-plugins
spec:
  plugins:
    - name: openid-connect
      enable: true
      config:
        client_id: "json-server"
        client_secret: "yyyy"
        discovery: "xxxx"
        scope: "openid profile roles address phone"
        bearer_only: true
        realm: "zzzz"
        redirect_uri: "/*"
        logout_path: "/logout"
---
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: json-server-upstream
spec:
  discovery:
    type: consul
    serviceName: json-server
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: json-server-route
spec:
  http:
    - name: json-server
      match:
        hosts:
          - json-server.apisix.apps.lab.com
        paths:
          - /*
      plugin_config_name: json-server-plugins
      upstreams:
        - name: json-server-upstream

But this configuration works very well (the same but with plugin definition embedded directly to the route)
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: json-server-upstream
spec:
  discovery:
    type: consul
    serviceName: json-server
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: json-server-route
spec:
  http:
    - name: json-server
      match:
        hosts:
          - json-server.apisix.apps.lab.com
        paths:
          - /*
      upstreams:
        - name: json-server-upstream
      plugins:
        - name: openid-connect
          enable: true
          config:
            client_id: "json-server"
            client_secret: "yyyy"
            discovery: "xxxx"
            scope: "openid profile roles address phone"
            bearer_only: true
            realm: "zzzz"
            redirect_uri: "/*"
            logout_path: "/logout"

I also tried the first example (which doesn't work) and then I manually changed the configuration by adding "plugin_config_id": "246f7509" and everything worked perfectly.
Expected Behavior
A route with references to plugin and upstream as required by the CRD configurations
Example:
{
  "uris": [
    "/*"
  ],
  "name": "consul_json-server-route_json-server",
  "desc": "Created by apisix-ingress-controller, DO NOT modify it manually",
  "hosts": [
    "json-server.apisix.apps.lab.com"
  ],
  **"plugin_config_id": "246f7509",**
  **"upstream_id": "72c79c84",**
  "labels": {
    "managed-by": "apisix-ingress-controller"
  },
  "status": 1
}

But I get this result
{
  "uris": [
    "/*"
  ],
  "name": "consul_json-server-route_json-server",
  "desc": "Created by apisix-ingress-controller, DO NOT modify it manually",
  "hosts": [
    "json-server.apisix.apps.lab.com"
  ],
  **"upstream_id": "72c79c84",**
  "labels": {
    "managed-by": "apisix-ingress-controller"
  },
  "status": 1
}

Error Logs
Nothing relevant
Steps to Reproduce
Followed this official apisix tutorial but with Consul discovery type.
Environment

APISIX 3.2 & APISIX INGRESS 1.6.1
kubernetes

Hi, @unreal-altran , this bug seems to have been fixed in the latest version. I reproduced this bug in version 1.6.0, but when I try to use lastest version of apisix-ingress-controller, it works well.

Hi, @jiangfucheng yes I can confirm that with the 1.6.1 version of the ingress controller the problem does not occur.
I noticed during my last test that currently the Apisix helm charts  has a dependency to an older version of the ingress controller. I think it should be changed to 0.11.6 that has the ingress appVersion: 1.6.1 :-)


Adding angular material instructions

PR Checklist
Please check if your PR fulfills the following requirements:

[ ] The commit message follows our guidelines: https://github.com/analogjs/analog/blob/main/CONTRIBUTING.md#-commit-message-guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[x] Documentation content changes
[ ] Other... Please describe:

Which package are you modifying?

[ ] vite-plugin-angular
[ ] vite-plugin-nitro
[ ] astro-angular
[ ] create-analog
[ ] router
[x] platform
[ ] content
[ ] nx-plugin
[ ] trpc

What is the current behavior?

Closes #
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[ ] No

Other information
[optional] What gif best describes this PR or how it makes you feel?

Thanks @cskiwi! Should there be a step for adding provideAnimations() to the app.config.ts?

Thanks! @allcontributors add @cskiwi for doc


Overflow of Caption fixed with added new class and by defining it in …

#273  of Caption fixed with added new class and by defining it in css file as well
Before:

After:

responsive as well:

Thanks for your contribution :)


CWE-295 in go SDK，client message was malformed, however, in the server tls authentication, it was found that Go's crypto\x509\x509.go passed authenticated.

What version of Go are you using (go version)?

$ go version 1.14.6
$ grpc version  1.26.0

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env

What did you do?
A certification validation error occurred when using Defensics Fuzz that is a tool for checking security vulnerabilities to test the gRPC service port with SSL authentation.
We have used the Defensics Fuzz Testing Tool to test the grpc service port. The tool distorted the client certificate signature content and tried to establish communication. The packet capture showed that the client message was malformed, however, in the server tls authentication, it was found that Go's crypto\x509\x509.go passed authentication.
We highly suspected it that a flaw in Go language tls authentication
Please confirm whether the go language has this error?

What did you expect to see?
authentication failed
What did you see instead?
authentication passed

@wangting1995 without a way to reproduce your issue we cannot investigate. Please update the issue to include a self contained program that demonstrates the issue. Thank you.

@wangting1995 please email more info to<EMAIL_ADDRESS>Please do not post it here.
More: https://golang.org/security
cc @FiloSottile
@gopherbot remove WaitingForInfo

Closing this as there is not enough information for an investigation. Please follow golang.org/security if you have a report.


Fix bug estimate used gas and estimation for WeightV2

Solves https://github.com/paritytech/frontier/issues/1100
Some changes that we included on Moonbeam were missing upstream:

Currently pallet-ethereum PostDispatchInfo included the legacy/standard used gas but the block includes the effective gas data. This is an issue when producing blocks, as we hint the block builder way more available resources left than actually are.
Gas estimation using binary search need to be updated to WeightV2 (basically the ethereum call runtime api).

Just tested this one and it indeed fixes #1100, thanks!
Let me try to add a test to it so we avoid missing that in the future.

Hey @tgmichel I'm have hard time adding a test for this one =/
What I've done so far was to add this one to the frame/ethereum/tests/legacy.rs (but my idea is to create a separated file to include the other Transactions types too):
#[test]
fn validated_block_gas_limit_works() {
	let (pairs, mut ext) = new_test_ext_with_initial_balance(1, 100_000_000_000);
	let alice = &pairs[0];

	let mut transaction = legacy_erc20_creation_unsigned_transaction();

	ext.execute_with(|| {

		System::set_block_number(1);

		// Probe transaction, to get its real gas_used.
		let signed = transaction.sign(&alice.private_key);
		assert_ok!(crate::ValidatedTransaction::<Test>::apply(alice.address, signed));

		Ethereum::on_finalize(System::block_number());

		let block = <CurrentBlock<Test>>::get().unwrap();
		let mut next_block_gas_used = U256::zero();

		System::set_block_number(2);

		// Being sure that we send sufficient transactions to surpass block_gas_limit.
		while next_block_gas_used <= (block.header.gas_limit + block.header.gas_used) {
			let signed = transaction.sign(&alice.private_key);
			assert_ok!(crate::ValidatedTransaction::<Test>::apply(alice.address, signed));
			transaction.nonce += U256::one();
			next_block_gas_used += block.header.gas_used;
		}

		Ethereum::on_finalize(System::block_number());

		let block = <CurrentBlock<Test>>::get().unwrap();
		assert_eq!(block.header.gas_used, block.header.gas_limit);
	});
}

I left the assert_eq!(...) so it can outputs the values for inspection:
---- tests::legacy::validated_block_gas_limit_works stdout ----
thread 'tests::legacy::validated_block_gas_limit_works' panicked at 'assertion failed: `(left == right)`
  left: `151119462`,
 right: `150000000`', frame/ethereum/src/tests/legacy.rs:549:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

As Ethereum::store_block() (called by Ethereum::on_finalize()) is not checking for cumulative_gas <= block_gas_limit I think that is done one step earlier, by the pool logic maybe? Am I right on this one?
If so I'll move the test to somewhere else, maybe adding it to primitives/evm/src/validation.rs.

@arturgontijo thank you, I thought you already had a working test sorry :|
IMO if we want to verify that configured block gas limit works as expected with WeightV2 we need to test two type of transactions: ref time heavy, and proof size heavy, so in either case we make sure we account the right UsedGas component. I think this types of tests are easier on typescript.

@tgmichel no man, still studying the whole code base.
Regarding the TS test, I see that we have a TODO that could be exactly what we need:
https://github.com/paritytech/frontier/blob/master/ts-tests/tests/test-gas.ts#L75
Do you mind writing the test(s) for it? I think you would do way way faster than me, avoiding this being blocked. =)

Tests pass locally but fail on github machine, caused by the MAX_PROPOSAL_DURATION harcoded value to 10 seconds in substrate (which IMO makes no sense):
https://github.com/paritytech/substrate/blob/ebe8dcd3be6583faea1ca0b38441a39e2e02ce23/client/consensus/manual-seal/src/seal_block.rs#L127
10 seconds is not enough in our case, where we need to process 75M gas worth of transactions in the CI. Deadline is reached while applying the transaction pool iterator, and an arbitrary amount of transactions are included instead reaching the configured limit.

Does heavier transactions help here?
I mean, transactions that spend more than 21_000 gas?
Something like this:
    mapping(address => uint) public map;

    // n=1      30k
    // n=10     37k
    // n=100    100k
    // n=250    205k
    // n=500    380k
    // n=1000   745k
    function storageLoop(
        uint16 n,
        address _to,
        uint _amount
    ) public {
        for (uint16 i = 0; i < n; i++) {
            map[_to] += _amount;
        }
    }

When can we merge this? I hope it can be included in the polkadot-v0.9.43 branch. :)

I will work on fixing the tests today

@sorpaas we should cherry-pick this PR to polkadot-v0.9.43 branch.

@sorpaas we should cherry-pick this PR to polkadot-v0.9.43 branch.

also to polkadot-v0.9.42 branch


Improve hint for cmake not installed in Linux (code 127)

Thanks for this! Could you run cargo fmt?


Add Automated CI/CD pipeline for building PCF control via GitHub Actions

OneNote Web
Build
	Navigate to main component folder
		Change Manifest.xml - version #
		npm run build
		Navigate to Solution Folder (Solution\controlComponent\ folder)
			If it doesn't exist - Create Solution \ controlComponent (follow steps in create your first…)
				RUN: pac solution init --publisher-name Samples --publisher-prefix samples
				RUN: pac solution add-reference --path ..\..\
				FIRST TIME ONLY: msbuild /t:restore
		Update src\other\Solution.xml
			Change Version Number (match Manifest.xml)
		msbuild
			//builds the debug version - unmanaged, etc. - into the bin\debug folder
		msbuild /p:configuration=Release
			// builds Release (managed) version into bin\release folder

Release
	Rename RichTextTinyMCE5Component.zip >>> RichTextTinyMCE5Component_v{version #}_{managed/unmanaged}.zip
		_unmanaged = debug folder
		_managed = release folder
	Upload and stamp release version in GitHub

Deploy
	- Go to environment
	- Solutions: Import (upload solution_managed.zip)
	- Verify (in app) that it was upgraded

https://github.com/microsoft/powercat-code-components/blob/main/.github/workflows/create-release.yml


Declare Argument including validation.

Resolves #605

Fails

:no_entry_sign:
Issue marked as feature should have docs commit

  Generated by :no_entry_sign: dangerJS against ee47856635098fa8c8ab846b5ee672e5ce274a0f

:tada: This PR is included in version 5.0.0-rc.3 :tada:
The release is available on:

5.0.0-rc.3
GitHub release

Your semantic-release bot :package::rocket:


Added a Prediction using kmeans

Added a Folder in which a Prediction using k-means Clustering Algorithms is shown

This Folder includes a dataset and a .ipynb file
The Dataset is included for ease of Developer contributing further.
kmeans.ipynb file is well explained by me to understand various steps of Prediction.

Please Review my PR and merge it

``congratulations on your hactoberfest contribution. if you like this repo then give it a star. Happy hacking..

Can you please label this PR as hacktoberfest-accepted?

``congratulations on your hactoberfest contribution. if you like this repo then give it a star. Happy hacking..

Sure ✌


Mandated git_tag for non-opence remote feedstock

Checklist before submitting

[ ] Did you read the contributor guide?
[ ] Did you update any affected documentation?
[ ] Did you write any tests to validate this change?

Description
Fixes # (issue).
Review process to land

All tests and other checks must succeed.
At least one maintainer must review and approve.
If any  maintainer requests changes, they must be addressed.

Codecov Report

Merging #138 (af6da2b) into main (84cfdfa) will decrease coverage by 0.14%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##             main     #138      +/-   ##
==========================================
- Coverage   96.25%   96.11%   -0.14%
==========================================
  Files          23       23
  Lines        2104     2108       +4
==========================================
+ Hits         2025     2026       +1
- Misses         79       82       +3

Flag
Coverage Δ

unittests
96.11% <80.00%> (-0.14%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

open_ce/build_tree.py
96.34% <66.67%> (-0.76%)
:arrow_down:

open_ce/__init__.py
100.00% <100.00%> (ø)

open_ce/errors.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Codecov Report

Merging #138 (af6da2b) into main (84cfdfa) will decrease coverage by 0.14%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##             main     #138      +/-   ##
==========================================
- Coverage   96.25%   96.11%   -0.14%
==========================================
  Files          23       23
  Lines        2104     2108       +4
==========================================
+ Hits         2025     2026       +1
- Misses         79       82       +3

Flag
Coverage Δ

unittests
96.11% <80.00%> (-0.14%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

open_ce/build_tree.py
96.34% <66.67%> (-0.76%)
:arrow_down:

open_ce/__init__.py
100.00% <100.00%> (ø)

open_ce/errors.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Memory use is increasing as the times of call in function scipy.stats._sobol._categorize

My issue is about memory leak in function scipy.stats._sobol._categorize.
As the below code shows, if I input illegal input to the function, the memory use is increasing as the number of calls.
Reproducing code example:
import sys
import tracemalloc
import numpy
import scipy.stats._sobol

arg_list = []
arg_list.append(-87.11965751389266)
arg_list.append(17.04812581438513)
arg_list.append(-3)
arg_list.append(-87)
arg_list.append(-14)
arg_list.append(37)
arg_list.append(-63)
arg_list.append(-73)
arg_list.append(-83)
arg_list.append(-14)
arg_list.append(96)
arg_list.append(70)

memory_recorder = []
tracemalloc.start()
start_snapshot = tracemalloc.take_snapshot()
for i in range(0, 100):
  try:
    scipy.stats._sobol._categorize(numpy.array([arg_list[0], arg_list[1]]),
                                   numpy.array([arg_list[2], arg_list[3], arg_list[4], arg_list[5], arg_list[6]]),
                                   numpy.array([arg_list[7], arg_list[8], arg_list[9], arg_list[10], arg_list[11]]))
  except:
    pass
  if i % 10 == 0:
    one_snapshot = tracemalloc.take_snapshot()
    memory_recorder.append(one_snapshot.compare_to(start_snapshot, 'lineno')[0])

for i in range(2, len(memory_recorder)):
  if str(memory_recorder[i].traceback) == str(memory_recorder[1].traceback):
    print(memory_recorder[i].traceback)
    print(memory_recorder[i].size)

tracemalloc.stop()

Error message:
test.py:32
9594
test.py:32
13690
test.py:32
17482
test.py:32
21482
test.py:32
25274
test.py:32
29322
test.py:32
33114
test.py:32
37210

Scipy/Numpy/Python version information:
scipy 1.7.0

@Stardust1225 thanks for the report. _sobol._categorize is a private function, is there actually any way that a public function can pass such invalid input to it? Normally the input validation in public functions should catch this.

Yeah if there is no leak when called by a public function I would not say that it's a concern. This is really not meant to be used elsewhere.

So this function is only called once here:
base_draws = self.engine.random(n).ravel()
p_cumulative = np.empty_like(self.pvals, dtype=float)
_fill_p_cumulative(np.array(self.pvals, dtype=float), p_cumulative)
sample = np.zeros_like(self.pvals, dtype=int)
_categorize(base_draws, p_cumulative, sample)

The types are explicitely given for all parameters excepts base_draws. But all engines return float arrays. To call this function with a list of integers here, you would need to make your own class sampling only integers and this would fail. But I am not convince that this would make any sense to try to do this.

I've reported the leak to cython (cython/cython#4296) but agree this particular function shouldn't cause any issues. The issue @tupui points out with base_draws doesn't matter because it's the first argument so no other memory views will have been constructed yet.

Thanks for having a deeper look @peterbell10


WIP Fix gcr support

Q
A

Bug fix?
yes

New feature?
no

API breaks?
no

Deprecations?
no

License
Apache 2.0

What's in this PR?
For GCP auth in docker config contains JSON string (ServiceAccount), not a password in plaintext.

Hi there @leominov !
Have you tried the GCR part from here? https://github.com/banzaicloud/bank-vaults/tree/master/docs/mutating-webhook#registry-access

Oh, shi, you right :)

It's little bit weird, but ok.

Would it be simpler with this PR? If yes, I'm happy to merge :), I was just asking if you have checked the method we had earlier.


Request for new Toggle and synonyms

We need the following new toggle for the smart device that we are developing.
New toggle
Name: burst
Synonyms: burst, spray, disperse, release
Thanks!
Edit: Also, what is the estimated turn around time on these types of requests?

I tend to process new requests every Wednesday, and it takes a few days after that for the changes to roll out, depending on the server build schedule.

Can I also please request an update to the toggle in the original post to add the synonym "mist".
Update to toggle
Name: burst
Synonyms: burst, spray, disperse, release, mist

This update has been submitted, and should roll out in a few days.


fix: initialize the uninitialized

While building all demos I found that several colors are uninitialized, and where throwing an error when building.
Sample "error":
pimoroni-pico/examples/stellar_unicorn/stellar_lava_lamp.cpp: In function 'int main()':
pimoroni-pico/examples/stellar_unicorn/stellar_lava_lamp.cpp:128:27: error: 'bright_r' may be used uninitialized [-Werror=maybe-uninitialized]
  128 |           graphics.set_pen(bright_r, bright_g, bright_b);
      |           ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pimoroni-pico/examples/stellar_unicorn/stellar_lava_lamp.cpp:113:13: note: 'bright_r' was declared here
  113 |     uint8_t bright_r, bright_g, bright_b;
      |             ^~~~~~~~

I could have added the build flag for dismissing that warning, but I felt more explicit is better.
Let me know what you think.

Thank you, more explicit is indeed better. I try to build with Werror and Wall where possible, but we’re still targeting a pretty old compiler 😬


Lookup user by (Email/PhoneNumber/UUID)

There should be a way to search for a user using their email or phone number. The nodejs library can search for a user for each provider.
A real use case would be that before authenticating a user she would like to be able to hit the database to validate if the email is associated with any user. Another example is that if the user logs in with a phone number, they will not be able to know if the user is already registered. I think that if they want to convert firebase as a first class citizen to flutter, they should have a firebase admin sdk for dart, otherwise we might think that the project does not have great interest or support for flutter.

Hi @andrefedev
Is this possible using firebase sdk for android or ios?  if yes, can you please link it?
Thank you

I don't understand your question, are you suggesting to use native Firebase SDK libraries for Android / iOS in a Flutter project?

Hi @andrefedev
I am asking if this can be done with Firebase without using Flutter on native platforms or it's just nodejs library that can do this

Hello - this is an admin only feature so won't land in this library (which is for clients).


Review Invenio Helm Charts

PR: [when the PR is created, put a link to it here.  DO NOT use the "linked PR" feature]
User Test Scripts: [when there are user test scripts available link them here]

Review the Helm charts for Invenio v12, and determine their current suitability for Imperial deployment, and identify any work required to bring them up to standard
Acceptance Criteria
List the criteria which must be met for the issue to be considered complete

A set of subsequent issues created which identify specific work to be done

We've successfully installed both the vanilla demo site and branded dev instance of the Imperial repo on a cluster (on DigitalOcean) so we've navigated through the community helm charts and their limitations. Documentation will feed into our dev guide for Imperial. Closing.