Issue with new wheels, people can't install 0.2.0 with CUDA 11.8

Awesome work on the 0.2.0 release and the wheels, PanQiWei!  Thousands of new people are trying AutoGPTQ today and that is amazing.
Got an issue that's affecting some of them:
Describe the bug
People trying to run pip install auto-gptq or pip install auto-gptq==0.2.0 are getting the follow errors:
Requested auto-gptq==0.2.0 from https://files.pythonhosted.org/packages/b1/f9/97153ae5cf926f96fd37e61424a1bb58e0c9991cc220b2e17390fb8bde97/auto_gptq-0.2.0.tar.gz has inconsistent version: expected '0.2.0', but metadata has '0.2.0+cu1180'
ERROR: Could not find a version that satisfies the requirement auto-gptq==0.2.0 (from versions: 0.0.4, 0.0.5, 0.1.0, 0.2.0)

Full log:
Found existing installation: auto-gptq 0.1.0
Uninstalling auto-gptq-0.1.0:
  Successfully uninstalled auto-gptq-0.1.0
Using pip 23.1.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting auto-gptq==0.2.0
  Using cached auto_gptq-0.2.0.tar.gz (47 kB)
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info
  writing /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/SOURCES.txt'
  /usr/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  reading manifest file '/tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Discarding https://files.pythonhosted.org/packages/b1/f9/97153ae5cf926f96fd37e61424a1bb58e0c9991cc220b2e17390fb8bde97/auto_gptq-0.2.0.tar.gz (from https://pypi.org/simple/auto-gptq/) (requires-python:>=3.8.0): Requested auto-gptq==0.2.0 from https://files.pythonhosted.org/packages/b1/f9/97153ae5cf926f96fd37e61424a1bb58e0c9991cc220b2e17390fb8bde97/auto_gptq-0.2.0.tar.gz has inconsistent version: expected '0.2.0', but metadata has '0.2.0+cu1180'
ERROR: Could not find a version that satisfies the requirement auto-gptq==0.2.0 (from versions: 0.0.4, 0.0.5, 0.1.0, 0.2.0)
ERROR: No matching distribution found for auto-gptq==0.2.0

Software version
Example of one user with the problem:

ubuntu 22.04
nvidia/cuda:11.8.0-devel-ubuntu22.04 container which includes a CUDA Version 11.8.0

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

To Reproduce
pip install auto-gptq

Expected behavior
Installs auto-gptq 0.2.0 + cu118

Also downloading the prebuilt wheels does not seem to work at least with the syntax mentioned in the README
 > [3/5] RUN pip install auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl: #0 0.448 WARNING: Requirement 'auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl' looks like a filename, but the file does not exist #0 0.462 Processing /auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl #0 0.472 ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl'
It seems to expect an already downloaded package.

Also downloading the prebuilt wheels does not seem to work at least with the syntax mentioned in the README
 > [3/5] RUN pip install auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl: #0 0.448 WARNING: Requirement 'auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl' looks like a filename, but the file does not exist #0 0.462 Processing /auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl #0 0.472 ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/auto_gptq-0.2.0+cu118-cp310-cp310-linux_x86_64.whl'
It seems to expect an already downloaded package.

You need to execute the command in a directory where the wheel is downloaded and saved.

That user is me btw. I Was using it in the colab. I red from @TheBloke Model Card That for Above cuda version 12.0 Should Compile Fro The Source Code. I did it. but the Version it installed is 0.1.0. When i Run Tom's falcon gptq it Shows me RefineWeb model not Supported Error. I Made a Issue on Tom's Repo and He Told me to install from pip. I Did but Even if the pip project shows version 2, When I Install it Installs Only Version 1. After Some Conversation with Tom. I Compiled from the Source Code. It Succesfully compiled and Installed version version 0.2.0+cuda1180.
But How did it compiled cuda118 when my colab cuda version is 12

@TheBloke Hi, I can install successfully using pip install auto-gptq on both my local computer and cloud server, but I also re-implement your problem when adding environment variable CUDA_VERSION=11.8 before pip command. So I think if you also have added the environment variable, you can just remove it.

It's not a problem for me personally. But I have had several support requests about it this morning, from people trying to use AutoGPTQ from Google Colab and Docker containers - e @kumpulak is using Docker.
I can inform users to unset CUDA_VERSION but is possible to fix whatever problem is causing this issue so this is not necessary going forward. Otherwise I expect it's going to generate a lot of support requests.  I've already had four messages about it this morning.

@TheFaheem you don't have CUDA toolkit 12.0 installed, otherwise it wouldn't work.  You have 11.8.  You can see your CUDA toolkit version by running:
nvcc --version

eg:
[pytorch2] tomj@a10:/workspace $ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

@TheFaheem you don't have CUDA toolkit 12.0 installed, otherwise it wouldn't work. You have 11.8. You can see your CUDA toolkit version by running:
nvcc --version

eg:
[pytorch2] tomj@a10:/workspace $ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

But, While Running !nvidia-smi
It Shows Cuda Version: 12.0

Yes, that is the version supported by your GPU driver.  But you have CUDA toolkit 11.8 installed and that is fine.  It is the same for me:
[pytorch2] tomj@a10:/workspace $ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
[pytorch2] tomj@a10:/workspace $ nvidia-smi
Fri Jun  2 10:28:46 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A10          On   | 00000000:06:00.0 Off |                    0 |
|  0%   32C    P8    16W / 150W |      0MiB / 23028MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

@TheBloke Hi, I can install successfully using pip install auto-gptq on both my local computer and cloud server, but I also re-implement your problem when adding environment variable CUDA_VERSION=11.8 before pip command. So I think if you also have added the environment variable, you can just remove it.

It's not a problem for me personally. But I have had several support requests about it this morning, from people trying to use AutoGPTQ from Google Colab and Docker containers - eg @kumpulak is using Docker and @TheFaheem is using Google Colab.
I can inform users to unset CUDA_VERSION but is possible to fix whatever problem is causing this issue so this is not necessary going forward. Otherwise I expect it's going to generate a lot of support requests. I've already had four messages about it this morning.

I just fix the problem that users set CUDA_VERSION when install auto-gptq, I will release a patch fix later.

I just fix the problem that users set CUDA_VERSION when install auto-gptq, I will release a patch fix later.

Thank you so much!

@PanQiWei Can You Please Explain What are These?
```WARNING:accelerate.utils.modeling:The safetensors archive passed at /root/.cache/huggingface/hub/models--TheBloke--WizardLM-Uncensored-Falcon-7B-GPTQ/snapshots/83fd597a3332323e06fe883f680829a498d9fa9f/gptq_model-4bit-64g.safetensors does not contain metadata. Make sure to save your model with the save_pretrained method. Defaulting to 'pt' metadata.
WARNING:auto_gptq.modeling._base:can't get model's sequence length from model config, will set to 4096.
WARNING:auto_gptq.modeling._base:RWGPTQForCausalLM hasn't fused attention module yet, will skip inject fused attention.
WARNING:auto_gptq.modeling._base:RWGPTQForCausalLM hasn't fused mlp module yet, will skip inject fused mlp.

And is there any parameter or option to stream the output. did you implemented any generator function?

Yes, that is the version supported by your GPU driver. But you have CUDA toolkit 11.8 installed and that is fine. It is the same for me:
[pytorch2] tomj@a10:/workspace $ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
[pytorch2] tomj@a10:/workspace $ nvidia-smi
Fri Jun  2 10:28:46 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A10          On   | 00000000:06:00.0 Off |                    0 |
|  0%   32C    P8    16W / 150W |      0MiB / 23028MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Ah... Thanks!. I Thought it's Showing The installed Cuda Version.

Oh yeah, I was going to raise another issue for that @PanQiWei
People are quite confused by all the WARNINGs that get printed, which actually are just information
I think it would be a good idea to print these messages as logger.info instead.  And to hide the accelerate message WARNING:accelerate.utils.modeling:
People think something is wrong, when it's actually all fine.
In the case of these messages:
WARNING:auto_gptq.modeling._base:RWGPTQForCausalLM hasn't fused attention module yet, will skip inject fused attention.
WARNING:auto_gptq.modeling._base:RWGPTQForCausalLM hasn't fused mlp module yet, will skip inject fused mlp.

Perhaps these messages should only be printed if the user actually passed inject_fused_attention=True or inject_fused_mlp=True.  But not when it gets set by default?

You are right, I should improve the warnings and set some arguments' default value to None and reset to proper value internally if users not manually specify them.

And is there any parameter or option to stream the output. did you implemented any generator function?

There is no streaming code in AutoGPTQ at the moment.
But you could use third party software like text-generation-webui.   That can provide an API, and that API has a streaming option. See example API script here:  https://github.com/oobabooga/text-generation-webui/blob/main/api-example-stream.py

And is there any parameter or option to stream the output. did you implemented any generator function?

@TheFaheem auto-gptq is compatiple with hf transformer's TextGenerationPipeline, so it's streamer should also be used for auto-gptq's models, but I haven't try it yet.

I can confirm unsetting the CUDA_VERSION works. My Dockerfile installs it like this now:
RUN unset CUDA_VERSION && pip install auto-gptq==0.2.0

@PanQiWei @TheBloke When i Tried to Compile AutoGPTQ in kaggle nb i goth the following:
Cloning into 'AutoGPTQ'...
remote: Enumerating objects: 2114, done.
remote: Counting objects: 100% (445/445), done.
remote: Compressing objects: 100% (240/240), done.
remote: Total 2114 (delta 278), reused 244 (delta 194), pack-reused 1669
Receiving objects: 100% (2114/2114), 7.41 MiB | 17.94 MiB/s, done.
Resolving deltas: 100% (1408/1408), done.
/kaggle/working/AutoGPTQ
Processing /kaggle/working/AutoGPTQ
  Preparing metadata (setup.py) ... done
Collecting accelerate>=0.19.0 (from auto-gptq==0.2.0)
  Downloading accelerate-0.19.0-py3-none-any.whl (219 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 219.1/219.1 kB 5.3 MB/s eta 0:00:0000:01
Requirement already satisfied: datasets in /opt/conda/lib/python3.10/site-packages (from auto-gptq==0.2.0) (2.1.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.10/site-packages (from auto-gptq==0.2.0) (1.23.5)
Collecting rouge (from auto-gptq==0.2.0)
  Downloading rouge-1.0.1-py3-none-any.whl (13 kB)
Requirement already satisfied: torch>=1.13.0 in /opt/conda/lib/python3.10/site-packages (from auto-gptq==0.2.0) (2.0.0)
Requirement already satisfied: safetensors in /opt/conda/lib/python3.10/site-packages (from auto-gptq==0.2.0) (0.3.1)
Requirement already satisfied: transformers>=4.26.1 in /opt/conda/lib/python3.10/site-packages (from auto-gptq==0.2.0) (4.30.0.dev0)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from accelerate>=0.19.0->auto-gptq==0.2.0) (21.3)
Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate>=0.19.0->auto-gptq==0.2.0) (5.9.3)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from accelerate>=0.19.0->auto-gptq==0.2.0) (5.4.1)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->auto-gptq==0.2.0) (3.12.0)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->auto-gptq==0.2.0) (4.5.0)
Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->auto-gptq==0.2.0) (1.12)
Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->auto-gptq==0.2.0) (3.1)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->auto-gptq==0.2.0) (3.1.2)
Requirement already satisfied: huggingface-hub<1.0,>=0.14.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->auto-gptq==0.2.0) (0.14.1)
Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->auto-gptq==0.2.0) (2023.5.5)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->auto-gptq==0.2.0) (2.28.2)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->auto-gptq==0.2.0) (0.13.3)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.26.1->auto-gptq==0.2.0) (4.64.1)
Requirement already satisfied: pyarrow>=5.0.0 in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (10.0.1)
Requirement already satisfied: dill in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (0.3.6)
Requirement already satisfied: pandas in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (1.5.3)
Requirement already satisfied: xxhash in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (3.2.0)
Requirement already satisfied: multiprocess in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (0.70.14)
Requirement already satisfied: fsspec[http]>=2021.05.0 in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (2023.5.0)
Requirement already satisfied: aiohttp in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (3.8.4)
Requirement already satisfied: responses<0.19 in /opt/conda/lib/python3.10/site-packages (from datasets->auto-gptq==0.2.0) (0.18.0)
Requirement already satisfied: six in /opt/conda/lib/python3.10/site-packages (from rouge->auto-gptq==0.2.0) (1.16.0)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (23.1.0)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (2.1.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (6.0.4)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (4.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (1.9.1)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (1.3.3)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp->datasets->auto-gptq==0.2.0) (1.3.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->accelerate>=0.19.0->auto-gptq==0.2.0) (3.0.9)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->auto-gptq==0.2.0) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->auto-gptq==0.2.0) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers>=4.26.1->auto-gptq==0.2.0) (2023.5.7)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.13.0->auto-gptq==0.2.0) (2.1.2)
Requirement already satisfied: python-dateutil>=2.8.1 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->auto-gptq==0.2.0) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->auto-gptq==0.2.0) (2023.3)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.13.0->auto-gptq==0.2.0) (1.3.0)
Building wheels for collected packages: auto-gptq
  Building wheel for auto-gptq (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [95 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.10
      creating build/lib.linux-x86_64-3.10/auto_gptq
      copying auto_gptq/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq
      creating build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/gpt_bigcode.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/llama.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/moss.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/gpt_neox.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/_const.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/bloom.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/_base.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/_utils.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/opt.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/rw.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/gpt2.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/codegen.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/auto.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      copying auto_gptq/modeling/gptj.py -> build/lib.linux-x86_64-3.10/auto_gptq/modeling
      creating build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks
      copying auto_gptq/eval_tasks/sequence_classification_task.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks
      copying auto_gptq/eval_tasks/text_summarization_task.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks
      copying auto_gptq/eval_tasks/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks
      copying auto_gptq/eval_tasks/_base.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks
      copying auto_gptq/eval_tasks/language_modeling_task.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks
      creating build/lib.linux-x86_64-3.10/auto_gptq/quantization
      copying auto_gptq/quantization/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/quantization
      copying auto_gptq/quantization/quantizer.py -> build/lib.linux-x86_64-3.10/auto_gptq/quantization
      copying auto_gptq/quantization/gptq.py -> build/lib.linux-x86_64-3.10/auto_gptq/quantization
      creating build/lib.linux-x86_64-3.10/auto_gptq/utils
      copying auto_gptq/utils/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/utils
      copying auto_gptq/utils/import_utils.py -> build/lib.linux-x86_64-3.10/auto_gptq/utils
      copying auto_gptq/utils/data_utils.py -> build/lib.linux-x86_64-3.10/auto_gptq/utils
      creating build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/qlinear_triton.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/qlinear.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/qlinear_old.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/fused_llama_mlp.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/fused_llama_attn.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/fused_gptj_attn.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      copying auto_gptq/nn_modules/_fused_base.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules
      creating build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks/_utils
      copying auto_gptq/eval_tasks/_utils/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks/_utils
      copying auto_gptq/eval_tasks/_utils/generation_utils.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks/_utils
      copying auto_gptq/eval_tasks/_utils/classification_utils.py -> build/lib.linux-x86_64-3.10/auto_gptq/eval_tasks/_utils
      creating build/lib.linux-x86_64-3.10/auto_gptq/nn_modules/triton_utils
      copying auto_gptq/nn_modules/triton_utils/mixin.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules/triton_utils
      copying auto_gptq/nn_modules/triton_utils/__init__.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules/triton_utils
      copying auto_gptq/nn_modules/triton_utils/kernels.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules/triton_utils
      copying auto_gptq/nn_modules/triton_utils/custom_autotune.py -> build/lib.linux-x86_64-3.10/auto_gptq/nn_modules/triton_utils
      running build_ext
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/kaggle/working/AutoGPTQ/setup.py", line 91, in <module>
          setup(
        File "/opt/conda/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/opt/conda/lib/python3.10/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/opt/conda/lib/python3.10/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/opt/conda/lib/python3.10/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/conda/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 343, in run
          self.run_command("build")
        File "/opt/conda/lib/python3.10/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/conda/lib/python3.10/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/conda/lib/python3.10/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/opt/conda/lib/python3.10/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/conda/lib/python3.10/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/conda/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/opt/conda/lib/python3.10/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
          _build_ext.build_ext.run(self)
        File "/opt/conda/lib/python3.10/distutils/command/build_ext.py", line 340, in run
          self.build_extensions()
        File "/opt/conda/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 499, in build_extensions
          _check_cuda_version(compiler_name, compiler_version)
        File "/opt/conda/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 387, in _check_cuda_version
          raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
      RuntimeError:
      The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.8). Please make sure to use the same CUDA versions.

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for auto-gptq
  Running setup.py clean for auto-gptq
Failed to build auto-gptq
ERROR: Could not build wheels for auto-gptq, which is required to install pyproject.toml-based projects

I used !nvcc --version to Check The Cuda version it shows this:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon_Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0

How Should I Get Rid of These and Compile Successfully!

This is because the cuda version used in pytorch isn't compatible with the one that computer installed, because the major version is different. In this case, you can only install using the cu118 pre-compiled wheel, install from source will fail.

This is because the cuda version used in pytorch isn't compatible with the one that computer installed, because the major version is different. In this case, you can only install using the cu118 pre-compiled wheel, install from source will fail. Or you can also first compile pytorch from sorce, then install auto-gptq from source.

Can You Please Elaborate More and me What Should I Do More Clearly?

when the cuda's major version of pytorch used and computer installed are different, you can't compile auto-gptq from source code, instead, you can use pre-compiled wheel to install.

when the cuda's major version of pytorch used and computer installed are different, you can't compile auto-gptq from source code, instead, you can use pre-compiled wheel to install.

Yes, I Solved it using PreCompiled wheel

when the cuda's major version of pytorch used and computer installed are different, you can't compile auto-gptq from source code, instead, you can use pre-compiled wheel to install.

Yes, I Solved it using PreCompiled wheel

@TheFaheem can you list the steps you followed, this would be helpful to conclude.

when the cuda's major version of pytorch used and computer installed are different, you can't compile auto-gptq from source code, instead, you can use pre-compiled wheel to install.

Yes, I Solved it using PreCompiled wheel

@TheFaheem can you list the steps you followed, this would be helpful to conclude.

Here You Go MyFriend!
Get the link of the Precompiled wheel from the latest release and install like this
!pip install {link to the wheel}

@PanQiWei When i'm Using The TextGenerationpipeline,
I Got The Following Warning:
The model 'RWGPTQForCausalLM' is not supported for .

But it runs fine. Why Does This Shows up?

Also How Does This Precompiled Wheel https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.1/auto_gptq-0.2.1+cu118-cp310-cp310-linux_x86_64.whl works when my cuda version is 12.1 ?
@TheBloke @PanQiWei

@PanQiWei When i'm Using The TextGenerationpipeline,
I Got The Following Warning:
The model 'RWGPTQForCausalLM' is not supported for .

But it runs fine. Why Does This Shows up?

This is a warning raised from hf transformers for models that not officially supported by them, but you can just ignore it in auto-gptq.

Also How Does This Precompiled Wheel https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.1/auto_gptq-0.2.1+cu118-cp310-cp310-linux_x86_64.whl works when my cuda version is 12.1 ?

This is because the backward compatibility of cuda

@PanQiWei When i'm Using The TextGenerationpipeline,
I Got The Following Warning:
The model 'RWGPTQForCausalLM' is not supported for .

But it runs fine. Why Does This Shows up?

This is a warning raised from hf transformers for models that not officially supported by them, but you can just ignore it in auto-gptq.

Also How Does This Precompiled Wheel https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.1/auto_gptq-0.2.1+cu118-cp310-cp310-linux_x86_64.whl works when my cuda version is 12.1 ?

This is because the backward compatibility of cuda

Ahh... Is this All. Thanks For Clarifying This!

And Please Can You create a inference example script using hf pipeline?

here is a simple example of using auto-gptq quantized model with hf pipeline, for more advanced usage, you can turn to hf's tutorials and documents.

close this issue for the main problem in here is solved. anyone that have other questions or suggestions can raise in a new issue. ❤

here is a simple example of using auto-gptq quantized model with hf pipeline, for more advanced usage, for now you can turn to hf's tutorials and documents.
Systematic tutorials and example scripts for using auto-gptq will be continued added in with the development progress.

@PanQiWei One LAst Question.
Is 1 tokens per second is normal for GPTQ Inference. i think it's Not.
This is Why I Asked You For Inference Script. So Here is my Script:
import torch
from transformers import AutoTokenizer, TextGenerationPipeline
from auto_gptq import AutoGPTQForCausalLM
from huggingface_hub import hf_hub_download
from transformers import GenerationConfig
import time

#model_path = hf_hub_download(repo_id="TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ", filename="gptq_model-4bit-64g.safetensors")

# Download the model from HF and store it locally, then reference its location here:
#quantized_model_dir = model_path

from transformers import AutoTokenizer, TextGenerationPipeline

tokenizer = AutoTokenizer.from_pretrained(
    "TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ",
    use_fast=False
)
model = AutoGPTQForCausalLM.from_quantized(
    "TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ",
    use_triton=False,
    use_safetensors=True,
    device="cuda:0",
    trust_remote_code=True,
    max_memory={i: "13 GIB" for i in range(torch.cuda.device_count())}
)

#pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer, device_map="auto")

prompt = "Write a story about alpaca"
prompt_template = f"### Instruction: {prompt}\n### Response:"

start = time.time()
tokens = tokenizer(prompt_template, return_tensors="pt").to(model.device)
gen_config = GenerationConfig(max_new_tokens=256, temperature=0.3, top_k=35, top_p=0.90, pad_token_id=tokenizer.eos_token_id)
output = model.generate(inputs=tokens.input_ids, generation_config=gen_config)
print(tokenizer.decode(output[0]))

delay = time.time()
total_time = (delay - start)
time_per_token = total_time / 256

# Calculate tokens per second
tokens_per_second = 256 / total_time

# Print the results
print("Total inference time: {:.2f} ms".format(total_time))
print("Number of tokens generated: {}".format(256))
print("Time per token: {:.2f} ms/token".format(time_per_token))
print("Tokens per second: {:.2f} token/s".format(tokens_per_second))

This is What The Output is,
### Instruction: Write a story about alpaca
### Response:Once upon a time, in a small village nestled in the mountains, there lived a young girl named Maya. Maya was known for her love of animals, especially alpacas. She spent most of her days tending to the village's small herd of alpacas, helping to groom and feed them.
One day, Maya received a letter in the mail from a far-off land. The letter was from a group of scientists who were studying alpacas and had discovered something amazing. They had found a way to use alpaca wool to create a new type of fabric that was both warm and waterproof.
Maya was thrilled at the prospect of using her beloved alpacas to help the world. She immediately set out to learn more about the new fabric and how it could be used. She spent months studying and experimenting, and eventually, she came up with a plan to create a line of clothing made entirely from the new fabric.
With the help of her friends and family, Maya began to weave the fabric into clothing, scarves, and even blankets. The clothing was not only warm and waterproof, but it was also incredibly soft and comfortable.
Maya's success

Total inference time: 210.78 ms
Number of tokens generated: 256
Time per token: 0.82 ms/token
Tokens per second: 1.21 token/s

ImportError: libcudart.so.12: cannot open shared object file: No such file or directory

@lucasjinreal See the issue you opened


feat: Deprecate old bidderNeedsIdentityVerification in favor of more nuanced method for determining idv needs

Blocked by #6404 because those commits are in here. https://github.com/artsy/force/pull/6405/commits/2c345b6704930bd971c5673d3a05bf39c0c46da5 is the relevant commit.
In the process of doing #6342, #6366 and https://github.com/artsy/force/pull/6404 I found our existing bidderNeedsIdentityVerification method confusing and ended up writing some duplicative logic for calculating a series of interdependent values. Eventually it these values came to live together in a new adjacent function which returns all of them in a sack:
interface BidderQualifications {
  /** Presence of bidder */
  registrationAttempted: boolean
  /** Bidder is qualified */
  qualifiedForBidding: boolean
  /** Sale requires identity verification but the user does not have it */
  userLacksIdentityVerification: boolean
  /** User's pending identity verification, if present */
  pendingIdentityVerification: {
    internalID: string
  } | null
  /** Whether user should be prompted to verify ID, that is:
   * - they are not qualified
   * - they are lacking idv for a sale that requires it
   * - they have an available pendingIdentityVerification
   */
  shouldPromptIdVerification: boolean
}

This PR removes the older method from the final places it was used. Current tests are still passing fine and the call signatures of the removed and new methods match (with the exception of the previously still snake-cased bidder, which is null since these call sites are for registration flows) so this should just be a refactor that removes ambiguity from a helper file..

@erikdstock - can this be closed / reopened when needed?


[Question] FS0072: Type inference fails between Task and Task<'T>

ref https://github.com/rspeele/TaskBuilder.fs/issues/2
use this task builder: https://github.com/rspeele/TaskBuilder.fs/blob/6aa7ef6e47caf9f0af56413c4bb6d256763ffbaf/TaskBuilder.fs
and this F# code consuming the builder:
open System.Threading.Tasks
open FSharp.Control.Tasks.ContextSensitive

type X() =
    member this.Num = 0

let xTask : Task<X> = task { return X() }

let badTask =
    task {
        let! x = xTask
        return x.Num + 1
        //      ^ FS0072: Lookup on object of indeterminate type based on
        //        information prior to this program point.
    }

This caused the author the remove direct support for Task, and requires the user to wrap a non generic Task with a helper (unitTask), see https://github.com/rspeele/TaskBuilder.fs/commit/eeb42e52fce7d082dae1d25f2b297f9849b16f4b.
I can still reproduce this issue with the latest nightly. Do you think the type inference could be improved to support this? Or is there maybe a workaround that could be applied to the ce-builder, so that the Task<'T> overload is prioritized?

@0x53A This is either a language suggestion or a stackoverflow question
There is a hack to prioritize overloads by making them extension methods, see https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/FSharp.Core/Query.fsi#L349 for an example.  However it may not help with an interaction with type inference like this

For me FS72 is never clear between bug-fixing and language suggestion (e.g. https://github.com/Microsoft/visualfsharp/issues/3170).
It would actually expect the above code snippet to compile, and my first reaction was that this must be a bug, but I'm sure you can find a quote in the spec which says this is by design. =)
Your tip with the extension methods seems to work, I will try to create a PR against the repo.
Thank you!
Should I close this and create a new issue at fs-lang-suggestions, or keep this open?

Should I close this and create a new issue at fs-lang-suggestions, or keep this open?

Close and open a suggestion issue

@0x53A You are correct that https://github.com/Microsoft/visualfsharp/pull/3173 should really have had an RFC though it was the most borderline of these cases I know of.  If you could create one that would be great.


Please add you adapter to our repository

We saw your adapter and if it is ready please request that it is added to the repository. With this it becomes available for all users. Please consult https://github.com/ioBroker/ioBroker.repositories#add-a-new-adapter-to-the-latest-repository and the following sections for more details.
If you need help, contact us in the forum https://forum.iobroker.net/ or in the developer chat https://github.com/iobroker-community-adapters/info .
Thank you for your support.

Asses


Improve test files to cover more test cases

create for each module min, max and scenario parameter files to cover more testcases and test the deployment of every module aspect

add checklist to PR template, to maintain parameter files (if new parameter is added)

Split the issue into issues per namespace

I'm updating just the Azurefirewall one right now while I was updating it here #1328

As modules are moved and partly enforced in AVM, closing this


API Interceptor usecases

High level notes:

API Interceptors intercept configured Rancher API calls placed by UI/clients
API Interceptors can function as API controllers that can allow/deny certain APIs or APIs with certain data.
API Interceptors can modify the API command's JSON payload that will be processed by Rancher
Admin can specify multiple interceptors for an API request
Rancher behavior in case multiple interceptors are configured for an API:

When a Rancher API is called, each interceptor will be invoked one after the other
If one interceptor reports failure/rejection, then no more interceptor will be invoked further and the API will be terminated
If one interceptor returns a modified JSON payload, then this modified payload will be sent to the subsequent interceptor. The payload sent back if any by the last interceptor, will be used to process the API command at Rancher.
For API processing to proceed, all interceptors should report success

How to configure API Interceptors

Admin can configure the API Interceptors using Rancher UI under Admin -> Settings -> Advanced Settings
Edit the setting 'api.interceptor.config' and set the value to a json config content like below:

{
	"requestInterceptors": [{
		"type": "authTokenValidator",
		"paths": ["/v2-beta/projects", "/v2-beta/projects/{path:.*}"],
		"endpoint": "",
		"methods": ["post", "delete"],
		"secretToken": ""
	}, {
		"type": "http",
		"paths": ["/v2-beta/projects/{path:.*}"],
		"endpoint": "http://localhost:8092/external-interceptor",
		"methods": ["post", "delete", "get"],
		"secretToken": "pqrs"
	}, {
		"type": "http",
		"paths": ["/v2-beta/projects/{path:.*}"],
		"endpoint": "http://localhost:8094/another-external-interceptor",
		"methods": ["post", "delete", "get"],
		"secretToken": "xyz"
	}],

	"destinations": [{
		"paths": ["/v1/services/{id:.*}"],
		"destinationURL": "http://...:8080/"
	}]
}

Specify an array requestInterceptors[] where each interceptor defines:

type (Use "http" for an interceptor that is being served by an http endpoint)
endpoint (http endpoint that should be called)
paths[]  (array of Rancher API request paths that should be intercepted)
methods[] (array of HTTP methods to narrow down the APIs to be intercepted by method)
secretToken (that is used to generate a HMAC signature sent to the Interceptor in request header "X-API-Auth-Signature")

Also destinations[] can be used to specify the final endpoint to send the API to after all interceptors are invoked.

Keep destinations[] empty so that by default all APIs get executed by the Rancher server.

Example configurations:

To add an interceptor for say Create Stack API for a  particular environment (say "1a5"), interceptor config will be:

"requestInterceptors": [{
		"type": "http",
		"paths": ["/v2-beta/projects/1a5/stack"],
		"endpoint": "http://localhost:8092/external-interceptor",
		"methods": ["post"],
		"secretToken": "pqrs"
	}]

To add an interceptor for say Create Service API under a particular environment (say "1a5"), specify path as below:

"requestInterceptors": [{
		"type": "http",
		"paths": ["/v2-beta/projects/1a5/service"],
		"endpoint": "http://localhost:8092/external-interceptor",
		"methods": ["post"],
		"secretToken": "pqrs"
	}]

To add an interceptor for any API requests under a particular environment, specify path using regex as below:

"requestInterceptors": [{
		"type": "http",
		"paths": ["/v2-beta/projects/1a5/{path:.*}"],
		"endpoint": "http://localhost:8092/external-interceptor",
		"methods": ["post", "delete", "get"],
		"secretToken": "pqrs"
	}]

What is the payload sent to an Interceptor?

When Rancher calls an "http" Interceptor, it will POST following data to the interceptor endpoint:

POST http://localhost:8092/external-interceptor
request headers:
"X-API-Auth-Signature": <HMAC signature of the payload>
request JSON body
{
   UUID:
   APIPath: "/v2-beta/projects/1a5/stack"
   envID: "1a5"
   headers: {} // The original headers (or the headers returned by the last interceptor)
   body: {} // The original API body (or the body returned by the last interceptor)
}

To get the Rancher accountID if any is associated with the request, always specify the "authTokenValidator" Rancher's Interceptor as the first Interceptor in your config:

	"requestInterceptors": [{
		"type": "authTokenValidator",
		"paths": ["/v2-beta/projects", "/v2-beta/projects/{path:.*}"],
		"endpoint": "",
		"methods": ["post", "delete"],
		"secretToken": ""
	}, {}]

The "authTokenValidator" is a Rancher internal interceptor that will insert the account associated with a request, into the headers: {} in the payload. It will add a header "X-API-Account-Id" under the headers: {} that will give the accountId.

I verified various use cases for API interceptor, found different bugs including:

Paths bugs - https://github.com/rancher/rancher/issues/7818
Error message display - https://github.com/rancher/rancher/issues/7773
Timeout error - https://github.com/rancher/rancher/issues/7833
Add HTTP method to the payload (enhancement): https://github.com/rancher/rancher/issues/7823


Double-check in-app / brew updates

See https://github.com/getferdi/ferdi/issues/19#issuecomment-533257404:

I tried both, but neither the in-app update checker or brew cask outdated is reporting any available updates to install.

Copied from here:

So while Ferdi still doesn't show up when I run brew cask outdated, I do seem to be able get the newest version by running brew cask upgrade ferdi.
It's downloading 5.3.3 right now. Perhaps brew cask outdated is broken, or casks just work differently?

The auto-updater on mac currently does not work due to:
Build-in updater: The build-in updater uses the electron-updater package. This package requires the updated application to be signed in order to work. As Ferdi releases are currently not yet signed, this method won't work.
Brew cask: brew cask outdated shows a list of outdated casks that do now auto-update themselves. Ferdi however, does auto-update and thus doesn't get shown.
You'll have to run brew cask outdated --greedy in order to see if Ferdi is outdated.

@vantezzen Thanks for the explanations! Awaiting  https://github.com/getferdi/ferdi/pull/52 then.


Question: re: separating into multiple pages

I know this goes against the definition of Slate as stated in the beginning, but our API is ridiculously enormous, and I would love to know if there is a way to have each section display on a new page (rather than one long page).
Thanks!
Chris
Here is an example of someone using it like that:
https://www.fullcontact.com/developer/docs/

You can approximate this by creating subdirectories within your /slate/source/ directory, and placing a separate index.html.md file per leaf subdirectory. All the referenced content files would still be stored together in /slate/source/includes, but you'd selectively reference only those files you want per index.html.md instance.

:+1: yup, that's how to do it! Thanks for helping out @bc-mkatz.

I'm trying this out, but not getting things to match the desired example (https://www.fullcontact.com/developer/docs/).

Where in the layout.erb file should the  go? I've tried putting it in several places within the , but the results are always the same: First in the left column is the footer, then the "sub-folders/sub-items", and then the primary content of the parent's index.html.md.

As well, the ToC entries are dark blue underlined (changing to purple underlined after visiting) instead of inheriting the standard styling (which I modified, as well as added more ToC levels).

Is there perhaps a user forum where I should post my questions?

@msemp: It looks like we removed this. But while we were using it, we evidently put it at the

 tag's default location within layout.erb, as shown in the excerpt below.
For color styling, make sure you're investigating (or overriding) the variables in the
.../source/stylesheets/_variables.scss file.
Our current frustration with Slate's whole TOC approach is that we can't expose multiple subdirectories (with full accordion expand/collapse behavior) within the same TOC. Slate/Tocify apparently is just not designed to do this. So we're limited in our ability to present our content in separate subdirectories, while providing easy navigation among them. And that means we're limited in our ability to create discrete URLs – since each URL requires a separate subdirectory – which means we're limited in SEO performance.
This has become a huge frustration. We'd really like to isolate a lot of our content into separate landing and procedural pages. But that appears to be a non-starter within these constraints. Hope you can soon build out the UI you want!
 <% if current_page.data.search %>

          <input type="text" class="search" id="input-search" placeholder="Search">

      <% end %>

Products

Orders

Customers

      <% if current_page.data.toc_footers %>

          <% current_page.data.toc_footers.each do |footer| %>

<%= footer %>

          <% end %>

      <% end %>

        <%= yield %>
        <% current_page.data.includes && current_page.data.includes.each do |include| %>
          <%= partial "includes/#{include}" %>
        <% end %>

@bc-mkatz: I wasn't able to get it to work, but the consensus was to change the UI experience anyway, for other reasons, and this rendered the point moot for now :-).
But I would like to confirm that the id values in the  elements can be any text, not necessarily labeled "-link"?
Thanks very much for your follow-up!

Yes, the "-link" strings were just placeholder examples.

@bc-mkatz This approach would make search ineffective because it would only search in the current "group". For example if I click on /subdir/thing2/ link and then search, I would not get results from thing1 or thing3.

@aaani: Interesting. We've been disappointed with the quality of results from Slate's default lunr search engine. Are you telling me that search is silently scoped to the current directory? If so, do you know of a flag/setting to override this, and open up sitewide search? (Including things 1–3 in your example above.)

@bc-mkatz I apologize I wasn't aware it's using Lunr. I (wrongly) assumed the search would only be scoped to the current page.

Is there a way to make lunr search other files too?


FAQ Accordion card

https://www.frontendmentor.io/challenges/faq-accordion-card-XlyjD0Oam

https://github.com/mikhael7/frontendmentor/commit/4dd356bbfa10415d94026a7bd2aa6b3c49b68538


Improve Shell Completion documentation

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

nice, was able to install it with this hints :+1:


3d Skin layer

Then searching for a player the skin doesn't include the 3d layer as it used this link: https://minotar.net/bust/mrkavatch/180.png instead of this one: https://minotar.net/armor/bust/mrkavatch/180.png
Not the missing "armor"part

Fixed in fd5e8c399a4f9bd9373a18c4e67f278cf678ddb3


JSX syntax highlighting mysteriously stopped working today

All the of syntax highlighting for jsx files is no longer working. I tried uninstalling and then reinstalling the package and I also tried uninstalling Atom and then reinstalling it. No of these attempts solved the issue. Everything is up to date as well.

What version of atom are you running? I noticed this too after updating to 1.9.1 or 1.9.2 (OSX if it matters)

Same issue here, 1.9.2 OSX.

Having the same issue. What's the last know good version of Atom that works?

So in Atom 1.8 React syntax highlighting works - https://github.com/atom/atom/releases/tag/v1.8.0

+1 I'm using 1.9.2... just started happening yesterday for me. My guess would be that it worked in 1.9.1 but I can't confirm that.

I did however install the language-babel plugin after this happened and it appears to highlight everything properly in 1.9.2... but it also shows a notice to not have both plugins active at the same time...

I took @andersryanc 's advice and installed language-babel. That works perfectly fine. Another solution is to use a package called https://atom.io/packages/language-javascript-jsx. Bother language-babel and language-javascript-jsx will solve this issue until the react package is fixed.

I started having the same issue today. @andersryanc 's advice worked for me.

Same here, syntax highlighting broken and language-babel solved it :+1:

Should be fixed https://github.com/orktes/atom-react/issues/188


Typing CS documentation contains mistake about the room_id

In https://matrix.org/docs/spec/client_server/latest#id49, the part:

but still have a room_id key

is not correct.
Also in the sample of https://matrix.org/docs/spec/client_server/latest#m-typing:
{
    "content": {
        "user_ids": [
            "@alice:matrix.org",
            "@bob:example.com"
        ]
    },
    "room_id": "!jEsUZKDJdhlrceRyVU:example.org",
    "type": "m.typing"
}

contains room_id field, which is actually not the case.
The typing event is already in the rooms.join.{room_id}.ephemeral.events list of the sync response, so room_id is not necessary in the event content. And it's actually not there. The issue is just in the documentation.

This probably needs clarifying in the /sync section rather than in the typing module


code.tizen.org - example - gallery not working

Visit
https://code.tizen.org/demos?path=1.0%2Fexamples%2Fmobile%2FUIComponents%2Fcomponents%2Fgallery%2Fgallery.html
Try to slide
Observe:

unable to slide image

PR has been merged

Issue still is reproducible on
https://code.tizen.org/demos?path=1.0%2Fexamples%2Fmobile%2FUIComponents%2Fcomponents%2Fgallery%2Fgallery.html

Yes, but helper is not created due to fail in library-creator.
AWS is running Design Editor with this bug. On my local machine everything is correct.
Bug:

OK, I'll verify it with next code.tizen.org update

works well


Guide for managing-database-transactions

Purpose

Adds a guide for managing-database-transactions to docs/content/src/learn/guides/database-integrations folder.

resolves #374

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


AKS Basic LoadBalancer can't be used

Rancher Server Setup

Rancher version: 2.6.2
Installation option (Docker install/Helm Chart):

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc):

Proxy/Cert Details:

Information about the Cluster

Kubernetes version:
Cluster Type (Local/Downstream): Local and downstream

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider): Local cluster 3 nodes, downstream cluster AKS 1.20 provisioned by Rancher. Basic LB selected and no availability zones selected for the node group.

Describe the bug

AKS clusters cannot be created with the Basic LB sku on 2.6.2. Abbot has noticed that the LB sku dropdown is greyed out until he unclicks all of the availability zones from the first node group, adds a node group, then removes the first again. At this point, it lets him select Basic. Continuing on results in this error:
error failed to create cluster: containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=0 – Original Error: Code="SLBRequiredForAvailabilityZone" Message="Standard Load Balancer is required for availability zone." Target="networkProfile.loadBalancerSku"

Abbot cannot create clusters without a public IP in front of the LB, apparently Standard ones will always do that.
To Reproduce

None
Troubleshooting
We tried editing the cluster yaml after the fact to make everything reflect, but that did not change anything.
SURE-3685

We think that what's happening here is an empty slice of Availability Zones is being passed to AKS instead of nil, so AKS thinks there are availability zones even though there aren't. This would cause the error because if availability zones are specified, you have to use a Standard Load Balancer instead of a basic one.

Ticket #36959 - Test Results
With Docker on single-node instance:
Reproduced on rancher v2.6.2:

Fresh install of rancher v2.6.2
Provision a downstream AKS cluster with k8s v1.20.15 and a Basic LB; [un-click availability zones; add node group, delete node group, select LB as Basic]
Reproduced - Failure to provision downstream AKS cluster

Screenshot:

Verified on rancher v2.6-851ff2f401b2c932869407cb75751806bae72776-head:

Fresh install of rancher v2.6-head
Provision a downstream AKS cluster with k8s v1.20.15 and Basic LB;
Verified - Successfully provisioned AKS cluster

Screenshot:


Define and implement most important privileges

Define potential database privileges for Apinf user roles. Prioritize them and implement the most significant ones for MVP.
Definition of done

[x] Make a list of user privileges per user role

[x] ApiBackends collection
[x] ElasticSearch analytics data

[ ] Implement 3-5 most important database privileges.

I have created a document describing our system permissions.

I am working on this issue, starting with user bookmarks. I would like to work with a pair programming partner on the Elastic analytics query (filtering to a given UserID or API ID). There is also a proposed schema change to the apiBackends collection, which is modified in a pending pull-request.


Support proper initialization order for static SDS clusters

Title: Support proper initialization order for static SDS clusters
Description:
If a static cluster is using SDS (fetching certificates from remote), and it is using envoy_grpc which requires another static sds cluster.  It needs to initialize sds cluster first before initializing the cluster using sds.

@qiwzhang what is the current behavior. Will the cluster using SDS retry getting certificates from the remote, so that eventually after the SDS cluster is setup the cluster using SDS will succeed?

Yes,  subscription will retry right away after the first failure.  the cluster of using SDS will get the certificate. I even have an integration test to test this case.  https://github.com/envoyproxy/envoy/blob/master/test/integration/sds_dynamic_integration_test.cc#L367


Science Track Content

Science track content as defined by SM.

Was this one finished?
On 2 Jul 2015 22:48, "lukestanley"<EMAIL_ADDRESS>wrote:

Merged #43 https://github.com/PyconUK/pyconuk.org/pull/43.
—
Reply to this email directly or view it on GitHub
https://github.com/PyconUK/pyconuk.org/pull/43#event-346907038.

@snim2, are you asking because you don't think it's finished?

I thought there was one empty talk slot that we were still discussing?
On 3 Jul 2015 00:27, "Peter Inglesby"<EMAIL_ADDRESS>wrote:

@snim2 https://github.com/snim2, are you asking because you don't think
it's finished?
—
Reply to this email directly or view it on GitHub
https://github.com/PyconUK/pyconuk.org/pull/43#issuecomment-118191502.

On 3 July 2015 at 02:08, Sarah Mount<EMAIL_ADDRESS>wrote:

I thought there was one empty talk slot that we were still discussing?

Yeah no problem. When the talk is picked I will put it in.

For some reason I thought I saw a commit which added an actual talk in the empty slot. Maybe that got reverted or perhaps I just misread the commit message. No matter, the file that was merged seems fine.
I'll get back to you about that slot later today, I have an idea, but I need to liaise with one of the track attendees...


First draft of Inconsistencies spec

Related Issue
Fixes #3583
Description
Draft spec for dealing with inconsistencies. Probably not quite finished yet, but complete enough to begin discussions.
Microsoft Reviewers: Open in CodeFlow

Please add this one too for new line behavipur for empty text block Across web and mobile.
{
"type": "TextBlock",
"text": ""
},
Link : https://github.com/microsoft/AdaptiveCards/issues/3702

#3702 is effectively a duplicate of #641, which is in the list :)

In reply to: 571427833 [](ancestors = 571427833)


Problem with TypeScript

"npm i --save-dev @types/react-textfit" Doesn't work..

Commenting as I've had the same issue and have been unable to resolve.

basically if its lacks of types just make your own types or just declare them as 'any' for now:

create file react-textfit.d.ts in your typings folder with should be configured with your tsconfig
declare module 'react-textfit';
save and thats all

const Textfit=  require('react-textfit');

and install  npm i --save @types/node
and wait until ts is supported is what i do always when i intergrate jsx component without definition type.


XYPlot appears to require a consistant number of children

For both area and line charts it appears that there is a render issue such that react-vis requries a consitant number of elements. For instance, say we were render three line series:
<XYPlot> <series 1> <series 2> <series 3> </XYPlot>
and then add grid lines
<XYPlot> <grid> <grid> <series 1> <series 2> <series 3> </XYPlot>
More likely than not series 3 would not appear

Fixed in #75


Missing dependency for Tomcat 8.x causes crash

Deploying CAS/master on Tomcat 7.0.20, causes a crash, with the following error:
Provider com.ctc.wstx.stax.WstxInputFactory cannot be located.
This seems to be required by the hibernate validator. This pull adds the missing dependency explicitly. I have also excluded few other dependencies for logging and caching APIs to reduce conflicts.
Tested deployment on both TC7 and 8.

+1


Virus total : PUA.OSX.Adware on v1.8.1

Hi,
On Virustotal, the lastest version 1.8.1 for macOS (Apple M1/M2) is reporting the PUA.OSX.Adware.
What do you think ?
Is this a false positive ?
Thanks

Yes - it's not uncommon unfortunately for Virustotal to report these false positives.


[UDP Proxy] Add onSessionComplete for UDP session filters

Commit Message:
Additional Description: Add onSessionComplete filter callback that's called before writing access log
Risk Level: low
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:

/retest

/retest

/retest

@yanavlasov Can you, please, take a look?


Remove extra P at line 41

Not sure if this 'P' is needed.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


GPL-542 Actually we need parent/child relationships LHR stockplates from Samples extraction

User story
GPL-542 | As SRA for CGaP (Laura) I want to maintain the parent/child relationship when plates are created in Sequencescape through Samples Extraction LIMS as it would be good to have consistency between LHR stock plates created by SS or Samples extraction
Who are the primary contacts for this story
Laura L (CGaP)
Eduardo
And?
Acceptance criteria
To be considered successful the solution must allow:

[ ] add a list of acceptance criteria here
[ ] ...

Dependencies
This story is blocked by the following dependencies:

#<issue_no.>
sanger/#<issue_no.>

Additional context
Laura L writes: Thursday, 18 June 2020 at 18:08
Maintain the parent/child relationship when plates are created in Sequencescape through Samples Extraction LIMS. As it is now all the LHR stock plates that were created prior to using Samples extraction have parent/child relationships in SS, whereas those created through samples extraction do not. This wasn’t deemed an issue but it would be good to have consistency and might have implications in tracking/reporting? We did talk about this a while back at stand-up and I think we said add it to the bottom of the backlog but not sure anyone was tasked with this so it may have been forgotten.

Discussed on stand up 22nd July: "...422 and 424 are dependent on GPL-542 being fixed, so we think that will need to be done." Agreed that

Will roll out 422 & 424 into production, cant be used but is safe (more work rolling back or putting in additional warnings to user)
This story GPL-542 given a low priority put below; Sentinel, LH Crawler Cambridge & improvements & More data to MLWH (reporting) at time of decision.


fix mnemonicsToEntropy mnemonic length check

This will close #300

Hello! thanks for this PR, please apply for a grant for this fix: https://gitcoin.co/issue/skywinder/web3swift/300/100026188

I applied and updated my branch with the latest changes from the develop branch.


Update Edge Introduction pictures

Summary of the changes / Why this is an improvement

updated Edge Intro pictures
removed :scale: 50%

Checklist

[x] Link to issue this PR refers to (if applicable): https://github.com/crate/cloud-docs/issues/43
[x] CLA is signed

Hi. Was it just some conversion using oxipng, or are those real updates? If the latter is the case, I would like to suggest to switch to a better style/paradigm of having screenshots in the documentation.
https://thisisimportant.net/posts/screenshots-in-documentation/
-- via: https://news.ycombinator.com/item?id=38639629
My proposal is to Use simplified screenshots exclusively.

Hi. I see where you're coming from.
Screenshots are highly debatable topic, I agree that we could do them better. For one, we could (should?) do less of them. I am guilty of this myself, it's easy to rely on them.
I added "screenshot overhaul" to my backlog and will consider your suggestion. Obscuring parts sound like a good idea to me.

Thanks!

Obscuring parts sound like a good idea to me.

Not only obscuring, but also cropping down to the element of interest.


Add version: cthoeing.PasswordTech version 3.4.4

Pull request has been created with Komac v1.6.0 :rocket:
Microsoft Reviewers: Open in CodeFlow

@wingetbot waivers Add Validation-Unapproved-URL


Switch from develop to main

Since develop is just being used as main, and releases as stored as tags, remove develop and use main as main branch.
Acceptance Criteria:

update main with the latest code
ensure branch protection rules are still active
ensure GHA workflows are up-to-date for the new main branch name

Resolved by #639


How to make that dirty lens effect shown on the readme ?..

 Eg, the one from https://github.com/Unity-Technologies/PostProcessing.

Ok thanks :)


A question about use " MSE loss"

Thank you very much for sharing the well organized code. I am trying to build a work on your code. I found that the code worked fine with the cross entropy loss you defined earlier, but I got the error "RuntimeError: CUDA error: device-side assert triggered" when I used my own MSE loss：
def ignore_label(scores, labels, ignore=None, valid=None, scene_id=None):
    if ignore is None:
        return scores, labels
    if(valid is None):
        valid = labels != ignore
    vscores = scores[valid]
    vlabels = labels[valid]
    return vscores, vlabels

@LOSSES.register_module('MSELoss')
class MSELoss(nn.Module):
    def __init__(
        self,
        pred="c_pred",
        target="c_target",
        valid_index=None,
        size_average=None,
        reduce=None,
        reduction="mean",
        loss_weight=1.0,
        ignore_index=20
    ):
        super(MSELoss, self).__init__()
        self.loss_weight = loss_weight
        self.ignore_index = ignore_index
        self.pred = pred
        self.target = target
        self.valid_index = valid_index
        self.loss = nn.MSELoss(
            size_average=size_average,
            reduce=reduce,
            reduction=reduction
        )

    def forward(self, points):
        pred = points[self.pred]
        target = points[self.target]
        if(self.valid_index):
            valid = points[self.valid_index]
            pred, target = ignore_label(pred,target,self.ignore_index,valid)

        loss = self.loss(pred, target) * self.loss_weight
        return loss

I tried to split the data from 102400 to 51200, and even 25600, and still got this error.

Traceback (most recent call last):
File "/root/models/Pointcept_DM5/tools/train.py", line 68, in
main()
File "/root/models/Pointcept_DM5/tools/train.py", line 55, in main
launch(
File "/root/models/Pointcept_DM5/pointcept/engines/launch.py", line 89, in launch
main_func(*cfg) # 单卡训练
File "/root/models/Pointcept_DM5/tools/train.py", line 27, in main_worker
trainer.train()
File "/root/models/Pointcept_DM5/pointcept/engines/train.py", line 165, in train
self.run_step()
File "/root/models/Pointcept_DM5/pointcept/engines/train.py", line 236, in run_step
output_dict = self.model(input_dict,input_dict)
File "/root/packages/anconda3/envs/pt3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/packages/anconda3/envs/pt3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, kwargs)
File "/root/models/Pointcept_DM5/pointcept/models/default.py", line 572, in forward
point['c_pred'] = point['c_pred'][valid]
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
This problem is very troubling to me, can you give me some suggestions？ Thank you.

Thank you very much for sharing the well organized code. I am trying to build a work on your code. I found that the code worked fine with the cross entropy loss you defined earlier, but I got the error "RuntimeError: CUDA error: device-side assert triggered" when I used my own MSE loss：
def ignore_label(scores, labels, ignore=None, valid=None, scene_id=None):
    if ignore is None:
        return scores, labels
    if(valid is None):
        valid = labels != ignore
    vscores = scores[valid]
    vlabels = labels[valid]
    return vscores, vlabels

@LOSSES.register_module('MSELoss')
class MSELoss(nn.Module):
    def __init__(
        self,
        pred="c_pred",
        target="c_target",
        valid_index=None,
        size_average=None,
        reduce=None,
        reduction="mean",
        loss_weight=1.0,
        ignore_index=20
    ):
        super(MSELoss, self).__init__()
        self.loss_weight = loss_weight
        self.ignore_index = ignore_index
        self.pred = pred
        self.target = target
        self.valid_index = valid_index
        self.loss = nn.MSELoss(
            size_average=size_average,
            reduce=reduce,
            reduction=reduction
        )

    def forward(self, points):
        pred = points[self.pred]
        target = points[self.target]
        if(self.valid_index):
            valid = points[self.valid_index]
            pred, target = ignore_label(pred,target,self.ignore_index,valid)

        loss = self.loss(pred, target) * self.loss_weight
        return loss

I tried to split the data from 102400 to 51200, and even 25600, and still got this error.  Traceback (most recent call last): File "/root/models/Pointcept_DM5/tools/train.py", line 68, in main() File "/root/models/Pointcept_DM5/tools/train.py", line 55, in main launch( File "/root/models/Pointcept_DM5/pointcept/engines/launch.py", line 89, in launch main_func(*cfg) # 单卡训练 File "/root/models/Pointcept_DM5/tools/train.py", line 27, in main_worker trainer.train() File "/root/models/Pointcept_DM5/pointcept/engines/train.py", line 165, in train self.run_step() File "/root/models/Pointcept_DM5/pointcept/engines/train.py", line 236, in run_step output_dict = self.model(input_dict,input_dict) File "/root/packages/anconda3/envs/pt3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/root/packages/anconda3/envs/pt3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "/root/models/Pointcept_DM5/pointcept/models/default.py", line 572, in forward point['c_pred'] = point['c_pred'][valid] RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
This problem is very troubling to me, can you give me some suggestions？ Thank you.

I changed the output channel to 3(N,3) in the original code, and calculated the MSE with "coord"(N,3) together with the output, but still got the following error:
*[06/05 08:34:31 pointcept]: Train: [1/100][212/4800] Data 0.003 (0.004) Batch 0.275 (0.295) Remain 39:16:00 loss: 0.0016 Lr: 0.00060
[06/05 08:34:32 pointcept]: Train: [1/100][213/4800] Data 0.005 (0.004) Batch 0.256 (0.294) Remain 39:14:32 loss: 0.0011 Lr: 0.00060
[06/05 08:34:32 pointcept]: Train: [1/100][214/4800] Data 0.003 (0.004) Batch 0.260 (0.294) Remain 39:13:13 loss: 0.0013 Lr: 0.00060
Traceback (most recent call last):
File "/root/models/Pointcept_DM5/tools/train.py", line 68, in
main()
File "/root/models/Pointcept_DM5/tools/train.py", line 55, in main
launch(
File "/root/models/Pointcept_DM5/pointcept/engines/launch.py", line 89, in launch
main_func(cfg) # 单卡训练
File "/root/models/Pointcept_DM5/tools/train.py", line 27, in main_worker
trainer.train()
File "/root/models/Pointcept_DM5/pointcept/engines/train.py", line 165, in train
self.run_step()
File "/root/models/Pointcept_DM5/pointcept/engines/train.py", line 240, in run_step
self.scaler.scale(loss).backward() # 后向传播
File "/root/packages/anconda3/envs/pt3/lib/python3.8/site-packages/torch/_tensor.py", line 492, in backward
torch.autograd.backward(
File "/root/packages/anconda3/envs/pt3/lib/python3.8/site-packages/torch/autograd/init.py", line 251, in backward
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
This is really confusing...  Can you try to reproduce this? This shouldn't require much code changes.

Oh, I also encounter a similar issue in some settings. I suspect it is caused by pooling. I rewrote it in my development version, but still have not boarder verified. I can attach the experimental version to this issue tomorrow.

May I confirm which model you are using?

May I confirm which model you are using?

Thank you for you reply. I am using version 1.5.2 Pointcept (downloaded directly from GitHub).  However, in practice, I used the output (N, 3) of Point Transformer V3 (channel 20 becomes channel 3) to align the original coordinates (N, 3) by MSE, which is not similar problem occurred. I modified the code and then this error occurred. I don't think I changed much, I just replaced the input features. Would you like to help me look at the code? If possible, I can send my current code to your email. Thank you.

May I confirm which model you are using?

Thank you for you reply. I am using version 1.5.2 Pointcept (downloaded directly from GitHub). However, in practice, I used the output (N, 3) of Point Transformer V3 (channel 20 becomes channel 3) to align the original coordinates (N, 3) by MSE, which is not similar problem occurred. I modified the code and then this error occurred. I don't think I changed much, I just replaced the input features. Would you like to help me look at the code? If possible, I can send my current code to your email. Thank you.

If possible, please leave your email address where you can be contacted. Thank you.

May I confirm which model you are using?

Or you can try this: concatenate the coordinates (N,3) with a random variable (N,3) is as input to the network. The output is aligned to the original coordinates (N,3) by MSE. Similar errors will occur after training for several epochs on ScanNet.

I am also not sure, but try to replace the pooling code with the following (not rely on code, but rely on grid_coord)
class SerializedPooling(PointModule):
    def __init__(
        self,
        in_channels,
        out_channels,
        stride=2,
        norm_layer=None,
        act_layer=None,
        reduce="max",
        shuffle_orders=True,
        traceable=True,  # record parent and cluster
    ):
        super().__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels

        self.stride = stride
        assert reduce in ["sum", "mean", "min", "max"]
        self.reduce = reduce
        self.shuffle_orders = shuffle_orders
        self.traceable = traceable

        self.proj = nn.Linear(in_channels, out_channels)
        if norm_layer is not None:
            self.norm = PointSequential(norm_layer(out_channels))
        if act_layer is not None:
            self.act = PointSequential(act_layer())

    def serialized_forward(self, point: Point):
        pooling_depth = (math.ceil(self.stride) - 1).bit_length()
        if pooling_depth > point.serialized_depth:
            pooling_depth = 0
        assert {
            "serialized_code",
            "serialized_order",
            "serialized_inverse",
            "serialized_depth",
        }.issubset(
            point.keys()
        ), "Run point.serialization() point cloud before SerializedPooling"

        code = point.serialized_code >> pooling_depth * 3
        code_, cluster, counts = torch.unique(
            code[0],
            sorted=True,
            return_inverse=True,
            return_counts=True,
        )
        # indices of point sorted by cluster, for torch_scatter.segment_csr
        _, indices = torch.sort(cluster)
        # index pointer for sorted point, for torch_scatter.segment_csr
        idx_ptr = torch.cat([counts.new_zeros(1), torch.cumsum(counts, dim=0)])
        # head_indices of each cluster, for reduce attr e.g. code, batch
        head_indices = indices[idx_ptr[:-1]]
        # generate down code, order, inverse
        code = code[:, head_indices]
        order = torch.argsort(code)
        inverse = torch.zeros_like(order).scatter_(
            dim=1,
            index=order,
            src=torch.arange(0, code.shape[1], device=order.device).repeat(
                code.shape[0], 1
            ),
        )

        if self.shuffle_orders:
            perm = torch.randperm(code.shape[0])
            code = code[perm]
            order = order[perm]
            inverse = inverse[perm]

        # collect information
        point_dict = Dict(
            feat=torch_scatter.segment_csr(
                self.proj(point.feat)[indices], idx_ptr, reduce=self.reduce
            ),
            coord=torch_scatter.segment_csr(
                point.coord[indices], idx_ptr, reduce="mean"
            ),
            grid_coord=point.grid_coord[head_indices] >> pooling_depth,
            serialized_code=code,
            serialized_order=order,
            serialized_inverse=inverse,
            serialized_depth=point.serialized_depth - pooling_depth,
            batch=point.batch[head_indices],
        )

        if "condition" in point.keys():
            point_dict["condition"] = point.condition
        if "context" in point.keys():
            point_dict["context"] = point.context

        if self.traceable:
            point_dict["pooling_inverse"] = cluster
            point_dict["pooling_parent"] = point
        point = Point(point_dict)
        if self.norm is not None:
            point = self.norm(point)
        if self.act is not None:
            point = self.act(point)
        point.sparsify()
        return point

    def grid_forward(self, point: Point):
        if "grid_coord" in point.keys():
            grid_coord = point.grid_coord
        elif {"coord", "grid_size"}.issubset(point.keys()):
            grid_coord = torch.div(
                point.coord - point.coord.min(0)[0],
                point.grid_size,
                rounding_mode="trunc",
            ).int()
        else:
            raise AssertionError(
                "[gird_coord] or [coord, grid_size] should be include in the Point"
            )
        grid_coord = torch.div(grid_coord, self.stride, rounding_mode="trunc")
        grid_coord, cluster, counts = torch.unique(
            grid_coord,
            sorted=True,
            return_inverse=True,
            return_counts=True,
            dim=0,
        )
        # indices of point sorted by cluster, for torch_scatter.segment_csr
        _, indices = torch.sort(cluster)
        # index pointer for sorted point, for torch_scatter.segment_csr
        idx_ptr = torch.cat([counts.new_zeros(1), torch.cumsum(counts, dim=0)])
        # head_indices of each cluster, for reduce attr e.g. code, batch
        head_indices = indices[idx_ptr[:-1]]
        point_dict = Dict(
            feat=torch_scatter.segment_csr(
                self.proj(point.feat)[indices], idx_ptr, reduce=self.reduce
            ),
            coord=torch_scatter.segment_csr(
                point.coord[indices], idx_ptr, reduce="mean"
            ),
            grid_coord=grid_coord,
            batch=point.batch[head_indices],
        )
        if "condition" in point.keys():
            point_dict["condition"] = point.condition
        if "context" in point.keys():
            point_dict["context"] = point.context

        if self.traceable:
            point_dict["pooling_inverse"] = cluster
            point_dict["pooling_parent"] = point
        order = point.order
        point = Point(point_dict)
        if self.norm is not None:
            point = self.norm(point)
        if self.act is not None:
            point = self.act(point)
        point.serialization(order=order, shuffle_orders=self.shuffle_orders)
        point.sparsify()
        return point

    def forward(self, point: Point):
        # if self.stride == 2 ** (math.ceil(self.stride) - 1).bit_length():
        #     return self.serialized_forward(point)
        # else:
        #     return self.grid_forward(point)
        return self.grid_forward(point)

Give me a feedback if this work

I am also not sure, but try to replace the pooling code with the following (not rely on code, but rely on grid_coord)

Thank you very much. I will now replace this code and try it.

I am also not sure, but try to replace the pooling code with the following (not rely on code, but rely on grid_coord)

Thank you very much. I will now replace this code and try it.
I found the following problems:

Add this line in Point.serialization()

Add this line in Point.serialization()

Thank you very much. It can be run. However, it may take several epochs before similar errors appear. So, we need to wait for a while.

I am also not sure, but try to replace the pooling code with the following (not rely on code, but rely on grid_coord)
class SerializedPooling(PointModule):
    def __init__(
        self,
        in_channels,
        out_channels,
        stride=2,
        norm_layer=None,
        act_layer=None,
        reduce="max",
        shuffle_orders=True,
        traceable=True,  # record parent and cluster
    ):
        super().__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels

        self.stride = stride
        assert reduce in ["sum", "mean", "min", "max"]
        self.reduce = reduce
        self.shuffle_orders = shuffle_orders
        self.traceable = traceable

        self.proj = nn.Linear(in_channels, out_channels)
        if norm_layer is not None:
            self.norm = PointSequential(norm_layer(out_channels))
        if act_layer is not None:
            self.act = PointSequential(act_layer())

    def serialized_forward(self, point: Point):
        pooling_depth = (math.ceil(self.stride) - 1).bit_length()
        if pooling_depth > point.serialized_depth:
            pooling_depth = 0
        assert {
            "serialized_code",
            "serialized_order",
            "serialized_inverse",
            "serialized_depth",
        }.issubset(
            point.keys()
        ), "Run point.serialization() point cloud before SerializedPooling"

        code = point.serialized_code >> pooling_depth * 3
        code_, cluster, counts = torch.unique(
            code[0],
            sorted=True,
            return_inverse=True,
            return_counts=True,
        )
        # indices of point sorted by cluster, for torch_scatter.segment_csr
        _, indices = torch.sort(cluster)
        # index pointer for sorted point, for torch_scatter.segment_csr
        idx_ptr = torch.cat([counts.new_zeros(1), torch.cumsum(counts, dim=0)])
        # head_indices of each cluster, for reduce attr e.g. code, batch
        head_indices = indices[idx_ptr[:-1]]
        # generate down code, order, inverse
        code = code[:, head_indices]
        order = torch.argsort(code)
        inverse = torch.zeros_like(order).scatter_(
            dim=1,
            index=order,
            src=torch.arange(0, code.shape[1], device=order.device).repeat(
                code.shape[0], 1
            ),
        )

        if self.shuffle_orders:
            perm = torch.randperm(code.shape[0])
            code = code[perm]
            order = order[perm]
            inverse = inverse[perm]

        # collect information
        point_dict = Dict(
            feat=torch_scatter.segment_csr(
                self.proj(point.feat)[indices], idx_ptr, reduce=self.reduce
            ),
            coord=torch_scatter.segment_csr(
                point.coord[indices], idx_ptr, reduce="mean"
            ),
            grid_coord=point.grid_coord[head_indices] >> pooling_depth,
            serialized_code=code,
            serialized_order=order,
            serialized_inverse=inverse,
            serialized_depth=point.serialized_depth - pooling_depth,
            batch=point.batch[head_indices],
        )

        if "condition" in point.keys():
            point_dict["condition"] = point.condition
        if "context" in point.keys():
            point_dict["context"] = point.context

        if self.traceable:
            point_dict["pooling_inverse"] = cluster
            point_dict["pooling_parent"] = point
        point = Point(point_dict)
        if self.norm is not None:
            point = self.norm(point)
        if self.act is not None:
            point = self.act(point)
        point.sparsify()
        return point

    def grid_forward(self, point: Point):
        if "grid_coord" in point.keys():
            grid_coord = point.grid_coord
        elif {"coord", "grid_size"}.issubset(point.keys()):
            grid_coord = torch.div(
                point.coord - point.coord.min(0)[0],
                point.grid_size,
                rounding_mode="trunc",
            ).int()
        else:
            raise AssertionError(
                "[gird_coord] or [coord, grid_size] should be include in the Point"
            )
        grid_coord = torch.div(grid_coord, self.stride, rounding_mode="trunc")
        grid_coord, cluster, counts = torch.unique(
            grid_coord,
            sorted=True,
            return_inverse=True,
            return_counts=True,
            dim=0,
        )
        # indices of point sorted by cluster, for torch_scatter.segment_csr
        _, indices = torch.sort(cluster)
        # index pointer for sorted point, for torch_scatter.segment_csr
        idx_ptr = torch.cat([counts.new_zeros(1), torch.cumsum(counts, dim=0)])
        # head_indices of each cluster, for reduce attr e.g. code, batch
        head_indices = indices[idx_ptr[:-1]]
        point_dict = Dict(
            feat=torch_scatter.segment_csr(
                self.proj(point.feat)[indices], idx_ptr, reduce=self.reduce
            ),
            coord=torch_scatter.segment_csr(
                point.coord[indices], idx_ptr, reduce="mean"
            ),
            grid_coord=grid_coord,
            batch=point.batch[head_indices],
        )
        if "condition" in point.keys():
            point_dict["condition"] = point.condition
        if "context" in point.keys():
            point_dict["context"] = point.context

        if self.traceable:
            point_dict["pooling_inverse"] = cluster
            point_dict["pooling_parent"] = point
        order = point.order
        point = Point(point_dict)
        if self.norm is not None:
            point = self.norm(point)
        if self.act is not None:
            point = self.act(point)
        point.serialization(order=order, shuffle_orders=self.shuffle_orders)
        point.sparsify()
        return point

    def forward(self, point: Point):
        # if self.stride == 2 ** (math.ceil(self.stride) - 1).bit_length():
        #     return self.serialized_forward(point)
        # else:
        #     return self.grid_forward(point)
        return self.grid_forward(point)

Give me a feedback if this work

Thank you very much. Now the code can train normally. At the same time, I found that this pooling did not affect performance.

Fundamentally, the two pooling strategies are the same. The former relies on existing serialization codes. I will change the pooling code to the later one as some other researcher want to change the default serialization pattern, but it will also affect the pooling process.

Hi @Gofinge,
the grid_forward wrongly pools points from different batches. The problem appears after the torch.unique which doesn't know about the batch.
It could be easily solved by pre-shifting the batch indexes and removing them later:
        ....
        grid_coord = grid_coord | point.batch.view(-1, 1) << 16
        grid_coord, cluster, counts = torch.unique(
            grid_coord,
            sorted=True,
            return_inverse=True,
            return_counts=True,
            dim=0,
        )
        grid_coord = grid_coord & 0xFFFF
        ....

Ah sorry,
my comment was based on your suggestion in this issue:

https://github.com/Pointcept/PointTransformerV3/issues/59#issuecomment-2151455021

As far as I know, there is no grid-pooling for ptv3, or am I wrong?

@jotix16 Oh, yes. I remember that now. Thanks for your reminding.
BTW: Fundermantially Pooling used in PTv3 is GridPool. The two implementations are actually the same.

Exactly, but as you mentioned somewhere, in SerializedPooling you can scale the grid only with powers of two.
GridPooling is more flexible but you have to re-serialize.
Thank you for getting back to me so quickly, I just added the comment in case someone needs to use the grid pooling from this issue.

Exactly, but as you mentioned somewhere, in SerializedPooling you can scale the grid only with powers of two.
GridPooling is more flexible but you have to re-serialize.
Thank you for getting back to me so quickly, I just added the comment in case someone needs to use the grid pooling from this issue.

Hi, have you used PT v3 to train on a single 24G GPU? I set batch size=2, num work=4, and kept everything else the same. I found that the performance of PT v3 fluctuated greatly (76.10~77.23). I fixed the random seed. But the loss results are not consistent. Do you know why this is? How can I improve or set it up to make the performance of PT v3 stable?