Error installing in Automatic1111

Here is the error in the console:
Error running install.py for extension D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "d:\repos\stable-diffusion-webui\venv\Scripts\python.exe" "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stdout: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
*** Collecting tensorrt==9.0.1.post11.dev4
***   Downloading https://pypi.nvidia.com/tensorrt/tensorrt-9.0.1.post11.dev4.tar.gz (18 kB)
***   Preparing metadata (setup.py): started
***   Preparing metadata (setup.py): finished with status 'done'
*** Building wheels for collected packages: tensorrt
***   Building wheel for tensorrt (setup.py): started
***   Building wheel for tensorrt (setup.py): still running...
***   Building wheel for tensorrt (setup.py): finished with status 'done'
***   Created wheel for tensorrt: filename=tensorrt-9.0.1.post11.dev4-py2.py3-none-any.whl size=17618 sha256=e059e2b3b7dd7ecf4c805ab6f2b4589ddb43b0959bfa66178fa0d01559ba1ef8
***   Stored in directory: c:\users\X\appdata\local\pip\cache\wheels\d1\6d\71\f679d0d23a60523f9a05445e269bfd0bcd1c5272097fa931df
*** Successfully built tensorrt
*** Installing collected packages: tensorrt
*** Successfully installed tensorrt-9.0.1.post11.dev4
*** Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
*** Collecting polygraphy
***   Downloading polygraphy-0.49.0-py2.py3-none-any.whl (327 kB)
***      -------------------------------------- 327.9/327.9 kB 4.1 MB/s eta 0:00:00
*** Installing collected packages: polygraphy
*** Successfully installed polygraphy-0.49.0
*** Collecting protobuf==3.20.2
***   Downloading protobuf-3.20.2-cp310-cp310-win_amd64.whl (904 kB)
***      -------------------------------------- 904.0/904.0 kB 4.4 MB/s eta 0:00:00
*** Installing collected packages: protobuf
***   Attempting uninstall: protobuf
***     Found existing installation: protobuf 3.20.0
***     Uninstalling protobuf-3.20.0:
***       Successfully uninstalled protobuf-3.20.0
*** TensorRT is not installed! Installing...
*** Installing nvidia-cudnn-cu11
*** Installing tensorrt
*** removing nvidia-cudnn-cu11
*** Polygraphy is not installed! Installing...
*** Installing polygraphy
*** GS is not installed! Installing...
*** Installing protobuf
***
*** stderr: A matching Triton is not available, some optimizations will not be enabled.
*** Error caught was: No module named 'triton'
*** d:\repos\stable-diffusion-webui\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
***   rank_zero_deprecation(
***
*** [notice] A new release of pip available: 22.2.1 -> 23.2.1
*** [notice] To update, run: d:\repos\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip
***
*** [notice] A new release of pip available: 22.2.1 -> 23.2.1
*** [notice] To update, run: d:\repos\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip
*** ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'D:\\repos\\stable-diffusion-webui\\venv\\Lib\\site-packages\\google\\~rotobuf\\internal\\_api_implementation.cp310-win_amd64.pyd'
*** Check the permissions.
***
***
*** [notice] A new release of pip available: 22.2.1 -> 23.2.1
*** [notice] To update, run: d:\repos\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip
*** Traceback (most recent call last):
***   File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 30, in <module>***     install()
***   File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 19, in install
***     launch.run_pip("install protobuf==3.20.2", "protobuf", live=True)
***   File "d:\repos\stable-diffusion-webui\modules\launch_utils.py", line 138, in run_pip
***     return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live)
***   File "d:\repos\stable-diffusion-webui\modules\launch_utils.py", line 115, in run
***     raise RuntimeError("\n".join(error_bits))
*** RuntimeError: Couldn't install protobuf.
*** Command: "d:\repos\stable-diffusion-webui\venv\Scripts\python.exe" -m pip install protobuf==3.20.2 --prefer-binary
*** Error code: 1

And then when I restarted the webui, I got these popups:

What does that mean?

And then when I try to export the default engine, I get this error in the console:
{'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
Disabling attention optimization
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Traceback (most recent call last):
  File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 84, in export_onnx
    torch.onnx.export(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 506, in export
    _export(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 1548, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 1113, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 989, in _create_jit_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\onnx\utils.py", line 893, in _trace_and_get_graph_from_model
    trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 1268, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 127, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\jit\_trace.py", line 118, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "D:\repos\stable-diffusion-webui\modules\sd_unet.py", line 91, in UNetModel_forward
    return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
  File "D:\repos\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 789, in forward
    emb = self.time_embed(t_emb)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\container.py", line 217, in forward
    input = module(input)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1488, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "D:\repos\stable-diffusion-webui\extensions-builtin\Lora\networks.py", line 429, in network_Linear_forward
    return originals.Linear_forward(self, input)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1117, in call_function
    prediction = await utils.async_iteration(iterator)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 350, in async_iteration
    return await iterator.__anext__()
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 343, in __anext__
    return await anyio.to_thread.run_sync(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 326, in run_sync_iterator_async
    return next(iterator)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 695, in gen_wrapper
    yield from f(*args, **kwargs)
  File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 154, in export_unet_to_trt
    export_onnx(
  File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "C:\Users\hesad\AppData\Local\Programs\Python\Python310\lib\_sitebuiltins.py", line 26, in __call__
    raise SystemExit(code)
SystemExit: None

Hello!
What packages have you installed? Anything updated from auto1111 default packages?
Your 2nd error might be due to the failed installation.

Here are my other installed extensions. Pretty standard stuff. Otherwise, it is all default. (I also already updated my NVIDIA drivers to 537.58 too. I have a RTX 3060.)

I should mention I'm on the latest version of Auto1111: version: v1.6.0  •  python: 3.10.6  •  torch: 2.0.1+cu118  •  xformers: 0.0.20  •  gradio: 3.41.2  •  checkpoint: 1240e811e2

We also used the lastest version of Auto1111.
Does the first error occur again when you restart the webui?
Have you tried manually installing protobuf?

If I restart the webui, I get the popups again, and this in the console:
Requirement already satisfied: protobuf==3.20.2 in .\venv\lib\site-packages (3.20.2)
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: onnx-graphsurgeon in .\venv\lib\site-packages (0.3.27)
Requirement already satisfied: onnx in .\venv\lib\site-packages (from onnx-graphsurgeon) (1.14.1)
Requirement already satisfied: numpy in .\venv\lib\site-packages (from onnx-graphsurgeon) (1.23.5)
Requirement already satisfied: protobuf>=3.20.2 in .\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (3.20.2)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in .\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (4.5.0)
GS is not installed! Installing...
Installing protobuf
Installing onnx-graphsurgeon
UI Config not initialized

I have not tried manually installing protobuf.

You can see the packages that a required and how to install them in the install.py in the root dir.
The pop up errors are due to PyTorch having their own cudnn libraries, but TensorRT requires them aswell and installs them too. We remove that package after installing the tensorrt wheel. (see here: https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/blob/d8eee382158cd7373e44fe7ac9265aa985a8fd29/install.py#L9)

I shouldn't have to install anything manually... Why are there separate cudnn libraries? They seem to be conflicting... Or did TensorRT fail to remove the package after installing the tensorrt wheel?

Should I delete the extension and try to install it again?

No you shouldn't, that's why we have the install.py that auto1111 runs. It might have failed to uninstall... Can you also sent me a pip freeze ?

what is a pip freeze, and how do I do it?

Should I delete the extension and try to install it again?

You can, maybe it makes a difference.

Ok, I deleted the extension folder, and restarted webui. Then I tried reinstalling the extension. I did not get the console error messages this time about protobuf (probably because it is already installed). Just this:
Requirement already satisfied: protobuf==3.20.2 in .\venv\lib\site-packages (3.20.2)
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: onnx-graphsurgeon in .\venv\lib\site-packages (0.3.27)
Requirement already satisfied: numpy in .\venv\lib\site-packages (from onnx-graphsurgeon) (1.23.5)
Requirement already satisfied: onnx in .\venv\lib\site-packages (from onnx-graphsurgeon) (1.14.1)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in .\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (4.5.0)
Requirement already satisfied: protobuf>=3.20.2 in .\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (3.20.2)
GS is not installed! Installing...
Installing protobuf
Installing onnx-graphsurgeon
UI Config not initialized

Then when I restarted the webui, I got the popup errors again, the same console messages above, and when trying to export the default engine I get the same console error: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

I can see that the packages are installed that our extension needs, but cudnn is still installed.
The extension won't uninstall it because it only checks if tensorrt is installed, and if it is, it won't uninstall cudnn.... might need a code change here. You can try to run the following in the venv of automatic1111 :
python -m pip uninstall -y nvidia-cudnn-cu11

Tried running that, and got this:
WARNING: Ignoring invalid distribution -rotobuf (d:\repos\stable-diffusion-webui\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -rotobuf (d:\repos\stable-diffusion-webui\venv\lib\site-packages)
Found existing installation: nvidia-cudnn-cu11 <IP_ADDRESS>
Uninstalling nvidia-cudnn-cu11-<IP_ADDRESS>:
  Successfully uninstalled nvidia-cudnn-cu11-<IP_ADDRESS>
ERROR: Exception:
Traceback (most recent call last):
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_internal\commands\uninstall.py", line 103, in run
    uninstall_pathset.commit()
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_internal\req\req_uninstall.py", line 424, in commit
    self._moved_paths.commit()
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_internal\req\req_uninstall.py", line 277, in commit
    save_dir.cleanup()
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_internal\utils\temp_dir.py", line 173, in cleanup
    rmtree(self._path)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 326, in wrapped_f
    return self(f, *args, **kw)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 406, in __call__
    do = self.iter(retry_state=retry_state)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 362, in iter
    raise retry_exc.reraise()
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 195, in reraise
    raise self.last_attempt.result()
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 451, in result
    return self.__get_result()
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 403, in __get_result
    raise self._exception
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 409, in __call__
    result = fn(*args, **kwargs)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\pip\_internal\utils\misc.py", line 128, in rmtree
    shutil.rmtree(dir, ignore_errors=ignore_errors, onerror=rmtree_errorhandler)
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 749, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 614, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 619, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 617, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'D:\\repos\\stable-diffusion-webui\\venv\\Lib\\site-packages\\nvidia\\~udnn\\bin\\cudnn64_8.dll'

Restarted webui, and no more popup errors, but still get this when trying to export default engine: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

If I do a pip freeze now, I see this in the console:
WARNING: Ignoring invalid distribution -vidia-cudnn-cu11 (d:\repos\stable-diffusion-webui\venv\lib\site-packages)
WARNING: Ignoring invalid distribution -rotobuf (d:\repos\stable-diffusion-webui\venv\lib\site-packages)

And here is the current state of the pip freeze:
absl-py==1.4.0
accelerate==0.21.0
addict==2.4.0
aenum==3.1.12
aiofiles==23.2.1
aiohttp==3.8.4
aiosignal==1.3.1
altair==5.0.0
antlr4-python3-runtime==4.9.3
anyio==3.6.2
async-timeout==4.0.2
attrs==23.1.0
basicsr==1.4.2
beautifulsoup4==4.12.2
blendmodes==2022
blis==0.7.9
boltons==23.0.0
Brotli==1.1.0
cachetools==5.3.0
catalogue==2.0.8
certifi==2023.5.7
cffi==1.15.1
chardet==4.0.0
charset-normalizer==3.1.0
clean-fid==0.1.35
click==8.1.7
clip @ git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
colorama==0.4.6
confection==0.0.4
contourpy==1.0.7
cssselect2==0.7.0
cycler==0.11.0
cymem==2.0.7
deprecation==2.1.0
duckduckgo-search==3.9.3
dynamicprompts==0.29.0
einops==0.4.1
facexlib==0.3.0
fastapi==0.94.0
ffmpy==0.3.0
filelock==3.12.0
filterpy==1.4.5
flatbuffers==23.5.9
font-roboto==0.0.1
fonts==0.0.3
fonttools==4.39.4
freetype-py==2.3.0
frozenlist==1.3.3
fsspec==2023.5.0
ftfy==6.1.1
future==0.18.3
fvcore==0.1.5.post20221221
gdown==4.7.1
gfpgan==1.3.8
gitdb==4.0.10
GitPython==3.1.32
google-auth==2.18.1
google-auth-oauthlib==1.0.0
gradio==3.41.2
gradio_client==0.5.0
grpcio==1.54.2
h11==0.12.0
h2==4.1.0
hpack==4.0.0
httpcore==0.15.0
httpx==0.24.1
huggingface-hub==0.14.1
hyperframe==6.0.1
idna==2.10
imageio==2.28.1
importlib-resources==6.0.1
inflection==0.5.1
iopath==0.1.9
Jinja2==3.1.2
jsonmerge==1.8.0
jsonschema==4.17.3
kiwisolver==1.4.4
kornia==0.6.7
langcodes==3.3.0
lark==1.1.2
lazy_loader==0.2
lightning-utilities==0.8.0
linkify-it-py==2.0.2
llvmlite==0.40.0
lmdb==1.4.1
lpips==0.1.4
lxml==4.9.3
Markdown==3.4.3
markdown-it-py==2.2.0
MarkupSafe==2.1.2
matplotlib==3.7.1
mdit-py-plugins==0.3.3
mdurl==0.1.2
mediapipe==0.10.5
mpmath==1.3.0
multidict==6.0.4
murmurhash==1.0.9
networkx==3.1
numba==0.57.0
numpy==1.23.5
nvidia-cublas-cu11==<IP_ADDRESS>
nvidia-cuda-nvrtc-cu11==11.8.89
nvidia-cuda-runtime-cu11==11.8.89
oauthlib==3.2.2
omegaconf==2.2.3
onnx==1.14.1
onnx-graphsurgeon==0.3.27
open-clip-torch==2.20.0
opencv-contrib-python==<IP_ADDRESS>
opencv-python==<IP_ADDRESS>
orjson==3.8.12
packaging==23.1
pandas==2.0.1
pathy==0.10.1
piexif==1.1.3
Pillow==9.5.0
polygraphy==0.49.0
portalocker==2.7.0
preshed==3.0.8
protobuf==3.20.2
psutil==5.9.5
py-cpuinfo==9.0.0
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycairo==1.23.0
pycparser==2.21
pydantic==1.10.7
pydub==0.25.1
Pygments==2.15.1
pyparsing==3.0.9
pyrsistent==0.19.3
PySocks==1.7.1
python-dateutil==2.8.2
python-multipart==0.0.6
pytorch-lightning==1.9.4
pytz==2023.3
PyWavelets==1.4.1
pywin32==306
PyYAML==6.0
realesrgan==0.3.0
regex==2023.5.5
reportlab==4.0.0
requests==2.31.0
requests-oauthlib==1.3.1
resize-right==0.0.2
rich==13.6.0
rlPyCairo==0.2.0
rsa==4.9
safetensors==0.3.1
scikit-image==0.21.0
scipy==1.10.1
seaborn==0.13.0
semantic-version==2.10.0
Send2Trash==1.8.0
sentencepiece==0.1.99
six==1.16.0
smart-open==6.3.0
smmap==5.0.0
sniffio==1.3.0
socksio==1.0.0
sounddevice==0.4.6
soupsieve==2.4.1
spacy==3.5.3
spacy-legacy==3.0.12
spacy-loggers==1.0.4
srsly==2.4.6
starlette==0.26.1
svglib==1.5.1
sympy==1.12
tabulate==0.9.0
tb-nightly==2.14.0a20230520
tensorboard-data-server==0.7.0
tensorrt==9.0.1.post11.dev4
tensorrt-bindings==9.0.1.post11.dev4
tensorrt-libs==9.0.1.post11.dev4
termcolor==2.3.0
thinc==8.1.10
thop==0.1.1.post2209072238
tifffile==2023.4.12
timm==0.9.2
tinycss2==1.2.1
tokenizers==0.13.3
tomesd==0.1.3
tomli==2.0.1
toolz==0.12.0
torch==2.0.1+cu118
torchdiffeq==0.2.3
torchmetrics==0.11.4
torchsde==0.2.5
torchvision==0.15.2+cu118
tqdm==4.65.0
trampoline==0.1.2
transformers==4.30.2
typer==0.7.0
typing_extensions==4.5.0
tzdata==2023.3
uc-micro-py==1.0.2
ultralytics==8.0.195
urllib3==1.26.15
uvicorn==0.22.0
wasabi==1.1.1
wcwidth==0.2.6
webencodings==0.5.1
websockets==11.0.3
Werkzeug==2.3.4
xformers==0.0.20
yacs==0.1.8
yapf==0.33.0
yarl==1.9.2

What is that about "-vidia-cudnn-cu11" being an invalid distribution?

From my understanding, when pip is installing a package, it removes the first letter of the package name. When the installation completes, this extra file is removed. When the install crashes, the extra file isn't removed and is left as a fake package that is ignored. It's fine to leave it, but you can also navigate to venv\Lib\site-packages to remove it.

with python -m pip uninstall -y nvidia-cudnn-cu11 I also don't get more the error on startup. But I still get
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

I mean this error seems like a different one than the cudnn pop ups:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0!

So it appears like I no longer have nvidia-cudnn-cu11 installed, per the pip freeze... Is that what we want?

Popups errors solved with python -m pip uninstall -y nvidia-cudnn-cu11 but still get
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
when trying to generate images

since i struggle with this error myself and tried various things. this is what i found out by now:
delete and rebuild venv - no change
deactivate all extensions and reinstall tensorrt extension - no change
deactivate all  COMMANDLINE_ARGS - Error is gone and it builded the TensorRT engine
This were my COMMANDLINE_ARGS before = --api --xformers --medvram-sdxl --opt-sdp-attention --medvram --disable-safe-unpickle
so i guess that one of them was causing the problem, i will get them now one by one back and check which one caused the error.

I can still generate images without errors. It is only when I try to export the default engine that I get the erorr about expecting all tensors to be on the same device...

So it appears like I no longer have nvidia-cudnn-cu11 installed, per the pip freeze... Is that what we want?

Yes

my commandline args are: --autolaunch --update-check --xformers --no-half-vae --medvram --api

the problem is medvram. Solved!

Same errors as OP.

For my part, the python -m pip uninstall -y nvidia-cudnn-cu11 didn't seem to work as the extension was "not installed".
So instead I went to venv\Lib\site-packages and I removed the cudnn.dist-info & the cudd folder in the nvidia folder. It seems to be working fine. At least, I can start without errors, and I can start generating the engine. It seems to be generating without issues now.

It might be that VRAM limitations fall back to CPU tensors and therefore are not working with the Torch ONNX export.

I removed xformers arg and when exporting default engine still get error: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Will try removing medvram arg...

Nothing under SD Unet after generating engine completed?

For my part, the python -m pip uninstall -y nvidia-cudnn-cu11 didn't seem to work as the extension was "not installed".
So instead I went to venv\Lib\site-packages and I removed the cudnn.dist-info & the cudnn folder in the nvidia folder. It seems to be working fine. At least, I can start without errors, and I can start generating the engine. It seems to be generating without issues now.

Have you exported the engine and enabled it? See here:  https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT#how-to-use

but after exporting should I use medvram again?

I'm the only one when generate images with already selected engine gets this error?

I'm the only one when generate images with already selected engine gets this error?

I'm getting the same errors. Tried deleting venv folder, extensions folder, getting rid of any arguements in the web batch file. now the tab for tensorrt doesnt even appear

Following the steps in the discussion able to build an engine but it errors out at the saving step

For my part, the python -m pip uninstall -y nvidia-cudnn-cu11 didn't seem to work as the extension was "not installed".
So instead I went to venv\Lib\site-packages and I removed the cudnn.dist-info & the cudnn folder in the nvidia folder. It seems to be working fine. At least, I can start without errors, and I can start generating the engine. It seems to be generating without issues now.

Have you exported the engine and enabled it? See here: https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT#how-to-use

Yes, I was currently following the instructions actually, to see if "seems to be working" could be converted into "is working". I can confirm it works flawlessly! I was able to export an engine and confirm that it increases my generation speed by ~40%!
In summary, for me the issue was that during the installation, somehow it didn't uninstall cudnn, and cudnn was taking priority over the dlls in tensorrt. After removing cudnn manually, my problem was solved.

Ok, so I removed medvram, and that did seem to be causing the problem with the tensors being on the same device.  We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  known = {axis for axis in composite_axis if axis_name2known_length[axis] != _unknown_axis_length}
D:\repos\stable-diffusion-webui\venv\lib\site-packages\einops\einops.py:315: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  unknown = {axis for axis in composite_axis if axis_name2known_length[axis] == _unknown_axis_length}
D:\repos\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py:158: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1] == self.channels
D:\repos\stable-diffusion-webui\modules\sd_hijack_unet.py:26: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if a.shape[-2:] != b.shape[-2:]:
D:\repos\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py:109: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1] == self.channels
ERROR:asyncio:Exception in callback H11Protocol.timeout_keep_alive_handler()
handle: <TimerHandle when=7159.015 H11Protocol.timeout_keep_alive_handler()>
Traceback (most recent call last):
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 249, in _fire_event_triggered_transitions
    new_state = EVENT_TRIGGERED_TRANSITIONS[role][state][event_type]
KeyError: <class 'h11._events.ConnectionClosed'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 383, in timeout_keep_alive_handler
    self.conn.send(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 493, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 242, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 238, in process_event
    self._fire_event_triggered_transitions(role, event_type)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 251, in _fire_event_triggered_transitions
    raise LocalProtocolError(
h11._util.LocalProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE

I thought it might be the model I had selected, so I switched to the SD 1.5 model, and then got this error:
Loading model v1-5-pruned-emaonly.safetensors [6ce0161689] (2 out of 3)
Loading weights [6ce0161689] from D:\repos\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
Creating model from config: D:\repos\stable-diffusion-webui\configs\v1-inference.yaml
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[E] ONNX-Runtime is not installed, so constant folding may be suboptimal or not work at all.
    Consider installing ONNX-Runtime: D:\repos\stable-diffusion-webui\venv\Scripts\python.exe -m pip install onnxruntime
[I] Folding Constants | Pass 1
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[W] Falling back to `onnx.shape_inference` because `onnxruntime.tools.symbolic_shape_infer` either could not be loaded or did not run successfully.
    Note that using ONNX-Runtime for shape inference may be faster and require less memory.
    Consider installing ONNX-Runtime or setting POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to do so automatically.
[W] Attempting to run shape inference on a large model (1641.0 MiB). This may require a large amount of memory.
    If memory consumption becomes too high, the process may be killed. You may want to try disabling shape inference in that case.
[I]     Total Nodes | Original:  8992, After Folding:  6216 |  2776 Nodes Folded
[I] Folding Constants | Pass 2
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was:
No module named 'onnxruntime'
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[W] Attempting to run shape inference on a large model (1642.0 MiB). This may require a large amount of memory.
    If memory consumption becomes too high, the process may be killed. You may want to try disabling shape inference in that case.
[I]     Total Nodes | Original:  6216, After Folding:  6152 |    64 Nodes Folded
[I] Folding Constants | Pass 3
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was:
No module named 'onnxruntime'
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[I]     Total Nodes | Original:  6152, After Folding:  6152 |     0 Nodes Folded
*** API error: POST: http://<IP_ADDRESS>:7860/api/predict {'error': 'LocalProtocolError', 'detail': '', 'body': '', 'errors': "Can't send data when our state is ERROR"}
    Traceback (most recent call last):
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
        await self.app(scope, receive, _send)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 109, in __call__
        await response(scope, receive, send)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 270, in __call__
        async with anyio.create_task_group() as task_group:
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 662, in __aexit__
        raise exceptions[0]
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 273, in wrap
        await func()
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 134, in stream_response
        return await super().stream_response(send)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 255, in stream_response
        await send(
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 159, in _send
        await send(message)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 512, in send
        output = self.conn.send(event)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
        data_list = self.send_with_data_passthrough(event)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 483, in send_with_data_passthrough
        raise LocalProtocolError("Can't send data when our state is ERROR")
    h11._util.LocalProtocolError: Can't send data when our state is ERROR

---
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 428, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 109, in __call__
    await response(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 270, in __call__
    async with anyio.create_task_group() as task_group:
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 273, in wrap
    await func()
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 134, in stream_response
    return await super().stream_response(send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 255, in stream_response
    await send(
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 159, in _send
    await send(message)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 512, in send
    output = self.conn.send(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 483, in send_with_data_passthrough
    raise LocalProtocolError("Can't send data when our state is ERROR")
h11._util.LocalProtocolError: Can't send data when our state is ERROR
'AsyncRequest' object has no attribute '_json_response_data'
Applying attention optimization: sdp-no-mem... done.
Model loaded in 112.6s (load weights from disk: 15.3s, create model: 0.5s, apply weights to model: 96.5s).

I tried stopping and restarting the webui server. I made sure I am on the SD 1.5 model. If I try to export the default engine I get the same full error that I did above, ending in 'AsyncRequest' object has no attribute '_json_response_data'
I don't think it was the model I was on, as it is giving the same error with SD 1.5.

So now I can export model (with some out of memory errors on 12GB), but when I try to use I get again this error :
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm).
Thi error I got exporting with medvram but now I don't have it on startup

I thought I might try uninstalling cudnn manually as @Altrue did, but when I go to venv\Lib\site-packages I don't see folders for cudnn.dist-info & cudnn folders in the nvidia folder. Here is what I have in the nvidia folder:

It seems my error now might be this: No module named 'onnxruntime'

Should I try manually installing onnxruntime in the venv per python.exe -m pip install onnxruntime?

onnxruntime-gpu afaik, but it should be added in install.py if its the case

I already have onnx and onnx-graphsurgeon installed. Do I need onnxruntime or onnxruntime-gpu?

After all that, I just restarted the webui, and tried exporting the default engine again, and now it seems to be exporting... I did not install onnxruntime or onnxruntime-gpu...
It still output this error to the console, even though it continued with exporting this time.
ERROR:asyncio:Exception in callback H11Protocol.timeout_keep_alive_handler()
handle: <TimerHandle when=8572.921 H11Protocol.timeout_keep_alive_handler()>
Traceback (most recent call last):
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 249, in _fire_event_triggered_transitions
    new_state = EVENT_TRIGGERED_TRANSITIONS[role][state][event_type]
KeyError: <class 'h11._events.ConnectionClosed'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 383, in timeout_keep_alive_handler
    self.conn.send(event)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 493, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 242, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 238, in process_event
    self._fire_event_triggered_transitions(role, event_type)
  File "d:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 251, in _fire_event_triggered_transitions
    raise LocalProtocolError(
h11._util.LocalProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE

I'm the only one when generate images with already selected engine gets this error?

I'm getting the same errors. Tried deleting venv folder, extensions folder, getting rid of any arguements in the web batch file. now the tab for tensorrt doesnt even appear

SAME. tab disappeared, no matter what i do reinstall, upgrade requirement. nothing

I was having the "procedure entry point" error popups as well but deleting the cudnn folder in venv\Lib\site-packages\nvidia  fixed it. I'm able to generate images now.

I'm the only one when generate images with already selected engine gets this error?

I'm getting the same errors. Tried deleting venv folder, extensions folder, getting rid of any arguements in the web batch file. now the tab for tensorrt doesnt even appear

Same issue here. Removed TensorRT extension, deleted venv folder. Started WebUI, venv reinstalled, reinstalled TensorRT extension, now it doesn't appear in UI anymore. What's the procedure to fully uninstall/reinstall TensortRT without deleting the A1111 WebUI completely?
Error:
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: onnx-graphsurgeon in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (0.3.27)
Requirement already satisfied: numpy in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx-graphsurgeon) (1.23.5)
Requirement already satisfied: onnx in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx-graphsurgeon) (1.14.1)
Requirement already satisfied: protobuf>=3.20.2 in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (3.20.2)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (4.8.0)
GS is not installed! Installing...
Installing protobuf
Installing onnx-graphsurgeon
UI Config not initialized
Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
*** Error loading script: trt.py
    Traceback (most recent call last):
      File "D:\Scratch\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\Scratch\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in <module>
        import ui_trt
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in <module>
        from exporter import export_onnx, export_trt
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in <module>
        from utilities import Engine
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in <module>
        import tensorrt as trt
      File "D:\Scratch\stable-diffusion-webui\venv\lib\site-packages\tensorrt\__init__.py", line 18, in <module>
        from tensorrt_bindings import *
    ModuleNotFoundError: No module named 'tensorrt_bindings'

i also foudn that if i remove the file i generated, then the extention disappear from the UI.
restoring those deleted files will make the extention appear again... wtf lol

I'm the only one when generate images with already selected engine gets this error?

I'm getting the same errors. Tried deleting venv folder, extensions folder, getting rid of any arguements in the web batch file. now the tab for tensorrt doesnt even appear

Same issue here. Removed TensorRT extension, deleted venv folder. Started WebUI, venv reinstalled, reinstalled TensorRT extension, now it doesn't appear in UI anymore. What's the procedure to fully uninstall/reinstall TensortRT without deleting the A1111 WebUI completely?
Error:
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: onnx-graphsurgeon in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (0.3.27)
Requirement already satisfied: numpy in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx-graphsurgeon) (1.23.5)
Requirement already satisfied: onnx in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx-graphsurgeon) (1.14.1)
Requirement already satisfied: protobuf>=3.20.2 in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (3.20.2)
Requirement already satisfied: typing-extensions>=<IP_ADDRESS> in d:\scratch\stable-diffusion-webui\venv\lib\site-packages (from onnx->onnx-graphsurgeon) (4.8.0)
GS is not installed! Installing...
Installing protobuf
Installing onnx-graphsurgeon
UI Config not initialized
Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
*** Error loading script: trt.py
    Traceback (most recent call last):
      File "D:\Scratch\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\Scratch\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in <module>
        import ui_trt
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in <module>
        from exporter import export_onnx, export_trt
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in <module>
        from utilities import Engine
      File "D:\Scratch\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in <module>
        import tensorrt as trt
      File "D:\Scratch\stable-diffusion-webui\venv\lib\site-packages\tensorrt\__init__.py", line 18, in <module>
        from tensorrt_bindings import *
    ModuleNotFoundError: No module named 'tensorrt_bindings'

Deleting the stable-diffusion-webui-tensorrt extension in the extensions folder should be all you need to do.

So it finally exported the default engine for SD 1.5, and I see it in models\Unet-trt, but now SD Unet doesn't show it in the dropdown menu. It only shows automatic or none... Refreshing that list does nothing. I have tried restarting the UI. I will now restart webui server to see if that helps.

i also foudn that if i remove the models i converted then the extention disappear from the UI. restoring those deleted files will make the extention appear again... wtf lol

You removed the generated trt model from the Unet-trt folder?  If so you need to also remove the reference to that model in the model.json folder. Or if you only have one model generated, just delete the model file and the .json file both.

So it finally exported the default engine for SD 1.5, and I see it in models\Unet-trt, but now SD Unet doesn't show it in the dropdown menu. It only shows automatic or none... Refreshing that list does nothing. I have tried restarting the UI. I will now restart webui server to see if that helps.

Do you have a model file in the models/Unet-trt folder? If so, it might be that the model.json file didn't get properly written. I had that happen at one point.
If you open the model.json file and it is empty than that is the issue.

Yes, I have a model file in the models/Unet-trt folder, but the model.json file is empty, just {}. How do I regenerate the model.json file? Do I have to export the default engine again?

Yes, I have a model file in the models/Unet-trt folder, but the model.json file is empty, just {}. How do I regenerate the model.json file? Do I have to export the default engine again?

Normally you'd need to rebuild the profile, I don't think there's a clear way to regenerate the model.json file.
You can try deleting everything in the model.json file and pasting in this
{     "cc89": {         "v1-5-pruned-emaonly": [             {                 "filepath": "v1-5-pruned-emaonly_d7049739_cc89_sample=1x4x64x64+2x4x64x64+8x4x96x96-timesteps=1+2+8-encoder_hidden_states=1x77x768+2x77x768+8x154x768.trt",                 "config": {                     "profile": {                         "sample": [                             [                                 1,                                 4,                                 64,                                 64                             ],                             [                                 2,                                 4,                                 64,                                 64                             ],                             [                                 8,                                 4,                                 96,                                 96                             ]                         ],                         "timesteps": [                             [                                 1                             ],                             [                                 2                             ],                             [                                 8                             ]                         ],                         "encoder_hidden_states": [                             [                                 1,                                 77,                                 768                             ],                             [                                 2,                                 77,                                 768                             ],                             [                                 8,                                 154,                                 768                             ]                         ]                     },                     "static_shapes": false,                     "fp32": false,                     "inpaint": false,                     "refit": true,                     "lora": false,                     "vram": 0,                     "unet_hidden_dim": 4                 }             }         ]     } }

Hmm, I deleted everything in the model.json file, and pasted in that, and still nothing in the dropdown. I restarted UI, still nothing. Will now restart webui server.

what is this now

what is this now

Looks like you specified settings in your image generation that aren't supported by an engine you have built. What are you specifying for your image generation parameters?

I have two models in the folder, but the .json is empty as well.

what is this now

Looks like you specified settings in your image generation that aren't supported by an engine you have built. What are you specifying for your image generation parameters?

this sounds more like that the model i converted, wich was a custom size (512x768 batch 1) wasnt working as expected. So now i'm redoing it using default settings. i already used TensorRT a lot in the past, and i remeber that there were some trial and error in converting models.. maybe this is the case here.

Restarting the webui server, and got this error:
*** Error loading script: trt.py
    Traceback (most recent call last):
      File "D:\repos\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\repos\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in <module>
        import ui_trt
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 16, in <module>
        from model_manager import modelmanager, cc_major, TRT_MODEL_DIR
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\model_manager.py", line 219, in <module>
        modelmanager = ModelManager()
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\model_manager.py", line 35, in __init__
        self.update()
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\model_manager.py", line 67, in update
        for base_model, models in base_models.items():
    RuntimeError: dictionary changed size during iteration

Could be because we tried pasting in the values to model.json?

In my Nvidia folder there is a blank _init.py file

Restarting the webui server, and got this error:
*** Error loading script: trt.py
    Traceback (most recent call last):
      File "D:\repos\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts
        script_module = script_loading.load_module(scriptfile.path)
      File "D:\repos\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module
        module_spec.loader.exec_module(module)
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in <module>
        import ui_trt
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 16, in <module>
        from model_manager import modelmanager, cc_major, TRT_MODEL_DIR
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\model_manager.py", line 219, in <module>
        modelmanager = ModelManager()
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\model_manager.py", line 35, in __init__
        self.update()
      File "D:\repos\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\model_manager.py", line 67, in update
        for base_model, models in base_models.items():
    RuntimeError: dictionary changed size during iteration

Could be because we tried pasting in the values to model.json? Looks like it deleted everything in the model.json file when I restarted the webui server...

Could be, I wasn't really sure if that would work or not.

Ok, I'm going to delete everything in the models/Unet-trt and models/Unet-onnx folders, reboot webui server, and try exporting default engine again. Hopefully this time the model.json file is created correctly.

Exporting now... It is still reporting that ONNX-Runtime is not installed, and it looks like Polygraphy uses it, and it is giving a bunch of errors because of that, but the export continues. Do I want to manually install that module?

Exporting now... It is still reporting that ONNX-Runtime is not installed, and it looks like Polygraphy uses it, and it is giving a bunch of errors because of that, but the export continues. Do I want to manually install that module?

No it's fine. That error is "normal".

Export finished, and properly added to SD Unet dropdown. I can now use the TensorRT model, and it is indeed much faster on my 3060.
So it seems my errors were mostly caused by --medvram (which I still have not re-enabled), possibly conflicting with another extension (Photoshop plugin) using the --api. I had to manually uninstall cudnn via python -m pip uninstall -y nvidia-cudnn-cu11.
After all that, it seems to be working. I will try to reenable --medvram and --api commandline args and see if it still works.

with 12 gb I get W] Requested amount of GPU memory<PHONE_NUMBER>0 bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
[W] UNSUPPORTED_STATESkipping tactic 0 due to insufficient memory on requested size of<PHONE_NUMBER>0 detected for tactic 0x0000000000000000. during exporting.

with 12 gb I get W] Requested amount of GPU memory<PHONE_NUMBER>0 bytes) could not be allocated. There may not be enough free memory for allocation to succeed. [W] UNSUPPORTED_STATESkipping tactic 0 due to insufficient memory on requested size of<PHONE_NUMBER>0 detected for tactic 0x0000000000000000. during exporting.

I also got that error on my 3060 with 12gb vram, but the export continued, and seems to work.

with 12 gb I get W] Requested amount of GPU memory<PHONE_NUMBER>0 bytes) could not be allocated. There may not be enough free memory for allocation to succeed. [W] UNSUPPORTED_STATESkipping tactic 0 due to insufficient memory on requested size of<PHONE_NUMBER>0 detected for tactic 0x0000000000000000. during exporting.

If the engine generation continues after getting that message, its fine. The engine build process will try different "tactics" to build the TensorRT engine and some use more memory than others, if a given tactic can't be used the build should continue with a different tactic.  If the build fails to build after getting a memory error, then the specified engine is probably running out of VRAM.

Tried this time with sd 1.5 instead of sdXL and now its working. sd 1.5 from 3-4it/s to 10/11it/s batch size 1

Ok, I re-enabled --medvram and --api, and the TensorRT model I built before seems to still work. I do get this message when I use it:
[W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
Should I enable CUDA lazy loading? How would I do that?
With --medvram re-enabled, I cannot export any new engines though. I get this error as I did before:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
So something with the --medvram argument is conflicting here. I'll have to remove that commandline arg whenever I want to export new engines until fixed.

CUDA Lazy loading can help with memory usage, it's enabled using an environment variable:  SET CUDA_MODULE_LOADING=LAZY

CUDA Lazy loading can help with memory usage, it's enabled using an environment variable: SET CUDA_MODULE_LOADING=LAZY

I'm not sure exactly how much it helps in this case, but you can give it a shot.

Generating images with unet gives me this error  : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Any idea?

If I change the vae before generating image it seems to work.
BTW it's not faster :(

Generating images with unet gives me this error : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Any idea?

Are you trying to use SDXL?

As far as performance goes on my 3060, using SD 1.5 at 512x512 resolution, for a batch size of 1, I'm getting speed increases from about 3.9 it/s to 9 it/s, an increase of 2.3x! That's impressive.
But if I increase the batch size, something strange happens. With TRT set to 'none', the generation takes way longer now. A batch size of 4, without TRT (SD Unet set to 'none'), takes a minute and 40 seconds (1:40), whereas with a batch size of 1, generating 4 images takes only 25 seconds when TRT is off. What is going on here? Something seems wrong. Is the TRT UNET not deactivated correctly when TRT is set to 'none' for batch sizes greater than 1? With TRT active, a batch size of 4 is very fast, generating 4 images in only 8 seconds.

As far as performance goes on my 3060, using SD 1.5 at 512x512 resolution, for a batch size of 1, I'm getting speed increases from about 3.9 it/s to 9 it/s, an increase of 2.3x! That's impressive.
But if I increase the batch size, something strange happens. With TRT set to 'none', the generation takes way longer now. A batch size of 4, without TRT (SD Unet set to 'none'), takes a minute and 40 seconds (1:40), whereas with a batch size of 1, generating 4 images takes only 25 seconds when TRT is off. What is going on here? Something seems wrong. Is the TRT UNET not deactivated correctly when TRT is set to 'none' for batch sizes greater than 1? With TRT active, a batch size of 4 is very fast, generating 4 images in only 8 seconds.

You're likely hitting the memory limits of you GPU. Your --medvram and --xformers command line arguments previously were directed at allowing you to use less VRAM for image generation.  So without those active, you're probably hitting VRAM limitations that are slowing you down at larger batch sizes.
TensorRT is more memory efficient than vanilla PyTorch so it can generally handle larger batch sizes.  If you want to run larger batch sizes without using TensorRT you'll likely need to re-enable your previous memory optimizations.

Generating images with unet gives me this error : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Any idea?

Are you trying to use SDXL?

yes

Generating images with unet gives me this error : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Any idea?

Are you trying to use SDXL?

yes

I have the same issue with SDXL and TRT, works fine without

These performance numbers were with --medvram and --xformers enabled (although I'm using sdp-no-mem at the moment, not xformers).
If I disable the TensorRT extension, then my speeds are about 5 it/s with batch size 1 (4 images in 19 seconds), and 2.2 it/s with batch size of 4 (11 seconds). So even with a batch size of 1, the speeds are slower with the TensorRT extension enabled (5 it/s vs 3.9 it/s). So it seems there is a problem here...

Generating images with unet gives me this error : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Any idea?

Are you trying to use SDXL?

yes

The extension supports SDXL, but it relies on functionality that hasn't been implemented in the release branch. If you need to work with SDXL you'll need to use a Automatic1111 build from the Dev branch at the moment. Note that the Dev branch is not intended for production work and may break other things that you are currently using.

Generating images with unet gives me this error : RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Any idea?

Are you trying to use SDXL?

yes

I have the same issue with SDXL and TRT, works fine without

If i change the vae few times (enabled on the quick menù) it seems starting. Can't understand....

This is an error because it broke when downloading.
Copy from here Stable-Diffusion\venv\Lib\site-packages\torch\lib to here Stable-Diffusion\venv\Lib\site-packages\nvidia\cudnn\bin

cudnn

I don't have cudnn folder.

the TensorRT tab is still isn't appearing for me. After I uninstalled it the first time it no longer appears. Anyone facing the same issue?

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py     Traceback (most recent call last):       File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts         script_module = script_loading.load_module(scriptfile.path)       File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module         module_spec.loader.exec_module(module)       File "<frozen importlib._bootstrap_external>", line 883, in exec_module       File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed       File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in <module>         import ui_trt       File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in <module>         from exporter import export_onnx, export_trt       File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in <module>         from utilities import Engine       File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in <module>         import tensorrt as trt       File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt\__init__.py", line 18, in <module>         from tensorrt_bindings import *     ModuleNotFoundError: No module named 'tensorrt_bindings'

I got mine to export a model thing, but I get this every time I try to use it...
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
I saw up there someone else had the same problem.

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions?  If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

I got mine to export a model thing, but I get this every time I try to use it... RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) I saw up there someone else had the same problem.

If you are trying to use SDXL then this is currently expected.  The extension supports SDXL, but it relies on functionality that hasn't been implemented in the release branch of Automatic1111. If you need to work with SDXL you'll need to use a Automatic1111 build from the Dev branch at the moment. Note that the Dev branch is not intended for production work and may break other things that you are currently using.

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

i been doing that but its still giving me the nomodule found error and the tensorrt tab is still not appearing for me

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

That doesn't matter unfortunately. Many people have deleted the stable-diffusion-webui-tesnsorrt folder in extensions, and the tab refuses to show up again if installing from URL with this repository. We're all getting the same error log as posted here by users with the last line being: No module named 'tensorrt_bindings'
Something seems to be broken if uninstalling and trying to reinstall.

I got mine to export a model thing, but I get this every time I try to use it... RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) I saw up there someone else had the same problem.

If you are trying to use SDXL then this is currently expected. The extension supports SDXL, but it relies on functionality that hasn't been implemented in the release branch of Automatic1111. If you need to work with SDXL you'll need to use a Automatic1111 build from the Dev branch at the moment. Note that the Dev branch is not intended for production work and may break other things that you are currently using.

Thank you, didn't realize it wasn't working for SDXL yet.  Much appreciated.  Saves a headache.

Im installing from the guide that nvidia posted, and i get that error, but that error appears after i relaunched webui-user.bat i get an error that protobuf cannot be installed, when i relaunch it gets installed.

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

That doesn't matter unfortunately. Many people have deleted the stable-diffusion-webui-tesnsorrt folder in extensions, and the tab refuses to show up again if installing from URL with this repository. We're all getting the same error log as posted here by users with the last line being: No module named 'tensorrt_bindings'
Something seems to be broken if uninstalling and trying to reinstall.

im also getting this error, this extension is such a buggy mess

Hmm.. I re-enabled the TensorRT extension, and I can't replicate the slowdown with the batch size 4 that I had before. Not sure what was causing that.

the tensorrt bindings error doesnt have anything to do with where youre installing the extension from, ive installed everything manually and it still gives me that error

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

i been doing that but its still giving me the nomodule found error and the tensorrt tab is still not appearing for me

Can  you post a bit more of the error output that show more of the error message?

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

i been doing that but its still giving me the nomodule found error and the tensorrt tab is still not appearing for me

Can you post a bit more of the error output that show more of the error message?

That is the error message, the first message i get is after i install the extension from url. is this.
*** Error running install.py for extension E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe" "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stdout: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
*** Collecting tensorrt==9.0.1.post11.dev4
***   Using cached tensorrt-9.0.1.post11.dev4-py2.py3-none-any.whl
*** Installing collected packages: tensorrt
*** Successfully installed tensorrt-9.0.1.post11.dev4
*** Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
*** Collecting polygraphy
***   Using cached polygraphy-0.49.0-py2.py3-none-any.whl (327 kB)
*** Installing collected packages: polygraphy
*** Successfully installed polygraphy-0.49.0
*** Collecting protobuf==3.20.2
***   Using cached protobuf-3.20.2-cp310-cp310-win_amd64.whl (904 kB)
*** Installing collected packages: protobuf
***   Attempting uninstall: protobuf
***     Found existing installation: protobuf 3.20.0
***     Uninstalling protobuf-3.20.0:
***       Successfully uninstalled protobuf-3.20.0
*** TensorRT is not installed! Installing...
*** Installing nvidia-cudnn-cu11
*** Installing tensorrt
*** removing nvidia-cudnn-cu11
*** Polygraphy is not installed! Installing...
*** Installing polygraphy
*** GS is not installed! Installing...
*** Installing protobuf

*** stderr: A matching Triton is not available, some optimizations will not be enabled.
*** Error caught was: No module named 'triton'
*** E:\AIsoftware\Stable Diffusion\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: pytorch_lightning.utilities.distributed.rank_zero_only has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from pytorch_lightning.utilities instead.
***   rank_zero_deprecation(

*** [notice] A new release of pip available: 22.3.1 -> 23.3
*** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip

*** [notice] A new release of pip available: 22.3.1 -> 23.3
*** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip
*** ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'E:\AIsoftware\Stable Diffusion\venv\Lib\site-packages\google\~rotobuf\internal\_api_implementation.cp310-win_amd64.pyd'
*** Check the permissions.

*** [notice] A new release of pip available: 22.3.1 -> 23.3
*** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip
*** Traceback (most recent call last):
***   File "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 30, in
***     install()
***   File "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 19, in install
***     launch.run_pip("install protobuf==3.20.2", "protobuf", live=True)
***   File "E:\AIsoftware\Stable Diffusion\modules\launch_utils.py", line 138, in run_pip
***     return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live)
***   File "E:\AIsoftware\Stable Diffusion\modules\launch_utils.py", line 115, in run
***     raise RuntimeError("\n".join(error_bits))
*** RuntimeError: Couldn't install protobuf.
*** Command: "E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe" -m pip install protobuf==3.20.2 --prefer-binary
*** Error code: 1

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

i been doing that but its still giving me the nomodule found error and the tensorrt tab is still not appearing for me

Can you post a bit more of the error output that show more of the error message?

That is the error message, the first message i get is after i install the extension from url. is this.
*** Error running install.py for extension E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT. *** Command: "E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe" "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py" *** Error code: 1 *** stdout: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com *** Collecting tensorrt==9.0.1.post11.dev4 *** Using cached tensorrt-9.0.1.post11.dev4-py2.py3-none-any.whl *** Installing collected packages: tensorrt *** Successfully installed tensorrt-9.0.1.post11.dev4 *** Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com *** Collecting polygraphy *** Using cached polygraphy-0.49.0-py2.py3-none-any.whl (327 kB) *** Installing collected packages: polygraphy *** Successfully installed polygraphy-0.49.0 *** Collecting protobuf==3.20.2 *** Using cached protobuf-3.20.2-cp310-cp310-win_amd64.whl (904 kB) *** Installing collected packages: protobuf *** Attempting uninstall: protobuf *** Found existing installation: protobuf 3.20.0 *** Uninstalling protobuf-3.20.0: *** Successfully uninstalled protobuf-3.20.0 *** TensorRT is not installed! Installing... *** Installing nvidia-cudnn-cu11 *** Installing tensorrt *** removing nvidia-cudnn-cu11 *** Polygraphy is not installed! Installing... *** Installing polygraphy *** GS is not installed! Installing... *** Installing protobuf
*** stderr: A matching Triton is not available, some optimizations will not be enabled. *** Error caught was: No module named 'triton' *** E:\AIsoftware\Stable Diffusion\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: pytorch_lightning.utilities.distributed.rank_zero_only has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from pytorch_lightning.utilities instead. *** rank_zero_deprecation(
*** [notice] A new release of pip available: 22.3.1 -> 23.3 *** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip
*** [notice] A new release of pip available: 22.3.1 -> 23.3 *** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip *** ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'E:\AIsoftware\Stable Diffusion\venv\Lib\site-packages\google~rotobuf\internal_api_implementation.cp310-win_amd64.pyd' *** Check the permissions.
*** [notice] A new release of pip available: 22.3.1 -> 23.3 *** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip *** Traceback (most recent call last): *** File "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 30, in *** install() *** File "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 19, in install *** launch.run_pip("install protobuf==3.20.2", "protobuf", live=True) *** File "E:\AIsoftware\Stable Diffusion\modules\launch_utils.py", line 138, in run_pip *** return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live) *** File "E:\AIsoftware\Stable Diffusion\modules\launch_utils.py", line 115, in run *** raise RuntimeError("\n".join(error_bits)) *** RuntimeError: Couldn't install protobuf. *** Command: "E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe" -m pip install protobuf==3.20.2 --prefer-binary *** Error code: 1

This is for the protobuf error that occurs on install correct?
Does protobuf get installed properly when the webui is reloaded?  I generally see it list the error when installing, but get properly installed when the webui is reloaded which is required to complete the extension installation.

Can anyone tell me whats wrong here? i tried to reinstall it multiple times and even tried to reinstall automatic1111 multiple times. I got it working for some odd reason on my old version, but i accidently deleted that.
Launching Web UI with arguments: --xformers *** Error loading script: trt.py Traceback (most recent call last): File "E:\AIsoftware\stable-diffusion-webui\modules\scripts.py", line 382, in load_scripts script_module = script_loading.load_module(scriptfile.path) File "E:\AIsoftware\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module module_spec.loader.exec_module(module) File "", line 883, in exec_module File "", line 241, in call_with_frames_removed File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in import ui_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in from exporter import export_onnx, export_trt File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in from utilities import Engine File "E:\AIsoftware\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in import tensorrt as trt File "E:\AIsoftware\stable-diffusion-webui\venv\lib\site-packages\tensorrt__init_.py", line 18, in from tensorrt_bindings import * ModuleNotFoundError: No module named 'tensorrt_bindings'

Are you installing using the "Available" tab within extensions? If so that hasn't yet been updated to point to this current version. You'll want to delete the stable-diffusion-webui-tesnsorrt folder found in the extensions folder. And the install using the Install from URL option.

i been doing that but its still giving me the nomodule found error and the tensorrt tab is still not appearing for me

Can you post a bit more of the error output that show more of the error message?

That is the error message, the first message i get is after i install the extension from url. is this.
*** Error running install.py for extension E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT. *** Command: "E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe" "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py" *** Error code: 1 *** stdout: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com *** Collecting tensorrt==9.0.1.post11.dev4 *** Using cached tensorrt-9.0.1.post11.dev4-py2.py3-none-any.whl *** Installing collected packages: tensorrt *** Successfully installed tensorrt-9.0.1.post11.dev4 *** Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com *** Collecting polygraphy *** Using cached polygraphy-0.49.0-py2.py3-none-any.whl (327 kB) *** Installing collected packages: polygraphy *** Successfully installed polygraphy-0.49.0 *** Collecting protobuf==3.20.2 *** Using cached protobuf-3.20.2-cp310-cp310-win_amd64.whl (904 kB) *** Installing collected packages: protobuf *** Attempting uninstall: protobuf *** Found existing installation: protobuf 3.20.0 *** Uninstalling protobuf-3.20.0: *** Successfully uninstalled protobuf-3.20.0 *** TensorRT is not installed! Installing... *** Installing nvidia-cudnn-cu11 *** Installing tensorrt *** removing nvidia-cudnn-cu11 *** Polygraphy is not installed! Installing... *** Installing polygraphy *** GS is not installed! Installing... *** Installing protobuf
*** stderr: A matching Triton is not available, some optimizations will not be enabled. *** Error caught was: No module named 'triton' *** E:\AIsoftware\Stable Diffusion\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: pytorch_lightning.utilities.distributed.rank_zero_only has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from pytorch_lightning.utilities instead. *** rank_zero_deprecation(
*** [notice] A new release of pip available: 22.3.1 -> 23.3 *** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip
*** [notice] A new release of pip available: 22.3.1 -> 23.3 *** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip *** ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'E:\AIsoftware\Stable Diffusion\venv\Lib\site-packages\google~rotobuf\internal_api_implementation.cp310-win_amd64.pyd' *** Check the permissions.
*** [notice] A new release of pip available: 22.3.1 -> 23.3 *** [notice] To update, run: E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe -m pip install --upgrade pip *** Traceback (most recent call last): *** File "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 30, in *** install() *** File "E:\AIsoftware\Stable Diffusion\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 19, in install *** launch.run_pip("install protobuf==3.20.2", "protobuf", live=True) *** File "E:\AIsoftware\Stable Diffusion\modules\launch_utils.py", line 138, in run_pip *** return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live) *** File "E:\AIsoftware\Stable Diffusion\modules\launch_utils.py", line 115, in run *** raise RuntimeError("\n".join(error_bits)) *** RuntimeError: Couldn't install protobuf. *** Command: "E:\AIsoftware\Stable Diffusion\venv\Scripts\python.exe" -m pip install protobuf==3.20.2 --prefer-binary *** Error code: 1

This is for the protobuf error that occurs on install correct?
Does protobuf get installed properly when the webui is reloaded? I generally see it list the error when installing, but get properly installed when the webui is reloaded which is required to complete the extension installation.

As far as i have seen so far, that is correct. protobuf gets installed after a reload. but then i get that first error i posted.

everything seems to work here finally but now there's a new weird error.
when using IMG2IMG but switching to another model that is not the converted one.

This is for the protobuf error that occurs on install correct?
Does protobuf get installed properly when the webui is reloaded? I generally see it list the error when installing, but get properly installed when the webui is reloaded which is required to complete the extension installation.

As far as i have seen so far, that is correct. protobuf gets installed after a reload. but then i get that first error i posted.

Can you try the following:
open a cmd window in the webui root directory (stable-diffusion-webui)
Issue the following:  venv\scripts\activate.bat
This should activate the venv virtual environment
issue the following command:
pip freeze
And post the results from pip freeze

This is for the protobuf error that occurs on install correct?
Does protobuf get installed properly when the webui is reloaded? I generally see it list the error when installing, but get properly installed when the webui is reloaded which is required to complete the extension installation.

As far as i have seen so far, that is correct. protobuf gets installed after a reload. but then i get that first error i posted.

Can you try the following: open a cmd window in the webui root directory (stable-diffusion-webui) Issue the following: venv\scripts\activate.bat
This should activate the venv virtual environment
issue the following command: pip freeze
And post the results from pip freeze

absl-py==1.4.0
accelerate==0.23.0
addict==2.4.0
aenum==3.1.15
aiofiles==23.2.1
aiohttp==3.8.4
aiosignal==1.3.1
albumentations==1.3.1
altair==5.1.1
annotated-types==0.5.0
antlr4-python3-runtime==4.9.3
anyio==3.7.1
asttokens==2.2.1
async-timeout==4.0.2
attrs==22.2.0
audioread==3.0.0
backcall==0.2.0
basicsr==1.4.2
bidict==0.22.1
blendmodes==2023
blobfile==2.0.1
boltons==23.0.0
cachetools==5.3.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==3.1.0
clean-fid==0.1.35
click==8.1.3
clip @ git+https://github.com/openai/CLIP.git@a9b1bf5920416aaeaec965c25dd9e8f98c864f16
colorama==0.4.6
comm==0.1.3
ConfigArgParse==1.5.3
contourpy==1.0.7
cycler==0.11.0
Cython==3.0.3
dash==2.9.2
dash-core-components==2.0.0
dash-html-components==2.0.0
dash-table==5.0.0
debugpy==1.6.6
decorator==5.1.1
deprecation==2.1.0
diffuser==0.0.1
diffusers==0.10.2
easydict==1.10
einops==0.6.1
exceptiongroup==1.1.3
executing==1.2.0
facexlib==0.3.0
fastapi==0.103.1
fastjsonschema==2.16.3
ffmpy==0.3.1
filelock==3.10.7
filterpy==1.4.5
fire==0.5.0
Flask==2.2.3
fonttools==4.39.3
frozenlist==1.3.3
fsspec==2023.9.1
ftfy==6.1.1
future==0.18.3
gfpgan==1.3.8
gitdb==4.0.10
GitPython==3.1.36
google-api-core==2.11.0
google-auth==2.17.3
google-auth-oauthlib==1.0.0
google-cloud-speech==2.19.0
google-cloud-texttospeech==2.14.1
googleapis-common-protos==1.59.0
gradio==3.41.2
gradio_client==0.5.0
grpcio==1.54.0
grpcio-status==1.54.0
h11==0.14.0
httpcore==0.18.0
httpx==0.25.0
huggingface-hub==0.17.1
humanize==4.6.0
idna==3.4
imageio==2.27.0
importlib-metadata==6.8.0
importlib-resources==6.0.1
inflect==7.0.0
inflection==0.5.1
insightface==0.7.3
ipykernel==6.22.0
ipython==8.12.0
ipywidgets==8.0.6
itsdangerous==2.1.2
jedi==0.18.2
Jinja2==3.1.2
joblib==1.3.2
jsonmerge==1.9.2
jsonschema==4.17.3
jupyter_client==8.1.0
jupyter_core==5.3.0
jupyterlab-widgets==3.0.7
keyboard==0.13.5
kiwisolver==1.4.4
kornia==0.7.0
lark==1.1.7
lazy_loader==0.2
librosa==0.10.1
lightning-utilities==0.9.0
llvmlite==0.40.1
lmdb==1.4.1
lxml==4.9.2
Markdown==3.4.4
MarkupSafe==2.1.2
matplotlib==3.7.1
matplotlib-inline==0.1.6
mpmath==1.3.0
msgpack==1.0.6rc1
multidict==6.0.4
mypy-extensions==1.0.0
nbformat==5.7.0
nest-asyncio==1.5.6
networkx==3.0
numba==0.57.1
numpy==1.24.2
oauthlib==3.2.2
omegaconf==2.3.0
onnx==1.14.1
open-clip-torch==2.20.0
open3d==0.17.0
openai==0.27.5
opencv-python==<IP_ADDRESS>
opencv-python-headless==<IP_ADDRESS>
orjson==3.9.7
packaging==23.0
pandas==2.1.0
parso==0.8.3
pickleshare==0.7.5
piexif==1.1.3
Pillow==10.0.1
platformdirs==3.10.0
playsound==1.3.0
plotly==5.14.0
pooch==1.7.0
prettytable==3.9.0
progressbar==2.5
prompt-toolkit==3.0.38
proto-plus==1.22.2
protobuf==3.20.3
psutil==5.9.4
pure-eval==0.2.2
pyasn1==0.5.0
pyasn1-modules==0.3.0
PyAudio==0.2.13
pycparser==2.21
pycryptodomex==3.17
pydantic==2.3.0
pydantic_core==2.6.3
pydub==0.25.1
pygame==2.4.0
Pygments==2.14.0
pyparsing==3.0.9
pyre-extensions==0.0.23
pyrsistent==0.19.3
PySoundFile==0.9.0.post1
python-dateutil==2.8.2
python-engineio==4.5.1
python-multipart==0.0.6
python-socketio==5.7.2
pytorch-lightning==2.0.9
pytz==2023.3.post1
PyWavelets==1.4.1
pywin32==306
PyYAML==6.0
pyzmq==25.0.2
qudida==0.0.4
realesrgan==0.3.0
regex==2023.3.23
requests==2.28.2
requests-oauthlib==1.3.1
resize-right==0.0.2
rotary-embedding-torch==0.3.0
rsa==4.9
safetensors==0.3.3
scikit-image==0.20.0
scikit-learn==1.3.0
scipy==1.10.1
semantic-version==2.10.0
sentencepiece==0.1.99
simpleaudio==1.0.4
six==1.16.0
smmap==5.0.0
sniffio==1.3.0
soundfile==0.12.1
soxr==0.3.6
stack-data==0.6.2
starlette==0.27.0
sympy==1.11.1
tb-nightly==2.15.0a20230915
tenacity==8.2.2
tensorboard-data-server==0.7.1
termcolor==2.2.0
threadpoolctl==3.2.0
tifffile==2023.3.21
timm==0.9.7
tokenizers==0.13.2
tomesd==0.1.3
tomli==2.0.1
toolz==0.12.0
torch==2.0.1+cu118
torchaudio==2.0.2
torchdiffeq==0.2.3
torchmetrics==1.1.2
torchsde==0.2.5
torchvision==0.15.2+cu118
tornado==6.2
TorToiSe==2.8.0
tqdm==4.65.0
traitlets==5.9.0
trampoline==0.1.2
transformers==4.30.2
typing-inspect==0.8.0
typing_extensions==4.7.1
tzdata==2023.3
Unidecode==1.3.6
urllib3==1.26.15
uvicorn==0.23.2
wcwidth==0.2.6
webrtcvad==2.0.10
websocket-client==1.6.1
websockets==11.0.3
Werkzeug==2.2.3
widgetsnbextension==4.0.7
yapf==0.40.1
yarl==1.9.2
zipp==3.15.0

all command line args off, no errors on installation, TensorRT tab is showing up.  when I try to generate default model from TensorRT WebUI just crashes.

This is for the protobuf error that occurs on install correct?
Does protobuf get installed properly when the webui is reloaded? I generally see it list the error when installing, but get properly installed when the webui is reloaded which is required to complete the extension installation.

As far as i have seen so far, that is correct. protobuf gets installed after a reload. but then i get that first error i posted.

Can you try the following: open a cmd window in the webui root directory (stable-diffusion-webui) Issue the following: venv\scripts\activate.bat
This should activate the venv virtual environment
issue the following command: pip freeze
And post the results from pip freeze

absl-py==1.4.0 accelerate==0.23.0 addict==2.4.0 aenum==3.1.15 aiofiles==23.2.1 aiohttp==3.8.4 aiosignal==1.3.1

Can you try the following:
open a cmd window in the webui root directory (stable-diffusion-webui)
Issue the following: venv\scripts\activate.bat
This should activate the venv virtual environment
issue the following commands:
pip install nvidia-cudnn-cu11==<IP_ADDRESS>
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4
pip uninstall -y nvidia-cudnn-cu11

absl-py==1.4.0 accelerate==0.23.0 addict==2.4.0 aenum==3.1.15 aiofiles==23.2.1 aiohttp==3.8.4 aiosignal==1.3.1

Can you try the following: open a cmd window in the webui root directory (stable-diffusion-webui) Issue the following: venv\scripts\activate.bat
This should activate the venv virtual environment (Make sure you see (venv) listed at the beginning of the command line)
issue the following commands:
pip install nvidia-cudnn-cu11==<IP_ADDRESS>
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4
pip uninstall -y nvidia-cudnn-cu11

Same error as before.

alright, i found out whats was causing the tensorrt bindings error
one of my extensions kept replacing protobuf (also "rotobuf") with the latest version on startup, after disabling all of them it seems to work properly

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

absl-py==1.4.0 accelerate==0.23.0 addict==2.4.0 aenum==3.1.15 aiofiles==23.2.1 aiohttp==3.8.4 aiosignal==1.3.1

Can you try the following: open a cmd window in the webui root directory (stable-diffusion-webui) Issue the following: venv\scripts\activate.bat
This should activate the venv virtual environment (Make sure you see (venv) listed at the beginning of the command line)
issue the following commands:
pip install nvidia-cudnn-cu11==<IP_ADDRESS>
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4
pip uninstall -y nvidia-cudnn-cu11

Same error as before.

Ah that might be my fault, I'm using the Automatic1111 self installer version which doesn't use a virtual environment so I was going from memory I think when you are in a venv you can't call pip directly.
So once the venv is activated you would need to issue.
python -m pip install nvidia-cudnn-cu11==<IP_ADDRESS> python -m pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 python -m pip uninstall -y nvidia-cudnn-cu11

alright, i found out whats was causing the tensorrt bindings error
one of my extensions kept replacing protobuf (also "rotobuf") with the latest version on startup, after disabling all of them it seems to work properly
so yeah just install everything manually and disable all your extensions and it should probably work lol

Thanks for that information. Would you mind sharing what extensions you had installed, so we can try to determine the best way to make things play nicely together.

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

can you give the commands you input to do it?

pip uninstall -y nvidia-cudnn-cu11

Same error as before.

Ah that might be my fault, I'm using the Automatic1111 self installer version which doesn't use a virtual environment so I was going from memory I think when you are in a venv you can't call pip directly.
So once the venv is activated you would need to issue.
python -m pip install nvidia-cudnn-cu11==<IP_ADDRESS>

python -m pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

python -m pip uninstall -y nvidia-cudnn-cu11

Still the same error, when trying to launch webui.bat.
Im trying the portalbe version now, to see if that somewhat fixes the issue.
The version i currently have installed is used with the git pull

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

can you give the commands you input to do it?

In webui root directory:
venv\scripts\activate.bat

pip uninstall tensorrt

pip cache purge

pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

pip uninstall -y nvidia-cudnn-cu11

pip uninstall -y nvidia-cudnn-cu11

Same error as before.

Ah that might be my fault, I'm using the Automatic1111 self installer version which doesn't use a virtual environment so I was going from memory I think when you are in a venv you can't call pip directly.
So once the venv is activated you would need to issue.
python -m pip install nvidia-cudnn-cu11==<IP_ADDRESS>

python -m pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

python -m pip uninstall -y nvidia-cudnn-cu11

Still the same error, when trying to launch webui.bat. Im trying the portalbe version now, to see if that somewhat fixes the issue. The version i currently have installed is used with the git pull

So i just tried it with the portable version method 1 from here https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs
and i get the same error when installing from URL

The error you mean here is the issue with the Tensor-bindings?

So i just tried it with the portable version method 1 from here https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs
and i get the same error when installing from URL

The error you mean here is the issue with the Tensor-bindings?

yes this error
No module 'xformers'. Proceeding without it.
*** Error loading script: trt.py
Traceback (most recent call last):
File "E:\AIsoftware\sdwebui\webui\modules\scripts.py", line 382, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "E:\AIsoftware\sdwebui\webui\modules\script_loading.py", line 10, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "E:\AIsoftware\sdwebui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\scripts\trt.py", line 10, in
import ui_trt
File "E:\AIsoftware\sdwebui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 10, in
from exporter import export_onnx, export_trt
File "E:\AIsoftware\sdwebui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 10, in
from utilities import Engine
File "E:\AIsoftware\sdwebui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 32, in
import tensorrt as trt
File "E:\AIsoftware\sdwebui\webui\venv\lib\site-packages\tensorrt_init.py", line 18, in
from tensorrt_bindings import *
ModuleNotFoundError: No module named 'tensorrt_bindings'

So i just tried it with the portable version method 1 from here https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs
and i get the same error when installing from URL

The error you mean here is the issue with the Tensor-bindings?

yes this error
No module named 'tensorrt_bindings'

Ok.  One other thing to try.
Open a command window.
Navigate to the sd.webui folder (for the self installer version)
issue:  environment.bat   (This should set the proper environment)
Then try below: (The --no-cache-dir should prevent any cached versions from being used.)
pip install nvidia-cudnn-cu11==<IP_ADDRESS> --no-cache-dir
pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4 --no-cache-dir

The error you mean here is the issue with the Tensor-bindings?

yes this error
No module named 'tensorrt_bindings'

Ok. One other thing to try.
Open a command window.

You can also look in sd.webui\system\python\Lib\site-packages and check the same.
The local version of site-packages should be getting used, but you may be pulling things in from your main Python installation which is causing issues.

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

can you give the commands you input to do it?

In webui root directory:
venv\scripts\activate.bat

pip uninstall tensorrt

pip cache purge

pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

pip uninstall -y nvidia-cudnn-cu11

Thanks for adding this.
After uninstalling, running the above purge and installing again it seems to be working. I still saw the "The procedure entry point ?destroyTensorDescriptor@ops@cudnn@@YA?AW4cudnnStatus_t......" popup occur four times on startup of the UI but it seems to function after clicking past them.

Can you look in c:\users\MYNAME\appdata\local\programs\python\python310\lib\site-packages and see if you have a rotobuf folder there, and delete it if so.
You can also look in sd.webui\system\python\Lib\site-packages and check the same.
The local version of site-packages should be getting used, but you may be pulling things in from your main Python installation which is causing issues.

Thats actually good to know. but aparently i got it installed now.
in the portable version. i havent tried this in the other version i have installed.
activate.bat in venv/scripts/
running pip upgrade command to 23.3 version
pip install nvidia-cudnn-cu11==<IP_ADDRESS> --no-cache-dir
pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir
pip uninstall -y nvidia-cudnn-cu11
then i ran webui.bat and installed the extension from url
used the restart button in the ui
and it started without errors this time. t

Can you look in c:\users\MYNAME\appdata\local\programs\python\python310\lib\site-packages and see if you have a rotobuf folder there, and delete it if so.
You can also look in sd.webui\system\python\Lib\site-packages and check the same.
The local version of site-packages should be getting used, but you may be pulling things in from your main Python installation which is causing issues.

Thats actually good to know. but aparently i got it installed now. in the portable version. i havent tried this in the other version i have installed. activate.bat in venv/scripts/ running pip upgrade command to 23.3 version
pip install nvidia-cudnn-cu11==<IP_ADDRESS> --no-cache-dir pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir pip uninstall -y nvidia-cudnn-cu11
then i ran webui.bat and installed the extension from url used the restart button in the ui and it started without errors this time. t

Great, glad it's working for you

Can you look in c:\users\MYNAME\appdata\local\programs\python\python310\lib\site-packages and see if you have a rotobuf folder there, and delete it if so.
You can also look in sd.webui\system\python\Lib\site-packages and check the same.
The local version of site-packages should be getting used, but you may be pulling things in from your main Python installation which is causing issues.

Thats actually good to know. but aparently i got it installed now. in the portable version. i havent tried this in the other version i have installed. activate.bat in venv/scripts/ running pip upgrade command to 23.3 version
pip install nvidia-cudnn-cu11==<IP_ADDRESS> --no-cache-dir pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir pip uninstall -y nvidia-cudnn-cu11
then i ran webui.bat and installed the extension from url used the restart button in the ui and it started without errors this time. t

Great, glad it's working for you

Thanks a lot for the help. Hoping other people with the same issue can find the answers here.

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

can you give the commands you input to do it?

In webui root directory:
venv\scripts\activate.bat

pip uninstall tensorrt

pip cache purge

pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

pip uninstall -y nvidia-cudnn-cu11

Thanks for adding this. After uninstalling, running the above purge and installing again it seems to be working. I still saw the "The procedure entry point ?destroyTensorDescriptor@ops@cudnn@@ya?AW4cudnnStatus_t......" popup occur four times on startup of the UI but it seems to function after clicking past them.

odd.. im getting this error when im installing now and the tab still doesnt appear. this is the only extension i have as well
*** Error running install.py for extension D:\Dream\AI\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT.
*** Command: "D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe" "D:\Dream\AI\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py"
*** Error code: 1
*** stdout: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com
*** Collecting tensorrt==9.0.1.post11.dev4
***   Using cached tensorrt-9.0.1.post11.dev4-py2.py3-none-any.whl
*** Installing collected packages: tensorrt
*** Successfully installed tensorrt-9.0.1.post11.dev4
*** Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
*** Collecting polygraphy
***   Downloading polygraphy-0.49.0-py2.py3-none-any.whl (327 kB)
***      -------------------------------------- 327.9/327.9 kB 6.8 MB/s eta 0:00:00
*** Installing collected packages: polygraphy
*** Successfully installed polygraphy-0.49.0
*** Collecting protobuf==3.20.2
***   Downloading protobuf-3.20.2-cp310-cp310-win_amd64.whl (904 kB)
***      -------------------------------------- 904.0/904.0 kB 8.2 MB/s eta 0:00:00
*** Installing collected packages: protobuf
***   Attempting uninstall: protobuf
***     Found existing installation: protobuf 3.20.0
***     Uninstalling protobuf-3.20.0:
***       Successfully uninstalled protobuf-3.20.0
*** no module 'xformers'. Processing without...
*** no module 'xformers'. Processing without...
*** No module 'xformers'. Proceeding without it.
*** TensorRT is not installed! Installing...
*** Installing nvidia-cudnn-cu11
*** Installing tensorrt
*** removing nvidia-cudnn-cu11
*** Polygraphy is not installed! Installing...
*** Installing polygraphy
*** GS is not installed! Installing...
*** Installing protobuf

*** stderr: D:\Dream\AI\stable-diffusion-webui\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: pytorch_lightning.utilities.distributed.rank_zero_only has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from pytorch_lightning.utilities instead.
***   rank_zero_deprecation(

*** [notice] A new release of pip available: 22.2.1 -> 23.3
*** [notice] To update, run: D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip

*** [notice] A new release of pip available: 22.2.1 -> 23.3
*** [notice] To update, run: D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip
*** ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'D:\Dream\AI\stable-diffusion-webui\venv\Lib\site-packages\google\~rotobuf\internal\_api_implementation.cp310-win_amd64.pyd'
*** Check the permissions.

*** [notice] A new release of pip available: 22.2.1 -> 23.3
*** [notice] To update, run: D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip
*** Traceback (most recent call last):
***   File "D:\Dream\AI\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 30, in
***     install()
***   File "D:\Dream\AI\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py", line 19, in install
***     launch.run_pip("install protobuf==3.20.2", "protobuf", live=True)
***   File "D:\Dream\AI\stable-diffusion-webui\modules\launch_utils.py", line 138, in run_pip
***     return run(f'"{python}" -m pip {command} --prefer-binary{index_url_line}', desc=f"Installing {desc}", errdesc=f"Couldn't install {desc}", live=live)
***   File "D:\Dream\AI\stable-diffusion-webui\modules\launch_utils.py", line 115, in run
***     raise RuntimeError("\n".join(error_bits))
*** RuntimeError: Couldn't install protobuf.
*** Command: "D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe" -m pip install protobuf==3.20.2 --prefer-binary
*** Error code: 1

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

can you give the commands you input to do it?

In webui root directory:
venv\scripts\activate.bat

pip uninstall tensorrt

pip cache purge

pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

pip uninstall -y nvidia-cudnn-cu11

Thanks for adding this. After uninstalling, running the above purge and installing again it seems to be working. I still saw the "The procedure entry point ?destroyTensorDescriptor@ops@cudnn@@ya?AW4cudnnStatus_t......" popup occur four times on startup of the UI but it seems to function after clicking past them.

odd.. im getting this error when im installing now and the tab still doesnt appear. this is the only extension i have as well
*** Error running install.py for extension D:\Dream\AI\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT. *** Command: "D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe" "D:\Dream\AI\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\install.py" *** Error code: 1 *** stdout: Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com *** Collecting tensorrt==9.0.1.post11.dev4 *** Using cached tensorrt-9.0.1.post11.dev4-py2.py3-none-any.whl *** Installing collected packages: tensorrt *** Successfully installed tensorrt-9.0.1.post11.dev4 *** Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com *** Collecting polygraphy *** Downloading polygraphy-0.49.0-py2.py3-none-any.whl (327 kB) *** -------------------------------------- 327.9/327.9 kB 6.8 MB/s eta 0:00:00 *** Installing collected packages: polygraphy *** Successfully installed polygraphy-0.49.0 *** Collecting protobuf==3.20.2 *** Downloading protobuf-3.20.2-cp310-cp310-win_amd64.whl (904 kB) *** -------------------------------------- 904.0/904.0 kB 8.2 MB/s eta 0:00:00 *** Installing collected packages: protobuf *** Attempting uninstall: protobuf *** Found existing installation: protobuf 3.20.0 *** Uninstalling protobuf-3.20.0: *** Successfully uninstalled protobuf-3.20.0 *** no module 'xformers'. Processing without... *** no module 'xformers'. Processing without... *** No module 'xformers'. Proceeding without it. *** TensorRT is not installed! Installing... *** Installing nvidia-cudnn-cu11 *** Installing tensorrt *** removing nvidia-cudnn-cu11 *** Polygraphy is not installed! Installing... *** Installing polygraphy *** GS is not installed! Installing... *** Installing protobuf
*** stderr: D:\Dream\AI\stable-diffusion-webui\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: pytorch_lightning.utilities.distributed.rank_zero_only has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from pytorch_lightning.utilities instead. *** rank_zero_deprecation(
*** [notice] A new release of pip available: 22.2.1 -> 23.3 *** [notice] To update, run: D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip
*** [notice] A new release of pip available: 22.2.1 -> 23.3 *** [notice] To update, run: D:\Dream\AI\stable-diffusion-webui\venv\Scripts\python.exe -m pip install --upgrade pip *** ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'D:\Dream\AI\stable-diffusion-webui\venv\Lib\site-packages\google~rotobuf\internal_api_implementation.cp310-win_amd64.pyd' *** Check the permissions.