Failed to 'convert conformer encoder with layernorm plugin'

Describe the bug
The script run_streaming_small_model.sh failed at stage 2
To Reproduce
Follow https://github.com/wenet-e2e/wenet/tree/main/runtime/gpu/tensorrt
docker pull soar97/triton-wenet:22.12
docker run -it --rm --name "wenet_trt_test" --gpus all --shm-size 1g --net host soar97/triton-wenet:22.12
git clone https://github.com/wenet-e2e/wenet.git
pip3 install -r requirements.txt

cd wenet/runtime/gpu/tensorrt
pip3 install -r requirements.txt

pip3 install nvidia-pyindex

bash run_streaming_small_model.sh

Expected behavior
Succeed to run the script: run_streaming_small_model.sh
Screenshots
Desktop (please complete the following information):

OS: ubuntu 20.04
Version 20.04

Additional context
logs:
repalce onnx ops with layernorm plugin
[I] RUNNING | Command: /usr/local/bin/polygraphy surgeon sanitize /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/u2pp_aishell2_onnx/encoder_fp16.onnx --fold-constant -o /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV2.onnx
[I] Loading model: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/u2pp_aishell2_onnx/encoder_fp16.onnx
[I] Original Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=('B', 16, 256)],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 456 Initializer(s) ----

    ---- 2653 Node(s) ----

[I] Folding Constants | Pass 1
2023-12-08 09:03:20.556508006 [W:onnxruntime:, unsqueeze_elimination.cc:20 Apply] UnsqueezeElimination cannot remove node /embed/Unsqueeze_2
2023-12-08 09:03:20.556545117 [W:onnxruntime:, unsqueeze_elimination.cc:20 Apply] UnsqueezeElimination cannot remove node /Unsqueeze_2
[I]     Total Nodes | Original:  2653, After Folding:  1810 |   843 Nodes Folded
[I] Folding Constants | Pass 2
[I]     Total Nodes | Original:  1810, After Folding:  1810 |     0 Nodes Folded
[I] Saving ONNX model to: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV2.onnx
[I] New Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=('B', 16, 256)],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 1168 Initializer(s) ----

    ---- 1810 Node(s) ----

[I] PASSED | Runtime: 4.426s | Command: /usr/local/bin/polygraphy surgeon sanitize /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/u2pp_aishell2_onnx/encoder_fp16.onnx --fold-constant -o /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV2.onnx
[W] Found distinct tensors that share the same name:
[id:<PHONE_NUMBER>19440] Variable (chunk_out): (shape=['B', 16, 256], dtype=float16)
[id:<PHONE_NUMBER>12032] Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
Note: Producer node(s) of first tensor:
[/after_norm/Add_1 (Add)
	Inputs: [
		Variable (/after_norm/Mul_output_0): (shape=['B', 16, 256], dtype=float16)
		Constant (encoder.after_norm.bias): (shape=(256,), dtype=float16)
	]
	Outputs: [
		Variable (chunk_out): (shape=['B', 16, 256], dtype=float16)
	]]
Producer node(s) of second tensor:
[LayerNormN-72 (LayerNorm)
	Inputs: [
		Variable (LayerNormV-71): (shape=None, dtype=float32)
		Constant (LayerNormB-72): (shape=(256,), dtype=float32)
		Constant (LayerNormA-72): (shape=(256,), dtype=float32)
	]
	Outputs: [
		Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
	]]
[W] Found distinct tensors that share the same name:
[id:<PHONE_NUMBER>19440] Variable (chunk_out): (shape=['B', 16, 256], dtype=float16)
[id:<PHONE_NUMBER>12032] Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
Note: Producer node(s) of first tensor:
[]
Producer node(s) of second tensor:
[LayerNormN-72 (LayerNorm)
	Inputs: [
		Variable (LayerNormV-71): (shape=None, dtype=float32)
		Constant (LayerNormB-72): (shape=(256,), dtype=float32)
		Constant (LayerNormA-72): (shape=(256,), dtype=float32)
	]
	Outputs: [
		Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
	]]
[I] RUNNING | Command: /usr/local/bin/polygraphy surgeon sanitize /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx --fold-constant -o /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[I] Loading model: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx
[W] Falling back to `onnx.shape_inference` because `onnxruntime.tools.symbolic_shape_infer` either could not be loaded or did not run successfully.
    Note that using ONNX-Runtime for shape inference may be faster and require less memory.
    Consider installing ONNX-Runtime or setting POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to do so automatically.
[W] ONNX shape inference exited with an error:
    [ShapeInferenceError] (op_type:MatMul, node name: /encoders.0/feed_forward_macaron/w_1/MatMul): [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)
[I] Loading model: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx
[I] Original Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=()],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 1022 Initializer(s) ----

    ---- 1226 Node(s) ----

[I] Folding Constants | Pass 1
[W] ONNX shape inference exited with an error:
    [ShapeInferenceError] (op_type:MatMul, node name: /encoders.0/feed_forward_macaron/w_1/MatMul): [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)
[I]     Total Nodes | Original:  1226, After Folding:  1226 |     0 Nodes Folded
[W] ONNX shape inference exited with an error:
    [ShapeInferenceError] (op_type:MatMul, node name: /encoders.0/feed_forward_macaron/w_1/MatMul): [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)
[I] Saving ONNX model to: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[I] New Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=()],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 1022 Initializer(s) ----

    ---- 1226 Node(s) ----

[I] PASSED | Runtime: 3.356s | Command: /usr/local/bin/polygraphy surgeon sanitize /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx --fold-constant -o /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
convert conformer encoder with layernorm plugin
&&&& RUNNING TensorRT.trtexec [TensorRT v8501] # /usr/src/tensorrt/bin/trtexec --fp16 --onnx=/mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx --minShapes=chunk_xs:1x67x80,chunk_lens:1,offset:1x1,att_cache:1x12x4x80x128,cnn_cache:1x12x256x7,cache_mask:1x1x80 --optShapes=chunk_xs:16x67x80,chunk_lens:16,offset:16x1,att_cache:16x12x4x80x128,cnn_cache:16x12x256x7,cache_mask:16x1x80 --maxShapes=chunk_xs:32x67x80,chunk_lens:32,offset:32x1,att_cache:32x12x4x80x128,cnn_cache:32x12x256x7,cache_mask:32x1x80 --plugins=/mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so --saveEngine=/mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoder_fp16.plan
[12/08/2023-09:03:27] [I] === Model Options ===
[12/08/2023-09:03:27] [I] Format: ONNX
[12/08/2023-09:03:27] [I] Model: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[12/08/2023-09:03:27] [I] Output:
[12/08/2023-09:03:27] [I] === Build Options ===
[12/08/2023-09:03:27] [I] Max batch: explicit batch
[12/08/2023-09:03:27] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[12/08/2023-09:03:27] [I] minTiming: 1
[12/08/2023-09:03:27] [I] avgTiming: 8
[12/08/2023-09:03:27] [I] Precision: FP32+FP16
[12/08/2023-09:03:27] [I] LayerPrecisions:
[12/08/2023-09:03:27] [I] Calibration:
[12/08/2023-09:03:27] [I] Refit: Disabled
[12/08/2023-09:03:27] [I] Sparsity: Disabled
[12/08/2023-09:03:27] [I] Safe mode: Disabled
[12/08/2023-09:03:27] [I] DirectIO mode: Disabled
[12/08/2023-09:03:27] [I] Restricted mode: Disabled
[12/08/2023-09:03:27] [I] Build only: Disabled
[12/08/2023-09:03:27] [I] Save engine: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoder_fp16.plan
[12/08/2023-09:03:27] [I] Load engine:
[12/08/2023-09:03:27] [I] Profiling verbosity: 0
[12/08/2023-09:03:27] [I] Tactic sources: Using default tactic sources
[12/08/2023-09:03:27] [I] timingCacheMode: local
[12/08/2023-09:03:27] [I] timingCacheFile:
[12/08/2023-09:03:27] [I] Heuristic: Disabled
[12/08/2023-09:03:27] [I] Preview Features: Use default preview flags.
[12/08/2023-09:03:27] [I] Input(s)s format: fp32:CHW
[12/08/2023-09:03:27] [I] Output(s)s format: fp32:CHW
[12/08/2023-09:03:27] [I] Input build shape: chunk_xs=1x67x80+16x67x80+32x67x80
[12/08/2023-09:03:27] [I] Input build shape: chunk_lens=1+16+32
[12/08/2023-09:03:27] [I] Input build shape: cache_mask=1x1x80+16x1x80+32x1x80
[12/08/2023-09:03:27] [I] Input build shape: offset=1x1+16x1+32x1
[12/08/2023-09:03:27] [I] Input build shape: cnn_cache=1x12x256x7+16x12x256x7+32x12x256x7
[12/08/2023-09:03:27] [I] Input build shape: att_cache=1x12x4x80x128+16x12x4x80x128+32x12x4x80x128
[12/08/2023-09:03:27] [I] Input calibration shapes: model
[12/08/2023-09:03:27] [I] === System Options ===
[12/08/2023-09:03:27] [I] Device: 0
[12/08/2023-09:03:27] [I] DLACore:
[12/08/2023-09:03:27] [I] Plugins: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so
[12/08/2023-09:03:27] [I] === Inference Options ===
[12/08/2023-09:03:27] [I] Batch: Explicit
[12/08/2023-09:03:27] [I] Input inference shape: att_cache=16x12x4x80x128
[12/08/2023-09:03:27] [I] Input inference shape: cnn_cache=16x12x256x7
[12/08/2023-09:03:27] [I] Input inference shape: offset=16x1
[12/08/2023-09:03:27] [I] Input inference shape: cache_mask=16x1x80
[12/08/2023-09:03:27] [I] Input inference shape: chunk_lens=16
[12/08/2023-09:03:27] [I] Input inference shape: chunk_xs=16x67x80
[12/08/2023-09:03:27] [I] Iterations: 10
[12/08/2023-09:03:27] [I] Duration: 3s (+ 200ms warm up)
[12/08/2023-09:03:27] [I] Sleep time: 0ms
[12/08/2023-09:03:27] [I] Idle time: 0ms
[12/08/2023-09:03:27] [I] Streams: 1
[12/08/2023-09:03:27] [I] ExposeDMA: Disabled
[12/08/2023-09:03:27] [I] Data transfers: Enabled
[12/08/2023-09:03:27] [I] Spin-wait: Disabled
[12/08/2023-09:03:27] [I] Multithreading: Disabled
[12/08/2023-09:03:27] [I] CUDA Graph: Disabled
[12/08/2023-09:03:27] [I] Separate profiling: Disabled
[12/08/2023-09:03:27] [I] Time Deserialize: Disabled
[12/08/2023-09:03:27] [I] Time Refit: Disabled
[12/08/2023-09:03:27] [I] NVTX verbosity: 0
[12/08/2023-09:03:27] [I] Persistent Cache Ratio: 0
[12/08/2023-09:03:27] [I] Inputs:
[12/08/2023-09:03:27] [I] === Reporting Options ===
[12/08/2023-09:03:27] [I] Verbose: Disabled
[12/08/2023-09:03:27] [I] Averages: 10 inferences
[12/08/2023-09:03:27] [I] Percentiles: 90,95,99
[12/08/2023-09:03:27] [I] Dump refittable layers:Disabled
[12/08/2023-09:03:27] [I] Dump output: Disabled
[12/08/2023-09:03:27] [I] Profile: Disabled
[12/08/2023-09:03:27] [I] Export timing to JSON file:
[12/08/2023-09:03:27] [I] Export output to JSON file:
[12/08/2023-09:03:27] [I] Export profile to JSON file:
[12/08/2023-09:03:27] [I]
[12/08/2023-09:03:27] [I] === Device Information ===
[12/08/2023-09:03:27] [I] Selected Device: NVIDIA GeForce GTX 1080 Ti
[12/08/2023-09:03:27] [I] Compute Capability: 6.1
[12/08/2023-09:03:27] [I] SMs: 28
[12/08/2023-09:03:27] [I] Compute Clock Rate: 1.645 GHz
[12/08/2023-09:03:27] [I] Device Global Memory: 11170 MiB
[12/08/2023-09:03:27] [I] Shared Memory per SM: 96 KiB
[12/08/2023-09:03:27] [I] Memory Bus Width: 352 bits (ECC disabled)
[12/08/2023-09:03:27] [I] Memory Clock Rate: 5.505 GHz
[12/08/2023-09:03:27] [I]
[12/08/2023-09:03:27] [I] TensorRT version: 8.5.1
[12/08/2023-09:03:27] [I] Loading supplied plugin library: /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so
[12/08/2023-09:03:27] [I] [TRT] [MemUsageChange] Init CUDA: CPU +9, GPU +0, now: CPU 22, GPU 144 (MiB)
[12/08/2023-09:03:29] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +122, GPU +22, now: CPU 199, GPU 166 (MiB)
[12/08/2023-09:03:29] [I] Start parsing network model
[12/08/2023-09:03:29] [I] [TRT] ----------------------------------------------------------------
[12/08/2023-09:03:29] [I] [TRT] Input filename:   /mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[12/08/2023-09:03:29] [I] [TRT] ONNX IR version:  0.0.8
[12/08/2023-09:03:29] [I] [TRT] Opset version:    14
[12/08/2023-09:03:29] [I] [TRT] Producer name:    pytorch
[12/08/2023-09:03:29] [I] [TRT] Producer version: 1.13.0
[12/08/2023-09:03:29] [I] [TRT] Domain:
[12/08/2023-09:03:29] [I] [TRT] Model version:    0
[12/08/2023-09:03:29] [I] [TRT] Doc string:
[12/08/2023-09:03:29] [I] [TRT] ----------------------------------------------------------------
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:375: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/08/2023-09:03:29] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:29] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:29] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:29] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:29] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:29] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:29] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:29] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:29] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:29] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:30] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:30] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:30] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:30] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:31] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:31] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:31] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:31] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:32] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:32] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:32] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:32] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:32] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:32] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:32] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:32] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:32] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:32] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:32] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:32] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:32] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:32] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:32] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:32] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:32] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:32] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:32] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:32] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:32] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:32] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:32] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:32] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:33] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:33] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:33] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:33] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:33] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:33] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:33] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:33] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:33] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:33] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:33] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:33] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:33] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:33] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:33] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:33] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:33] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:33] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:33] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:33] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:33] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:33] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:33] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:33] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:34] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:34] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:34] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:34] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:34] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:34] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:34] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:34] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:34] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:34] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:34] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:34] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:34] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:34] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:34] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:34] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:34] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:34] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:34] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:34] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:34] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:34] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:34] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:34] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:35] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:35] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:35] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:35] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:35] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:35] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:35] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:35] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:35] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:35] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:35] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:35] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:35] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:35] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:35] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:35] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:35] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:35] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:35] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:35] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:36] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:36] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:36] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:36] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:36] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:36] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:36] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:36] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:37] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:37] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:37] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:37] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:37] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:37] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:37] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:37] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:37] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:37] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:37] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:37] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:37] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:37] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:37] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:37] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:37] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:37] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:37] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:37] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:38] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:38] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:38] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:38] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:38] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:38] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:38] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:38] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:39] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:39] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:39] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:39] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:39] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:39] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:39] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:39] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:39] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:39] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:39] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:39] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:39] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:39] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:39] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:39] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:40] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:40] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:40] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:40] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:40] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:40] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:40] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:40] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:40] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:40] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:40] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:40] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:41] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:41] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:41] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:41] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:41] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/08/2023-09:03:41] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/08/2023-09:03:41] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/08/2023-09:03:41] [I] [TRT] Successfully created plugin: LayerNorm
[12/08/2023-09:03:41] [I] Finish parsing network model
[12/08/2023-09:03:45] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +6, GPU +12, now: CPU 340, GPU 178 (MiB)
[12/08/2023-09:03:45] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +1, GPU +10, now: CPU 341, GPU 188 (MiB)
[12/08/2023-09:03:45] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[12/08/2023-09:04:42] [E] Error[2]: [pluginV2DynamicExtRunner.cpp::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed. )
[12/08/2023-09:04:42] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[12/08/2023-09:04:42] [E] Engine could not be created from network
[12/08/2023-09:04:42] [E] Building engine failed
[12/08/2023-09:04:42] [E] Failed to create engine from model or file.
[12/08/2023-09:04:42] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # /usr/src/tensorrt/bin/trtexec --fp16 --onnx=/mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx --minShapes=chunk_xs:1x67x80,chunk_lens:1,offset:1x1,att_cache:1x12x4x80x128,cnn_cache:1x12x256x7,cache_mask:1x1x80 --optShapes=chunk_xs:16x67x80,chunk_lens:16,offset:16x1,att_cache:16x12x4x80x128,cnn_cache:16x12x256x7,cache_mask:16x1x80 --maxShapes=chunk_xs:32x67x80,chunk_lens:32,offset:32x1,att_cache:32x12x4x80x128,cnn_cache:32x12x256x7,cache_mask:32x1x80 --plugins=/mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so --saveEngine=/mnt/xt/8T/CODES/Audio/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoder_fp16.plan

Could you try export with '--verbose' and attached detailed log?
Or I suggest to use latest tensorrt ( >= 8.6), and onnx opset e.g. 17, since it would support layernorm plugin internally. So, we could skip the stage https://github.com/wenet-e2e/wenet/blob/main/runtime/gpu/tensorrt/run_streaming_small_model.sh#L56-L64 and remove '--plugin' option.

@yuekaizhang Thank you for your replay.  I achieved my aim by following https://github.com/wenet-e2e/wenet/blob/main/runtime/gpu/README.md. If any update on docs about  using tensorrt as backend, I will try again.

@yuekaizhang could you plz take a look at this log?
[12/19/2023-12:45:27] [V] [TRT] Casting to type: float16
[12/19/2023-12:45:27] [V] [TRT] Registering layer: /Cast_7 for ONNX node: /Cast_7
[12/19/2023-12:45:27] [V] [TRT] Registering tensor: log_probs_1131 for ONNX tensor: log_probs
[12/19/2023-12:45:27] [V] [TRT] /Cast_7 [Cast] outputs: [log_probs -> (-1, 16, 10)[HALF]],
[12/19/2023-12:45:27] [V] [TRT] Marking log_probs_1131 as output: log_probs
[12/19/2023-12:45:27] [V] [TRT] Marking log_probs_idx_1130 as output: log_probs_idx
[12/19/2023-12:45:27] [V] [TRT] Marking chunk_out_1125 as output: chunk_out
[12/19/2023-12:45:27] [V] [TRT] Marking chunk_out_lens_1126 as output: chunk_out_lens
[12/19/2023-12:45:27] [V] [TRT] Marking r_offset_1129 as output: r_offset
[12/19/2023-12:45:27] [V] [TRT] Marking r_att_cache_1123 as output: r_att_cache
[12/19/2023-12:45:27] [V] [TRT] Marking r_cnn_cache_1124 as output: r_cnn_cache
[12/19/2023-12:45:27] [V] [TRT] Marking r_cache_mask_534 as output: r_cache_mask
[12/19/2023-12:45:27] [I] Finish parsing network model
[12/19/2023-12:45:27] [E] Error[4]: [network.cpp::validate::3076] Error Code 4: Internal Error (cnn_cache: for dimension number 3 in profile 0 does not match network definition (got min=7, opt=7, max=7), expected min=opt=max=14).)
[12/19/2023-12:45:27] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[12/19/2023-12:45:27] [E] Engine could not be created from network
[12/19/2023-12:45:27] [E] Building engine failed
[12/19/2023-12:45:27] [E] Failed to create engine from model or file.
[12/19/2023-12:45:27] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # /usr/src/tensorrt/bin/trtexec --fp16 --onnx=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/encoderV4.onnx --minShapes=chunk_xs:1x67x80,chunk_lens:1,offset:1x1,att_cache:1x12x4x80x128,cnn_cache:1x12x256x7,cache_mask:1x1x80 --optShapes=chunk_xs:16x67x80,chunk_lens:16,offset:16x1,att_cache:16x12x4x80x128,cnn_cache:16x12x256x7,cache_mask:16x1x80 --maxShapes=chunk_xs:32x67x80,chunk_lens:32,offset:32x1,att_cache:32x12x4x80x128,cnn_cache:32x12x256x7,cache_mask:32x1x80 --plugins=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/LayerNorm.so --saveEngine=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/encoder_fp16.plan --verbose

"Error Code 4: Internal Error (cnn_cache: for dimension number 3 in profile 0 does not match network definition (got min=7, opt=7, max=7), expected min=opt=max=14).)"
Looks like you are using conv filter with 31, rather than 15, please change 7 with 14 for cnn_cache,  /usr/src/tensorrt/bin/trtexec --fp16 --onnx=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/encoderV4.onnx --minShapes=chunk_xs:1x67x80,chunk_lens:1,offset:1x1,att_cache:1x12x4x80x128,cnn_cache:1x12x256x7,cache_mask:1x1x80 --optShapes=chunk_xs:16x67x80,chunk_lens:16,offset:16x1,att_cache:16x12x4x80x128,cnn_cache:16x12x256x7,cache_mask:16x1x80 --maxShapes=chunk_xs:32x67x80,chunk_lens:32,offset:32x1,att_cache:32x12x4x80x128,cnn_cache:32x12x256x7,cache_mask:32x1x80 --plugins=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/LayerNorm.so --saveEngine=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/encoder_fp16.plan --verbose

Thank you for your reply.
I tried again with 14 instead of 7. It made more progress, but still failed.
[12/20/2023-12:40:54] [V] [TRT] Tactic: 0x000000000000001c Time: 0.00356148
[12/20/2023-12:40:54] [V] [TRT] Tactic: 0x000000000000001d Time: 0.00377095
[12/20/2023-12:40:54] [V] [TRT] Fastest Tactic: 0x000000000000000f Time: 0.00324706
[12/20/2023-12:40:54] [V] [TRT] --------------- Timing Runner: PWN(/embed/out/out.0/Add, PWN(/embed/pos_enc/Constant_output_0 + (Unnamed Layer* 75) [Shuffle], /embed/pos_enc/Mul)) (PointWise)
[12/20/2023-12:40:54] [V] [TRT] PointWise has no valid tactics for this config, skipping
[12/20/2023-12:40:54] [V] [TRT] >>>>>>>>>>>>>>> Chose Runner Type: PointWiseV2 Tactic: 0x000000000000000f
[12/20/2023-12:40:54] [V] [TRT] =============== Computing costs for
[12/20/2023-12:40:54] [V] [TRT] *************** Autotuning format combination: Float(4096,256,1), Float(1), Float(1) -> Float(4096,256,1) ***************
[12/20/2023-12:40:54] [V] [TRT] --------------- Timing Runner: LayerNormN-0 (PluginV2)
[12/20/2023-12:40:54] [V] [TRT] Deleting timing cache: 845 entries, served 18643 hits since creation.
[12/20/2023-12:40:54] [E] Error[2]: [pluginV2DynamicExtRunner.cpp::execute::115] Error Code 2: Internal Error (Assertion status == kSTATUS_SUCCESS failed. )
[12/20/2023-12:40:54] [E] Error[2]: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[12/20/2023-12:40:54] [E] Engine could not be created from network
[12/20/2023-12:40:54] [E] Building engine failed
[12/20/2023-12:40:54] [E] Failed to create engine from model or file.
[12/20/2023-12:40:54] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # /usr/src/tensorrt/bin/trtexec --fp16 --onnx=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/encoderV4.onnx --minShapes=chunk_xs:1x67x80,chunk_lens:1,offset:1x1,att_cache:1x12x4x80x128,cnn_cache:1x12x256x14,cache_mask:1x1x80 --optShapes=chunk_xs:16x67x80,chunk_lens:16,offset:16x1,att_cache:16x12x4x80x128,cnn_cache:16x12x256x14,cache_mask:16x1x80 --maxShapes=chunk_xs:32x67x80,chunk_lens:32,offset:32x1,att_cache:32x12x4x80x128,cnn_cache:32x12x256x14,cache_mask:32x1x80 --plugins=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/LayerNorm.so --saveEngine=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet//models/multi_cn_unified_conformer_exp/exp_streaming_trt/encoder_fp16.plan --verbose

The steps to reproduce

clone the repo, and the commit I used is 1f8e1c51f4f685282b30077642c701837265925f
download the pre-trained model: multi_cn_unified_conformer_exp.tar.gz
convert to onnxmodel_dir=${PWD}/models/multi_cn_unified_conformer_exp
onnx_model_dir=${model_dir}/onnx/
mkdir $onnx_model_dir

python3 wenet/bin/export_onnx_gpu.py --config=$model_dir/train.yaml --checkpoint=$model_dir/final.pt --cmvn_file=$model_dir/global_cmvn --ctc_weight=0.5 --output_onnx_dir=$onnx_model_dir --fp16 --streaming
cp $model_dir/train.yaml $onnx_model_dir/
#cp $model_dir/words.txt $onnx_model_dir/
cp $model_dir/units.txt $onnx_model_dir/words.txt

tensorrt testdocker pull soar97/triton-wenet:22.12
docker run -it --rm --name "wenet_trt_test" --gpus all -v /mnt/xt/8T:/mnt/xt/8T/ --shm-size 1g --net host soar97/triton-wenet:22.12

# inside the docker container
WENET_DIR=/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/
onnx_model_dir=${WENET_DIR}/models/multi_cn_unified_conformer_exp/onnx
outputs_dir=${WENET_DIR}/models/multi_cn_unified_conformer_exp/exp_streaming_trt
mkdir $outputs_dir
model_repo_path=${WENET_DIR}/runtime/gpu/tensorrt/model_repo_stateful_trt

# compile LayerNormPlugin
cd ${WENET_DIR}/runtime/gpu/tensorrt/LayerNormPlugin
make clean
make
cp LayerNorm.so $outputs_dir
cd ..

# repalce onnx ops with layernorm plugin
polygraphy surgeon sanitize $onnx_model_dir/encoder_fp16.onnx --fold-constant -o $outputs_dir/encoderV2.onnx
python3 replace_layernorm.py --input_onnx $outputs_dir/encoderV2.onnx \
                             --output_onnx $outputs_dir/encoderV3.onnx
polygraphy surgeon sanitize $outputs_dir/encoderV3.onnx --fold-constant -o $outputs_dir/encoderV4.onnx

# convert conformer encoder with layernorm plugin
MIN_BATCH=1
OPT_BATCH=16
MAX_BATCH=32
/usr/src/tensorrt/bin/trtexec \
    --fp16 \
    --onnx=$outputs_dir/encoderV4.onnx \
    --minShapes=chunk_xs:${MIN_BATCH}x67x80,chunk_lens:${MIN_BATCH},offset:${MIN_BATCH}x1,att_cache:${MIN_BATCH}x12x4x80x128,cnn_cache:${MIN_BATCH}x12x256x14,cache_mask:${MIN_BATCH}x1x80 \
    --optShapes=chunk_xs:${OPT_BATCH}x67x80,chunk_lens:${OPT_BATCH},offset:${OPT_BATCH}x1,att_cache:${OPT_BATCH}x12x4x80x128,cnn_cache:${OPT_BATCH}x12x256x14,cache_mask:${OPT_BATCH}x1x80 \
    --maxShapes=chunk_xs:${MAX_BATCH}x67x80,chunk_lens:${MAX_BATCH},offset:${MAX_BATCH}x1,att_cache:${MAX_BATCH}x12x4x80x128,cnn_cache:${MAX_BATCH}x12x256x14,cache_mask:${MAX_BATCH}x1x80 \
    --plugins=$outputs_dir/LayerNorm.so \
    --saveEngine=$outputs_dir/encoder_fp16.plan \
    --verbose

gin

Would you mind trying https://github.com/wenet-e2e/wenet/blob/3eb9a8579d65a32b606aa04c89bdfcaca10d220b/runtime/gpu/tensorrt/run_streaming_small_model.sh first? I am not sure if multi_cn_unified_conformer_exp.tar.gz works.
Also, I suggest to use latest tensorrt without layernorm plugin.

Or I suggest to use latest tensorrt ( >= 8.6), and onnx opset e.g. 17, since it would support layernorm plugin internally. So, we could skip the stage https://github.com/wenet-e2e/wenet/blob/main/runtime/gpu/tensorrt/run_streaming_small_model.sh#L56-L64 and remove '--plugin' option.

Thank you. I'll try run_streaming_small_model.sh in few days.
I cannot use tensorrt>= 8.6, because my target device is a jetson nx board, with JetPack 5.1.2 only includes TensorRT 8.5.2.
I run the scripts in x86 PC with a 1080Ti gpu.

onnx

using http://mobvoi-speech-public.ufile.ucloud.cn/public/wenet/aishell2/20210618_u2pp_conformer_exp.tar.gz

repalce onnx ops with layernorm plugin
root@ubt:/mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/LayerNormPlugin# polygraphy surgeon sanitize $onnx_model_dir/encoder_fp16.onnx --fold-constant -o $outputs_dir/encoderV2.onnx
[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[I] RUNNING | Command: /usr/local/bin/polygraphy surgeon sanitize /mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/models/20210618_u2pp_conformer_exp/onnx//encoder_fp16.onnx --fold-constant -o /mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV2.onnx
[I] Loading model: /mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/models/20210618_u2pp_conformer_exp/onnx/encoder_fp16.onnx
[I] Original Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=('B', 16, 256)],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 456 Initializer(s) ----

    ---- 2653 Node(s) ----

[I] Folding Constants | Pass 1
2023-12-21 12:44:50.040462676 [W:onnxruntime:, unsqueeze_elimination.cc:20 Apply] UnsqueezeElimination cannot remove node /embed/Unsqueeze_2
2023-12-21 12:44:50.040493041 [W:onnxruntime:, unsqueeze_elimination.cc:20 Apply] UnsqueezeElimination cannot remove node /Unsqueeze_2
[I]     Total Nodes | Original:  2653, After Folding:  1810 |   843 Nodes Folded
[I] Folding Constants | Pass 2
[I]     Total Nodes | Original:  1810, After Folding:  1810 |     0 Nodes Folded
[I] Saving ONNX model to: /mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV2.onnx
[I] New Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=('B', 16, 256)],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 1168 Initializer(s) ----

    ---- 1810 Node(s) ----

[I] PASSED | Runtime: 4.063s | Command: /usr/local/bin/polygraphy surgeon sanitize /mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/models/20210618_u2pp_conformer_exp/onnx//encoder_fp16.onnx --fold-constant -o /mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV2.onnx

root@ubt://mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt# python3 replace_layernorm.py --input_onnx $outputs_dir/encoderV2.onnx \
>                                --output_onnx $outputs_dir/encoderV3.onnx \
>                                || exit 1
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Found distinct tensors that share the same name:
[id:<PHONE_NUMBER>16096] Variable (chunk_out): (shape=['B', 16, 256], dtype=float16)
[id:<PHONE_NUMBER>08688] Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
Note: Producer node(s) of first tensor:
[/after_norm/Add_1 (Add)
	Inputs: [
		Variable (/after_norm/Mul_output_0): (shape=['B', 16, 256], dtype=float16)
		Constant (encoder.after_norm.bias): (shape=(256,), dtype=float16)
	]
	Outputs: [
		Variable (chunk_out): (shape=['B', 16, 256], dtype=float16)
	]]
Producer node(s) of second tensor:
[LayerNormN-72 (LayerNorm)
	Inputs: [
		Variable (LayerNormV-71): (shape=None, dtype=float32)
		Constant (LayerNormB-72): (shape=(256,), dtype=float32)
		Constant (LayerNormA-72): (shape=(256,), dtype=float32)
	]
	Outputs: [
		Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
	]]
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Found distinct tensors that share the same name:
[id:<PHONE_NUMBER>16096] Variable (chunk_out): (shape=['B', 16, 256], dtype=float16)
[id:<PHONE_NUMBER>08688] Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
Note: Producer node(s) of first tensor:
[]
Producer node(s) of second tensor:
[LayerNormN-72 (LayerNorm)
	Inputs: [
		Variable (LayerNormV-71): (shape=None, dtype=float32)
		Constant (LayerNormB-72): (shape=(256,), dtype=float32)
		Constant (LayerNormA-72): (shape=(256,), dtype=float32)
	]
	Outputs: [
		Variable (chunk_out): (shape=[], dtype=<class 'numpy.float16'>)
	]]

root@ubt://mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt# polygraphy surgeon sanitize $outputs_dir/encoderV3.onnx --fold-constant -o $outputs_dir/encoderV4.onnx
[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[I] RUNNING | Command: /usr/local/bin/polygraphy surgeon sanitize //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx --fold-constant -o //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[I] Loading model: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx
[W] Falling back to `onnx.shape_inference` because `onnxruntime.tools.symbolic_shape_infer` either could not be loaded or did not run successfully.
    Note that using ONNX-Runtime for shape inference may be faster and require less memory.
    Consider installing ONNX-Runtime or setting POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to do so automatically.
[W] ONNX shape inference exited with an error:
    [ShapeInferenceError] (op_type:MatMul, node name: /encoders.0/feed_forward_macaron/w_1/MatMul): [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)
[I] Loading model: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx
[I] Original Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=()],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 1022 Initializer(s) ----

    ---- 1226 Node(s) ----

[I] Folding Constants | Pass 1
[W] ONNX shape inference exited with an error:
    [ShapeInferenceError] (op_type:MatMul, node name: /encoders.0/feed_forward_macaron/w_1/MatMul): [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)
[I]     Total Nodes | Original:  1226, After Folding:  1226 |     0 Nodes Folded
[W] ONNX shape inference exited with an error:
    [ShapeInferenceError] (op_type:MatMul, node name: /encoders.0/feed_forward_macaron/w_1/MatMul): [TypeInferenceError] Inferred elem type differs from existing elem type: (1) vs (10)
[I] Saving ONNX model to: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[I] New Model:
    Name: torch_jit | ONNX Opset: 14

    ---- 6 Graph Input(s) ----
    {chunk_xs [dtype=float16, shape=('B', 67, 80)],
     chunk_lens [dtype=int32, shape=('B',)],
     offset [dtype=int64, shape=('B', 1)],
     att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 8 Graph Output(s) ----
    {log_probs [dtype=float16, shape=('B', 16, 10)],
     log_probs_idx [dtype=int64, shape=('B', 16, 10)],
     chunk_out [dtype=float16, shape=()],
     chunk_out_lens [dtype=int32, shape=('B',)],
     r_offset [dtype=int64, shape=('B', 1)],
     r_att_cache [dtype=float16, shape=('B', 12, 4, 80, 128)],
     r_cnn_cache [dtype=float16, shape=('B', 12, 256, 7)],
     r_cache_mask [dtype=float16, shape=('B', 1, 80)]}

    ---- 1022 Initializer(s) ----

    ---- 1226 Node(s) ----

[I] PASSED | Runtime: 3.078s | Command: /usr/local/bin/polygraphy surgeon sanitize //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV3.onnx --fold-constant -o //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx

convert conformer encoder with layernorm plugin
      $trtexec \
>           --fp16 \
>           --onnx=$outputs_dir/encoderV4.onnx \
>           --minShapes=chunk_xs:${MIN_BATCH}x67x80,chunk_lens:${MIN_BATCH},offset:${MIN_BATCH}x1,att_cache:${MIN_BATCH}x12x4x80x128,cnn_cache:${MIN_BATCH}x12x256x7,cache_mask:${MIN_BATCH}x1x80 \
>           --optShapes=chunk_xs:${OPT_BATCH}x67x80,chunk_lens:${OPT_BATCH},offset:${OPT_BATCH}x1,att_cache:${OPT_BATCH}x12x4x80x128,cnn_cache:${OPT_BATCH}x12x256x7,cache_mask:${OPT_BATCH}x1x80 \
>           --maxShapes=chunk_xs:${MAX_BATCH}x67x80,chunk_lens:${MAX_BATCH},offset:${MAX_BATCH}x1,att_cache:${MAX_BATCH}x12x4x80x128,cnn_cache:${MAX_BATCH}x12x256x7,cache_mask:${MAX_BATCH}x1x80 \
>           --plugins=$outputs_dir/LayerNorm.so \
>           --saveEngine=$outputs_dir/encoder_fp16.plan
&&&& RUNNING TensorRT.trtexec [TensorRT v8501] # /usr/src/tensorrt/bin/trtexec --fp16 --onnx=//mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx --minShapes=chunk_xs:1x67x80,chunk_lens:1,offset:1x1,att_cache:1x12x4x80x128,cnn_cache:1x12x256x7,cache_mask:1x1x80 --optShapes=chunk_xs:16x67x80,chunk_lens:16,offset:16x1,att_cache:16x12x4x80x128,cnn_cache:16x12x256x7,cache_mask:16x1x80 --maxShapes=chunk_xs:32x67x80,chunk_lens:32,offset:32x1,att_cache:32x12x4x80x128,cnn_cache:32x12x256x7,cache_mask:32x1x80 --plugins=//mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so --saveEngine=//mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoder_fp16.plan
[12/21/2023-12:57:49] [I] === Model Options ===
[12/21/2023-12:57:49] [I] Format: ONNX
[12/21/2023-12:57:49] [I] Model: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[12/21/2023-12:57:49] [I] Output:
[12/21/2023-12:57:49] [I] === Build Options ===
[12/21/2023-12:57:49] [I] Max batch: explicit batch
[12/21/2023-12:57:49] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[12/21/2023-12:57:49] [I] minTiming: 1
[12/21/2023-12:57:49] [I] avgTiming: 8
[12/21/2023-12:57:49] [I] Precision: FP32+FP16
[12/21/2023-12:57:49] [I] LayerPrecisions:
[12/21/2023-12:57:49] [I] Calibration:
[12/21/2023-12:57:49] [I] Refit: Disabled
[12/21/2023-12:57:49] [I] Sparsity: Disabled
[12/21/2023-12:57:49] [I] Safe mode: Disabled
[12/21/2023-12:57:49] [I] DirectIO mode: Disabled
[12/21/2023-12:57:49] [I] Restricted mode: Disabled
[12/21/2023-12:57:49] [I] Build only: Disabled
[12/21/2023-12:57:49] [I] Save engine: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoder_fp16.plan
[12/21/2023-12:57:49] [I] Load engine:
[12/21/2023-12:57:49] [I] Profiling verbosity: 0
[12/21/2023-12:57:49] [I] Tactic sources: Using default tactic sources
[12/21/2023-12:57:49] [I] timingCacheMode: local
[12/21/2023-12:57:49] [I] timingCacheFile:
[12/21/2023-12:57:49] [I] Heuristic: Disabled
[12/21/2023-12:57:49] [I] Preview Features: Use default preview flags.
[12/21/2023-12:57:49] [I] Input(s)s format: fp32:CHW
[12/21/2023-12:57:49] [I] Output(s)s format: fp32:CHW
[12/21/2023-12:57:49] [I] Input build shape: chunk_xs=1x67x80+16x67x80+32x67x80
[12/21/2023-12:57:49] [I] Input build shape: chunk_lens=1+16+32
[12/21/2023-12:57:49] [I] Input build shape: cache_mask=1x1x80+16x1x80+32x1x80
[12/21/2023-12:57:49] [I] Input build shape: offset=1x1+16x1+32x1
[12/21/2023-12:57:49] [I] Input build shape: cnn_cache=1x12x256x7+16x12x256x7+32x12x256x7
[12/21/2023-12:57:49] [I] Input build shape: att_cache=1x12x4x80x128+16x12x4x80x128+32x12x4x80x128
[12/21/2023-12:57:49] [I] Input calibration shapes: model
[12/21/2023-12:57:49] [I] === System Options ===
[12/21/2023-12:57:49] [I] Device: 0
[12/21/2023-12:57:49] [I] DLACore:
[12/21/2023-12:57:49] [I] Plugins: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so
[12/21/2023-12:57:49] [I] === Inference Options ===
[12/21/2023-12:57:49] [I] Batch: Explicit
[12/21/2023-12:57:49] [I] Input inference shape: att_cache=16x12x4x80x128
[12/21/2023-12:57:49] [I] Input inference shape: cnn_cache=16x12x256x7
[12/21/2023-12:57:49] [I] Input inference shape: offset=16x1
[12/21/2023-12:57:49] [I] Input inference shape: cache_mask=16x1x80
[12/21/2023-12:57:49] [I] Input inference shape: chunk_lens=16
[12/21/2023-12:57:49] [I] Input inference shape: chunk_xs=16x67x80
[12/21/2023-12:57:49] [I] Iterations: 10
[12/21/2023-12:57:49] [I] Duration: 3s (+ 200ms warm up)
[12/21/2023-12:57:49] [I] Sleep time: 0ms
[12/21/2023-12:57:49] [I] Idle time: 0ms
[12/21/2023-12:57:49] [I] Streams: 1
[12/21/2023-12:57:49] [I] ExposeDMA: Disabled
[12/21/2023-12:57:49] [I] Data transfers: Enabled
[12/21/2023-12:57:49] [I] Spin-wait: Disabled
[12/21/2023-12:57:49] [I] Multithreading: Disabled
[12/21/2023-12:57:49] [I] CUDA Graph: Disabled
[12/21/2023-12:57:49] [I] Separate profiling: Disabled
[12/21/2023-12:57:49] [I] Time Deserialize: Disabled
[12/21/2023-12:57:49] [I] Time Refit: Disabled
[12/21/2023-12:57:49] [I] NVTX verbosity: 0
[12/21/2023-12:57:49] [I] Persistent Cache Ratio: 0
[12/21/2023-12:57:49] [I] Inputs:
[12/21/2023-12:57:49] [I] === Reporting Options ===
[12/21/2023-12:57:49] [I] Verbose: Disabled
[12/21/2023-12:57:49] [I] Averages: 10 inferences
[12/21/2023-12:57:49] [I] Percentiles: 90,95,99
[12/21/2023-12:57:49] [I] Dump refittable layers:Disabled
[12/21/2023-12:57:49] [I] Dump output: Disabled
[12/21/2023-12:57:49] [I] Profile: Disabled
[12/21/2023-12:57:49] [I] Export timing to JSON file:
[12/21/2023-12:57:49] [I] Export output to JSON file:
[12/21/2023-12:57:49] [I] Export profile to JSON file:
[12/21/2023-12:57:49] [I]
[12/21/2023-12:57:49] [I] === Device Information ===
[12/21/2023-12:57:49] [I] Selected Device: NVIDIA GeForce GTX 1080 Ti
[12/21/2023-12:57:49] [I] Compute Capability: 6.1
[12/21/2023-12:57:49] [I] SMs: 28
[12/21/2023-12:57:49] [I] Compute Clock Rate: 1.645 GHz
[12/21/2023-12:57:49] [I] Device Global Memory: 11170 MiB
[12/21/2023-12:57:49] [I] Shared Memory per SM: 96 KiB
[12/21/2023-12:57:49] [I] Memory Bus Width: 352 bits (ECC disabled)
[12/21/2023-12:57:49] [I] Memory Clock Rate: 5.505 GHz
[12/21/2023-12:57:49] [I]
[12/21/2023-12:57:49] [I] TensorRT version: 8.5.1
[12/21/2023-12:57:49] [I] Loading supplied plugin library: //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/LayerNorm.so
[12/21/2023-12:57:49] [I] [TRT] [MemUsageChange] Init CUDA: CPU +9, GPU +0, now: CPU 22, GPU 149 (MiB)
[12/21/2023-12:57:51] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +122, GPU +22, now: CPU 199, GPU 171 (MiB)
[12/21/2023-12:57:51] [I] Start parsing network model
[12/21/2023-12:57:51] [I] [TRT] ----------------------------------------------------------------
[12/21/2023-12:57:51] [I] [TRT] Input filename:   //mnt/xt/8T/CODES/gitlab-ubt/husky/wenet/runtime/gpu/tensorrt/exp_streaming_trt/encoderV4.onnx
[12/21/2023-12:57:51] [I] [TRT] ONNX IR version:  0.0.8
[12/21/2023-12:57:51] [I] [TRT] Opset version:    14
[12/21/2023-12:57:51] [I] [TRT] Producer name:    pytorch
[12/21/2023-12:57:51] [I] [TRT] Producer version: 1.13.0
[12/21/2023-12:57:51] [I] [TRT] Domain:
[12/21/2023-12:57:51] [I] [TRT] Model version:    0
[12/21/2023-12:57:51] [I] [TRT] Doc string:
[12/21/2023-12:57:51] [I] [TRT] ----------------------------------------------------------------
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:375: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:51] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:51] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:51] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:51] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:52] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:52] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:52] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:52] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:53] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:53] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:53] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:53] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:54] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:54] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:54] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:54] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:55] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:55] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:55] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:55] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:56] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:56] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:56] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:56] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:56] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:56] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:56] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:56] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:56] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:56] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:56] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:56] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:56] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:56] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:56] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:56] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:56] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:56] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:56] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:56] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:57] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:57] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:57] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:57] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:57] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:57] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:57] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:57] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:57] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:57] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:57] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:57] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:57] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:57] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:57] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:57] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:58] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:58] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:58] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:58] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:58] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:58] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:58] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:58] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:58] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:58] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:58] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:58] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:58] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:58] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:58] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:58] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:59] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:59] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:59] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:59] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:59] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:59] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:59] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:59] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:59] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:59] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:59] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:59] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:59] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:59] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:59] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:59] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:57:59] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:57:59] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:57:59] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:57:59] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:58:00] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:58:00] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:58:00] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:58:00] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:58:00] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:58:00] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:58:00] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:58:00] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:58:00] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:58:00] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:58:00] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:58:00] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:58:00] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:58:00] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:58:00] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:58:00] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:58:00] [I] [TRT] No importer registered for op: LayerNorm. Attempting to import as plugin.
[12/21/2023-12:58:00] [I] [TRT] Searching for plugin: LayerNorm, plugin_version: 1, plugin_namespace:
[12/21/2023-12:58:00] [W] [TRT] parsers/onnx/builtin_op_importers.cpp:5225: Attribute epsilon not found in plugin node! Ensure that the plugin creator has a default value defined or the engine may fail to build.
[12/21/2023-12:58:00] [I] [TRT] Successfully created plugin: LayerNorm
[12/21/2023-12:58:01] [I] Finish parsing network model
[12/21/2023-12:58:04] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +6, GPU +12, now: CPU 340, GPU 183 (MiB)
[12/21/2023-12:58:04] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +1, GPU +10, now: CPU 341, GPU 193 (MiB)
[12/21/2023-12:58:04] [I] [TRT] Local timing cache in use.  Ensure that the plugin creator has a default value defined or the engine may fail to build.

Does it matter?