ubuntu tensorRT5.1.5.0 sample yolov3轉onnx轉trt，pytorch轉onnx轉trt推理

tensorRT5.1.5.0安裝

官網安裝並解壓：https://developer.nvidia.com/nvidia-tensorrt-5x-download
添加環境變量：

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/wxy/TensorRT-5.1.5.0/lib

pip安裝python版本，裝在你的python環境下

cd TensorRT-5.1.5.0/python 
pip install tensorrt-5.1.5.0-cp36-none-linux_x86_64.whl
##驗證是否成功
import tensorrt

yolov3轉onnx轉trt

yolov3轉onnx
官方推薦python2，修改一下代碼python3也可以運行，yolov3_to_onnx.py修改

1.parse_cfg_file函數裏面remainder = cfg_file.read().decode()
2.if sys.version_info[0] > 2:這兩句話判斷python版本的註釋掉
3.download_file函數md5哈希驗證取消掉就可以了

執行python yolov3_to_onnx.py

onnx轉trt
會出現’NoneType’ object has no attribute ‘serialize’，根據報錯排查問題，可能是顯存不夠，也可能是初始化失敗等等。

pytorch轉trt推理

總體有兩種思路：
1.pytorch轉onnx轉trt推理，類似於上面yolov3的例子
2.onnx-tensorRT包
pytorch轉onnx轉trt推理
https://blog.csdn.net/qq_33120609/article/details/96565184?depth_1-utm_source=distribute.pc_relevant.none-task&utm_source=distribute.pc_relevant.none-task
pytorch轉onnx

import torch
model = './model_300.pkl'
dummy_input = torch.randn(batch_size, 3, 300, 300, device='cuda')
model = torch.load(model)
torch.onnx.export(model, dummy_input,"mymodel.onnx" , verbose=False)

測試ONNX model 是否與PyTorch model 輸出一致

import onnx_tensorrt.backend as backend
import cv2
import onnx
import numpy as np
​
​
model = onnx.load("model_300.onnx")
engine = backend.prepare(model, device='CUDA:0')
path = '../Net/test.jpg'
img = cv2.imread(path)
print(img.shape)
img = cv2.resize(img,(300,300))
img = img.transpose(2,0,1)
img = np.ascontiguousarray(img)
img = img[np.newaxis,:]
print(img.shape)
input_data= img.astype(np.float32)
data =np.random.random(size=(3, 300, 300))
#data = data.transpose(2,0,1)
data = data[np.newaxis,:]
#input_data=data.astype(np.float32)
#input_data = np.random.random(size=(1, 3, 300, 300)).astype(np.float32)
output_data = engine.run(input_data)
print(output_data)

onnx-tensorRT包安裝
我下載的是tensorRT是5.1.50版本的，注意裝v5.1版本的代碼，
https://github.com/onnx/onnx-tensorrt
https://github.com/onnx/onnx-tensorrt/tree/5.1

git clone --recursive https://github.com/onnx/onnx-tensorrt.git
git check 55d75dec4f289d1c7a63d5cf578b4c6ab441c03e
git checkout -b branch
mkdir build
cd build
cmake .. -DTENSORRT_ROOT=<tensorrt_install_dir>
OR
cmake .. -DTENSORRT_ROOT=<tensorrt_install_dir> -DGPU_ARCHS="61"
make -j8
sudo make install

onnx2trt model_300.onnx -o my_engine.trt

bug
(1)importError: No module named onnx_tensorrt.backend
去onnx-tensorrt 目錄
(2)while trans to ONNX/TensorRT，[8] Assertion failed: get_shape_size(new_shape) == get_shape_size(tensor.getDimensions())
修改 變換時view中的 -1
(3)ImportError: /usr/local/lib/libnvonnxparser.so.0: undefined symbol: _ZNK6google8protobuf7Message11GetTypeNameB5cxx11Ev
這可能幫助你
(4) 如果你使用SSD或者類似,maxpool(ceil_mode) 不支持
修改用padding代替