TensorRT是NVIDIA推出的一款深度學習推理優化器，可以方便深度學習模型在低成本的單個GPU機器或者一些嵌入式平臺上的部署，這裏介紹TensorRT在ubuntu16.04下的安裝。
TensorRT下載地址：https://developer.nvidia.com/nvidia-tensorrt-5x-download 下載需要登錄NVIDIA帳號
可以點擊下載自己cuda對應的TensorRT版本，比如TensorRT-5.0.2.6.Ubuntu-16.04.4.x86_64-gnu.cuda-9.0.cudnn7.3.tar.gz
需要注意的是TensorRT的版本需要和cuda以及cudnn一致，TensorRT-5.0.2.6.對應cuda9.0和cudnn7.3。解壓Tensorrt下載文件，並添加環境變量（這裏的$HOME爲TensrRT-5.0.26所在目錄）
vim ~/.bashrc
export LD_LIBRARY_PATH=$HOME/TensorRT-5.0.2.6/lib:$LD_LIBRARY_PATH
export CUDA_INSTALL_DIR=/usr/local/cuda-9.0
export CUDNN_INSTALL_DIR=/usr/local/cuda-9.0
source ~/.bashrc
然後進入解壓目錄，安裝TensorRT
#install TensorRT
cd $HOME/tensorrt-5.0.2.6/python
sudo pip3 install tensorrt-5.0.2.6-py2.py3-none-any.whl
#install UFF
cd ../uff
sudo pip3 install uff-0.5.5-py2.py3-none-any.whl
#install graphsurgeon
cd ../graphsurgeon
sudo pip3 install graphsurgeon-0.3.2-py2.py3-none-any.whl
測試TensorRT是否安裝成功，進入Python編輯器導入tensorrt和uff模塊
import tensorrt
import uff
print(tensorrt.__version__)
print(uff.__version__)
常見報錯問題
ImportError: libnvinfer.so.5: cannot open shared object file: Nosuch file or directory
ImportError: libnvonnxparser.so.0: cannot open shared object file:No such file or directory
ImportError: libnvonnxparser_runtime.so.0: cannot open shared objectfile: No such file or directory
ImportError: libnvparsers.so.5: cannot open shared object file: Nosuch file or directory
ImportError: libnvinfer_plugin.so.5: cannot open shared object file:No such file or directory
解決辦法 如下
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvinfer.so.5 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvonnxparser.so.0 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvonnxparser_runtime.so.0 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvparsers.so.5 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/ libnvinfer_plugin.so.5/usr/lib/
另外通過命令：which convert-to-uff 可以輸出uff的安裝路徑
運行sample測試下TensorRT是否安裝成功
cd TensorRT-5.1.2.2/samples/sampleMNIST
make clean
make
cd /TensorRT-XXX/bin
./sample_mnist
顯示如下結果，說明TensorRT已正常安裝
[I] Output:
0:
1:
2:
3:
4:
5:
6:
7:
8: **********
9:
&&&& PASSED TensorRT.sample_mnist # ./sample_mnist