TensorRT是NVIDIA推出的一款深度学习推理优化器,可以方便深度学习模型在低成本的单个GPU机器或者一些嵌入式平台上的部署,这里介绍TensorRT在ubuntu16.04下的安装。
TensorRT下载地址:https://developer.nvidia.com/nvidia-tensorrt-5x-download 下载需要登录NVIDIA帐号
可以点击下载自己cuda对应的TensorRT版本,比如TensorRT-5.0.2.6.Ubuntu-16.04.4.x86_64-gnu.cuda-9.0.cudnn7.3.tar.gz
需要注意的是TensorRT的版本需要和cuda以及cudnn一致,TensorRT-5.0.2.6.对应cuda9.0和cudnn7.3。解压Tensorrt下载文件,并添加环境变量(这里的$HOME为TensrRT-5.0.26所在目录)
vim ~/.bashrc
export LD_LIBRARY_PATH=$HOME/TensorRT-5.0.2.6/lib:$LD_LIBRARY_PATH
export CUDA_INSTALL_DIR=/usr/local/cuda-9.0
export CUDNN_INSTALL_DIR=/usr/local/cuda-9.0
source ~/.bashrc
然后进入解压目录,安装TensorRT
#install TensorRT
cd $HOME/tensorrt-5.0.2.6/python
sudo pip3 install tensorrt-5.0.2.6-py2.py3-none-any.whl
#install UFF
cd ../uff
sudo pip3 install uff-0.5.5-py2.py3-none-any.whl
#install graphsurgeon
cd ../graphsurgeon
sudo pip3 install graphsurgeon-0.3.2-py2.py3-none-any.whl
测试TensorRT是否安装成功,进入Python编辑器导入tensorrt和uff模块
import tensorrt
import uff
print(tensorrt.__version__)
print(uff.__version__)
常见报错问题
ImportError: libnvinfer.so.5: cannot open shared object file: Nosuch file or directory
ImportError: libnvonnxparser.so.0: cannot open shared object file:No such file or directory
ImportError: libnvonnxparser_runtime.so.0: cannot open shared objectfile: No such file or directory
ImportError: libnvparsers.so.5: cannot open shared object file: Nosuch file or directory
ImportError: libnvinfer_plugin.so.5: cannot open shared object file:No such file or directory
解决办法 如下
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvinfer.so.5 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvonnxparser.so.0 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvonnxparser_runtime.so.0 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/libnvparsers.so.5 /usr/lib/
sudo cp $HOME/TensorRT-5.0.2.6/targets/x86_64-linux-gnu/lib/ libnvinfer_plugin.so.5/usr/lib/
另外通过命令:which convert-to-uff 可以输出uff的安装路径
运行sample测试下TensorRT是否安装成功
cd TensorRT-5.1.2.2/samples/sampleMNIST
make clean
make
cd /TensorRT-XXX/bin
./sample_mnist
显示如下结果,说明TensorRT已正常安装
[I] Output:
0:
1:
2:
3:
4:
5:
6:
7:
8: **********
9:
&&&& PASSED TensorRT.sample_mnist # ./sample_mnist