本文已參與「新人創作禮」活動，一起開啓掘金創作之路。
一、參考資料
Jetson 系列——基於yolov5和deepsort的多目標頭部識別，跟蹤，使用tensorrt和c++加速
二、相關介紹
2.1 重要說明
==該項目能部署在Jetson系列的產品，也能部署在X86 服務器中。==
2.2 項目結構
.
├── assets
│ └── yolosort.gif
├── build # 編譯的文件夾
│ ├── CMakeCache.txt
│ ├── CMakeFiles
│ ├── cmake_install.cmake
│ ├── libdeepsort.so
│ ├── libyolov5_trt.so
│ ├── Makefile
│ └── yolosort
├── CMakeLists_deepsort-tensorrt_win10.txt
├── CMakeLists.txt
├── CMakeLists_yolov5-deepsort-tensorrt_win10.txt
├── deepsort # deepsort源碼
│ ├── include
│ └── src
├── resources # 存放engine引擎的文件夾
├── include
│ └── manager.hpp
├── LICENSE
├── README.md
├── requirementes-gpu.txt
├── src
│ ├── main.cpp # 入口函數
│ └── manager.cpp
└── yolo # yolo源碼
├── include
└── src
複製代碼2.3 主項目
2.4 生成 onnx 文件
2.5 生成 deepsort.engine
2.6 生成 yolov5s.engine
TensorRT實現yolov5推理加速（一） tensorrtx/yolov5
三、實驗環境
3.1 系統環境
由於博主在PC主機上測試部分代碼，以下實驗環境爲PC主機的環境，以後有條件，我再測試一下Jetson TX2的效果。
Environment
Operating System + Version: Ubuntu + 16.04
GPU Type: GeForce GTX1650,4GB
Nvidia Driver Version: 470.63.01
CUDA Version: 10.2.300
CUDNN Version: 7.6.5
Python Version (if applicable): 3.6.14
virtualenv：20.13.0
gcc：7.5.0
g++：7.5.0
複製代碼3.2 requirements-gpu.txt
absl-py==1.0.0
appdirs==1.4.4
backcall==0.2.0
cached-property==1.5.2
cachetools==4.2.4
certifi==2021.10.8
chardet==4.0.0
charset-normalizer==2.0.10
cycler==0.11.0
Cython==0.29.26
dataclasses==0.8
decorator==4.4.2
distlib==0.3.4
easydict==1.9
filelock==3.4.1
flake8==4.0.1
future==0.18.2
gdown==3.10.1
google-auth==2.3.3
google-auth-oauthlib==0.4.6
graphsurgeon @ file:///home/yichao/360Downloads/TensorRT-7.1.3.4/graphsurgeon/graphsurgeon-0.4.5-py2.py3-none-any.whl
grpcio==1.43.0
h5py==3.1.0
idna==3.3
imageio==2.13.5
importlib-metadata==4.2.0
importlib-resources==5.4.0
imutil==0.2.5
ipdb==0.13.9
ipython==7.16.3
ipython-genutils==0.2.0
isort==4.3.21
jedi==0.17.2
kiwisolver==1.3.1
Mako==1.1.6
Markdown==3.3.5
MarkupSafe==2.0.1
matplotlib==3.3.4
mccabe==0.6.1
networkx==2.5.1
numpy==1.19.5
oauthlib==3.1.1
opencv-python==4.5.5.62
pandas==1.1.5
parso==0.7.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.4.0
platformdirs==2.4.0
prompt-toolkit==3.0.24
protobuf==3.19.3
ptyprocess==0.7.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycodestyle==2.8.0
pycuda==2020.1
pyflakes==2.4.0
Pygments==2.11.2
pyparsing==3.0.6
PySocks==1.7.1
python-dateutil==2.8.2
pytools==2021.2.9
pytz==2021.3
PyWavelets==1.1.1
PyYAML==6.0
requests==2.27.1
requests-oauthlib==1.3.0
rsa==4.8
scikit-image==0.17.2
scipy==1.5.4
seaborn==0.11.2
six==1.16.0
tb-nightly==2.8.0a20220117
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorrt @ file:///home/yichao/360Downloads/TensorRT-7.1.3.4/python/tensorrt-7.1.3.4-cp36-none-linux_x86_64.whl
tifffile==2020.9.3
toml==0.10.2
torch @ file:///home/yichao/%E4%B8%8B%E8%BD%BD/torch-1.9.0%2Bcu102-cp36-cp36m-linux_x86_64.whl
torchvision @ file:///home/yichao/%E4%B8%8B%E8%BD%BD/torchvision-0.10.0%2Bcu102-cp36-cp36m-linux_x86_64.whl
tqdm==4.62.3
traitlets==4.3.3
typing_extensions==4.0.1
urllib3==1.26.8
virtualenv==20.13.0
wcwidth==0.2.5
Werkzeug==2.0.2
yacs==0.1.8
yapf==0.32.0
zipp==3.6.0
複製代碼四、生成 onnx 文件
4.1 下載github代碼
git clone https://github.com/RichardoMrMu/deepsort-tensorrt.git
複製代碼4.2 下載預訓練模型
ckpt.t7，46MB
下載完成後，將ckpt.t7放到 deep_sort_pytorch/deep_sort/deep/checkpoint/ckpt.t7 路徑。
4.3 生成 onnx 文件
# 拷貝文件
cp {deepsort-tensorrt}/exportOnnx.py {deep_sort_pytorch}/
cd deep_sort_pytorch
# 生成onnx文件
python3 exportOnnx.py
# 拷貝onnx文件到對應目錄
mv {deep_sort_pytorch}/deepsort.onnx {deepsort-tensorrt}/resources
複製代碼(venv) yichao@yichao:~/下載/deep_sort_pytorch$ time python exportOnnx.py
==> Exporting model to ONNX format at 'deepsort.onnx'
/home/yichao/MyDocuments/Yolov5_DeepSort_Pytorch/venv/lib/python3.6/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
real 0m26.103s
user 0m7.749s
sys 0m2.463s
deepsort.onnx，44.7MB
複製代碼4.3.1 顯卡佔用情況
Fri Jan 21 15:03:09 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |
| 27% 30C P0 17W / 75W | 1419MiB / 3903MiB | 2% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1481 G /usr/lib/xorg/Xorg 264MiB |
| 0 N/A N/A 10705 C python 1151MiB |
+-----------------------------------------------------------------------------+
複製代碼五、生成 deepsort.engine
5.1 拷貝onnx文件到對應目錄
mv {deep_sort_pytorch}/deepsort.onnx {deepsort-tensorrt}/resources
複製代碼5.2 生成 CMakeLists.txt 文件
cd {deepsort-tensorrt}
# cmake
mkdir build
cd build
cmake ..
複製代碼5.3 修改 CMakeLists.txt 文件
deepsort-tensorrt/CMakeLists.txt 添加tensorRT庫。
# tensorRT
include_directories(/home/yichao/360Downloads/TensorRT-7.1.3.4/include/)
link_directories(/home/yichao/360Downloads/TensorRT-7.1.3.4/lib/)
複製代碼deepsort-tensorrt/CMakeLists.txt 添加Eigen庫。
include_directories(/usr/include/eigen3)
複製代碼5.4 編譯
make -j8
複製代碼5.5 生成engine引擎
deepsort.onnx 的路徑：/home/yichao/下載/deepsort-tensorrt/resources/deepsort.onnx。 deepsort.engine 的路徑：/home/yichao/下載/deepsort-tensorrt/resources/deepsort.engine。
./onnx2engine ../resources/deepsort.onnx ../resources/deepsort.engine
複製代碼(venv) yichao@yichao:~/下載/deepsort-tensorrt/build$ time ./onnx2engine ../resources/deepsort.onnx ../resources/deepsort.engine
----------------------------------------------------------------
Input filename: ../resources/deepsort.onnx
ONNX IR version: 0.0.6
Opset version: 10
Producer name: pytorch
Producer version: 1.9
Domain:
Model version: 0
Doc string:
----------------------------------------------------------------
[01/21/2022-15:10:32] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
==============
| SUCCESS! |
==============
real 1m58.582s
user 0m27.711s
sys 0m3.474s
deepsort.engine，24.6MB
複製代碼5.5.1 顯存佔用情況
Fri Jan 21 15:11:04 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |
| 27% 34C P0 44W / 75W | 1255MiB / 3903MiB | 100% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1481 G /usr/lib/xorg/Xorg 264MiB |
| 0 N/A N/A 11074 C ./onnx2engine 987MiB |
+-----------------------------------------------------------------------------+
複製代碼5.6 測試引擎是否生成
./demo ../resource/deepsort.engine ../resources/track.txt
複製代碼六、主項目
6.1 下載github代碼
# 下載github代碼
git clone https://github.com/RichardoMrMu/yolov5-deepsort-tensorrt.git
複製代碼6.2 拷貝engine到對應目錄
yolov5-deepsort-tensorrt/engine/yolov5s.engine
yolov5-deepsort-tensorrt/engine/deepsort.engine
複製代碼6.3 修改配置
yolov5-deepsort-tensorrt/src/main.cpp，修改engine的路徑。
// before you cmake and make, please change ./src/main.cpp char* yolo_engine = ""; char* sort_engine = ""; to your own path
char* yolo_engine = "yolov5-deepsort-tensorrt/engine/yolov5s.engine";
char* sort_engine = "yolov5-deepsort-tensorrt/engine/deepsort.engine";
// 設置攝像頭索引
frame = capture.open(0);
// 顯示檢測結果
// 在 yolov5-deepsort-tensorrt/src/manager.cpp 文件中，取消註釋
showDetection(frame,det); # 取消註釋
複製代碼6.4 生成 CMakeLists.txt 文件
cd yolov5-deepsort-tensorrt
# cmake
mkdir build
cd build
cmake ..
複製代碼6.5 修改 CMakeLists.txt 文件
deepsort-tensorrt/CMakeLists.txt 添加tensorRT庫。
# tensorRT
include_directories(/home/yichao/360Downloads/TensorRT-7.1.3.4/include)
link_directories(/home/yichao/360Downloads/TensorRT-7.1.3.4/lib/)
複製代碼deepsort-tensorrt/CMakeLists.txt 添加Eigen庫。
include_directories(/usr/include/eigen3)
複製代碼6.6 編譯
make -j8
複製代碼6.7 運行
# 運行
./yolosort
複製代碼(venv) yichao@yichao:~/下載/yolov5-deepsort-tensorrt/build$ ./yolosort
yolov5_trt_create ...
yolov5_trt_create cuda engine...
yolov5_trt_create buffer ...
yolov5_trt_create stream ...
yolov5_trt_create done ...
create yolov5-trt , instance = 0x25a10a0
delay_proress:3ms, delay_infer:7ms
delay_infer:33ms
delay_proress:6ms, delay_infer:6ms
delay_infer:21ms
delay_proress:3ms, delay_infer:6ms
delay_infer:16ms
delay_proress:12ms, delay_infer:7ms
delay_infer:29ms
delay_proress:4ms, delay_infer:7ms
delay_infer:19ms
delay_proress:12ms, delay_infer:7ms
delay_infer:30ms
delay_proress:10ms, delay_infer:7ms
delay_infer:26ms
delay_proress:5ms, delay_infer:7ms
delay_infer:18ms
...
複製代碼delay_proress：yolov5預處理時間； delay_infer：yolov5目標檢測時間； delay_infer：總共的時間。
6.7.1 顯存佔用情況
Fri Jan 21 14:37:18 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01 Driver Version: 470.63.01 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:01:00.0 On | N/A |
| 27% 31C P0 18W / 75W | 1082MiB / 3903MiB | 16% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1481 G /usr/lib/xorg/Xorg 264MiB |
| 0 N/A N/A 9950 C ./yolosort 814MiB |
+-----------------------------------------------------------------------------+
複製代碼七、可能存在的問題
Q1：yaml.load()函數缺少參數錯誤
運行 python exportOnnx.py 出錯。
(venv) yichao@yichao:~/下載/deep_sort_pytorch$ python exportOnnx.py
Traceback (most recent call last):
File "exportOnnx.py", line 22, in <module>
cfg.merge_from_file(args.config_deepsort)
File "/home/yichao/下載/deep_sort_pytorch/utils/parser.py", line 23, in merge_from_file
self.update(yaml.load(fo.read()))
TypeError: load() missing 1 required positional argument: 'Loader'
複製代碼錯誤原因：
yaml.load() 函數的參數不對，缺少 Loader 對象。
yaml.load(CLIN,Loader=yaml.FullLoader) # 加上Loader=yaml.FullLoader 避免警告。
注：yaml版本5.1之後棄用，YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated.
解決辦法：
修改 deep_sort_pytorch/utils/parser.py 中的代碼。
cfg_dict.update(yaml.load(fo.read()))
改成
cfg_dict.update(yaml.load(fo, Loader=yaml.FullLoader))
複製代碼Q2：缺少 NInfer.h 文件
In file included from /home/yichao/下載/deepsort-tensorrt/include/deepsort.h:6:0,
from /home/yichao/下載/deepsort-tensorrt/src/deepsort.cpp:1:
/home/yichao/下載/deepsort-tensorrt/include/featuretensor.h:7:10: fatal error: NvInfer.h: 沒有那個文件或目錄
#include <NvInfer.h>
^~~~~~~~~~~
compilation terminated.
CMakeFiles/deepsort.dir/build.make:75: recipe for target 'CMakeFiles/deepsort.dir/src/deepsort.cpp.o' failed
make[2]: *** [CMakeFiles/deepsort.dir/src/deepsort.cpp.o] Error 1
CMakeFiles/deepsort.dir/build.make:103: recipe for target 'CMakeFiles/deepsort.dir/src/featuretensor.cpp.o' failed
make[2]: *** [CMakeFiles/deepsort.dir/src/featuretensor.cpp.o] Error 1
CMakeFiles/Makefile2:86: recipe for target 'CMakeFiles/deepsort.dir/all' failed
make[1]: *** [CMakeFiles/deepsort.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
複製代碼錯誤原因：
NvInfer.h 頭文件屬於 TensorRT 下的一個專有頭文件，在編譯C++ 代碼時需要找到它。
解決辦法：
deepsort-tensorrt/CMakeLists.txt，增加tensorRT的依賴庫
# tensorRT
include_directories(/home/yichao/360Downloads/TensorRT-7.1.3.4/include)
link_directories(/home/yichao/360Downloads/TensorRT-7.1.3.4/lib/)
複製代碼Q3：缺少 Eigen/Core 文件
/home/yichao/360Downloads/TensorRT-7.1.3.4/include/NvInfer.h:3339:22: note: declared here
class TRT_DEPRECATED IOutputDimensionsFormula
^~~~~~~~~~~~~~~~~~~~~~~~
/home/yichao/360Downloads/TensorRT-7.1.3.4/include/NvInfer.h:5565:81: warning: ‘IPluginLayer’ is deprecated [-Wdeprecated-declarations]
ITensor* const* inputs, int nbInputs, IPluginExt& plugin) TRTNOEXCEPT = 0;
^
/home/yichao/360Downloads/TensorRT-7.1.3.4/include/NvInfer.h:3373:22: note: declared here
class TRT_DEPRECATED IPluginLayer : public ILayer
^~~~~~~~~~~~
CMakeFiles/Makefile2:86: recipe for target 'CMakeFiles/deepsort.dir/all' failed
make[1]: *** [CMakeFiles/deepsort.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
複製代碼解決辦法：重新make
make -j8
複製代碼/home/yichao/下載/deepsort-tensorrt/include/datatype.h:28:10: fatal error: Eigen/Core: 沒有那個文件或目錄
#include <Eigen/Core>
^~~~~~~~~~~~
compilation terminated.
CMakeFiles/deepsort.dir/build.make:75: recipe for target 'CMakeFiles/deepsort.dir/src/deepsort.cpp.o' failed
make[2]: *** [CMakeFiles/deepsort.dir/src/deepsort.cpp.o] Error 1
CMakeFiles/Makefile2:86: recipe for target 'CMakeFiles/deepsort.dir/all' failed
make[1]: *** [CMakeFiles/deepsort.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
複製代碼錯誤原因：
編譯找不到Eigen
解決辦法：
1. 安裝eigen3，默認安裝在 /usr/include/eigen3
sudo apt-get install libeigen3-dev
2. deepsort-tensorrt/CMakeLists.txt，增加eigen3的依賴庫
include_directories(/usr/include/eigen3)
# 如果還不成功，嘗試創建軟鏈接
sudo ln -s /usr/include/eigen3/Eigen /usr/include/Eigen
3. 重新編譯
make clean
make -j8
複製代碼Q4：打不開攝像頭
(venv) yichao@yichao:~/下載/yolov5-deepsort-tensorrt/build$ ./yolosort
yolov5_trt_create ...
yolov5_trt_create cuda engine...
yolov5_trt_create buffer ...
yolov5_trt_create stream ...
yolov5_trt_create done ...
create yolov5-trt , instance = 0x295e000
GStreamer: Error opening bin: no element "0"
can not open
複製代碼(venv) yichao@yichao:~/下載/yolov5-deepsort-tensorrt/build$ ./yolosort
yolov5_trt_create ...
yolov5_trt_create cuda engine...
yolov5_trt_create buffer ...
yolov5_trt_create stream ...
yolov5_trt_create done ...
create yolov5-trt , instance = 0x186d000
GStreamer: Error opening bin: empty pipeline not allowed
can not open
複製代碼錯誤原因：
在 yolov5-deepsort-tensorrt/src/main.cpp 文件中，未設置攝像頭索引或者設置攝像頭失敗。
frame = capture.open(""); // 未設置索引
frame = capture.open("0"); // 設置失敗，索引爲int類型
解決辦法：
在 yolov5-deepsort-tensorrt/src/main.cpp 文件中，設置攝像頭索引。
frame = capture.open(0);from：https://juejin.cn/post/7103907324682567694