DeepStream 6.0 update
This commit is contained in:
unknown
354
readme.md
354
readme.md
@@ -1,83 +1,234 @@
|
||||
# DeepStream-Yolo
|
||||
NVIDIA DeepStream SDK 5.1 configuration for YOLO models
|
||||
|
||||
# New update for DeepStream 6.0 and TensorRT is coming soon, stay tuned!
|
||||
NVIDIA DeepStream SDK 6.0 configuration for YOLO models
|
||||
|
||||
### Future updates (comming soon, stay tuned)
|
||||
|
||||
* New documentation for custom model
|
||||
* New documentation for multiple models
|
||||
* DeepStream tutorials
|
||||
* Native PyTorch support (YOLOv5 and YOLOR)
|
||||
* Native PP-YOLO support
|
||||
* Models benchmark
|
||||
|
||||
**NOTE**: The support for YOLOv5 was removed in this current update. If you want the old repo version, please use the commit 297e0e9 and DeepStream 5.1 requirements.
|
||||
|
||||
### Improvements on this repository
|
||||
|
||||
* Darknet CFG params parser (it doesn't need to edit nvdsparsebbox_Yolo.cpp or another file for native models)
|
||||
* Support for new_coords, beta_nms and scale_x_y params
|
||||
* Support for new models that aren't supported in official DeepStream SDK YOLO.
|
||||
* Support for layers that aren't supported in official DeepStream SDK YOLO.
|
||||
* Support for activations that aren't supported in official DeepStream SDK YOLO.
|
||||
* Support for Convolutional groups
|
||||
* **Support for INT8 calibration** (it isn't available for YOLOv5 models)
|
||||
* **Support for non square models**
|
||||
* Support for new models
|
||||
* Support for new layers types
|
||||
* Support for new activations
|
||||
* Support for convolutional groups
|
||||
* Support for INT8 calibration
|
||||
* Support for non square models
|
||||
|
||||
##
|
||||
|
||||
Tutorial
|
||||
### Getting started
|
||||
|
||||
* [Requirements](#requirements)
|
||||
* [Tested models](#tested-models)
|
||||
* [dGPU installation](#dgpu-installation)
|
||||
* [Basic usage](#basic-usage)
|
||||
* [INT8 calibration](#int8-calibration)
|
||||
* [Configuring to your custom model](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/customModels.md)
|
||||
* [Multiple YOLO inferences](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/multipleInferences.md)
|
||||
|
||||
TensorRT conversion
|
||||
* Native (tested models below)
|
||||
* [YOLOv4x-Mish](https://github.com/AlexeyAB/darknet) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4x-mish.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4x-mish.weights)]
|
||||
* [YOLOv4-CSP](https://github.com/WongKinYiu/ScaledYOLOv4/tree/yolov4-csp) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4-csp.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-csp.weights)]
|
||||
* [YOLOv4](https://github.com/AlexeyAB/darknet) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights)]
|
||||
* [YOLOv4-Tiny](https://github.com/AlexeyAB/darknet) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4-tiny.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights)]
|
||||
* [YOLOv3-SPP](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3-spp.cfg)] [[weights](https://pjreddie.com/media/files/yolov3-spp.weights)]
|
||||
* [YOLOv3](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg)] [[weights](https://pjreddie.com/media/files/yolov3.weights)]
|
||||
* [YOLOv3-Tiny-PRN](https://github.com/WongKinYiu/PartialResidualNetworks) [[cfg](https://raw.githubusercontent.com/WongKinYiu/PartialResidualNetworks/master/cfg/yolov3-tiny-prn.cfg)] [[weights](https://github.com/WongKinYiu/PartialResidualNetworks/raw/master/model/yolov3-tiny-prn.weights)]
|
||||
* [YOLOv3-Tiny](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3-tiny.cfg)] [[weights](https://pjreddie.com/media/files/yolov3-tiny.weights)]
|
||||
* [YOLOv3-Lite](https://github.com/dog-qiuqiu/MobileNet-Yolo) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/MobileNet-Yolo/master/MobileNetV2-YOLOv3-Lite/COCO/MobileNetV2-YOLOv3-Lite-coco.cfg)] [[weights](https://github.com/dog-qiuqiu/MobileNet-Yolo/raw/master/MobileNetV2-YOLOv3-Lite/COCO/MobileNetV2-YOLOv3-Lite-coco.weights)]
|
||||
* [YOLOv3-Nano](https://github.com/dog-qiuqiu/MobileNet-Yolo) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/MobileNet-Yolo/master/MobileNetV2-YOLOv3-Nano/COCO/MobileNetV2-YOLOv3-Nano-coco.cfg)] [[weights](https://github.com/dog-qiuqiu/MobileNet-Yolo/raw/master/MobileNetV2-YOLOv3-Nano/COCO/MobileNetV2-YOLOv3-Nano-coco.weights)]
|
||||
* [YOLO-Fastest 1.1](https://github.com/dog-qiuqiu/Yolo-Fastest) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/Yolo-Fastest/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1-xl.cfg)] [[weights](https://github.com/dog-qiuqiu/Yolo-Fastest/raw/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1-xl.weights)]
|
||||
* [YOLO-Fastest-XL 1.1](https://github.com/dog-qiuqiu/Yolo-Fastest) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/Yolo-Fastest/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1.cfg)] [[weights](https://github.com/dog-qiuqiu/Yolo-Fastest/raw/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1.weights)]
|
||||
* [YOLOv2](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov2.cfg)] [[weights](https://pjreddie.com/media/files/yolov2.weights)]
|
||||
* [YOLOv2-Tiny](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov2-tiny.cfg)] [[weights](https://pjreddie.com/media/files/yolov2-tiny.weights)]
|
||||
|
||||
* External
|
||||
* [YOLOv5 5.0](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/YOLOv5-5.0.md)
|
||||
* [YOLOv5 4.0](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/YOLOv5-4.0.md)
|
||||
* [YOLOv5 3.X (3.0/3.1)](https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/YOLOv5-3.X.md)
|
||||
|
||||
Benchmark
|
||||
* [mAP/FPS comparison between models](#mapfps-comparison-between-models)
|
||||
|
||||
##
|
||||
|
||||
### Requirements
|
||||
* [NVIDIA DeepStream SDK 5.1](https://developer.nvidia.com/deepstream-sdk)
|
||||
* [DeepStream-Yolo Native](https://github.com/marcoslucianops/DeepStream-Yolo/tree/master/native) (for Darknet YOLO based models)
|
||||
* [DeepStream-Yolo External](https://github.com/marcoslucianops/DeepStream-Yolo/tree/master/external) (for PyTorch YOLOv5 based model)
|
||||
|
||||
* [Ubuntu 18.04](https://releases.ubuntu.com/18.04.6/)
|
||||
* [CUDA 11.4.3](https://developer.nvidia.com/cuda-toolkit)
|
||||
* [TensorRT 8.0 GA (8.0.1)](https://developer.nvidia.com/tensorrt)
|
||||
* [cuDNN >= 8.2](https://developer.nvidia.com/cudnn)
|
||||
* [NVIDIA Driver >= 470.63.01](https://www.nvidia.com.br/Download/index.aspx)
|
||||
* [NVIDIA DeepStream SDK 6.0](https://developer.nvidia.com/deepstream-sdk)
|
||||
* [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo)
|
||||
|
||||
##
|
||||
|
||||
### Tested models
|
||||
* [YOLOv4x-Mish](https://github.com/AlexeyAB/darknet) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4x-mish.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4x-mish.weights)]
|
||||
* [YOLOv4-CSP](https://github.com/WongKinYiu/ScaledYOLOv4/tree/yolov4-csp) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4-csp.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-csp.weights)]
|
||||
* [YOLOv4](https://github.com/AlexeyAB/darknet) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights)]
|
||||
* [YOLOv4-Tiny](https://github.com/AlexeyAB/darknet) [[cfg](https://raw.githubusercontent.com/AlexeyAB/darknet/master/cfg/yolov4-tiny.cfg)] [[weights](https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights)]
|
||||
* [YOLOv3-SPP](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3-spp.cfg)] [[weights](https://pjreddie.com/media/files/yolov3-spp.weights)]
|
||||
* [YOLOv3](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg)] [[weights](https://pjreddie.com/media/files/yolov3.weights)]
|
||||
* [YOLOv3-Tiny-PRN](https://github.com/WongKinYiu/PartialResidualNetworks) [[cfg](https://raw.githubusercontent.com/WongKinYiu/PartialResidualNetworks/master/cfg/yolov3-tiny-prn.cfg)] [[weights](https://github.com/WongKinYiu/PartialResidualNetworks/raw/master/model/yolov3-tiny-prn.weights)]
|
||||
* [YOLOv3-Tiny](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3-tiny.cfg)] [[weights](https://pjreddie.com/media/files/yolov3-tiny.weights)]
|
||||
* [YOLOv3-Lite](https://github.com/dog-qiuqiu/MobileNet-Yolo) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/MobileNet-Yolo/master/MobileNetV2-YOLOv3-Lite/COCO/MobileNetV2-YOLOv3-Lite-coco.cfg)] [[weights](https://github.com/dog-qiuqiu/MobileNet-Yolo/raw/master/MobileNetV2-YOLOv3-Lite/COCO/MobileNetV2-YOLOv3-Lite-coco.weights)]
|
||||
* [YOLOv3-Nano](https://github.com/dog-qiuqiu/MobileNet-Yolo) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/MobileNet-Yolo/master/MobileNetV2-YOLOv3-Nano/COCO/MobileNetV2-YOLOv3-Nano-coco.cfg)] [[weights](https://github.com/dog-qiuqiu/MobileNet-Yolo/raw/master/MobileNetV2-YOLOv3-Nano/COCO/MobileNetV2-YOLOv3-Nano-coco.weights)]
|
||||
* [YOLO-Fastest 1.1](https://github.com/dog-qiuqiu/Yolo-Fastest) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/Yolo-Fastest/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1-xl.cfg)] [[weights](https://github.com/dog-qiuqiu/Yolo-Fastest/raw/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1-xl.weights)]
|
||||
* [YOLO-Fastest-XL 1.1](https://github.com/dog-qiuqiu/Yolo-Fastest) [[cfg](https://raw.githubusercontent.com/dog-qiuqiu/Yolo-Fastest/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1.cfg)] [[weights](https://github.com/dog-qiuqiu/Yolo-Fastest/raw/master/ModelZoo/yolo-fastest-1.1_coco/yolo-fastest-1.1.weights)]
|
||||
* [YOLOv2](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov2.cfg)] [[weights](https://pjreddie.com/media/files/yolov2.weights)]
|
||||
* [YOLOv2-Tiny](https://github.com/pjreddie/darknet) [[cfg](https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov2-tiny.cfg)] [[weights](https://pjreddie.com/media/files/yolov2-tiny.weights)]
|
||||
|
||||
##
|
||||
|
||||
### dGPU installation
|
||||
|
||||
To install the DeepStream on dGPU (x86 platform), without docker, we need to do some steps to prepare the computer.
|
||||
|
||||
<details><summary>Open</summary>
|
||||
|
||||
#### 1. Disable Secure Boot in BIOS
|
||||
|
||||
<details><summary>If you are using a laptop with newer Intel/AMD processors, please update the kernel to newer version.</summary>
|
||||
|
||||
```
|
||||
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-headers-5.11.0-051100_5.11.0-051100.202102142330_all.deb
|
||||
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-headers-5.11.0-051100-generic_5.11.0-051100.202102142330_amd64.deb
|
||||
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-image-unsigned-5.11.0-051100-generic_5.11.0-051100.202102142330_amd64.deb
|
||||
wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-modules-5.11.0-051100-generic_5.11.0-051100.202102142330_amd64.deb
|
||||
sudo dpkg -i *.deb
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
#### 2. Install dependencies
|
||||
|
||||
```
|
||||
sudo apt-get install gcc make git libtool autoconf autogen pkg-config cmake
|
||||
sudo apt-get install python3 python3-dev python3-pip
|
||||
sudo apt install libssl1.0.0 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstrtspserver-1.0-0 libjansson4
|
||||
sudo apt-get install linux-headers-$(uname -r)
|
||||
```
|
||||
|
||||
**NOTE**: Install DKMS if you are using the default Ubuntu kernel
|
||||
|
||||
```
|
||||
sudo apt-get install dkms
|
||||
```
|
||||
|
||||
**NOTE**: Purge all NVIDIA driver, CUDA, etc.
|
||||
|
||||
#### 3. Disable Nouveau
|
||||
|
||||
```
|
||||
sudo nano /etc/modprobe.d/blacklist-nouveau.conf
|
||||
```
|
||||
|
||||
* Add
|
||||
|
||||
```
|
||||
blacklist nouveau
|
||||
options nouveau modeset=0
|
||||
```
|
||||
|
||||
* Run
|
||||
|
||||
```
|
||||
sudo update-initramfs -u
|
||||
```
|
||||
|
||||
#### 4. Reboot the computer
|
||||
|
||||
```
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
#### 5. Download and install NVIDIA Driver without xconfig
|
||||
|
||||
```
|
||||
wget https://us.download.nvidia.com/tesla/470.82.01/NVIDIA-Linux-x86_64-470.82.01.run
|
||||
sudo sh NVIDIA-Linux-x86_64-470.82.01.run
|
||||
```
|
||||
|
||||
**NOTE**: If you are using default Ubuntu kernel, enable the DKMS during the installation. Else, you can skip this driver installation and install the NVIDIA driver from CUDA runfile (next step).
|
||||
|
||||
#### 6. Download and install CUDA 11.4.3 without NVIDIA Driver
|
||||
|
||||
```
|
||||
wget https://developer.download.nvidia.com/compute/cuda/11.4.3/local_installers/cuda_11.4.3_470.82.01_linux.run
|
||||
sudo sh cuda_11.4.3_470.82.01_linux.run
|
||||
```
|
||||
|
||||
* Export environment variables
|
||||
|
||||
```
|
||||
nano ~/.bashrc
|
||||
```
|
||||
|
||||
* Add
|
||||
|
||||
```
|
||||
export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}
|
||||
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64\${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
|
||||
```
|
||||
|
||||
* Run
|
||||
|
||||
```
|
||||
source ~/.bashrc
|
||||
sudo ldconfig
|
||||
```
|
||||
|
||||
**NOTE**: If you are using a laptop with NVIDIA Optimius, run
|
||||
|
||||
```
|
||||
sudo apt-get install nvidia-prime
|
||||
sudo prime-select nvidia
|
||||
```
|
||||
|
||||
#### 7. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT 8.0 GA (8.0.1)
|
||||
|
||||
```
|
||||
echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" | sudo tee /etc/apt/sources.list.d/cuda-repo.list
|
||||
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
|
||||
sudo apt-key add 7fa2af80.pub
|
||||
sudo apt-get update
|
||||
sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb
|
||||
sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub
|
||||
sudo apt-get update
|
||||
sudo apt-get install libnvinfer8=8.0.1-1+cuda11.3 libnvinfer-plugin8=8.0.1-1+cuda11.3 libnvparsers8=8.0.1-1+cuda11.3 libnvonnxparsers8=8.0.1-1+cuda11.3 libnvinfer-bin=8.0.1-1+cuda11.3 libnvinfer-dev=8.0.1-1+cuda11.3 libnvinfer-plugin-dev=8.0.1-1+cuda11.3 libnvparsers-dev=8.0.1-1+cuda11.3 libnvonnxparsers-dev=8.0.1-1+cuda11.3 libnvinfer-samples=8.0.1-1+cuda11.3 libnvinfer-doc=8.0.1-1+cuda11.3
|
||||
```
|
||||
|
||||
#### 8. Download from [NVIDIA website](https://developer.nvidia.com/deepstream-sdk) and install the DeepStream SDK 6.0
|
||||
|
||||
```
|
||||
sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb
|
||||
rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin
|
||||
```
|
||||
|
||||
#### 9. Reboot the computer
|
||||
|
||||
```
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
##
|
||||
|
||||
### Basic usage
|
||||
|
||||
#### 1. Download the repo
|
||||
|
||||
```
|
||||
git clone https://github.com/marcoslucianops/DeepStream-Yolo.git
|
||||
cd DeepStream-Yolo/native
|
||||
cd DeepStream-Yolo
|
||||
```
|
||||
|
||||
Download cfg and weights files from your model and move to DeepStream-Yolo/native folder
|
||||
#### 2. Download cfg and weights files from your model and move to DeepStream-Yolo folder
|
||||
|
||||
Compile
|
||||
#### 3. Compile
|
||||
|
||||
* x86 platform
|
||||
|
||||
```
|
||||
CUDA_VER=11.1 make -C nvdsinfer_custom_impl_Yolo
|
||||
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
|
||||
```
|
||||
|
||||
* Jetson platform
|
||||
|
||||
```
|
||||
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo
|
||||
```
|
||||
|
||||
Edit config_infer_primary.txt for your model (example for YOLOv4)
|
||||
#### 4. Edit config_infer_primary.txt for your model (example for YOLOv4)
|
||||
|
||||
```
|
||||
[property]
|
||||
...
|
||||
@@ -103,12 +254,14 @@ num-detected-classes=80
|
||||
pre-cluster-threshold=0.25
|
||||
```
|
||||
|
||||
Run
|
||||
#### 5. Run
|
||||
|
||||
```
|
||||
deepstream-app -c deepstream_app_config.txt
|
||||
```
|
||||
|
||||
If you want to use YOLOv2 or YOLOv2-Tiny models, change, before run, deepstream_app_config.txt
|
||||
**NOTE**: If you want to use YOLOv2 or YOLOv2-Tiny models, change the deepstream_app_config.txt file before run it
|
||||
|
||||
```
|
||||
[primary-gie]
|
||||
enable=1
|
||||
@@ -118,53 +271,63 @@ nvbuf-memory-type=0
|
||||
config-file=config_infer_primary_yoloV2.txt
|
||||
```
|
||||
|
||||
Note: config_infer_primary.txt uses cluster-mode=4 and NMS = 0.45 (via code) when beta_nms isn't available (when beta_nms is available, NMS = beta_nms), while config_infer_primary_yoloV2.txt uses cluster-mode=2 and nms-iou-threshold=0.45 to set NMS.
|
||||
**NOTE**: The config_infer_primary.txt file uses cluster-mode=4 and NMS = 0.45 (via code) when beta_nms isn't available (when beta_nms is available, NMS = beta_nms), while the config_infer_primary_yoloV2.txt file uses cluster-mode=2 and nms-iou-threshold=0.45 to set NMS.
|
||||
|
||||
##
|
||||
|
||||
### INT8 calibration
|
||||
|
||||
Install OpenCV
|
||||
#### 1. Install OpenCV
|
||||
|
||||
```
|
||||
sudo apt-get install libopencv-dev
|
||||
```
|
||||
|
||||
Compile/recompile the nvdsinfer_custom_impl_Yolo lib with OpenCV support
|
||||
#### 2. Compile/recompile the nvdsinfer_custom_impl_Yolo lib with OpenCV support
|
||||
|
||||
* x86 platform
|
||||
|
||||
```
|
||||
cd DeepStream-Yolo/native
|
||||
CUDA_VER=11.1 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
|
||||
cd DeepStream-Yolo
|
||||
CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
|
||||
```
|
||||
|
||||
* Jetson platform
|
||||
|
||||
```
|
||||
cd DeepStream-Yolo/native
|
||||
cd DeepStream-Yolo
|
||||
CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
|
||||
```
|
||||
|
||||
For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to DeepStream-Yolo/native folder
|
||||
#### 3. For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to DeepStream-Yolo folder
|
||||
|
||||
##### Select 1000 random images from COCO dataset to run calibration
|
||||
|
||||
Select 1000 random images from COCO dataset to run calibration
|
||||
```
|
||||
mkdir calibration
|
||||
```
|
||||
|
||||
```
|
||||
for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \
|
||||
cp val2017/${jpg} calibration/; \
|
||||
cp ${jpg} calibration/; \
|
||||
done
|
||||
```
|
||||
|
||||
Create the calibration.txt file with all selected images
|
||||
##### Create the calibration.txt file with all selected images
|
||||
|
||||
```
|
||||
realpath calibration/*jpg > calibration.txt
|
||||
```
|
||||
|
||||
Set environment variables
|
||||
##### Set environment variables
|
||||
|
||||
```
|
||||
export INT8_CALIB_IMG_PATH=calibration.txt
|
||||
export INT8_CALIB_BATCH_SIZE=1
|
||||
```
|
||||
|
||||
Change config_infer_primary.txt file
|
||||
##### Change config_infer_primary.txt file
|
||||
|
||||
```
|
||||
...
|
||||
model-engine-file=model_b1_gpu0_fp32.engine
|
||||
@@ -173,7 +336,9 @@ model-engine-file=model_b1_gpu0_fp32.engine
|
||||
network-mode=0
|
||||
...
|
||||
```
|
||||
To
|
||||
|
||||
* To
|
||||
|
||||
```
|
||||
...
|
||||
model-engine-file=model_b1_gpu0_int8.engine
|
||||
@@ -183,70 +348,13 @@ network-mode=1
|
||||
...
|
||||
```
|
||||
|
||||
Run
|
||||
##### Run
|
||||
|
||||
```
|
||||
deepstream-app -c deepstream_app_config.txt
|
||||
```
|
||||
|
||||
Note: NVIDIA recommends at least 500 images to get a good accuracy. In this example I used 1000 images to get better accuracy (more images = more accuracy). Higher INT8_CALIB_BATCH_SIZE values will increase the accuracy and calibration speed. Set it according to you GPU memory. This process can take a long time. The calibration isn't available for YOLOv5 models.
|
||||
|
||||
##
|
||||
|
||||
### mAP/FPS comparison between models
|
||||
|
||||
<details><summary>Open</summary>
|
||||
|
||||
```
|
||||
valid = val2017 (COCO)
|
||||
NMS = 0.45 (changed to beta_nms when used in Darknet cfg file) / 0.6 (YOLOv5 models)
|
||||
pre-cluster-threshold = 0.001 (mAP eval) / 0.25 (FPS measurement)
|
||||
batch-size = 1
|
||||
FPS measurement display width = 1920
|
||||
FPS measurement display height = 1080
|
||||
NOTE: Used NVIDIA GTX 1050 (4GB Mobile) for evaluate. Used maintain-aspect-ratio=1 in config_infer file for YOLOv4 (with letter_box=1) and YOLOv5 models. For INT8 calibration, was used 1000 random images from val2017 (COCO) and INT8_CALIB_BATCH_SIZE=1.
|
||||
```
|
||||
|
||||
| TensorRT | Precision | Resolution | IoU=0.5:0.95 | IoU=0.5 | IoU=0.75 | FPS<br />(with display) | FPS<br />(without display) |
|
||||
|:---------------:|:---------:|:----------:|:------------:|:-------:|:--------:|:-----------------------:|:--------------------------:|
|
||||
| YOLOv5x 5.0 | FP32 | 640 | 0. | 0. | 0. | . | . |
|
||||
| YOLOv5l 5.0 | FP32 | 640 | 0. | 0. | 0. | . | . |
|
||||
| YOLOv5m 5.0 | FP32 | 640 | 0. | 0. | 0. | . | . |
|
||||
| YOLOv5s 5.0 | FP32 | 640 | 0. | 0. | 0. | . | . |
|
||||
| YOLOv5s 5.0 | FP32 | 416 | 0. | 0. | 0. | . | . |
|
||||
| YOLOv4x-MISH | FP32 | 640 | 0.461 | 0.649 | 0.499 | . | . |
|
||||
| YOLOv4x-MISH | **INT8** | 640 | 0.443 | 0.629 | 0.479 | . | . |
|
||||
| YOLOv4x-MISH | FP32 | 608 | 0.461 | 0.650 | 0.496 | . | . |
|
||||
| YOLOv4-CSP | FP32 | 640 | 0.443 | 0.632 | 0.477 | . | . |
|
||||
| YOLOv4-CSP | FP32 | 608 | 0.443 | 0.632 | 0.477 | . | . |
|
||||
| YOLOv4-CSP | FP32 | 512 | 0.437 | 0.625 | 0.471 | . | . |
|
||||
| YOLOv4-CSP | **INT8** | 512 | 0.414 | 0.601 | 0.447 | . | . |
|
||||
| YOLOv4 | FP32 | 640 | 0.492 | 0.729 | 0.547 | . | . |
|
||||
| YOLOv4 | FP32 | 608 | 0.499 | 0.739 | 0.551 | . | . |
|
||||
| YOLOv4 | **INT8** | 608 | 0.483 | 0.728 | 0.534 | . | . |
|
||||
| YOLOv4 | FP32 | 512 | 0.492 | 0.730 | 0.542 | . | . |
|
||||
| YOLOv4 | FP32 | 416 | 0.468 | 0.702 | 0.507 | . | . |
|
||||
| YOLOv3-SPP | FP32 | 608 | 0.412 | 0.687 | 0.434 | . | . |
|
||||
| YOLOv3 | FP32 | 608 | 0.378 | 0.674 | 0.389 | . | . |
|
||||
| YOLOv3 | **INT8** | 608 | 0.381 | 0.677 | 0.388 | . | . |
|
||||
| YOLOv3 | FP32 | 416 | 0.373 | 0.669 | 0.379 | . | . |
|
||||
| YOLOv2 | FP32 | 608 | 0.211 | 0.365 | 0.220 | . | . |
|
||||
| YOLOv2 | FP32 | 416 | 0.207 | 0.362 | 0.211 | . | . |
|
||||
| YOLOv4-Tiny | FP32 | 416 | 0.216 | 0.403 | 0.207 | . | . |
|
||||
| YOLOv4-Tiny | **INT8** | 416 | 0.203 | 0.385 | 0.192 | . | . |
|
||||
| YOLOv3-Tiny-PRN | FP32 | 416 | 0.168 | 0.381 | 0.126 | . | . |
|
||||
| YOLOv3-Tiny-PRN | **INT8** | 416 | 0.155 | 0.358 | 0.113 | . | . |
|
||||
| YOLOv3-Tiny | FP32 | 416 | 0.096 | 0.203 | 0.080 | . | . |
|
||||
| YOLOv2-Tiny | FP32 | 416 | 0.084 | 0.194 | 0.062 | . | . |
|
||||
| YOLOv3-Lite | FP32 | 416 | 0.169 | 0.356 | 0.137 | . | . |
|
||||
| YOLOv3-Lite | FP32 | 320 | 0.158 | 0.328 | 0.132 | . | . |
|
||||
| YOLOv3-Nano | FP32 | 416 | 0.128 | 0.278 | 0.099 | . | . |
|
||||
| YOLOv3-Nano | FP32 | 320 | 0.122 | 0.260 | 0.099 | . | . |
|
||||
| YOLO-Fastest-XL | FP32 | 416 | 0.160 | 0.342 | 0.130 | . | . |
|
||||
| YOLO-Fastest-XL | FP32 | 320 | 0.158 | 0.329 | 0.135 | . | . |
|
||||
| YOLO-Fastest | FP32 | 416 | 0.101 | 0.230 | 0.072 | . | . |
|
||||
| YOLO-Fastest | FP32 | 320 | 0.102 | 0.232 | 0.073 | . | . |
|
||||
|
||||
</details>
|
||||
**NOTE**: NVIDIA recommends at least 500 images to get a good accuracy. In this example I used 1000 images to get better accuracy (more images = more accuracy). Higher INT8_CALIB_BATCH_SIZE values will increase the accuracy and calibration speed. Set it according to you GPU memory. This process can take a long time.
|
||||
|
||||
##
|
||||
|
||||
@@ -254,15 +362,11 @@ NOTE: Used NVIDIA GTX 1050 (4GB Mobile) for evaluate. Used maintain-aspect-ratio
|
||||
|
||||
You can get metadata from deepstream in Python and C++. For C++, you need edit deepstream-app or deepstream-test code. For Python your need install and edit [deepstream_python_apps](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps).
|
||||
|
||||
You need manipulate NvDsObjectMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html)/[C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/Meta/_NvDsObjectMeta.html)), NvDsFrameMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html)/[C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/Meta/_NvDsFrameMeta.html)) and NvOSD_RectParams ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsOSD/NvOSD_RectParams.html)/[C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/OSD/Data_Structures/_NvOSD_FrameRectParams.html)) to get label, position, etc. of bboxs.
|
||||
You need manipulate NvDsObjectMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html)/[C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsObjectMeta.html)), NvDsFrameMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html)/[C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsFrameMeta.html)) and NvOSD_RectParams ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvOSD/NvOSD_RectParams.html)/[C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvOSD__RectParams.html)) to get label, position, etc. of bboxes.
|
||||
|
||||
In C++ deepstream-app application, your code need be in analytics_done_buf_prob function.
|
||||
In C++/Python deepstream-test application, your code need be in osd_sink_pad_buffer_probe/tiler_src_pad_buffer_probe function.
|
||||
|
||||
Python is slightly slower than C (about 5-10%).
|
||||
|
||||
##
|
||||
|
||||
This code is open-source. You can use as you want. :)
|
||||
|
||||
My projects: https://www.youtube.com/MarcosLucianoTV
|
||||
My projects: https://www.youtube.com/MarcosLucianoTV (new videos and tutorials comming soon)
|
||||
|
||||
Reference in New Issue
Block a user