# DeepStream-Yolo NVIDIA DeepStream SDK 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 configuration for YOLO models ### **I will be back with updates soon, I'm full of work from my jobs right now. Sorry for the delay.** ### Future updates * DeepStream tutorials * Dynamic batch-size * Segmentation model support * Classification model support ### Improvements on this repository * Darknet cfg params parser (no need to edit `nvdsparsebbox_Yolo.cpp` or other files) * Support for `new_coords` and `scale_x_y` params * Support for new models * Support for new layers * Support for new activations * Support for convolutional groups * Support for INT8 calibration * Support for non square models * New documentation for multiple models * YOLOv5 >= 2.0 support * YOLOR support * GPU YOLO Decoder [#138](https://github.com/marcoslucianops/DeepStream-Yolo/issues/138) * PP-YOLOE support * YOLOv7 support * Optimized NMS [#142](https://github.com/marcoslucianops/DeepStream-Yolo/issues/142) * Models benchmarks * YOLOv8 support * YOLOX support * PP-YOLOE+ support * YOLOv6 >= 2.0 support * **ONNX model support with GPU post-processing** * **YOLO-NAS support (ONNX)** ## ### Getting started * [Requirements](#requirements) * [Suported models](#supported-models) * [Benchmarks](#benchmarks) * [dGPU installation](#dgpu-installation) * [Basic usage](#basic-usage) * [Docker usage](#docker-usage) * [NMS configuration](#nms-configuration) * [INT8 calibration](#int8-calibration) * [YOLOv5 usage](docs/YOLOv5.md) * [YOLOv6 usage](docs/YOLOv6.md) * [YOLOv7 usage](docs/YOLOv7.md) * [YOLOv8 usage](docs/YOLOv8.md) * [YOLOR usage](docs/YOLOR.md) * [YOLOX usage](docs/YOLOX.md) * [PP-YOLOE / PP-YOLOE+ usage](docs/PPYOLOE.md) * [Using your custom model](docs/customModels.md) * [Multiple YOLO GIEs](docs/multipleGIEs.md) ## ### Requirements #### DeepStream 6.2 on x86 platform * [Ubuntu 20.04](https://releases.ubuntu.com/20.04/) * [CUDA 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=runfile_local) * [TensorRT 8.5 GA Update 1 (8.5.2.2)](https://developer.nvidia.com/nvidia-tensorrt-8x-download) * [NVIDIA Driver 525.85.12 (Data center / Tesla series) / 525.105.17 (TITAN, GeForce RTX / GTX series and RTX / Quadro series)](https://www.nvidia.com.br/Download/index.aspx) * [NVIDIA DeepStream SDK 6.2](https://developer.nvidia.com/deepstream-getting-started) * [GStreamer 1.16.3](https://gstreamer.freedesktop.org/) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.1.1 on x86 platform * [Ubuntu 20.04](https://releases.ubuntu.com/20.04/) * [CUDA 11.7 Update 1](https://developer.nvidia.com/cuda-11-7-1-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=runfile_local) * [TensorRT 8.4 GA (8.4.1.5)](https://developer.nvidia.com/nvidia-tensorrt-8x-download) * [NVIDIA Driver 515.65.01](https://www.nvidia.com.br/Download/index.aspx) * [NVIDIA DeepStream SDK 6.1.1](https://developer.nvidia.com/deepstream-sdk-download-tesla-archived) * [GStreamer 1.16.2](https://gstreamer.freedesktop.org/) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.1 on x86 platform * [Ubuntu 20.04](https://releases.ubuntu.com/20.04/) * [CUDA 11.6 Update 1](https://developer.nvidia.com/cuda-11-6-1-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=runfile_local) * [TensorRT 8.2 GA Update 4 (8.2.5.1)](https://developer.nvidia.com/nvidia-tensorrt-8x-download) * [NVIDIA Driver 510.47.03](https://www.nvidia.com.br/Download/index.aspx) * [NVIDIA DeepStream SDK 6.1](https://developer.nvidia.com/deepstream-sdk-download-tesla-archived) * [GStreamer 1.16.2](https://gstreamer.freedesktop.org/) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.0.1 / 6.0 on x86 platform * [Ubuntu 18.04](https://releases.ubuntu.com/18.04.6/) * [CUDA 11.4 Update 1](https://developer.nvidia.com/cuda-11-4-1-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=18.04&target_type=runfile_local) * [TensorRT 8.0 GA (8.0.1)](https://developer.nvidia.com/nvidia-tensorrt-8x-download) * [NVIDIA Driver 470.63.01](https://www.nvidia.com.br/Download/index.aspx) * [NVIDIA DeepStream SDK 6.0.1 / 6.0](https://developer.nvidia.com/deepstream-sdk-download-tesla-archived) * [GStreamer 1.14.5](https://gstreamer.freedesktop.org/) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.2 on Jetson platform * [JetPack 5.1.1 / 5.1](https://developer.nvidia.com/embedded/jetpack) * [NVIDIA DeepStream SDK 6.2](https://developer.nvidia.com/deepstream-sdk) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.1.1 on Jetson platform * [JetPack 5.0.2](https://developer.nvidia.com/embedded/jetpack-sdk-502) * [NVIDIA DeepStream SDK 6.1.1](https://developer.nvidia.com/embedded/deepstream-on-jetson-downloads-archived) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.1 on Jetson platform * [JetPack 5.0.1 DP](https://developer.nvidia.com/embedded/jetpack-sdk-501dp) * [NVIDIA DeepStream SDK 6.1](https://developer.nvidia.com/embedded/deepstream-on-jetson-downloads-archived) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) #### DeepStream 6.0.1 / 6.0 on Jetson platform * [JetPack 4.6.2](https://developer.nvidia.com/embedded/jetpack-sdk-462) * [NVIDIA DeepStream SDK 6.0.1 / 6.0](https://developer.nvidia.com/embedded/deepstream-on-jetson-downloads-archived) * [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) ## ### Suported models * [Darknet YOLO](https://github.com/AlexeyAB/darknet) * [MobileNet-YOLO](https://github.com/dog-qiuqiu/MobileNet-Yolo) * [YOLO-Fastest](https://github.com/dog-qiuqiu/Yolo-Fastest) * [YOLOv5 >= 2.0](https://github.com/ultralytics/yolov5) * [YOLOv6 >= 2.0](https://github.com/meituan/YOLOv6) * [YOLOv7](https://github.com/WongKinYiu/yolov7) * [YOLOv8](https://github.com/ultralytics/ultralytics) * [YOLOR](https://github.com/WongKinYiu/yolor) * [YOLOX](https://github.com/Megvii-BaseDetection/YOLOX) * [PP-YOLOE / PP-YOLOE+](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5/configs/ppyoloe) ## ### Benchmarks #### Config ``` board = NVIDIA Tesla V100 16GB (AWS: p3.2xlarge) batch-size = 1 eval = val2017 (COCO) sample = 1920x1080 video ``` **NOTE**: Used maintain-aspect-ratio=1 in config_infer file for Darknet (with letter_box=1) and PyTorch models. #### NMS config - Eval ``` nms-iou-threshold = 0.6 (Darknet and YOLOv8) / 0.65 (YOLOv5, YOLOv6, YOLOv7, YOLOR and YOLOX) / 0.7 (Paddle) pre-cluster-threshold = 0.001 topk = 300 ``` - Test ``` nms-iou-threshold = 0.45 / 0.7 (Paddle) pre-cluster-threshold = 0.25 topk = 300 ``` #### Results **NOTE**: * = PyTorch **NOTE**: ** = The YOLOv4 is trained with the trainvalno5k set, so the mAP is high on val2017 test | DeepStream | Precision | Resolution | IoU=0.5:0.95 | IoU=0.5 | IoU=0.75 | FPS
(without display) | |:------------------:|:---------:|:----------:|:------------:|:-------:|:--------:|:--------------------------:| | PP-YOLOE-x | FP16 | 640 | 0.506 | 0.681 | 0.551 | 116.54 | | PP-YOLOE-l | FP16 | 640 | 0.498 | 0.674 | 0.545 | 187.93 | | PP-YOLOE-m | FP16 | 640 | 0.476 | 0.646 | 0.522 | 257.42 | | PP-YOLOE-s (400) | FP16 | 640 | 0.422 | 0.589 | 0.463 | 465.23 | | YOLOv7-E6E | FP16 | 1280 | 0.476 | 0.648 | 0.521 | 47.82 | | YOLOv7-D6 | FP16 | 1280 | 0.479 | 0.648 | 0.520 | 60.66 | | YOLOv7-E6 | FP16 | 1280 | 0.471 | 0.640 | 0.516 | 73.05 | | YOLOv7-W6 | FP16 | 1280 | 0.444 | 0.610 | 0.483 | 110.29 | | YOLOv7-X* | FP16 | 640 | 0.496 | 0.679 | 0.536 | 162.31 | | YOLOv7* | FP16 | 640 | 0.476 | 0.660 | 0.518 | 237.79 | | YOLOv7-Tiny Leaky* | FP16 | 640 | 0.345 | 0.516 | 0.372 | 611.36 | | YOLOv7-Tiny Leaky* | FP16 | 416 | 0.328 | 0.493 | 0.348 | 633.73 | | YOLOv5x6 6.1 | FP16 | 1280 | 0.508 | 0.683 | 0.554 | 54.88 | | YOLOv5l6 6.1 | FP16 | 1280 | 0.494 | 0.668 | 0.540 | 87.86 | | YOLOv5m6 6.1 | FP16 | 1280 | 0.469 | 0.644 | 0.514 | 142.68 | | YOLOv5s6 6.1 | FP16 | 1280 | 0.399 | 0.581 | 0.438 | 271.19 | | YOLOv5n6 6.1 | FP16 | 1280 | 0.317 | 0.487 | 0.344 | 392.20 | | YOLOv5x 6.1 | FP16 | 640 | 0.470 | 0.652 | 0.513 | 152.99 | | YOLOv5l 6.1 | FP16 | 640 | 0.454 | 0.636 | 0.496 | 247.60 | | YOLOv5m 6.1 | FP16 | 640 | 0.421 | 0.604 | 0.458 | 375.06 | | YOLOv5s 6.1 | FP16 | 640 | 0.344 | 0.528 | 0.371 | 602.44 | | YOLOv5n 6.1 | FP16 | 640 | 0.247 | 0.413 | 0.256 | 629.04 | | YOLOv4** | FP16 | 608 | 0.497 | 0.739 | 0.549 | 206.23 | | YOLOv4-Tiny | FP16 | 416 | 0.215 | 0.402 | 0.205 | 634.69 | ## ### dGPU installation To install the DeepStream on dGPU (x86 platform), without docker, we need to do some steps to prepare the computer.
DeepStream 6.2 #### 1. Disable Secure Boot in BIOS #### 2. Install dependencies ``` sudo apt-get update sudo apt-get install gcc make git libtool autoconf autogen pkg-config cmake sudo apt-get install python3 python3-dev python3-pip sudo apt-get install dkms sudo apt install libssl1.1 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstreamer-plugins-base1.0-dev libgstrtspserver-1.0-0 libjansson4 libyaml-cpp-dev libjsoncpp-dev protobuf-compiler sudo apt-get install linux-headers-$(uname -r) ``` **NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path) ``` sudo nvidia-uninstall sudo $CUDA_PATH/bin/cuda-uninstaller sudo apt-get remove --purge '*nvidia*' sudo apt-get remove --purge '*cuda*' sudo apt-get remove --purge '*cudnn*' sudo apt-get remove --purge '*tensorrt*' sudo apt autoremove --purge && sudo apt autoclean && sudo apt clean ``` #### 3. Install CUDA Keyring ``` wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb sudo dpkg -i cuda-keyring_1.0-1_all.deb sudo apt-get update ``` #### 4. Download and install NVIDIA Driver
TITAN, GeForce RTX / GTX series and RTX / Quadro series
- Download ``` wget https://us.download.nvidia.com/XFree86/Linux-x86_64/525.105.17/NVIDIA-Linux-x86_64-525.105.17.run ```
Laptop * Run ``` sudo sh NVIDIA-Linux-x86_64-525.105.17.run --no-cc-version-check --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: This step will disable the nouveau drivers. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-525.105.17.run --no-cc-version-check --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: If you are using a laptop with NVIDIA Optimius, run ``` sudo apt-get install nvidia-prime sudo prime-select nvidia ```
Desktop * Run ``` sudo sh NVIDIA-Linux-x86_64-525.105.17.run --no-cc-version-check --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ``` **NOTE**: This step will disable the nouveau drivers. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-525.105.17.run --no-cc-version-check --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ```
Data center / Tesla series
- Download ``` wget https://us.download.nvidia.com/XFree86/Linux-x86_64/525.105.17/NVIDIA-Linux-x86_64-525.105.17.run ``` * Run ``` sudo sh NVIDIA-Linux-x86_64-510.47.03.run --no-cc-version-check --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ```
#### 5. Download and install CUDA ``` wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run sudo sh cuda_11.8.0_520.61.05_linux.run --silent --toolkit ``` * Export environment variables ``` echo $'export PATH=/usr/local/cuda-11.8/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc ``` #### 6. Install TensorRT ``` sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /" sudo apt-get update sudo apt-get install libnvinfer8=8.5.2-1+cuda11.8 libnvinfer-plugin8=8.5.2-1+cuda11.8 libnvparsers8=8.5.2-1+cuda11.8 libnvonnxparsers8=8.5.2-1+cuda11.8 libnvinfer-bin=8.5.2-1+cuda11.8 libnvinfer-dev=8.5.2-1+cuda11.8 libnvinfer-plugin-dev=8.5.2-1+cuda11.8 libnvparsers-dev=8.5.2-1+cuda11.8 libnvonnxparsers-dev=8.5.2-1+cuda11.8 libnvinfer-samples=8.5.2-1+cuda11.8 libcudnn8=8.7.0.84-1+cuda11.8 libcudnn8-dev=8.7.0.84-1+cuda11.8 python3-libnvinfer=8.5.2-1+cuda11.8 python3-libnvinfer-dev=8.5.2-1+cuda11.8 sudo apt-mark hold libnvinfer* libnvparsers* libnvonnxparsers* libcudnn8* python3-libnvinfer* tensorrt ``` #### 7. Download from [NVIDIA website](https://developer.nvidia.com/deepstream-getting-started) and install the DeepStream SDK DeepStream 6.2 for Servers and Workstations (.deb) ``` sudo apt-get install ./deepstream-6.2_6.2.0-1_amd64.deb rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin sudo ln -snf /usr/local/cuda-11.8 /usr/local/cuda ``` #### 8. Reboot the computer ``` sudo reboot ```
DeepStream 6.1.1 #### 1. Disable Secure Boot in BIOS #### 2. Install dependencies ``` sudo apt-get update sudo apt-get install gcc make git libtool autoconf autogen pkg-config cmake sudo apt-get install python3 python3-dev python3-pip sudo apt-get install dkms sudo apt-get install libssl1.1 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstreamer-plugins-base1.0-dev libgstrtspserver-1.0-0 libjansson4 libyaml-cpp-dev sudo apt-get install linux-headers-$(uname -r) ``` **NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path) ``` sudo nvidia-uninstall sudo $CUDA_PATH/bin/cuda-uninstaller sudo apt-get remove --purge '*nvidia*' sudo apt-get remove --purge '*cuda*' sudo apt-get remove --purge '*cudnn*' sudo apt-get remove --purge '*tensorrt*' sudo apt autoremove --purge && sudo apt autoclean && sudo apt clean ``` #### 3. Install CUDA Keyring ``` wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb sudo dpkg -i cuda-keyring_1.0-1_all.deb sudo apt-get update ``` #### 4. Download and install NVIDIA Driver
TITAN, GeForce RTX / GTX series and RTX / Quadro series
- Download ``` wget https://us.download.nvidia.com/XFree86/Linux-x86_64/515.65.01/NVIDIA-Linux-x86_64-515.65.01.run ```
Laptop * Run ``` sudo sh NVIDIA-Linux-x86_64-515.65.01.run --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: This step will disable the nouveau drivers. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-515.65.01.run --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: If you are using a laptop with NVIDIA Optimius, run ``` sudo apt-get install nvidia-prime sudo prime-select nvidia ```
Desktop * Run ``` sudo sh NVIDIA-Linux-x86_64-515.65.01.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ``` **NOTE**: This step will disable the nouveau drivers. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-515.65.01.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ```
Data center / Tesla series
- Download ``` wget https://us.download.nvidia.com/tesla/515.65.01/NVIDIA-Linux-x86_64-515.65.01.run ``` * Run ``` sudo sh NVIDIA-Linux-x86_64-515.65.01.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ```
#### 5. Download and install CUDA ``` wget https://developer.download.nvidia.com/compute/cuda/11.7.1/local_installers/cuda_11.7.1_515.65.01_linux.run sudo sh cuda_11.7.1_515.65.01_linux.run --silent --toolkit ``` * Export environment variables ``` echo $'export PATH=/usr/local/cuda-11.7/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.7/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc ``` #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT TensorRT 8.4 GA for Ubuntu 20.04 and CUDA 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6 and 11.7 DEB local repo Package ``` sudo dpkg -i nv-tensorrt-repo-ubuntu2004-cuda11.6-trt8.4.1.5-ga-20220604_1-1_amd64.deb sudo apt-key add /var/nv-tensorrt-repo-ubuntu2004-cuda11.6-trt8.4.1.5-ga-20220604/9a60d8bf.pub sudo apt-get update sudo apt-get install libnvinfer8=8.4.1-1+cuda11.6 libnvinfer-plugin8=8.4.1-1+cuda11.6 libnvparsers8=8.4.1-1+cuda11.6 libnvonnxparsers8=8.4.1-1+cuda11.6 libnvinfer-bin=8.4.1-1+cuda11.6 libnvinfer-dev=8.4.1-1+cuda11.6 libnvinfer-plugin-dev=8.4.1-1+cuda11.6 libnvparsers-dev=8.4.1-1+cuda11.6 libnvonnxparsers-dev=8.4.1-1+cuda11.6 libnvinfer-samples=8.4.1-1+cuda11.6 libcudnn8=8.4.1.50-1+cuda11.6 libcudnn8-dev=8.4.1.50-1+cuda11.6 python3-libnvinfer=8.4.1-1+cuda11.6 python3-libnvinfer-dev=8.4.1-1+cuda11.6 sudo apt-mark hold libnvinfer* libnvparsers* libnvonnxparsers* libcudnn8* tensorrt ``` #### 7. Download from [NVIDIA website](https://developer.nvidia.com/deepstream-getting-started) and install the DeepStream SDK DeepStream 6.1.1 for Servers and Workstations (.deb) ``` sudo apt-get install ./deepstream-6.1_6.1.1-1_amd64.deb rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin sudo ln -snf /usr/local/cuda-11.7 /usr/local/cuda ``` #### 8. Reboot the computer ``` sudo reboot ```
DeepStream 6.1 #### 1. Disable Secure Boot in BIOS #### 2. Install dependencies ``` sudo apt-get update sudo apt-get install gcc make git libtool autoconf autogen pkg-config cmake sudo apt-get install python3 python3-dev python3-pip sudo apt-get install dkms sudo apt-get install libssl1.1 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstrtspserver-1.0-0 libjansson4 libyaml-cpp-dev sudo apt-get install linux-headers-$(uname -r) ``` **NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path) ``` sudo nvidia-uninstall sudo $CUDA_PATH/bin/cuda-uninstaller sudo apt-get remove --purge '*nvidia*' sudo apt-get remove --purge '*cuda*' sudo apt-get remove --purge '*cudnn*' sudo apt-get remove --purge '*tensorrt*' sudo apt autoremove --purge && sudo apt autoclean && sudo apt clean ``` #### 3. Install CUDA Keyring ``` wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.0-1_all.deb sudo dpkg -i cuda-keyring_1.0-1_all.deb sudo apt-get update ``` #### 4. Download and install NVIDIA Driver
TITAN, GeForce RTX / GTX series and RTX / Quadro series
- Download ``` wget https://us.download.nvidia.com/XFree86/Linux-x86_64/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run ```
Laptop * Run ``` sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: This step will disable the nouveau drivers. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: If you are using a laptop with NVIDIA Optimius, run ``` sudo apt-get install nvidia-prime sudo prime-select nvidia ```
Desktop * Run ``` sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ``` **NOTE**: This step will disable the nouveau drivers. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ```
Data center / Tesla series
- Download ``` wget https://us.download.nvidia.com/tesla/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run ``` * Run ``` sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ```
#### 5. Download and install CUDA ``` wget https://developer.download.nvidia.com/compute/cuda/11.6.1/local_installers/cuda_11.6.1_510.47.03_linux.run sudo sh cuda_11.6.1_510.47.03_linux.run --silent --toolkit ``` * Export environment variables ``` echo $'export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc ``` #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT TensorRT 8.2 GA Update 4 for Ubuntu 20.04 and CUDA 11.0, 11.1, 11.2, 11.3, 11.4 and 11.5 DEB local repo Package ``` sudo dpkg -i nv-tensorrt-repo-ubuntu2004-cuda11.4-trt8.2.5.1-ga-20220505_1-1_amd64.deb sudo apt-key add /var/nv-tensorrt-repo-ubuntu2004-cuda11.4-trt8.2.5.1-ga-20220505/82307095.pub sudo apt-get update sudo apt-get install libnvinfer8=8.2.5-1+cuda11.4 libnvinfer-plugin8=8.2.5-1+cuda11.4 libnvparsers8=8.2.5-1+cuda11.4 libnvonnxparsers8=8.2.5-1+cuda11.4 libnvinfer-bin=8.2.5-1+cuda11.4 libnvinfer-dev=8.2.5-1+cuda11.4 libnvinfer-plugin-dev=8.2.5-1+cuda11.4 libnvparsers-dev=8.2.5-1+cuda11.4 libnvonnxparsers-dev=8.2.5-1+cuda11.4 libnvinfer-samples=8.2.5-1+cuda11.4 libnvinfer-doc=8.2.5-1+cuda11.4 libcudnn8-dev=8.4.0.27-1+cuda11.6 libcudnn8=8.4.0.27-1+cuda11.6 sudo apt-mark hold libnvinfer* libnvparsers* libnvonnxparsers* libcudnn8* tensorrt ``` #### 7. Download from [NVIDIA website](https://developer.nvidia.com/deepstream-sdk-download-tesla-archived) and install the DeepStream SDK DeepStream 6.1 for Servers and Workstations (.deb) ``` sudo apt-get install ./deepstream-6.1_6.1.0-1_amd64.deb rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin sudo ln -snf /usr/local/cuda-11.6 /usr/local/cuda ``` #### 8. Reboot the computer ``` sudo reboot ```
DeepStream 6.0.1 / 6.0 #### 1. Disable Secure Boot in BIOS
If you are using a laptop with newer Intel/AMD processors and your Graphics in Settings->Details->About tab is llvmpipe, please update the kernel. ``` wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-headers-5.11.0-051100_5.11.0-051100.202102142330_all.deb wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-headers-5.11.0-051100-generic_5.11.0-051100.202102142330_amd64.deb wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-image-unsigned-5.11.0-051100-generic_5.11.0-051100.202102142330_amd64.deb wget https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.11/amd64/linux-modules-5.11.0-051100-generic_5.11.0-051100.202102142330_amd64.deb sudo dpkg -i *.deb sudo reboot ```
#### 2. Install dependencies ``` sudo apt-get update sudo apt-get install gcc make git libtool autoconf autogen pkg-config cmake sudo apt-get install python3 python3-dev python3-pip sudo apt-get install libssl1.0.0 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav libgstrtspserver-1.0-0 libjansson4 sudo apt-get install linux-headers-$(uname -r) ``` **NOTE**: Install DKMS only if you are using the default Ubuntu kernel ``` sudo apt-get install dkms ``` **NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path) ``` sudo nvidia-uninstall sudo $CUDA_PATH/bin/cuda-uninstaller sudo apt-get remove --purge '*nvidia*' sudo apt-get remove --purge '*cuda*' sudo apt-get remove --purge '*cudnn*' sudo apt-get remove --purge '*tensorrt*' sudo apt autoremove --purge && sudo apt autoclean && sudo apt clean ``` #### 3. Install CUDA Keyring ``` wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-keyring_1.0-1_all.deb sudo dpkg -i cuda-keyring_1.0-1_all.deb sudo apt-get update ``` #### 4. Download and install NVIDIA Driver
TITAN, GeForce RTX / GTX series and RTX / Quadro series
- Download ``` wget https://us.download.nvidia.com/XFree86/Linux-x86_64/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run ```
Laptop * Run ``` sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: This step will disable the nouveau drivers. **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd ``` **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. **NOTE**: If you are using a laptop with NVIDIA Optimius, run ``` sudo apt-get install nvidia-prime sudo prime-select nvidia ```
Desktop * Run ``` sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ``` **NOTE**: This step will disable the nouveau drivers. **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. * Reboot ``` sudo reboot ``` * Install ``` sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ``` **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel.
Data center / Tesla series
- Download ``` wget https://us.download.nvidia.com/tesla/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run ``` * Run ``` sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd --run-nvidia-xconfig ``` **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel.
#### 5. Download and install CUDA ``` wget https://developer.download.nvidia.com/compute/cuda/11.4.1/local_installers/cuda_11.4.1_470.57.02_linux.run sudo sh cuda_11.4.1_470.57.02_linux.run --silent --toolkit ``` * Export environment variables ``` echo $'export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc ``` #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT TensorRT 8.0.1 GA for Ubuntu 18.04 and CUDA 11.3 DEB local repo package ``` sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub sudo apt-get update sudo apt-get install libnvinfer8=8.0.1-1+cuda11.3 libnvinfer-plugin8=8.0.1-1+cuda11.3 libnvparsers8=8.0.1-1+cuda11.3 libnvonnxparsers8=8.0.1-1+cuda11.3 libnvinfer-bin=8.0.1-1+cuda11.3 libnvinfer-dev=8.0.1-1+cuda11.3 libnvinfer-plugin-dev=8.0.1-1+cuda11.3 libnvparsers-dev=8.0.1-1+cuda11.3 libnvonnxparsers-dev=8.0.1-1+cuda11.3 libnvinfer-samples=8.0.1-1+cuda11.3 libnvinfer-doc=8.0.1-1+cuda11.3 libcudnn8-dev=8.2.1.32-1+cuda11.3 libcudnn8=8.2.1.32-1+cuda11.3 sudo apt-mark hold libnvinfer* libnvparsers* libnvonnxparsers* libcudnn8* tensorrt ``` #### 7. Download from [NVIDIA website](https://developer.nvidia.com/deepstream-sdk-download-tesla-archived) and install the DeepStream SDK * DeepStream 6.0.1 for Servers and Workstations (.deb) ``` sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb ``` * DeepStream 6.0 for Servers and Workstations (.deb) ``` sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb ``` * Run ``` rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin sudo ln -snf /usr/local/cuda-11.4 /usr/local/cuda ``` #### 8. Reboot the computer ``` sudo reboot ```
## ### Basic usage #### 1. Download the repo ``` git clone https://github.com/marcoslucianops/DeepStream-Yolo.git cd DeepStream-Yolo ``` #### 2. Download the `cfg` and `weights` files from [Darknet](https://github.com/AlexeyAB/darknet) repo to the DeepStream-Yolo folder #### 3. Compile the lib * DeepStream 6.2 on x86 platform ``` CUDA_VER=11.8 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.1.1 on x86 platform ``` CUDA_VER=11.7 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.1 on x86 platform ``` CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.0.1 / 6.0 on x86 platform ``` CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.2 / 6.1.1 / 6.1 on Jetson platform ``` CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.0.1 / 6.0 on Jetson platform ``` CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo ``` #### 4. Edit the `config_infer_primary.txt` file according to your model (example for YOLOv4) ``` [property] ... custom-network-config=yolov4.cfg model-file=yolov4.weights ... ``` #### 5. Run ``` deepstream-app -c deepstream_app_config.txt ``` **NOTE**: If you want to use YOLOv2 or YOLOv2-Tiny models, change the `deepstream_app_config.txt` file before run it ``` ... [primary-gie] ... config-file=config_infer_primary_yoloV2.txt ... ``` ## ### Docker usage * x86 platform ``` nvcr.io/nvidia/deepstream:6.2-devel nvcr.io/nvidia/deepstream:6.2-triton ``` * Jetson platform ``` nvcr.io/nvidia/deepstream-l4t:6.2-samples nvcr.io/nvidia/deepstream-l4t:6.2-triton ``` **NOTE**: To compile the `nvdsinfer_custom_impl_Yolo`, you need to install the g++ inside the container ``` apt-get install build-essential ``` **NOTE**: With DeepStream 6.2, the docker containers do not package libraries necessary for certain multimedia operations like audio data parsing, CPU decode, and CPU encode. This change could affect processing certain video streams/files like mp4 that include audio track. Please run the below script inside the docker images to install additional packages that might be necessary to use all of the DeepStreamSDK features: ``` /opt/nvidia/deepstream/deepstream/user_additional_install.sh ``` ## ### NMS Configuration To change the `nms-iou-threshold`, `pre-cluster-threshold` and `topk` values, modify the config_infer file and regenerate the model engine file ``` [class-attrs-all] nms-iou-threshold=0.45 pre-cluster-threshold=0.25 topk=300 ``` **NOTE**: It is important to regenerate the engine to get the max detection speed based on `pre-cluster-threshold` you set. **NOTE**: Lower `topk` values will result in more performance. **NOTE**: Make sure to set `cluster-mode=2` in the config_infer file. ## ### INT8 calibration #### 1. Install OpenCV ``` sudo apt-get install libopencv-dev ``` #### 2. Compile/recompile the `nvdsinfer_custom_impl_Yolo` lib with OpenCV support * DeepStream 6.2 on x86 platform ``` CUDA_VER=11.8 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.1.1 on x86 platform ``` CUDA_VER=11.7 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.1 on x86 platform ``` CUDA_VER=11.6 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.0.1 / 6.0 on x86 platform ``` CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.2 / 6.1.1 / 6.1 on Jetson platform ``` CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ``` * DeepStream 6.0.1 / 6.0 on Jetson platform ``` CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ``` #### 3. For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to DeepStream-Yolo folder * Select 1000 random images from COCO dataset to run calibration ``` mkdir calibration ``` ``` for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \ cp ${jpg} calibration/; \ done ``` * Create the `calibration.txt` file with all selected images ``` realpath calibration/*jpg > calibration.txt ``` * Set environment variables ``` export INT8_CALIB_IMG_PATH=calibration.txt export INT8_CALIB_BATCH_SIZE=1 ``` * Edit the `config_infer` file ``` ... model-engine-file=model_b1_gpu0_fp32.engine #int8-calib-file=calib.table ... network-mode=0 ... ``` To ``` ... model-engine-file=model_b1_gpu0_int8.engine int8-calib-file=calib.table ... network-mode=1 ... ``` * Run ``` deepstream-app -c deepstream_app_config.txt ``` **NOTE**: NVIDIA recommends at least 500 images to get a good accuracy. On this example, I used 1000 images to get better accuracy (more images = more accuracy). Higher `INT8_CALIB_BATCH_SIZE` values will result in more accuracy and faster calibration speed. Set it according to you GPU memory. This process can take a long time. ## ### Extract metadata You can get metadata from DeepStream using Python and C/C++. For C/C++, you can edit the `deepstream-app` or `deepstream-test` codes. For Python, your can install and edit [deepstream_python_apps](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps). Basically, you need manipulate the `NvDsObjectMeta` ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html) / [C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsObjectMeta.html)) `and NvDsFrameMeta` ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html) / [C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsFrameMeta.html)) to get the label, position, etc. of bboxes. ## My projects: https://www.youtube.com/MarcosLucianoTV