thebears/deepstream_yolo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add documentation for multiple models

Browse Source
This commit is contained in:
Marcos Luciano
2022-07-01 16:39:25 -03:00
parent f484a33648
commit 095696a296
4 changed files with 665 additions and 508 deletions
Download Patch File Download Diff File Expand all files Collapse all files

473
docs/customModels.md
View File

@@ -1,22 +1,14 @@
# How to use custom models in DeepStream # How to use custom models on deepstream-app
* [Requirements](#requirements) * [Directory tree](#directory-tree)
* [Editing files](#editing-files) * [Compile the lib](#compile-the-lib)
* [Compile lib](#compile-lib) * [Understanding and editing deepstream_app_config file](#understanding-and-editing-deepstream_app_config-file)
* [Understanding and editing deepstream_app_config](#understanding-and-editing-deepstream_app_config) * [Understanding and editing config_infer_primary file](#understanding-and-editing-config_infer_primary-file)
* [Understanding and editing config_infer_primary](#understanding-and-editing-config_infer_primary) * [Testing the model](#testing-the-model)
* [Testing model](#testing-model)
## ##
### Requirements ### Directory tree
* [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo)
* Pre-treined YOLO model in Darknet or PyTorch
##
### Editing files
#### 1. Download the repo #### 1. Download the repo
@@ -25,254 +17,249 @@ git clone https://github.com/marcoslucianops/DeepStream-Yolo.git
cd DeepStream-Yolo cd DeepStream-Yolo
``` ```
#### 2. Copy your labels file to DeepStream-Yolo directory and remane it to labels.txt #### 2. Copy the class names file to DeepStream-Yolo folder and remane it to `labels.txt`
#### 3. Copy the yolo.cfg and yolo.weights/yolo.wts files to DeepStream-Yolo directory #### 3. Copy the `cfg` and `weights`/`wts` files to DeepStream-Yolo folder
**NOTE**: It's important to keep the YOLO model reference (yolov4_, yolov5_, yolor_, etc) in you cfg and weights/wts file to generate the engine correctly. **NOTE**: It's important to keep the YOLO model reference (`yolov4_`, `yolov5_`, `yolor_`, etc) in you `cfg` and `weights`/`wts` files to generate the engine correctly.
## ##
### Compile lib ### Compile the lib
* x86 platform * DeepStream 6.1 on x86 platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo
``` ```
* Jetson platform * DeepStream 6.0.1 / 6.0 on x86 platform
``` ```
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.1 on Jetson platform
```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
```
* DeepStream 6.0.1 / 6.0 on Jetson platform
```
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo
```
## ##
### Understanding and editing deepstream_app_config ### Understanding and editing deepstream_app_config file
To understand and edit deepstream_app_config.txt file, read the [DeepStream Reference Application - Configuration Groups](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_ref_app_deepstream.html#configuration-groups) To understand and edit `deepstream_app_config.txt` file, read the [DeepStream Reference Application - Configuration Groups](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_ref_app_deepstream.html#configuration-groups)
* tiled-display
```
[tiled-display]
enable=1
# If you have 1 stream use 1/1 (rows/columns), if you have 4 streams use 2/2 or 4/1 or 1/4 (rows/columns)
rows=1
columns=1
# Resolution of tiled display
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0
```
* source
* Example for 1 source:
```
[source0]
enable=1
# 1=Camera (V4L2), 2=URI, 3=MultiURI, 4=RTSP, 5=Camera (CSI; Jetson only)
type=3
# Stream URL
uri=rtsp://192.168.1.2/Streaming/Channels/101/httppreview
# Number of sources copy (if > 1, edit rows/columns in tiled-display section; use type=3 for more than 1 source)
num-sources=1
gpu-id=0
cudadec-memtype=0
```
* Example for 1 duplcated source:
```
[source0]
enable=1
type=3
uri=rtsp://192.168.1.2/Streaming/Channels/101/
num-sources=2
gpu-id=0
cudadec-memtype=0
```
* Example for 2 sources:
```
[source0]
enable=1
type=3
uri=rtsp://192.168.1.2/Streaming/Channels/101/
num-sources=1
gpu-id=0
cudadec-memtype=0
[source1]
enable=1
type=3
uri=rtsp://192.168.1.3/Streaming/Channels/101/
num-sources=1
gpu-id=0
cudadec-memtype=0
```
* sink
```
[sink0]
enable=1
# 1=Fakesink, 2=EGL (nveglglessink), 3=Filesink, 4=RTSP, 5=Overlay (Jetson only)
type=2
# Indicates how fast the stream is to be rendered (0=As fast as possible, 1=Synchronously)
sync=0
gpu-id=0
nvbuf-memory-type=0
```
* streammux
```
[streammux]
gpu-id=0
# Boolean property to inform muxer that sources are live
live-source=1
batch-size=1
batched-push-timeout=40000
# Resolution of streammux
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0
```
* primary-gie
```
[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt
```
**NOTE**: Edit the `config-file` according to your YOLO model.
## ##
#### tiled-display ### Understanding and editing config_infer_primary file
``` To understand and edit `config_infer_primary.txt` file, read the [DeepStream Plugin Guide - Gst-nvinfer File Configuration Specifications](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications)
[tiled-display]
enable=1 * model-color-format
# If you have 1 stream use 1/1 (rows/columns), if you have 4 streams use 2/2 or 4/1 or 1/4 (rows/columns)
rows=1 ```
columns=1 # 0=RGB, 1=BGR, 2=GRAYSCALE
# Resolution of tiled display model-color-format=0
width=1280 ```
height=720
gpu-id=0 **NOTE**: Set it according to the number of channels in the `cfg` file (1=GRAYSCALE, 3=RGB).
nvbuf-memory-type=0
``` * custom-network-config
* Example for custom YOLOv4 model
```
custom-network-config=yolov4_custom.cfg
```
* model-file
* Example for custom YOLOv4 model
```
model-file=yolov4_custom.weights
```
* model-engine-file
* Example for `batch-size=1` and `network-mode=2`
```
model-engine-file=model_b1_gpu0_fp16.engine
```
* Example for `batch-size=1` and `network-mode=1`
```
model-engine-file=model_b1_gpu0_int8.engine
```
* Example for `batch-size=1` and `network-mode=0`
```
model-engine-file=model_b1_gpu0_fp32.engine
```
* Example for `batch-size=2` and `network-mode=0`
```
model-engine-file=model_b2_gpu0_fp32.engine
```
**NOTE**: To change the generated engine filename, you need to edit and rebuild the `nvdsinfer_model_builder.cpp` file (`/opt/nvidia/deepstream/deepstream/sources/libs/nvdsinfer/nvdsinfer_model_builder.cpp`, lines 825-827)
```
suggestedPathName =
modelPath + "_b" + std::to_string(initParams.maxBatchSize) + "_" +
devId + "_" + networkMode2Str(networkMode) + ".engine";
```
* batch-size
```
batch-size=1
```
* network-mode
```
# 0=FP32, 1=INT8, 2=FP16
network-mode=0
```
* num-detected-classes
```
num-detected-classes=80
```
**NOTE**: Set it according to number of classes in `cfg` file.
* interval
```
# Number of consecutive batches to be skipped
interval=0
```
## ##
#### source ### Testing the model
* Example for 1 source:
```
[source0]
enable=1
# 1=Camera (V4L2), 2=URI, 3=MultiURI, 4=RTSP, 5=Camera (CSI; Jetson only)
type=3
# Stream URL
uri=rtsp://192.168.1.2/Streaming/Channels/101/httppreview
# Number of sources copy (if > 1, edit rows/columns in tiled-display section; use type=3 for more than 1 source)
num-sources=1
gpu-id=0
cudadec-memtype=0
```
* Example for 1 duplcated source:
```
[source0]
enable=1
type=3
uri=rtsp://192.168.1.2/Streaming/Channels/101/
num-sources=2
gpu-id=0
cudadec-memtype=0
```
* Example for 2 sources:
```
[source0]
enable=1
type=3
uri=rtsp://192.168.1.2/Streaming/Channels/101/
num-sources=1
gpu-id=0
cudadec-memtype=0
[source1]
enable=1
type=3
uri=rtsp://192.168.1.3/Streaming/Channels/101/
num-sources=1
gpu-id=0
cudadec-memtype=0
```
##
#### sink
```
[sink0]
enable=1
# 1=Fakesink, 2=EGL (nveglglessink), 3=Filesink, 4=RTSP, 5=Overlay (Jetson only)
type=2
# Indicates how fast the stream is to be rendered (0=As fast as possible, 1=Synchronously)
sync=0
gpu-id=0
nvbuf-memory-type=0
```
##
#### streammux
```
[streammux]
gpu-id=0
# Boolean property to inform muxer that sources are live
live-source=1
batch-size=1
batched-push-timeout=40000
# Resolution of streammux
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0
```
##
#### primary-gie
```
[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt
```
**NOTE**: Choose the correct config_infer_primary based on your YOLO model.
##
### Understanding and editing config_infer_primary
To understand and edit config_infer_primary.txt file, read the [DeepStream Plugin Guide - Gst-nvinfer File Configuration Specifications](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications)
##
#### model-color-format
```
# 0=RGB, 1=BGR, 2=GRAYSCALE
model-color-format=0
```
**NOTE**: Set it accoding to number of channels in yolo.cfg file (1=GRAYSCALE, 3=RGB)
##
#### custom-network-config
* Example for custom YOLOv4 model
```
custom-network-config=yolov4_custom.cfg
```
##
#### model-file
* Example for custom YOLOv4 model
```
model-file=yolov4_custom.weights
```
##
#### model-engine-file
* Example for batch-size=1 and network-mode=2
```
model-engine-file=model_b1_gpu0_fp16.engine
```
* Example for batch-size=1 and network-mode=1
```
model-engine-file=model_b1_gpu0_int8.engine
```
* Example for batch-size=1 and network-mode=0
```
model-engine-file=model_b1_gpu0_fp32.engine
```
* Example for batch-size=2 and network-mode=0
```
model-engine-file=model_b2_gpu0_fp32.engine
```
##
#### batch-size
```
batch-size=1
```
##
#### network-mode
```
# 0=FP32, 1=INT8, 2=FP16
network-mode=0
```
##
#### num-detected-classes
```
num-detected-classes=80
```
**NOTE**: Set it according to number of classes in yolo.cfg file
##
#### interval
```
# Number of consecutive batches to be skipped
interval=0
```
##
### Testing model
``` ```
deepstream-app -c deepstream_app_config.txt deepstream-app -c deepstream_app_config.txt

223
docs/multipleGIEs.md Normal file
View File

@@ -0,0 +1,223 @@
# How to use multiple YOLO GIEs on DeepStream
**NOTE**: The `deepstream-app` does not support multiple primary GIEs. You can only use one YOLO model as primary GIE and the other YOLO models as secondary GIEs (infering the primary detected object). To use 2 or more YOLO models as primary GIE, you need to use a custom code.
* [Directory tree](#directory-tree)
* [Change the YoloLayer plugin version](#change-the-yololayer-plugin-version)
* [Compile the libs](#compile-the-libs)
* [Edit the config_infer_primary files](#edit-the-config_infer_primary-files)
* [Edit the deepstream_app_config file](#edit-the-deepstream_app_config-file)
* [Test](#test)
##
### Directory tree
#### 1. Download the repo
```
git clone https://github.com/marcoslucianops/DeepStream-Yolo.git
cd DeepStream-Yolo
```
#### 2. Create folders for the GIEs and copy the DeepStream-Yolo files to them
![image](multipleGIEs_tree.png)
#### 3. Copy the class names file to each GIE folder and remane it to `labels.txt`
#### 4. Copy the `cfg` and `weights`/`wts` files to each GIE folder
**NOTE**: It's important to keep the YOLO model reference (`yolov4_`, `yolov5_`, `yolor_`, etc) in you `cfg` and `weights`/`wts` files to generate the engine correctly.
##
### Change the YoloLayer plugin version
Edit the `yoloPlugins.h` file (line 53), in each GIE `nvdsinfer_custom_impl_Yolo` folder.
```
const char* YOLOLAYER_PLUGIN_VERSION {"1"};
```
To:
```
const char* YOLOLAYER_PLUGIN_VERSION {"2"};
```
**NOTE**: `gie2`: version = 2 / `gie3`: version = 3 / `gie4`: version = 4.
##
### Compile the libs
**NOTE**: Do it for each GIE folder, replacing the GIE folder name (`gie1/nvdsinfer_custom_impl_Yolo`).
* DeepStream 6.1 on x86 platform
```
CUDA_VER=11.6 make -C gie1/nvdsinfer_custom_impl_Yolo
```
* DeepStream 6.0.1 / 6.0 on x86 platform
```
CUDA_VER=11.4 make -C gie1/nvdsinfer_custom_impl_Yolo
```
* DeepStream 6.1 on Jetson platform
```
CUDA_VER=11.4 make -C gie1/nvdsinfer_custom_impl_Yolo
```
* DeepStream 6.0.1 / 6.0 on Jetson platform
```
CUDA_VER=10.2 make -C gie1/nvdsinfer_custom_impl_Yolo
```
##
### Edit the config_infer_primary files
**NOTE**: Edit the files according to the model you will use (YOLOv4, YOLOv5, YOLOR, etc).
**NOTE**: Do it for each GIE folder.
* Edit the path of the `cfg` file
Example for gie1
```
custom-network-config=gie1/yolo.cfg
```
Example for gie2
```
custom-network-config=gie2/yolo.cfg
```
* Edit the gie-unique-id
Example for gie1
```
gie-unique-id=1
```
Example for gie2
```
gie-unique-id=2
```
* Edit the process-mode
Example for primary inference engine
```
process-mode=1
```
Example for secondary inference engine (infering the primary detected object)
```
process-mode=2
```
**NOTE**: In the secondary inference, we need to set which gie it will use to operate
Add
```
operate-on-gie-id=1
```
To operate on specific class ids
```
operate-on-class-ids=0;1;2
```
* Edit batch-size
Example for primary inference engine
```
batch-size=1
```
Example for secondary inference engine (infering the primary detected object)
```
batch-size=16
```
### Edit the deepstream_app_config file
**NOTE**: Add the `secondary-gie` key after `primary-gie` key.
Example for 1 `secondary-gie` (2 inferences):
```
[secondary-gie0]
enable=1
gpu-id=0
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0
nvbuf-memory-type=0
config-file=gie2/config_infer_primary.txt
```
Example for 2 `secondary-gie` (3 inferences):
```
[secondary-gie0]
enable=1
gpu-id=0
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0
nvbuf-memory-type=0
config-file=gie2/config_infer_primary.txt
[secondary-gie1]
enable=1
gpu-id=0
gie-unique-id=3
operate-on-gie-id=1
operate-on-class-ids=0
nvbuf-memory-type=0
config-file=gie3/config_infer_primary.txt
```
**NOTE**: Remember to edit `primary-gie` key
```
[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt
```
To
```
[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=gie1/config_infer_primary.txt
```
##
### Test
```
deepstream-app -c deepstream_app_config.txt
```
**NOTE**: During test process, the engine files will be generated in the DeepStream-Yolo folder. When build process is done, for each GIE, move engine file to its respective folder (`gie1`, `gie2`, etc).

BIN
docs/multipleGIEs_tree.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

475
readme.md
View File

@@ -4,28 +4,29 @@ NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO models
### Future updates ### Future updates
* New documentation for multiple models * Models benchmarks
* DeepStream tutorials * DeepStream tutorials
* Native YOLOX support * YOLOX support
* Native PP-YOLO support * PP-YOLO support
* YOLOv6 support
* Dynamic batch-size * Dynamic batch-size
### Improvements on this repository ### Improvements on this repository
* Darknet CFG params parser (no need to edit nvdsparsebbox_Yolo.cpp or another file) * Darknet cfg params parser (no need to edit `nvdsparsebbox_Yolo.cpp` or other files)
* Support for new_coords, beta_nms and scale_x_y params * Support for `new_coords`, `beta_nms` and `scale_x_y` params
* Support for new models * Support for new models
* Support for new layers * Support for new layers
* Support for new activations * Support for new activations
* Support for convolutional groups * Support for convolutional groups
* Support for INT8 calibration * Support for INT8 calibration
* Support for non square models * Support for non square models
* Support for reorg, implicit and channel layers (YOLOR) * Support for `reorg`, `implicit` and `channel` layers (YOLOR)
* YOLOv5 4.0, 5.0, 6.0 and 6.1 native support * YOLOv5 4.0, 5.0, 6.0 and 6.1 support
* YOLOR native support * YOLOR support
* Models benchmarks (**outdated**)
* **GPU YOLO Decoder (moved from CPU to GPU to get better performance)** [#138](https://github.com/marcoslucianops/DeepStream-Yolo/issues/138) * **GPU YOLO Decoder (moved from CPU to GPU to get better performance)** [#138](https://github.com/marcoslucianops/DeepStream-Yolo/issues/138)
* **GPU Batched NMS** [#142](https://github.com/marcoslucianops/DeepStream-Yolo/issues/142) * **GPU Batched NMS** [#142](https://github.com/marcoslucianops/DeepStream-Yolo/issues/142)
* **New documentation for multiple models**
## ##
@@ -41,6 +42,7 @@ NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO models
* [NMS configuration](#nms-configuration) * [NMS configuration](#nms-configuration)
* [INT8 calibration](#int8-calibration) * [INT8 calibration](#int8-calibration)
* [Using your custom model](docs/customModels.md) * [Using your custom model](docs/customModels.md)
* [Multiple YOLO GIEs](docs/multipleGIEs.md)
## ##
@@ -125,7 +127,7 @@ sudo apt-get install libssl1.1 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0
sudo apt-get install linux-headers-$(uname -r) sudo apt-get install linux-headers-$(uname -r)
``` ```
**NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path). **NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path)
``` ```
sudo nvidia-uninstall sudo nvidia-uninstall
@@ -149,35 +151,35 @@ sudo apt-get update
* TITAN, GeForce RTX / GTX series and RTX / Quadro series * TITAN, GeForce RTX / GTX series and RTX / Quadro series
``` ```
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run wget https://us.download.nvidia.com/XFree86/Linux-x86_64/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run
``` ```
* Data center / Tesla series * Data center / Tesla series
``` ```
wget https://us.download.nvidia.com/tesla/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run wget https://us.download.nvidia.com/tesla/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run
``` ```
* Run * Run
``` ```
sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd
``` ```
**NOTE**: This step will disable the nouveau drivers. **NOTE**: This step will disable the nouveau drivers.
* Reboot * Reboot
``` ```
sudo reboot sudo reboot
``` ```
* Install * Install
``` ```
sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd
``` ```
**NOTE**: If you are using a laptop with NVIDIA Optimius, run **NOTE**: If you are using a laptop with NVIDIA Optimius, run
@@ -195,10 +197,9 @@ sudo sh cuda_11.6.1_510.47.03_linux.run --silent --toolkit
* Export environment variables * Export environment variables
``` ```
echo $'export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc echo $'export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc
``` ```
#### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT
@@ -256,13 +257,13 @@ sudo apt install libssl1.0.0 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-p
sudo apt-get install linux-headers-$(uname -r) sudo apt-get install linux-headers-$(uname -r)
``` ```
**NOTE**: Install DKMS only if you are using the default Ubuntu kernel. **NOTE**: Install DKMS only if you are using the default Ubuntu kernel
``` ```
sudo apt-get install dkms sudo apt-get install dkms
``` ```
**NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path). **NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path)
``` ```
sudo nvidia-uninstall sudo nvidia-uninstall
@@ -286,39 +287,39 @@ sudo apt-get update
* TITAN, GeForce RTX / GTX series and RTX / Quadro series * TITAN, GeForce RTX / GTX series and RTX / Quadro series
``` ```
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run wget https://us.download.nvidia.com/XFree86/Linux-x86_64/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run
``` ```
* Data center / Tesla series * Data center / Tesla series
``` ```
wget https://us.download.nvidia.com/tesla/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run wget https://us.download.nvidia.com/tesla/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run
``` ```
* Run * Run
``` ```
sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd
``` ```
**NOTE**: This step will disable the nouveau drivers. **NOTE**: This step will disable the nouveau drivers.
**NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel.
* Reboot * Reboot
``` ```
sudo reboot sudo reboot
``` ```
* Install * Install
``` ```
sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd
``` ```
**NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel.
**NOTE**: If you are using a laptop with NVIDIA Optimius, run **NOTE**: If you are using a laptop with NVIDIA Optimius, run
@@ -336,10 +337,9 @@ sudo sh cuda_11.4.1_470.57.02_linux.run --silent --toolkit
* Export environment variables * Export environment variables
``` ```
echo $'export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc echo $'export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc
``` ```
#### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT
@@ -356,22 +356,22 @@ sudo apt-get install libnvinfer8=8.0.1-1+cuda11.3 libnvinfer-plugin8=8.0.1-1+cud
* DeepStream 6.0.1 for Servers and Workstations (.deb) * DeepStream 6.0.1 for Servers and Workstations (.deb)
``` ```
sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb
``` ```
* DeepStream 6.0 for Servers and Workstations (.deb) * DeepStream 6.0 for Servers and Workstations (.deb)
``` ```
sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb
``` ```
* Run * Run
``` ```
rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin
sudo ln -snf /usr/local/cuda-11.4 /usr/local/cuda sudo ln -snf /usr/local/cuda-11.4 /usr/local/cuda
``` ```
#### 8. Reboot the computer #### 8. Reboot the computer
@@ -392,55 +392,41 @@ git clone https://github.com/marcoslucianops/DeepStream-Yolo.git
cd DeepStream-Yolo cd DeepStream-Yolo
``` ```
#### 2. Download cfg and weights files from your model and move to DeepStream-Yolo folder #### 2. Download the `cfg` and `weights` files from [Darknet](https://github.com/AlexeyAB/darknet) repo to the DeepStream-Yolo folder
#### 3. Compile lib #### 3. Compile the lib
* DeepStream 6.1 on x86 platform * DeepStream 6.1 on x86 platform
``` ```
CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.0.1 / 6.0 on x86 platform * DeepStream 6.0.1 / 6.0 on x86 platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.1 on Jetson platform * DeepStream 6.1 on Jetson platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.0.1 / 6.0 on Jetson platform * DeepStream 6.0.1 / 6.0 on Jetson platform
``` ```
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo
``` ```
#### 4. Edit config_infer_primary.txt for your model (example for YOLOv4) #### 4. Edit the `config_infer_primary.txt` file according to your model (example for YOLOv4)
``` ```
[property] [property]
... ...
# 0=RGB, 1=BGR, 2=GRAYSCALE
model-color-format=0
# YOLO cfg
custom-network-config=yolov4.cfg custom-network-config=yolov4.cfg
# YOLO weights
model-file=yolov4.weights model-file=yolov4.weights
# Generated TensorRT model (will be created if it doesn't exist)
model-engine-file=model_b1_gpu0_fp32.engine
# Model labels file
labelfile-path=labels.txt
# Batch size
batch-size=1
# 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
# Number of classes in label file
num-detected-classes=80
... ...
``` ```
@@ -450,15 +436,12 @@ num-detected-classes=80
deepstream-app -c deepstream_app_config.txt deepstream-app -c deepstream_app_config.txt
``` ```
**NOTE**: If you want to use YOLOv2 or YOLOv2-Tiny models, change the deepstream_app_config.txt file before run it **NOTE**: If you want to use YOLOv2 or YOLOv2-Tiny models, change the `deepstream_app_config.txt` file before run it
``` ```
... ...
[primary-gie] [primary-gie]
enable=1 ...
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV2.txt config-file=config_infer_primary_yoloV2.txt
... ...
``` ```
@@ -467,211 +450,179 @@ config-file=config_infer_primary_yoloV2.txt
### YOLOv5 usage ### YOLOv5 usage
**NOTE**: Make sure to change the YOLOv5 repo version to your model version before conversion. **NOTE**: Make sure to change the YOLOv5 repo version according to your model version before the conversion.
#### 1. Copy gen_wts_yoloV5.py from DeepStream-Yolo/utils to [ultralytics/yolov5](https://github.com/ultralytics/yolov5) folder #### 1. Copy the `gen_wts_yoloV5.py` file from `DeepStream-Yolo/utils` directory to the [YOLOv5](https://github.com/ultralytics/yolov5) folder
#### 2. Open the ultralytics/yolov5 folder #### 2. Open the YOLOv5 folder
#### 3. Download pt file from [ultralytics/yolov5](https://github.com/ultralytics/yolov5/releases/) website (example for YOLOv5n 6.1) #### 3. Download the `pt` file from [YOLOv5](https://github.com/ultralytics/yolov5/releases/) repo (example for YOLOv5n 6.1)
``` ```
wget https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5n.pt wget https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5n.pt
``` ```
#### 4. Generate cfg and wts files (example for YOLOv5n) #### 4. Generate the `cfg` and `wts` files (example for YOLOv5n)
``` ```
python3 gen_wts_yoloV5.py -w yolov5n.pt -c models/yolov5n.yaml python3 gen_wts_yoloV5.py -w yolov5n.pt -c models/yolov5n.yaml
``` ```
#### 5. Copy generated cfg and wts files to DeepStream-Yolo folder #### 5. Copy the generated `cfg` and `wts` files to the DeepStream-Yolo folder
#### 6. Open DeepStream-Yolo folder #### 6. Open the DeepStream-Yolo folder
#### 7. Compile lib #### 7. Compile the lib
* DeepStream 6.1 on x86 platform * DeepStream 6.1 on x86 platform
``` ```
CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.0.1 / 6.0 on x86 platform * DeepStream 6.0.1 / 6.0 on x86 platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.1 on Jetson platform * DeepStream 6.1 on Jetson platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.0.1 / 6.0 on Jetson platform * DeepStream 6.0.1 / 6.0 on Jetson platform
``` ```
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo
``` ```
#### 8. Edit config_infer_primary_yoloV5.txt for your model (example for YOLOv5n) #### 8. Edit the `config_infer_primary_yoloV5.txt` file according to your model (example for YOLOv5n)
``` ```
[property] [property]
... ...
# 0=RGB, 1=BGR, 2=GRAYSCALE
model-color-format=0
# CFG
custom-network-config=yolov5n.cfg custom-network-config=yolov5n.cfg
# WTS
model-file=yolov5n.wts model-file=yolov5n.wts
# Generated TensorRT model (will be created if it doesn't exist)
model-engine-file=model_b1_gpu0_fp32.engine
# Model labels file
labelfile-path=labels.txt
# Batch size
batch-size=1
# 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
# Number of classes in label file
num-detected-classes=80
... ...
``` ```
#### 8. Change the deepstream_app_config.txt file #### 9. Edit the `deepstream_app_config.txt` file
``` ```
... ...
[primary-gie] [primary-gie]
enable=1 ...
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV5.txt config-file=config_infer_primary_yoloV5.txt
``` ```
#### 9. Run #### 10. Run
``` ```
deepstream-app -c deepstream_app_config.txt deepstream-app -c deepstream_app_config.txt
``` ```
**NOTE**: For YOLOv5 P6 or custom models, check the gen_wts_yoloV5.py args and use them according to your model **NOTE**: For YOLOv5 P6, check the `gen_wts_yoloV5.py` file args and set them according to your model.
* Input weights (.pt) file path **(required)** * Input weights (.pt) file path
``` ```
-w or --weights -w or --weights
``` ```
* Input cfg (.yaml) file path * Input cfg (.yaml) file path
``` ```
-c or --yaml -c or --yaml
``` ```
* Inference size [size] or [height , weight] **(default [640] / [1280] if --p6)** * Inference size [size] or [height , weight]
``` Default: 640 / 1280 (if --p6)
-s or --size
```
Example for 1280 ```
-s or --size
```
``` * Example for 1280
-s 1280
```
or ```
-s 1280
```
``` or
-s 1280 1280
``` ```
-s 1280 1280
```
## ##
### YOLOR usage ### YOLOR usage
#### 1. Copy gen_wts_yolor.py from DeepStream-Yolo/utils to [yolor](https://github.com/WongKinYiu/yolor) folder #### 1. Copy the `gen_wts_yolor.py` file from `DeepStream-Yolo/utils` directory to the [YOLOR](https://github.com/WongKinYiu/yolor) folder
#### 2. Open the yolor folder #### 2. Open the YOLOR folder
#### 3. Download pt file from [yolor](https://github.com/WongKinYiu/yolor) website #### 3. Download the `pt` file from [YOLOR](https://github.com/WongKinYiu/yolor) repo
#### 4. Generate wts file (example for YOLOR-CSP) #### 4. Generate the `cfg` and `wts` files (example for YOLOR-CSP)
``` ```
python3 gen_wts_yolor.py -w yolor_csp.pt -c cfg/yolor_csp.cfg python3 gen_wts_yolor.py -w yolor_csp.pt -c cfg/yolor_csp.cfg
``` ```
#### 5. Copy generated cfg and wts files to DeepStream-Yolo folder #### 5. Copy the generated `cfg` and `wts` files to the DeepStream-Yolo folder
#### 6. Open DeepStream-Yolo folder #### 6. Open the DeepStream-Yolo folder
#### 7. Compile lib #### 7. Compile the lib
* DeepStream 6.1 on x86 platform * DeepStream 6.1 on x86 platform
``` ```
CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.0.1 / 6.0 on x86 platform * DeepStream 6.0.1 / 6.0 on x86 platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.1 on Jetson platform * DeepStream 6.1 on Jetson platform
``` ```
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo
``` ```
* DeepStream 6.0.1 / 6.0 on Jetson platform * DeepStream 6.0.1 / 6.0 on Jetson platform
``` ```
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo
``` ```
#### 8. Edit config_infer_primary_yolor.txt for your model (example for YOLOR-CSP) #### 8. Edit the `config_infer_primary_yolor.txt` file according to your model (example for YOLOR-CSP)
``` ```
[property] [property]
... ...
# 0=RGB, 1=BGR, 2=GRAYSCALE
model-color-format=0
# CFG
custom-network-config=yolor_csp.cfg custom-network-config=yolor_csp.cfg
# WTS
model-file=yolor_csp.wts model-file=yolor_csp.wts
# Generated TensorRT model (will be created if it doesn't exist)
model-engine-file=model_b1_gpu0_fp32.engine
# Model labels file
labelfile-path=labels.txt
# Batch size
batch-size=1
# 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
# Number of classes in label file
num-detected-classes=80
... ...
``` ```
#### 8. Change the deepstream_app_config.txt file #### 9. Edit the `deepstream_app_config.txt` file
``` ```
... ...
[primary-gie] [primary-gie]
enable=1 ...
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yolor.txt config-file=config_infer_primary_yolor.txt
``` ```
#### 9. Run #### 10. Run
``` ```
deepstream-app -c deepstream_app_config.txt deepstream-app -c deepstream_app_config.txt
@@ -683,12 +634,6 @@ deepstream-app -c deepstream_app_config.txt
To change the `iou-threshold`, `score-threshold` and `topk` values, modify the `config_nms.txt` file and regenerate the model engine file. To change the `iou-threshold`, `score-threshold` and `topk` values, modify the `config_nms.txt` file and regenerate the model engine file.
**NOTE**: Lower `topk` values will result in more performance.
**NOTE**: Make sure to set cluster-mode=4 in config_infer file.
**NOTE**: You are still able to change the `pre-cluster-threshold` values in the `config_infer.txt` file.
``` ```
[property] [property]
iou-threshold=0.45 iou-threshold=0.45
@@ -696,6 +641,12 @@ score-threshold=0.25
topk=300 topk=300
``` ```
**NOTE**: Lower `topk` values will result in more performance.
**NOTE**: Make sure to set `cluster-mode=4` in the config_infer file.
**NOTE**: You are still able to change the `pre-cluster-threshold` values in the config_infer files.
## ##
### INT8 calibration ### INT8 calibration
@@ -706,100 +657,96 @@ topk=300
sudo apt-get install libopencv-dev sudo apt-get install libopencv-dev
``` ```
#### 2. Compile/recompile the nvdsinfer_custom_impl_Yolo lib with OpenCV support #### 2. Compile/recompile the `nvdsinfer_custom_impl_Yolo` lib with OpenCV support
* DeepStream 6.1 on x86 platform * DeepStream 6.1 on x86 platform
``` ```
cd DeepStream-Yolo CUDA_VER=11.6 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
CUDA_VER=11.6 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ```
```
* DeepStream 6.0.1 / 6.0 on x86 platform * DeepStream 6.0.1 / 6.0 on x86 platform
``` ```
cd DeepStream-Yolo CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ```
```
* DeepStream 6.1 on Jetson platform * DeepStream 6.1 on Jetson platform
``` ```
cd DeepStream-Yolo CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ```
```
* DeepStream 6.0.1 / 6.0 on Jetson platform * DeepStream 6.0.1 / 6.0 on Jetson platform
``` ```
cd DeepStream-Yolo CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo
CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo ```
```
#### 3. For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to DeepStream-Yolo folder #### 3. For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to DeepStream-Yolo folder
##### Select 1000 random images from COCO dataset to run calibration * Select 1000 random images from COCO dataset to run calibration
``` ```
mkdir calibration mkdir calibration
``` ```
``` ```
for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \ for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \
cp ${jpg} calibration/; \ cp ${jpg} calibration/; \
done done
``` ```
##### Create the calibration.txt file with all selected images * Create the `calibration.txt` file with all selected images
``` ```
realpath calibration/*jpg > calibration.txt realpath calibration/*jpg > calibration.txt
``` ```
##### Set environment variables * Set environment variables
``` ```
export INT8_CALIB_IMG_PATH=calibration.txt export INT8_CALIB_IMG_PATH=calibration.txt
export INT8_CALIB_BATCH_SIZE=1 export INT8_CALIB_BATCH_SIZE=1
``` ```
##### Change config_infer_primary.txt file * Edit the `config_infer` file
``` ```
... ...
model-engine-file=model_b1_gpu0_fp32.engine model-engine-file=model_b1_gpu0_fp32.engine
#int8-calib-file=calib.table #int8-calib-file=calib.table
... ...
network-mode=0 network-mode=0
... ...
``` ```
* To To
``` ```
... ...
model-engine-file=model_b1_gpu0_int8.engine model-engine-file=model_b1_gpu0_int8.engine
int8-calib-file=calib.table int8-calib-file=calib.table
... ...
network-mode=1 network-mode=1
... ...
``` ```
##### Run * Run
``` ```
deepstream-app -c deepstream_app_config.txt deepstream-app -c deepstream_app_config.txt
``` ```
**NOTE**: NVIDIA recommends at least 500 images to get a good accuracy. In this example I used 1000 images to get better accuracy (more images = more accuracy). Higher INT8_CALIB_BATCH_SIZE values will increase the accuracy and calibration speed. Set it according to you GPU memory. This process can take a long time. **NOTE**: NVIDIA recommends at least 500 images to get a good accuracy. On this example, I used 1000 images to get better accuracy (more images = more accuracy). Higher `INT8_CALIB_BATCH_SIZE` values will result in more accuracy and faster calibration speed. Set it according to you GPU memory. This process can take a long time.
## ##
### Extract metadata ### Extract metadata
You can get metadata from deepstream in Python and C/C++. For C/C++, you need edit deepstream-app or deepstream-test code. For Python your need install and edit [deepstream_python_apps](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps). You can get metadata from DeepStream using Python and C/C++. For C/C++, you can edit the `deepstream-app` or `deepstream-test` codes. For Python, your can install and edit [deepstream_python_apps](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps).
Basically, you need manipulate NvDsObjectMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html)/[C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsObjectMeta.html)) and NvDsFrameMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html)/[C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsFrameMeta.html)) to get label, position, etc. of bboxes. Basically, you need manipulate the `NvDsObjectMeta` ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html) / [C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsObjectMeta.html)) `and NvDsFrameMeta` ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html) / [C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsFrameMeta.html)) to get the label, position, etc. of bboxes.
## ##