diff --git a/docs/customModels.md b/docs/customModels.md index 381af62..25049c1 100644 --- a/docs/customModels.md +++ b/docs/customModels.md @@ -1,22 +1,14 @@ -# How to use custom models in DeepStream +# How to use custom models on deepstream-app -* [Requirements](#requirements) -* [Editing files](#editing-files) -* [Compile lib](#compile-lib) -* [Understanding and editing deepstream_app_config](#understanding-and-editing-deepstream_app_config) -* [Understanding and editing config_infer_primary](#understanding-and-editing-config_infer_primary) -* [Testing model](#testing-model) +* [Directory tree](#directory-tree) +* [Compile the lib](#compile-the-lib) +* [Understanding and editing deepstream_app_config file](#understanding-and-editing-deepstream_app_config-file) +* [Understanding and editing config_infer_primary file](#understanding-and-editing-config_infer_primary-file) +* [Testing the model](#testing-the-model) ## -### Requirements - -* [DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) -* Pre-treined YOLO model in Darknet or PyTorch - -## - -### Editing files +### Directory tree #### 1. Download the repo @@ -25,254 +17,249 @@ git clone https://github.com/marcoslucianops/DeepStream-Yolo.git cd DeepStream-Yolo ``` -#### 2. Copy your labels file to DeepStream-Yolo directory and remane it to labels.txt +#### 2. Copy the class names file to DeepStream-Yolo folder and remane it to `labels.txt` -#### 3. Copy the yolo.cfg and yolo.weights/yolo.wts files to DeepStream-Yolo directory +#### 3. Copy the `cfg` and `weights`/`wts` files to DeepStream-Yolo folder -**NOTE**: It's important to keep the YOLO model reference (yolov4_, yolov5_, yolor_, etc) in you cfg and weights/wts file to generate the engine correctly. +**NOTE**: It's important to keep the YOLO model reference (`yolov4_`, `yolov5_`, `yolor_`, etc) in you `cfg` and `weights`/`wts` files to generate the engine correctly. ## -### Compile lib +### Compile the lib -* x86 platform +* DeepStream 6.1 on x86 platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo + ``` -* Jetson platform +* DeepStream 6.0.1 / 6.0 on x86 platform -``` -CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` + +* DeepStream 6.1 on Jetson platform + + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` + +* DeepStream 6.0.1 / 6.0 on Jetson platform + + ``` + CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo + ``` ## -### Understanding and editing deepstream_app_config +### Understanding and editing deepstream_app_config file -To understand and edit deepstream_app_config.txt file, read the [DeepStream Reference Application - Configuration Groups](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_ref_app_deepstream.html#configuration-groups) +To understand and edit `deepstream_app_config.txt` file, read the [DeepStream Reference Application - Configuration Groups](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_ref_app_deepstream.html#configuration-groups) + + +* tiled-display + + ``` + [tiled-display] + enable=1 + # If you have 1 stream use 1/1 (rows/columns), if you have 4 streams use 2/2 or 4/1 or 1/4 (rows/columns) + rows=1 + columns=1 + # Resolution of tiled display + width=1280 + height=720 + gpu-id=0 + nvbuf-memory-type=0 + ``` + +* source + + * Example for 1 source: + + ``` + [source0] + enable=1 + # 1=Camera (V4L2), 2=URI, 3=MultiURI, 4=RTSP, 5=Camera (CSI; Jetson only) + type=3 + # Stream URL + uri=rtsp://192.168.1.2/Streaming/Channels/101/httppreview + # Number of sources copy (if > 1, edit rows/columns in tiled-display section; use type=3 for more than 1 source) + num-sources=1 + gpu-id=0 + cudadec-memtype=0 + ``` + + * Example for 1 duplcated source: + + ``` + [source0] + enable=1 + type=3 + uri=rtsp://192.168.1.2/Streaming/Channels/101/ + num-sources=2 + gpu-id=0 + cudadec-memtype=0 + ``` + + * Example for 2 sources: + + ``` + [source0] + enable=1 + type=3 + uri=rtsp://192.168.1.2/Streaming/Channels/101/ + num-sources=1 + gpu-id=0 + cudadec-memtype=0 + + [source1] + enable=1 + type=3 + uri=rtsp://192.168.1.3/Streaming/Channels/101/ + num-sources=1 + gpu-id=0 + cudadec-memtype=0 + ``` + +* sink + + ``` + [sink0] + enable=1 + # 1=Fakesink, 2=EGL (nveglglessink), 3=Filesink, 4=RTSP, 5=Overlay (Jetson only) + type=2 + # Indicates how fast the stream is to be rendered (0=As fast as possible, 1=Synchronously) + sync=0 + gpu-id=0 + nvbuf-memory-type=0 + ``` + +* streammux + + ``` + [streammux] + gpu-id=0 + # Boolean property to inform muxer that sources are live + live-source=1 + batch-size=1 + batched-push-timeout=40000 + # Resolution of streammux + width=1920 + height=1080 + enable-padding=0 + nvbuf-memory-type=0 + ``` + +* primary-gie + + ``` + [primary-gie] + enable=1 + gpu-id=0 + gie-unique-id=1 + nvbuf-memory-type=0 + config-file=config_infer_primary.txt + ``` + + **NOTE**: Edit the `config-file` according to your YOLO model. ## -#### tiled-display +### Understanding and editing config_infer_primary file -``` -[tiled-display] -enable=1 -# If you have 1 stream use 1/1 (rows/columns), if you have 4 streams use 2/2 or 4/1 or 1/4 (rows/columns) -rows=1 -columns=1 -# Resolution of tiled display -width=1280 -height=720 -gpu-id=0 -nvbuf-memory-type=0 -``` +To understand and edit `config_infer_primary.txt` file, read the [DeepStream Plugin Guide - Gst-nvinfer File Configuration Specifications](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications) + +* model-color-format + + ``` + # 0=RGB, 1=BGR, 2=GRAYSCALE + model-color-format=0 + ``` + + **NOTE**: Set it according to the number of channels in the `cfg` file (1=GRAYSCALE, 3=RGB). + +* custom-network-config + + * Example for custom YOLOv4 model + + ``` + custom-network-config=yolov4_custom.cfg + ``` + +* model-file + + * Example for custom YOLOv4 model + + ``` + model-file=yolov4_custom.weights + ``` + +* model-engine-file + + * Example for `batch-size=1` and `network-mode=2` + + ``` + model-engine-file=model_b1_gpu0_fp16.engine + ``` + + * Example for `batch-size=1` and `network-mode=1` + + ``` + model-engine-file=model_b1_gpu0_int8.engine + ``` + + * Example for `batch-size=1` and `network-mode=0` + + ``` + model-engine-file=model_b1_gpu0_fp32.engine + ``` + + * Example for `batch-size=2` and `network-mode=0` + + ``` + model-engine-file=model_b2_gpu0_fp32.engine + ``` + + **NOTE**: To change the generated engine filename, you need to edit and rebuild the `nvdsinfer_model_builder.cpp` file (`/opt/nvidia/deepstream/deepstream/sources/libs/nvdsinfer/nvdsinfer_model_builder.cpp`, lines 825-827) + + ``` + suggestedPathName = + modelPath + "_b" + std::to_string(initParams.maxBatchSize) + "_" + + devId + "_" + networkMode2Str(networkMode) + ".engine"; + ``` + +* batch-size + + ``` + batch-size=1 + ``` + +* network-mode + + ``` + # 0=FP32, 1=INT8, 2=FP16 + network-mode=0 + ``` + +* num-detected-classes + + ``` + num-detected-classes=80 + ``` + + **NOTE**: Set it according to number of classes in `cfg` file. + +* interval + + ``` + # Number of consecutive batches to be skipped + interval=0 + ``` ## -#### source - -* Example for 1 source: - -``` -[source0] -enable=1 -# 1=Camera (V4L2), 2=URI, 3=MultiURI, 4=RTSP, 5=Camera (CSI; Jetson only) -type=3 -# Stream URL -uri=rtsp://192.168.1.2/Streaming/Channels/101/httppreview -# Number of sources copy (if > 1, edit rows/columns in tiled-display section; use type=3 for more than 1 source) -num-sources=1 -gpu-id=0 -cudadec-memtype=0 -``` - -* Example for 1 duplcated source: - -``` -[source0] -enable=1 -type=3 -uri=rtsp://192.168.1.2/Streaming/Channels/101/ -num-sources=2 -gpu-id=0 -cudadec-memtype=0 -``` - -* Example for 2 sources: - -``` -[source0] -enable=1 -type=3 -uri=rtsp://192.168.1.2/Streaming/Channels/101/ -num-sources=1 -gpu-id=0 -cudadec-memtype=0 - -[source1] -enable=1 -type=3 -uri=rtsp://192.168.1.3/Streaming/Channels/101/ -num-sources=1 -gpu-id=0 -cudadec-memtype=0 -``` - -## - -#### sink - -``` -[sink0] -enable=1 -# 1=Fakesink, 2=EGL (nveglglessink), 3=Filesink, 4=RTSP, 5=Overlay (Jetson only) -type=2 -# Indicates how fast the stream is to be rendered (0=As fast as possible, 1=Synchronously) -sync=0 -gpu-id=0 -nvbuf-memory-type=0 -``` - -## - -#### streammux - -``` -[streammux] -gpu-id=0 -# Boolean property to inform muxer that sources are live -live-source=1 -batch-size=1 -batched-push-timeout=40000 -# Resolution of streammux -width=1920 -height=1080 -enable-padding=0 -nvbuf-memory-type=0 -``` - -## - -#### primary-gie - -``` -[primary-gie] -enable=1 -gpu-id=0 -gie-unique-id=1 -nvbuf-memory-type=0 -config-file=config_infer_primary.txt -``` - -**NOTE**: Choose the correct config_infer_primary based on your YOLO model. - -## - -### Understanding and editing config_infer_primary - -To understand and edit config_infer_primary.txt file, read the [DeepStream Plugin Guide - Gst-nvinfer File Configuration Specifications](https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications) - -## - -#### model-color-format - -``` -# 0=RGB, 1=BGR, 2=GRAYSCALE -model-color-format=0 -``` - -**NOTE**: Set it accoding to number of channels in yolo.cfg file (1=GRAYSCALE, 3=RGB) - -## - -#### custom-network-config - -* Example for custom YOLOv4 model - -``` -custom-network-config=yolov4_custom.cfg -``` - -## - -#### model-file - -* Example for custom YOLOv4 model - -``` -model-file=yolov4_custom.weights -``` - -## - -#### model-engine-file - -* Example for batch-size=1 and network-mode=2 - -``` -model-engine-file=model_b1_gpu0_fp16.engine -``` - -* Example for batch-size=1 and network-mode=1 - -``` -model-engine-file=model_b1_gpu0_int8.engine -``` - -* Example for batch-size=1 and network-mode=0 - -``` -model-engine-file=model_b1_gpu0_fp32.engine -``` - -* Example for batch-size=2 and network-mode=0 - -``` -model-engine-file=model_b2_gpu0_fp32.engine -``` - -## - -#### batch-size - -``` -batch-size=1 -``` - -## - -#### network-mode - -``` -# 0=FP32, 1=INT8, 2=FP16 -network-mode=0 -``` - -## - -#### num-detected-classes - -``` -num-detected-classes=80 -``` - -**NOTE**: Set it according to number of classes in yolo.cfg file - -## - -#### interval - -``` -# Number of consecutive batches to be skipped -interval=0 -``` - -## - -### Testing model +### Testing the model ``` deepstream-app -c deepstream_app_config.txt diff --git a/docs/multipleGIEs.md b/docs/multipleGIEs.md new file mode 100644 index 0000000..bee0cd9 --- /dev/null +++ b/docs/multipleGIEs.md @@ -0,0 +1,223 @@ +# How to use multiple YOLO GIEs on DeepStream + +**NOTE**: The `deepstream-app` does not support multiple primary GIEs. You can only use one YOLO model as primary GIE and the other YOLO models as secondary GIEs (infering the primary detected object). To use 2 or more YOLO models as primary GIE, you need to use a custom code. + +* [Directory tree](#directory-tree) +* [Change the YoloLayer plugin version](#change-the-yololayer-plugin-version) +* [Compile the libs](#compile-the-libs) +* [Edit the config_infer_primary files](#edit-the-config_infer_primary-files) +* [Edit the deepstream_app_config file](#edit-the-deepstream_app_config-file) +* [Test](#test) + +## + +### Directory tree + +#### 1. Download the repo + +``` +git clone https://github.com/marcoslucianops/DeepStream-Yolo.git +cd DeepStream-Yolo +``` + +#### 2. Create folders for the GIEs and copy the DeepStream-Yolo files to them + +![image](multipleGIEs_tree.png) + +#### 3. Copy the class names file to each GIE folder and remane it to `labels.txt` + +#### 4. Copy the `cfg` and `weights`/`wts` files to each GIE folder + +**NOTE**: It's important to keep the YOLO model reference (`yolov4_`, `yolov5_`, `yolor_`, etc) in you `cfg` and `weights`/`wts` files to generate the engine correctly. + +## + +### Change the YoloLayer plugin version + +Edit the `yoloPlugins.h` file (line 53), in each GIE `nvdsinfer_custom_impl_Yolo` folder. + +``` +const char* YOLOLAYER_PLUGIN_VERSION {"1"}; +``` +To: +``` +const char* YOLOLAYER_PLUGIN_VERSION {"2"}; +``` + +**NOTE**: `gie2`: version = 2 / `gie3`: version = 3 / `gie4`: version = 4. + +## + +### Compile the libs + +**NOTE**: Do it for each GIE folder, replacing the GIE folder name (`gie1/nvdsinfer_custom_impl_Yolo`). + +* DeepStream 6.1 on x86 platform + + ``` + CUDA_VER=11.6 make -C gie1/nvdsinfer_custom_impl_Yolo + ``` + +* DeepStream 6.0.1 / 6.0 on x86 platform + + ``` + CUDA_VER=11.4 make -C gie1/nvdsinfer_custom_impl_Yolo + ``` + +* DeepStream 6.1 on Jetson platform + + ``` + CUDA_VER=11.4 make -C gie1/nvdsinfer_custom_impl_Yolo + ``` + +* DeepStream 6.0.1 / 6.0 on Jetson platform + + ``` + CUDA_VER=10.2 make -C gie1/nvdsinfer_custom_impl_Yolo + ``` + +## + +### Edit the config_infer_primary files + +**NOTE**: Edit the files according to the model you will use (YOLOv4, YOLOv5, YOLOR, etc). + +**NOTE**: Do it for each GIE folder. + +* Edit the path of the `cfg` file + + Example for gie1 + + ``` + custom-network-config=gie1/yolo.cfg + ``` + + Example for gie2 + + ``` + custom-network-config=gie2/yolo.cfg + ``` + +* Edit the gie-unique-id + + Example for gie1 + + ``` + gie-unique-id=1 + ``` + + Example for gie2 + + ``` + gie-unique-id=2 + ``` + +* Edit the process-mode + + Example for primary inference engine + + ``` + process-mode=1 + ``` + + Example for secondary inference engine (infering the primary detected object) + + ``` + process-mode=2 + ``` + + **NOTE**: In the secondary inference, we need to set which gie it will use to operate + + Add + ``` + operate-on-gie-id=1 + ``` + + To operate on specific class ids + + ``` + operate-on-class-ids=0;1;2 + ``` + +* Edit batch-size + + Example for primary inference engine + + ``` + batch-size=1 + ``` + + Example for secondary inference engine (infering the primary detected object) + + ``` + batch-size=16 + ``` + +### Edit the deepstream_app_config file + +**NOTE**: Add the `secondary-gie` key after `primary-gie` key. + +Example for 1 `secondary-gie` (2 inferences): + +``` +[secondary-gie0] +enable=1 +gpu-id=0 +gie-unique-id=2 +operate-on-gie-id=1 +operate-on-class-ids=0 +nvbuf-memory-type=0 +config-file=gie2/config_infer_primary.txt +``` + +Example for 2 `secondary-gie` (3 inferences): +``` +[secondary-gie0] +enable=1 +gpu-id=0 +gie-unique-id=2 +operate-on-gie-id=1 +operate-on-class-ids=0 +nvbuf-memory-type=0 +config-file=gie2/config_infer_primary.txt + +[secondary-gie1] +enable=1 +gpu-id=0 +gie-unique-id=3 +operate-on-gie-id=1 +operate-on-class-ids=0 +nvbuf-memory-type=0 +config-file=gie3/config_infer_primary.txt +``` + +**NOTE**: Remember to edit `primary-gie` key + +``` +[primary-gie] +enable=1 +gpu-id=0 +gie-unique-id=1 +nvbuf-memory-type=0 +config-file=config_infer_primary.txt +``` + +To +``` +[primary-gie] +enable=1 +gpu-id=0 +gie-unique-id=1 +nvbuf-memory-type=0 +config-file=gie1/config_infer_primary.txt +``` + +## + +### Test + +``` +deepstream-app -c deepstream_app_config.txt +``` + +**NOTE**: During test process, the engine files will be generated in the DeepStream-Yolo folder. When build process is done, for each GIE, move engine file to its respective folder (`gie1`, `gie2`, etc). diff --git a/docs/multipleGIEs_tree.png b/docs/multipleGIEs_tree.png new file mode 100644 index 0000000..1e712d2 Binary files /dev/null and b/docs/multipleGIEs_tree.png differ diff --git a/readme.md b/readme.md index 80b51bf..7ac5f42 100644 --- a/readme.md +++ b/readme.md @@ -4,28 +4,29 @@ NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO models ### Future updates -* New documentation for multiple models +* Models benchmarks * DeepStream tutorials -* Native YOLOX support -* Native PP-YOLO support +* YOLOX support +* PP-YOLO support +* YOLOv6 support * Dynamic batch-size ### Improvements on this repository -* Darknet CFG params parser (no need to edit nvdsparsebbox_Yolo.cpp or another file) -* Support for new_coords, beta_nms and scale_x_y params +* Darknet cfg params parser (no need to edit `nvdsparsebbox_Yolo.cpp` or other files) +* Support for `new_coords`, `beta_nms` and `scale_x_y` params * Support for new models * Support for new layers * Support for new activations * Support for convolutional groups * Support for INT8 calibration * Support for non square models -* Support for reorg, implicit and channel layers (YOLOR) -* YOLOv5 4.0, 5.0, 6.0 and 6.1 native support -* YOLOR native support -* Models benchmarks (**outdated**) +* Support for `reorg`, `implicit` and `channel` layers (YOLOR) +* YOLOv5 4.0, 5.0, 6.0 and 6.1 support +* YOLOR support * **GPU YOLO Decoder (moved from CPU to GPU to get better performance)** [#138](https://github.com/marcoslucianops/DeepStream-Yolo/issues/138) * **GPU Batched NMS** [#142](https://github.com/marcoslucianops/DeepStream-Yolo/issues/142) +* **New documentation for multiple models** ## @@ -41,6 +42,7 @@ NVIDIA DeepStream SDK 6.1 / 6.0.1 / 6.0 configuration for YOLO models * [NMS configuration](#nms-configuration) * [INT8 calibration](#int8-calibration) * [Using your custom model](docs/customModels.md) +* [Multiple YOLO GIEs](docs/multipleGIEs.md) ## @@ -125,7 +127,7 @@ sudo apt-get install libssl1.1 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0 sudo apt-get install linux-headers-$(uname -r) ``` -**NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path). +**NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path) ``` sudo nvidia-uninstall @@ -149,35 +151,35 @@ sudo apt-get update * TITAN, GeForce RTX / GTX series and RTX / Quadro series -``` -wget https://us.download.nvidia.com/XFree86/Linux-x86_64/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run -``` + ``` + wget https://us.download.nvidia.com/XFree86/Linux-x86_64/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run + ``` * Data center / Tesla series -``` -wget https://us.download.nvidia.com/tesla/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run -``` + ``` + wget https://us.download.nvidia.com/tesla/510.47.03/NVIDIA-Linux-x86_64-510.47.03.run + ``` * Run -``` -sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd -``` + ``` + sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd + ``` -**NOTE**: This step will disable the nouveau drivers. + **NOTE**: This step will disable the nouveau drivers. * Reboot -``` -sudo reboot -``` + ``` + sudo reboot + ``` * Install -``` -sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd -``` + ``` + sudo sh NVIDIA-Linux-x86_64-510.47.03.run --silent --disable-nouveau --dkms --install-libglvnd + ``` **NOTE**: If you are using a laptop with NVIDIA Optimius, run @@ -195,10 +197,9 @@ sudo sh cuda_11.6.1_510.47.03_linux.run --silent --toolkit * Export environment variables -``` -echo $'export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc -``` - + ``` + echo $'export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc + ``` #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT @@ -256,13 +257,13 @@ sudo apt install libssl1.0.0 libgstreamer1.0-0 gstreamer1.0-tools gstreamer1.0-p sudo apt-get install linux-headers-$(uname -r) ``` -**NOTE**: Install DKMS only if you are using the default Ubuntu kernel. +**NOTE**: Install DKMS only if you are using the default Ubuntu kernel ``` sudo apt-get install dkms ``` -**NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path). +**NOTE**: Purge all NVIDIA driver, CUDA, etc (replace $CUDA_PATH to your CUDA path) ``` sudo nvidia-uninstall @@ -286,39 +287,39 @@ sudo apt-get update * TITAN, GeForce RTX / GTX series and RTX / Quadro series -``` -wget https://us.download.nvidia.com/XFree86/Linux-x86_64/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run -``` + ``` + wget https://us.download.nvidia.com/XFree86/Linux-x86_64/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run + ``` * Data center / Tesla series -``` -wget https://us.download.nvidia.com/tesla/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run -``` + ``` + wget https://us.download.nvidia.com/tesla/470.129.06/NVIDIA-Linux-x86_64-470.129.06.run + ``` * Run -``` -sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd -``` + ``` + sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd + ``` -**NOTE**: This step will disable the nouveau drivers. + **NOTE**: This step will disable the nouveau drivers. -**NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. + **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. * Reboot -``` -sudo reboot -``` + ``` + sudo reboot + ``` * Install -``` -sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd -``` + ``` + sudo sh NVIDIA-Linux-x86_64-470.129.06.run --silent --disable-nouveau --dkms --install-libglvnd + ``` -**NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. + **NOTE**: Remove --dkms flag if you installed the 5.11.0 kernel. **NOTE**: If you are using a laptop with NVIDIA Optimius, run @@ -336,10 +337,9 @@ sudo sh cuda_11.4.1_470.57.02_linux.run --silent --toolkit * Export environment variables -``` -echo $'export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc -``` - + ``` + echo $'export PATH=/usr/local/cuda-11.4/bin${PATH:+:${PATH}}\nexport LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc && source ~/.bashrc + ``` #### 6. Download from [NVIDIA website](https://developer.nvidia.com/nvidia-tensorrt-8x-download) and install the TensorRT @@ -356,22 +356,22 @@ sudo apt-get install libnvinfer8=8.0.1-1+cuda11.3 libnvinfer-plugin8=8.0.1-1+cud * DeepStream 6.0.1 for Servers and Workstations (.deb) -``` -sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb -``` + ``` + sudo apt-get install ./deepstream-6.0_6.0.1-1_amd64.deb + ``` * DeepStream 6.0 for Servers and Workstations (.deb) -``` -sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb -``` + ``` + sudo apt-get install ./deepstream-6.0_6.0.0-1_amd64.deb + ``` * Run -``` -rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin -sudo ln -snf /usr/local/cuda-11.4 /usr/local/cuda -``` + ``` + rm ${HOME}/.cache/gstreamer-1.0/registry.x86_64.bin + sudo ln -snf /usr/local/cuda-11.4 /usr/local/cuda + ``` #### 8. Reboot the computer @@ -392,55 +392,41 @@ git clone https://github.com/marcoslucianops/DeepStream-Yolo.git cd DeepStream-Yolo ``` -#### 2. Download cfg and weights files from your model and move to DeepStream-Yolo folder +#### 2. Download the `cfg` and `weights` files from [Darknet](https://github.com/AlexeyAB/darknet) repo to the DeepStream-Yolo folder -#### 3. Compile lib +#### 3. Compile the lib * DeepStream 6.1 on x86 platform -``` -CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on x86 platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.1 on Jetson platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on Jetson platform -``` -CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo + ``` -#### 4. Edit config_infer_primary.txt for your model (example for YOLOv4) +#### 4. Edit the `config_infer_primary.txt` file according to your model (example for YOLOv4) ``` [property] ... -# 0=RGB, 1=BGR, 2=GRAYSCALE -model-color-format=0 -# YOLO cfg custom-network-config=yolov4.cfg -# YOLO weights model-file=yolov4.weights -# Generated TensorRT model (will be created if it doesn't exist) -model-engine-file=model_b1_gpu0_fp32.engine -# Model labels file -labelfile-path=labels.txt -# Batch size -batch-size=1 -# 0=FP32, 1=INT8, 2=FP16 mode -network-mode=0 -# Number of classes in label file -num-detected-classes=80 ... ``` @@ -450,15 +436,12 @@ num-detected-classes=80 deepstream-app -c deepstream_app_config.txt ``` -**NOTE**: If you want to use YOLOv2 or YOLOv2-Tiny models, change the deepstream_app_config.txt file before run it +**NOTE**: If you want to use YOLOv2 or YOLOv2-Tiny models, change the `deepstream_app_config.txt` file before run it ``` ... [primary-gie] -enable=1 -gpu-id=0 -gie-unique-id=1 -nvbuf-memory-type=0 +... config-file=config_infer_primary_yoloV2.txt ... ``` @@ -467,211 +450,179 @@ config-file=config_infer_primary_yoloV2.txt ### YOLOv5 usage -**NOTE**: Make sure to change the YOLOv5 repo version to your model version before conversion. +**NOTE**: Make sure to change the YOLOv5 repo version according to your model version before the conversion. -#### 1. Copy gen_wts_yoloV5.py from DeepStream-Yolo/utils to [ultralytics/yolov5](https://github.com/ultralytics/yolov5) folder +#### 1. Copy the `gen_wts_yoloV5.py` file from `DeepStream-Yolo/utils` directory to the [YOLOv5](https://github.com/ultralytics/yolov5) folder -#### 2. Open the ultralytics/yolov5 folder +#### 2. Open the YOLOv5 folder -#### 3. Download pt file from [ultralytics/yolov5](https://github.com/ultralytics/yolov5/releases/) website (example for YOLOv5n 6.1) +#### 3. Download the `pt` file from [YOLOv5](https://github.com/ultralytics/yolov5/releases/) repo (example for YOLOv5n 6.1) ``` wget https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5n.pt ``` -#### 4. Generate cfg and wts files (example for YOLOv5n) +#### 4. Generate the `cfg` and `wts` files (example for YOLOv5n) ``` python3 gen_wts_yoloV5.py -w yolov5n.pt -c models/yolov5n.yaml ``` -#### 5. Copy generated cfg and wts files to DeepStream-Yolo folder +#### 5. Copy the generated `cfg` and `wts` files to the DeepStream-Yolo folder -#### 6. Open DeepStream-Yolo folder +#### 6. Open the DeepStream-Yolo folder -#### 7. Compile lib +#### 7. Compile the lib * DeepStream 6.1 on x86 platform -``` -CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on x86 platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.1 on Jetson platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on Jetson platform -``` -CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo + ``` -#### 8. Edit config_infer_primary_yoloV5.txt for your model (example for YOLOv5n) +#### 8. Edit the `config_infer_primary_yoloV5.txt` file according to your model (example for YOLOv5n) ``` [property] ... -# 0=RGB, 1=BGR, 2=GRAYSCALE -model-color-format=0 -# CFG custom-network-config=yolov5n.cfg -# WTS model-file=yolov5n.wts -# Generated TensorRT model (will be created if it doesn't exist) -model-engine-file=model_b1_gpu0_fp32.engine -# Model labels file -labelfile-path=labels.txt -# Batch size -batch-size=1 -# 0=FP32, 1=INT8, 2=FP16 mode -network-mode=0 -# Number of classes in label file -num-detected-classes=80 ... ``` -#### 8. Change the deepstream_app_config.txt file +#### 9. Edit the `deepstream_app_config.txt` file ``` ... [primary-gie] -enable=1 -gpu-id=0 -gie-unique-id=1 -nvbuf-memory-type=0 +... config-file=config_infer_primary_yoloV5.txt ``` -#### 9. Run +#### 10. Run ``` deepstream-app -c deepstream_app_config.txt ``` -**NOTE**: For YOLOv5 P6 or custom models, check the gen_wts_yoloV5.py args and use them according to your model +**NOTE**: For YOLOv5 P6, check the `gen_wts_yoloV5.py` file args and set them according to your model. -* Input weights (.pt) file path **(required)** +* Input weights (.pt) file path -``` --w or --weights -``` + ``` + -w or --weights + ``` * Input cfg (.yaml) file path -``` --c or --yaml -``` + ``` + -c or --yaml + ``` -* Inference size [size] or [height , weight] **(default [640] / [1280] if --p6)** +* Inference size [size] or [height , weight] -``` --s or --size -``` + Default: 640 / 1280 (if --p6) -Example for 1280 + ``` + -s or --size + ``` -``` --s 1280 -``` + * Example for 1280 -or + ``` + -s 1280 + ``` -``` --s 1280 1280 -``` + or + + ``` + -s 1280 1280 + ``` ## ### YOLOR usage -#### 1. Copy gen_wts_yolor.py from DeepStream-Yolo/utils to [yolor](https://github.com/WongKinYiu/yolor) folder +#### 1. Copy the `gen_wts_yolor.py` file from `DeepStream-Yolo/utils` directory to the [YOLOR](https://github.com/WongKinYiu/yolor) folder -#### 2. Open the yolor folder +#### 2. Open the YOLOR folder -#### 3. Download pt file from [yolor](https://github.com/WongKinYiu/yolor) website +#### 3. Download the `pt` file from [YOLOR](https://github.com/WongKinYiu/yolor) repo -#### 4. Generate wts file (example for YOLOR-CSP) +#### 4. Generate the `cfg` and `wts` files (example for YOLOR-CSP) ``` python3 gen_wts_yolor.py -w yolor_csp.pt -c cfg/yolor_csp.cfg ``` -#### 5. Copy generated cfg and wts files to DeepStream-Yolo folder +#### 5. Copy the generated `cfg` and `wts` files to the DeepStream-Yolo folder -#### 6. Open DeepStream-Yolo folder +#### 6. Open the DeepStream-Yolo folder -#### 7. Compile lib +#### 7. Compile the lib * DeepStream 6.1 on x86 platform -``` -CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.6 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on x86 platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.1 on Jetson platform -``` -CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on Jetson platform -``` -CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo + ``` -#### 8. Edit config_infer_primary_yolor.txt for your model (example for YOLOR-CSP) +#### 8. Edit the `config_infer_primary_yolor.txt` file according to your model (example for YOLOR-CSP) ``` [property] ... -# 0=RGB, 1=BGR, 2=GRAYSCALE -model-color-format=0 -# CFG custom-network-config=yolor_csp.cfg -# WTS model-file=yolor_csp.wts -# Generated TensorRT model (will be created if it doesn't exist) -model-engine-file=model_b1_gpu0_fp32.engine -# Model labels file -labelfile-path=labels.txt -# Batch size -batch-size=1 -# 0=FP32, 1=INT8, 2=FP16 mode -network-mode=0 -# Number of classes in label file -num-detected-classes=80 ... ``` -#### 8. Change the deepstream_app_config.txt file +#### 9. Edit the `deepstream_app_config.txt` file ``` ... [primary-gie] -enable=1 -gpu-id=0 -gie-unique-id=1 -nvbuf-memory-type=0 +... config-file=config_infer_primary_yolor.txt ``` -#### 9. Run +#### 10. Run ``` deepstream-app -c deepstream_app_config.txt @@ -683,12 +634,6 @@ deepstream-app -c deepstream_app_config.txt To change the `iou-threshold`, `score-threshold` and `topk` values, modify the `config_nms.txt` file and regenerate the model engine file. -**NOTE**: Lower `topk` values will result in more performance. - -**NOTE**: Make sure to set cluster-mode=4 in config_infer file. - -**NOTE**: You are still able to change the `pre-cluster-threshold` values in the `config_infer.txt` file. - ``` [property] iou-threshold=0.45 @@ -696,6 +641,12 @@ score-threshold=0.25 topk=300 ``` +**NOTE**: Lower `topk` values will result in more performance. + +**NOTE**: Make sure to set `cluster-mode=4` in the config_infer file. + +**NOTE**: You are still able to change the `pre-cluster-threshold` values in the config_infer files. + ## ### INT8 calibration @@ -706,100 +657,96 @@ topk=300 sudo apt-get install libopencv-dev ``` -#### 2. Compile/recompile the nvdsinfer_custom_impl_Yolo lib with OpenCV support +#### 2. Compile/recompile the `nvdsinfer_custom_impl_Yolo` lib with OpenCV support * DeepStream 6.1 on x86 platform -``` -cd DeepStream-Yolo -CUDA_VER=11.6 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.6 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on x86 platform -``` -cd DeepStream-Yolo -CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.1 on Jetson platform -``` -cd DeepStream-Yolo -CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo + ``` * DeepStream 6.0.1 / 6.0 on Jetson platform -``` -cd DeepStream-Yolo -CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo -``` + ``` + CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo + ``` #### 3. For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to DeepStream-Yolo folder -##### Select 1000 random images from COCO dataset to run calibration +* Select 1000 random images from COCO dataset to run calibration -``` -mkdir calibration -``` + ``` + mkdir calibration + ``` -``` -for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \ - cp ${jpg} calibration/; \ -done -``` + ``` + for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \ + cp ${jpg} calibration/; \ + done + ``` -##### Create the calibration.txt file with all selected images +* Create the `calibration.txt` file with all selected images -``` -realpath calibration/*jpg > calibration.txt -``` + ``` + realpath calibration/*jpg > calibration.txt + ``` -##### Set environment variables +* Set environment variables -``` -export INT8_CALIB_IMG_PATH=calibration.txt -export INT8_CALIB_BATCH_SIZE=1 -``` + ``` + export INT8_CALIB_IMG_PATH=calibration.txt + export INT8_CALIB_BATCH_SIZE=1 + ``` -##### Change config_infer_primary.txt file +* Edit the `config_infer` file -``` -... -model-engine-file=model_b1_gpu0_fp32.engine -#int8-calib-file=calib.table -... -network-mode=0 -... -``` + ``` + ... + model-engine-file=model_b1_gpu0_fp32.engine + #int8-calib-file=calib.table + ... + network-mode=0 + ... + ``` -* To + To -``` -... -model-engine-file=model_b1_gpu0_int8.engine -int8-calib-file=calib.table -... -network-mode=1 -... -``` + ``` + ... + model-engine-file=model_b1_gpu0_int8.engine + int8-calib-file=calib.table + ... + network-mode=1 + ... + ``` -##### Run +* Run -``` -deepstream-app -c deepstream_app_config.txt -``` + ``` + deepstream-app -c deepstream_app_config.txt + ``` -**NOTE**: NVIDIA recommends at least 500 images to get a good accuracy. In this example I used 1000 images to get better accuracy (more images = more accuracy). Higher INT8_CALIB_BATCH_SIZE values will increase the accuracy and calibration speed. Set it according to you GPU memory. This process can take a long time. +**NOTE**: NVIDIA recommends at least 500 images to get a good accuracy. On this example, I used 1000 images to get better accuracy (more images = more accuracy). Higher `INT8_CALIB_BATCH_SIZE` values will result in more accuracy and faster calibration speed. Set it according to you GPU memory. This process can take a long time. ## ### Extract metadata -You can get metadata from deepstream in Python and C/C++. For C/C++, you need edit deepstream-app or deepstream-test code. For Python your need install and edit [deepstream_python_apps](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps). +You can get metadata from DeepStream using Python and C/C++. For C/C++, you can edit the `deepstream-app` or `deepstream-test` codes. For Python, your can install and edit [deepstream_python_apps](https://github.com/NVIDIA-AI-IOT/deepstream_python_apps). -Basically, you need manipulate NvDsObjectMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html)/[C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsObjectMeta.html)) and NvDsFrameMeta ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html)/[C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsFrameMeta.html)) to get label, position, etc. of bboxes. +Basically, you need manipulate the `NvDsObjectMeta` ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsObjectMeta.html) / [C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsObjectMeta.html)) `and NvDsFrameMeta` ([Python](https://docs.nvidia.com/metropolis/deepstream/python-api/PYTHON_API/NvDsMeta/NvDsFrameMeta.html) / [C/C++](https://docs.nvidia.com/metropolis/deepstream/sdk-api/struct__NvDsFrameMeta.html)) to get the label, position, etc. of bboxes. ##