Using DeepStack with NVIDIA GPU

DeepStack GPU Version serves requests 5 - 20 times faster than the CPU version if you have an NVIDIA GPU.


Before you install the GPU Version, you need to follow the steps below.

Step 1: Setup NVIDIA Drivers and CUDA

Install the NVIDIA Driver

GUIDE: Nvidia Driver Install

Install the CUDA Toolkit

GUIDE: Install CUDA Toolkit

Step 2: Install NVIDIA Docker

The native docker engine does not support GPU access from containers, however nvidia-docker2 modifies your docker install to support GPU access.

Run the commands below to modify the docker engine

curl -s -L | \
sudo apt-key add -

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)

curl -s -L$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update

sudo apt-get install -y nvidia-docker2

sudo pkill -SIGHUP dockerd

If you run into issues, you can refer to this GUIDE

Step 3: Install DeepStack GPU Version

sudo docker pull deepquestai/deepstack:gpu

RUN DeepStack with GPU Access

Once the above steps are complete, when you run deepstack, add the args –rm –runtime=nvidia

sudo docker run --rm --runtime=nvidia -e VISION-FACE=True -v localstorage:/datastore \
-p 80:5000 deepquestai/deepstack:gpu

In the above example, activated only the FACE Api. You can activate multiple endpoints simultaneously as seen below

sudo docker run --rm --runtime=nvidia -e VISION-FACE=True -e VISION-DETECTION=True \
-v localstorage:/datastore -p 80:5000 deepquestai/deepstack:gpu

Here we activated two endpoints at the same time, note that the more endpoints you activate, the more the memory usage of DeepStack. GPUs have limited memory, hence, you should only activate the features you need. Your system can hang if the memory is overloaded