Deepstack GPU installation problem and Cudnn? - RTX 3080 no performance increase compared to only CPU

S

smithjames

n3wb
Jun 26, 2022
4
0
-
I have a rather high performance system but with too slow AI. Strange, but I did not see any performance increase after added a GPU-card, Nvidia Geforce RTX 3080 12GB (8960 cuda cores). It is difficult to install deepstack GPU in windows 11. I don't think I managed with Nvidia Cudnn installation?, and perhaps also other parts?

Previously without having an GPU I wsa using the deepstack version for CPU with performance about 300-600 ms + about 300 ms for face recognition and about 5000 ms when many people moved in many cameras at the same time. Now with RTX3080, about the same performance. I was expected 50-100 ms range.

My system
=============================================
Intel i9 11900K, 64 GB RAM, Geforce RTX 3080 12 GB, 4K Monitor
500 GB M.2 NVMe PCIe 4.0 + 24 TB SATA for storange
Windows 11

Cameras: 10 x Hikvision 4K 25 FPS, max quality and bitrate.

Recording: Continously with 4K 25 FPS on all cameras (direct to disc) + trigger

AI settings:
Mode: Hihg
Timout: 15 sec
Facial recognition on
Save unknown faces

When triggerd:
Hi-res JPEG files
Settings: Analys 5 pictures, one every 1 sec. (my goal is to wastly increase here for for improved face recognition, e.g. 250 ms and 40 pictures).
see screen shot for settings.

My "failed"(?) deepstack GPU installation
=============================================

Installation:
1. Updating windows 11 and restarted the computer

2. Installed the latest Nvidia driver - Download the latest official NVIDIA drivers

3a. Installed Visual Studio 2022 community version, included all packages that could be selected
3b. Installed Nvidia CUDA Toolkit - Installation Guide Windows :: CUDA Toolkit Documentation
Version . 11.7.0

4a. Installation of Nvidia Cudnn - Installation Guide :: NVIDIA Deep Learning cuDNN Documentation, Followed the "3. Installing cuDNN on Windows"
I had to registred an Nividia developer account just to be able to download the Cudnn.
Version 8.4.1.50

4b. Intalled Zlib

4c. Moved the Cudnn to the C:\Program Files\NVIDIA\CUDNN\v8.4.1.50

4d. Changed the Environment Variables

No error messanges and everything good so for.

4d. "Include cudnn.lib in your Visual Studio project. -> ..."
PROBLEM. I did not manage here, I just went to install deepstack in step 5 below.
Did anyone seccess with this? How to include cubnn.lin in Visual studio project? or can it be skipped?

5. Installed deepstack, the "GPU version for windows" - Using DeepStack with Windows 10 (CPU and GPU)
Verions: DeepStack-Installer-GPU-2022.01.1.exe

6. No changed settings in Blue iris, no error messages but also no performance increase having added the RTX3080 card.
 

Attachments

  • trigger-AI.png
    trigger-AI.png
    87.5 KB · Views: 23
You definitely have something screwy going on with your system. People with much less machines are seeing lower CPU times than you are. Maybe it is related to Win11 or maybe not.

Uncheck the use mainstream if available - Deepstack downrezes the photo so checking this box only increases your time.

While not related, suggest you uncheck the auto-flag box because these will not be deleted based on your clips and archiving settings and soon your system will stop recording because the drive is full.

Are you running the substreams?

Have you opened up Task Manager to see if the GPU is being utilized?


I can tell that you have not done every optimization in the wiki. Start with that.

Optimizing Blue Iris's CPU Usage

Blue Iris uses a lot of CPU time. In most Blue Iris installations, your CPU is the limiting factor which determines how many cameras you can have. As such, it is extremely important to optimize your configuration to ensure you can get the most out of your system while consuming the least...
ipcamtalk.com ipcamtalk.com



No reason to run 25FPS. That is just wasting storage space and running up the CPU% and not getting you anything of value. Police only care about a clean image capture, not smooth video. Shutter speed is more important than FPS.

Hollywood movies are only shot at 24FPS, so we do not need more than that for our mobile devices and tablets LOL.

15FPS with an iframe Interval of 15 at 8192 bitrate is more than enough.

Match the coding and FPS for the substream.

Watch these, for most of us, it isn't annoying until below 10FPS




 
  • Like
Reactions: Flintstone61 and stxrc
Thanks for input. But are there really no reasons to run 25 FPS? I am asking beause I want to have high quaility photos for the purpose of both making possible IDs of thiefs as well as for improved AI facial recognition. I have also seen recommendations from other NVR-systems to use 25-30 FPS for AI facial recognization.

With this high performance system I am hoping to ne able to run 25 FPS with AI facing detection with many AI frames per second and during relative long time (yet not implemented, now only 1 sec x 5 pictures). So the 25 FPS is not for having smooth videos.

Perhaps I have missunderstood but will not the 25 FPS not improve facial quality for IDs and AI detection?

I more or less followed the optimizing wiki already, but are not using 15 FPS.

  • I use direct-to-disc 4K (25 FPS, max quality and bit rate, I-frame to 25) continuously 24/7 with 10 hikvision cameras
  • I use substreams 640x480 (25 FPS, max quality and bit rate, I-frame to 25)
  • I use Hardware accelerated decoding: Intel + VPP
  • H.265 is used for everything. I expect to be able to store about 30 days for videos.
  • I do not limit the preview frame rate (it is set to 30)
  • Video scaling: Use "Fast" scaling

I will definitely uncheck the use mainstream if available, if the 640x480 substream is enough ?

The utilizing of GPU in Task manager, I have not checked so much but seems to be low, 10-20% perhaps.

I think you are saying that even without the GPU installed I should have better performance for the beginning?
So maybe win 11 is the problem then.

The Cudnn installation did not went according to the guide, could that be the problem?
 
Forget most of what you know about photography cameras. These are video surveillance cameras and a horse of another color. FPS isn't critical for sharp stills, exposure, shutter speed, is what counts. Switch to H264 rather than H265 encoding. H265 can result in additional blurring. I'm sure wittaj will post an excellent example why that is. The short answer is that 264 just works better for surveillance and results in minimal, at best, disk space savings. Bit rate can also be dropped from maximums to 8192 for a 4MP camera. I doubt you'll see much difference in that drop. Shut off hardware acceleration. It can actually add CPU utilization rather than lower it when using sub streams due to the CPU "handing off" operations needed for hardware acceleration.
 
  • Like
Reactions: looney2ns
As I and @sebastiantombs said, shutter speed is much more important than FPS. I set up a camera for catching license plates. The car is in and out of the frame in under 1 second. I am using 8FPS with a shutter speed of 1/2000.

At night, we have to run a very fast shutter speed (1/2,000) and in B/W with IR and the image will be black. All you will see are head/tail lights and the plate. Some people can get away with color if they have enough street lights, but most of us cannot. Here is a representative sample of plates I get at night of vehicles traveling about 45MPH at 175 feet from my 2MP 5241-Z12E camera running 8FPS:

1642810698566.png



My Deepstack works just fine with D1 resolution with a bitrate of 256 for most of my cameras at 15FPS.


Turn off Hardware Acceleration - it uses more CPU% to offload the video to the the GPU than the savings the GPU provides. Hardware acceleration isn't in the wiki anymore because substreams is way more efficient.

H264 will result in better images. H265 in theory provides more storage as it compresses differently, but part of that compression means it macro blocks big areas of the image that it thinks isn't moving. However, it also takes more processing power of the already small CPU in the camera and that can be problematic if someone is maxing out the camera and then it stutters.

In theory it is supposed to need 30% less storage than H264, but most of us have found it isn't that much. Mine was less than few minutes per day. And to my eye and others that I showed clips to and just said do you like video 1 or video 2 better, everyone thought the H264 provided a better image.

The left image is H264, so all the blocks are the same size corresponding to the resolution of the camera. H265 takes areas that it doesn't think has motion and makes them into bigger blocks and in doing so lessens the resolution yet increases the CPU demand to develop these larger blocks.

In theory H265 is supposed to need half the bitrate because of the macroblocking. But if there is a lot of motion in the image, then it becomes a pixelated mess. The only way to get around that is a higher bitrate. But if you need to run the same bitrate for H265 as you do H264, then the storage savings is zero. Storage is computed based on multiplying bitrate, FPS, and resolution.

1638584913822.png




In my testing I have one camera that sees a parked car in front of my house. H265 sees that the car isn't moving, so it macroblocks the whole car and surrounding area. Then the car owner walked up to the car and got in and the motion is missed because the macroblock being so large. Or if it catches it, because the bitrate is low, it is a pixelated mess during the critical capture point and by the time H265 adjusts to there is now motion, the ideal capture is missed.

In my case, the car is clear and defined in H264, but is blurry and soft edges in H265.

H265 is one of those theory things that sounds good, but reality use is much different.

As always, YMMV. But do not use Codec with BI or you may have trouble, so just stick with H264 without the H or B or + after.



And if you really want it for facial recognition like to identify specific people, most of us have found that facial recognition is more gimmicky and novelty than anything else. If you have to put in 5 or 10 or 15 or 35 pictures or more of yourself in the system for it to recognize it is you...then you shouldn't expect much.... My success rate was under 5% so I moved on to other things LOL.

It can work in certain situations like a business that requires everyone to stop in front of the camera and the camera is at head height. Outside of that, the percentage of being accurate is probably not going to be super high. You will get a lot of false "confirmations" doing a search.

Someone here posted once how horrible it was inside his house identifying his neighbors and others as him. Another guy his kids and wife were being tagged as him.

Unless you spend the big bucks that casinos and airports have LOL.

Heck even in ideal situations like a business with the camera at ideal height and optimal lighting it fails....

What do you think of Dahua's "Uyghur Warnings"?

Good luck. Wish you no ill will. I'm not here to argue with people simply to contribute when and where I can. This is also followed by lending my views on current technology and how it impacts all of us. It's undisputed that splitting of the atom changed the world but once again man decided to...
ipcamtalk.com ipcamtalk.com
 
  • Like
Reactions: sebastiantombs
Thanks for input, talking about several interesting subjects here. But I have not yet got an reply about Deepstack for GPU, did anyone managed to install it?

Sebastiantombs you mentioned in another thread that you using the GPU version, how did you install it and did you install also cudNN ?

Anyone else using Windows 11 ?

Regarding the performance: I makes sence that the shutter speed and exposure times is first priority. I except my highend? Hikvision cameras to fulfill that? DS-2CD2087G2 and DS-2CD2186G2. But I did not find the shutter speed info or how to adjust it.

The exposure time default was 1/12 which I assume would allow only max 12 FPS ? I guess I have to increase exposure times e.g. 1/15 or 1/25 ?

Nice example for FPS and plates, but I am not convinced that could straigth off be same for faces. If you would happen to an example of FPR vs facial detection as well I would be happy if you could share it. I will try out the H264 instead of H265 and remove hardware acceleration (by the way it is still in the wiki page). Good pointing out and my HDD might be too small then.
 
You have to take your cameras (even high end ones) off of default/auto settings and dial them in to your field of view. Shutter should me minimum 1/60s to reduce motion blur at night. Faster if you can.

Yeah, they added Hardware acceleration back but there is a caveat that it has been problematic for many users once Deepatack was introduced. Many here no longer run it because of the problems and it just doesn't save CPU% anymore now with substreams.

Many of us have the GPU version working. Have you confirmed in Task Manager if the GPU is working and using CUDA?
 
Yes I checked the task manager and the GPU is max 25% and given that there was no performance increase with the GPU there is something strange. The deepstack GPU installation failed me?
Or if it is windows 11 perhaps? Would you you advice me to do?

Sebastiantombs I also followed the same instruction, do you remember the setup of Cudnn? I run into difficulties with: "Include cudnn.lib in your Visual Studio project. -> ..." PROBLEM. I did not manage here, I just went to install deepstack in step 5 below. Did anyone seccess with this? How to include cubnn.lin in Visual studio project? or can it be skipped?
 
It's been a while and I don't remember for sure, but it is not necessary to create a project in Visual Studio. Make sure you moved all the appropriate library files to the appropriate folders. It sounds like DS isn't actually seeing or using that card and it should be capable of providing detections in the 50ms range when working properly.
 
It doesn't matter, H254 or H265, in terms of DeepStack. DS only "sees" a static image sent from BI for analysis so encoding never enters into this part of the equation.
 
You must log in or register to reply here.
Top Bottom