DeepStack: Server error 100

Skispcs

Getting the hang of it
Joined
Jun 10, 2014
Messages
104
Reaction score
30
I have been getting the error DeepStack: Server error 100 recently when Blue Iris detects motion.
ANy idea what to look at to continue to troubleshoot this error.
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
That means DS is timing out, detection time is taking too long. My guess is you're running the CPU version. What do DS analysis details show in terms of detection time?
 

Skispcs

Getting the hang of it
Joined
Jun 10, 2014
Messages
104
Reaction score
30
No, I am running the GPU version on a GTX 1650 which normally runs at 0% Utilization with a few spikes when an image is submitted.

CPU is an I7 4790K running below 50% util and the onboard intel GPU is handling the decoding at around 28% most of the time.

DS times were running a few 100 ms the last time I checked them. However I am currently running an older build of BI because of the bugs I was encountering with the newest version(s).
I will probably wait until the Stable branch is updated before I upgrade again.

Final note, these Server 100 errors are new, they only recently started happening so my intuition is that is is not related to the version of BI that I am running.
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
I'm on BI 5.4.9.18 with an i7-6700K and a GTX970. I haven't seen a "100" error since I switched to the GPU version months ago. If those errors have started recently with no other changes happening with the system I'd guess something in the system itself, hardware related, has changed.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,448
Reaction score
47,572
Location
USA
Yep, sounds like something changed or something carried over from a downgrade - for example, did you run custom models and then that isn't available in the version you are running now? I had that issue once when I rolled back. When you downgraded, is BI thinking that it is running the CPU version of DS? I would try with uninstalling and reinstalling DS first. Then look and see if Windows updated something.

What version of BI are you running.

But based on what you are saying, I suspect you are not using the substreams and with a 4th gen CPU you need to take advantage of every opportunity to bring down CPU.

Do EVERY optimization in the wiki and you will see a major drop in the CPU. Substreams are a must, especially once you get over 4MP cameras.

A member here runs 50 cameras on a 4th generation CPU at 30%, so I doubt you are running more cameras than that?

And by EVERY, I mean EVERY. Too many people come here complaining of high CPU usage and claim they have done every optimization in the wiki and once they post screenshots, we see they are not using the substreams and that is probably one of the biggest CPU savers. Do not skip one because you think it isn't important or won't make that big of a deal. Even dropping frame rate a few FPS can make a big difference. No reason to run more than 15FPS, and many us have cams running at 10 to 12 FPS.

If you do not understand what something does in BI, then ask. Too many people also change a setting not realizing what it does and actually makes their performance worse.

Please post a screenshot of your BI camera status page that shows FPS, MP/s, etc.

 

Skispcs

Getting the hang of it
Joined
Jun 10, 2014
Messages
104
Reaction score
30
I have not changed anything in BI in months.
I do not think the problem is CPU usage. I am below 50% and BI is only around 38% most of the time.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,448
Reaction score
47,572
Location
USA
Yeah, but below 50% can turn into the CPU maxing out with a lot of wind motion and sending more than the normal amount of images to DS and then it times out, especially if there is a bottleneck between the mobo and the graphics card or the GPU is not strong enough to handle all the DS at the same time.
 
Last edited:

Skispcs

Getting the hang of it
Joined
Jun 10, 2014
Messages
104
Reaction score
30
Yeah, but below 50% can turn into maxing out with a lot of wind motion and sending more than the normal amount of images to DS and then it times out, especially if there is a bottleneck between the mobo and the graphics card or the GPU is not strong enough to handle all the DS at the same time.
No, I don't think so man.
gpu util.PNG
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,448
Reaction score
47,572
Location
USA
Of course right now it is 0% you don't have any action - watch that little graphic and go have someone trigger your cams and see what happens....does it spike to 100 immediately and then time out or does it jump to 10% and still times out. Or maybe it stays 0% when motion is added indicating the GPU isn't working. There is a big difference in what is causing the timeout between both of those scenarios.

I have a 4th gen that sips along at under 10% CPU but when DS was introduced it would max out the CPU if too many cameras where triggering it.

As an example, just this week I added a GPU to my 4th gen.

I wanted to offload OpenALPR to it.

My GPU graph showed 0% like yours.

I add OpenALPR to it and it hums at 5% GPU. Then a car goes by and it spiked to 100% and the OpenALPR timed out.

Do not underestimate the amount of load it takes to process an image in an AI application.
 
Last edited:

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,690
Location
New Jersey
The point of optimizing BI is to lower CPU utilization not GPU utilization. While your CPU may be sitting below 50% when you look at it, it can easily spike past 75% when motion occurs on multiple cameras. While a GPU does unload the CPU with DS, there still is a load from Python on the CPU that needs to be taken into account. Heck, I'm running 21 cameras, ~225MP/s, and the CPU is peaking at around 30% with motion/DS and normally sits around 20%. The heat savings alone on the CPU make it worthwhile to optimize.
 

Skispcs

Getting the hang of it
Joined
Jun 10, 2014
Messages
104
Reaction score
30
The point of optimizing BI is to lower CPU utilization not GPU utilization. While your CPU may be sitting below 50% when you look at it, it can easily spike past 75% when motion occurs on multiple cameras.
Understand what you are saying. That is NOT happening here.
I am watching task manager as the cameras are triggered and neither the CPU nor the GPU are spiking more than a few percent even when multiple cameras are detecting motion.
CPU Util.PNG Spikes here are when I had someone move in back and forth through multiple cameras. GPU spike posted before is two images passed to DS after walking in front of those two cameras.
 

Skispcs

Getting the hang of it
Joined
Jun 10, 2014
Messages
104
Reaction score
30
Of course right now it is 0% you don't have any action - watch that little graphic and go have someone trigger your cams and see what happens....does it spike to 100 immediately and then time out or does it jump to 10% and still times out. Or maybe it stays 0% when motion is added indicating the GPU isn't working. There is a big difference in what is causing the timeout between both of those scenarios.

I have a 4th gen that sips along at under 10% CPU but when DS was introduced it would max out the CPU if too many cameras where triggering it.

As an example, just this week I added a GPU to my 4th gen.

I wanted to offload OpenALPR to it.

My GPU graph showed 0% like yours.

I add OpenALPR to it and it hums at 5% GPU. Then a car goes by and it spiked to 100% and the OpenALPR timed out.

Do not underestimate the amount of load it takes to process an image in an AI application.
That little spike in the GPU graph is two cameras passing 4 images each to the GPU as someone walked to the front door.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,448
Reaction score
47,572
Location
USA
OK so it looks like that is ruled out then.

What BI version are you running now and what version were you at that you rolled back away from?

Have you done the ole "reboot the computer" and see if it knocks out some gremlins that popped in? It is amazing how many things that fixes LOL. As a course of action, I usually restart my system every couple of weeks after I log into it just to make sure something isn't stuck in RAM.

Can you go back in the logs or settings and see the last time Windows did an update. Despite our best efforts to prevent that, sometimes Windows overrides our settings :banghead:
 
Top