DeepStack Temp files growing / not being deleted in the DeepStack Temp folder

[brettcp]

I've been running the latest BI5 and the integrated AI for a while now (I always keep BI updated).. yesterday I noticed my server had run out of disk space, after a bit of investigation I found that DeepStack's temp folder had grown to 132,000+ files, consuming 160GB of space. The folder is "C:\Users\<myuser>\AppData\Local\Temp\DeepStack". I restarted BI but the files were not purged, so I purged them manually, rebooted, then yesterday un-installed and re-installed DeepStack (CPU version), but this morning there are 54 items in that folder. I can watch the folder and see that files are created, then shortly after deleted (which is what I'd expect), but it seems that other files are left behind and not deleted as the folder is slowly growing.

Any ideas here? System has plenty of power, sits at around 25% CPU and 18% memory use with BI and AI/DeepStack running, 10 IP cameras. Even when there's a fair amount of load, the CPU only jumps up to around 40-50% use at most.

Thanks!

 

[KenAllen15]

Are you on the latest version of blue iris? I had that issue (files were in the redis subfolder), but updating BI fixed it.

 

[brettcp]

Are you on the latest version of blue iris? I had that issue (files were in the redis subfolder), but updating BI fixed it.

Yes, I always keep it up to date. As of today I'm on 5.4.9.6 (dated 7/8/21).

 

[sebastiantombs]

You made me look. About 7500 files and 900MB in there. Sorting by date, it looks like about a weeks worth of files. BI 5.4.9.6, GPU version of DS, 20 cameras but not all of them run on DS.

 

[brettcp]

You made me look. About 7500 files and 900MB in there. Sorting by date, it looks like about a weeks worth of files. BI 5.4.9.6, GPU version of DS, 20 cameras but not all of them run on DS.

I have 336 files as of this morning (just 24 hrs after my last post/update from yesterday). I just reported it to Blue Iris support and referenced this thread.

 

[sebastiantombs]

I just checked again and right now there are no files in the temp directory. I wonder if that's related to switching to the GPU version?

 

[wittaj]

Weird - my temp file is empty. I am running the CPU version.

 

[brettcp]

I just checked again and right now there are no files in the temp directory. I wonder if that's related to switching to the GPU version?

Perhaps... I am running the CPU version. Seems like its not cleaning up after itself properly as my files continue to grow in qty. If it were a permission issue, I'd assume it wouldn't be able to delete anything from that folder (and it does clean up the majority of the files it creates there) - it just tends to leave some behind.

 

[sebastiantombs]

I think that's a glitch specific to your installation. Maybe permissions or maybe a corrupt file in DeepStack.

 

[brettcp]

I think that's a glitch specific to your installation. Maybe permissions or maybe a corrupt file in DeepStack.

Could be.. but I did un-install DeepStack (then cleaned up its leftover files), rebooted, then re-installed it. I have not yet re-installed BI5. Will see what BI support has to say about it.

 

[brettcp]

This appears to be fixed in 5.4.9.8. I purged the temp folder, updated to 5.4.9.8, restarted, and its been ~20 hours now and Deepstack's temp folder is empty and being cleaned up/maintained as expected. I never received a response after submitting a ticket to Blue Iris on Sunday, but it does appear the issue is fixed in 5.4.9.8.

 

[brettcp]

Update: Still appears to be fixed in my particular case with 5.4.9.8 (for now, at least), but I did receive a reply from Blue Iris support:

"We are aware of the issue but no info on the root cause or fix. Need to investigate with DeepStack."

They also mentioned a workaround:

"Unfortunately what I have isn’t so much a fix as it is a workaround. It’s just using Windows Task Scheduler to call a batch file that runs ‘del C:\DeepStack\redis\temp-*.rdb’ every hour."

 

[brettcp]

Looks like this bug is back on recent builds.. had 12,000+ files sitting in there from the last 9 days.. I'm just going to setup a scheduled task to delete them at this point.

 

[bigscout79]

I agree, I'm seeing it today.

Also, it looks like:
c:\users\<name>\AppData\Local\Temp\DeepStack is filling up too.

using a TaskScheduler job for now..

 

[KenAllen15]

I agree, I'm seeing it today.

Also, it looks like:
c:\users\<name>\AppData\Local\Temp\DeepStack is filling up too.

using a TaskScheduler job for now..

Mine appears to be in /windows/temp/Deepstack about 3gb per hour!

Edit: I stopped the deepstack server and the folder is still filling up makes me think it's a blue iris issue and not a deepstack issue.

 

[spammenotinoz]

I don't see the issue in GPU version 2021.09.01, so if you have not already suggest upgrading deepstack.

 

[icpilot]

Mine appears to be in /windows/temp/Deepstack about 3gb per hour!

Edit: I stopped the deepstack server and the folder is still filling up makes me think it's a blue iris issue and not a deepstack issue.

Tagging this thread as I've been battling unexpected C:/ growth for quite some time. I checked both directories mentioned in this thread and the growth isn't coming from those directories. This machine is only used for BI and OpenALPR. The runaway growth is somewhere, but I am still trying to identify it.

 

[brettcp]

Tagging this thread as I've been battling unexpected C:/ growth for quite some time. I checked both directories mentioned in this thread and the growth isn't coming from those directories. This machine is only used for BI and OpenALPR. The runaway growth is somewhere, but I am still trying to identify it.

Look at tools like WinDirStat or Space Monger, they should be able to tell you where the growth is coming from.

 

[icpilot]

Look at tools like WinDirStat or Space Monger, they should be able to tell you where the growth is coming from.

I found it - thanks to you posting BI's response to this issue. In my case those temp files are in directory C:\DeepStack\redis - which is what BI had shown in their response to you.

And thanks too, for the reference to those tools. Downloading them now ....

 

[brettcp]

33k files accumulated in the last 72 hours, ouch! Hope it gets fixed soon.