Drive free space is inaccurate

Jeremy Boelter

Jeremy Boelter

n3wb
Nov 5, 2018
8
5
Frigid North
Hello!

Hoping someone here can help me through this -- i'm not sure when this started (within the last year, probably), but i'm getting frequent disk over-allotment errors and inaccurate free space reporting in BI.

I'll see the following warning when looking at the disk stats in the WebUI:

1697039669750.png

Prior to repair, the storage tab looks like:
1697039731117.png
(overflow on the G drive visualization? also almost the entire drive is allocated for BI, so i'm not sure where the green portion is coming from)

And after a repair:
1697039846064.png

The only solution i've found to date is to completely delete the \db folder and let it rebuild. When it regenerates the db, it typically will only find a few .bvr files (maybe just \new?) and i lose most of my history. I end up nuking everything (all storage and \db) every month or so and starting over.

Here are my storage configs:

1697040001239.png

1697040010687.png

1697040017414.png

1697040024712.png

And disk allocation in windows:

1697040048232.png

Any help here would be greatly appreciated -- I can attach logs / provide other screens or context if needed.

Thanks!

Jeremy
 
Why are you moving the video files around? That just wastes CPU cycles and bogs down the IO functions. Use that 16TB drive to store NEW and let it delete when it gets full. Same goes with drive I.

I am guessing that all of that constant movement of files is getting things over allocated during that move process.
 
  • Like
Reactions: kklee
samplenhold said:
Why are you moving the video files around? That just wastes CPU cycles and bogs down the IO functions. Use that 16TB drive to store NEW and let it delete when it gets full. Same goes with drive I.

I am guessing that all of that constant movement of files is getting things over allocated during that move process.
Click to expand...

Thanks for the quick response -- using multiple drives as such:

H: = NVME for High IO, with the intent of near-instant retrieval of recent alerts and previews on the webui (correct me if i'm wrong there)
G: = External USB 3.0 for long-term storage
I: = 2nd External USB 3.0 for longer-term storage

It's enterprise hw so i'm really not concerned about burning cycles... tons of headroom--specifically two 24 core Xeon 2690v3s running @ around 18% load.

While I appreciate the notion of being frugal with resources, it's really not a limiting factor in this case. It appears this is a common scenario that 'Storage' is an external / network destination--both are discussed frequently here and also called out in the BI documentation.

Is it possible this setting may have an impact?
1697052576556.png

Perhaps lowering that size so we have more frequent but less sustained writing?
 
  • Like
Reactions: bp2008
Make sure you don't have system restore turned on for those drives and set the recycle bin to zero.
 
  • Like
Reactions: JNDATHP
I've had similar problems to what you showed there, thankfully not every month, but still too often. October 2nd I discovered most of my cameras hadn't recorded anything for the past 4.5 days because Blue Iris forgot about most of the clips and let the disk get full. I went through much the same process trying to fix it. Repair/Regenerate took SEVEN HOURS and didn't fix anything. I had to delete the database and regenerate it from scratch. Took another seven hours. Meanwhile no recordings could happen because the rebuild procedure disables recording while it is active.

This is one of the reasons I run a second Blue Iris system on a different machine and do not update it at the same time as the main system.

I like Blue Iris, but I can't make excuses for the clip database. It is very unpleasant (that is the diplomatic way of saying it).
 
  • Like
Reactions: giomania, looney2ns and actran
Jeremy Boelter said:
It's enterprise hw so i'm really not concerned about burning cycles... tons of headroom--specifically two 24 core Xeon 2690v3s running @ around 18% load.
Click to expand...
How many cams and what are your kB/s and MP/s rates? For all of that hardware, and at 18% load?

I have an i7-8700 with 22 cams running at 20100 kB/s and 560 MP/s at only 16%.
 
samplenhold said:
How many cams and what are your kB/s and MP/s rates? For all of that hardware, and at 18% load?

I have an i7-8700 with 22 cams running at 20100 kB/s and 560 MP/s at only 16%.
Click to expand...
Looks like I'm closer to 15% when the BI console isn't open -- that's total, btw. BI is only using about 8%.
22 cams, 15600 kB/s and 520 MP/s
 
bp2008 said:
I've had similar problems to what you showed there, thankfully not every month, but still too often. October 2nd I discovered most of my cameras hadn't recorded anything for the past 4.5 days because Blue Iris forgot about most of the clips and let the disk get full. I went through much the same process trying to fix it. Repair/Regenerate took SEVEN HOURS and didn't fix anything. I had to delete the database and regenerate it from scratch. Took another seven hours. Meanwhile no recordings could happen because the rebuild procedure disables recording while it is active.

This is one of the reasons I run a second Blue Iris system on a different machine and do not update it at the same time as the main system.

I like Blue Iris, but I can't make excuses for the clip database. It is very unpleasant (that is the diplomatic way of saying it).
Click to expand...
Curious how many clips/images you have stored that it takes that long to rebuilt. I can perform a rebuild in a few minuets.
He does need to completely replace the DB though...
 
I have Database Rebuild scheduled for every night. A rebuild take literally seconds (4TB) with this regime and so poses little threat to footage capture and I have no issues. Maybe a more regular routine revuild is needed?
 
JNDATHP said:
Not sure how to do this; perhaps you could provide the answers?
Click to expand...
If you right click on the recycle bin > properties, you'll get options that allow you to adjust sizing or just disable all together:
1697083557190.png

System restore can be turned off by searching the start menu for "system protection" & click on "create a restore point". Click "configure" and you can configure / disable from that dialog:
1697083874996.png
 
  • Like
Reactions: JNDATHP
CCTVCam said:
I have Database Rebuild scheduled for every night. A rebuild take literally seconds (4TB) with this regime and so poses little threat to footage capture and I have no issues. Maybe a more regular routine revuild is needed?
Click to expand...
good call -- I forgot to mention that in my original post. I rebuild nightly as well.
 
  • Like
Reactions: CCTVCam
Maybe a build issue? Might be worth posting a build number. Mike Lud is usually around somewhere so might know something.
 
fenderman said:
Curious how many clips/images you have stored that it takes that long to rebuilt. I can perform a rebuild in a few minuets.
He does need to completely replace the DB though...
Click to expand...

1697118346558.png

Actually the slowness was probably due to an HDD malfunction. I remember this normally taking only 20 minutes or so.

1697118359468.png
 
  • Like
Reactions: fenderman
Jeremy Boelter said:
5.8.0.1 currently, i see .0.4 is available, might try that too.
Click to expand...

I will say beware changing builds if running CPAI. I've encountered all kinds of problems doing it and have had to wipe the whole system and reinstall windows every time. I'd only change if others are running the build you intend to move to sucecssfully, or use a system back up utility to backup your entire system including windows 1st so you can go back without data loss.
 
samplenhold said:
So I am curious. You have more records and files than me, but your index time is way less than mine. Anyone with any ideas?

View attachment 174621
Click to expand...

My database is on a SATA SSD. What is yours on? But I also remember index being slower, more like yours, in the past. I'm on 5.8.0.4. Maybe recent updates sped it up.
 
You must log in or register to reply here.
Top Bottom