Massive number of files in alerts directory

looney2ns

IPCT Contributor
Joined
Sep 25, 2016
Messages
15,606
Reaction score
22,833
Location
Evansville, In. USA
Save myself lots of money in potential liability costs.

Consider real example. A friend of mine got sued by a person who claims they slipped in ice outside of his house 5 months AFTER THE FACT. The person won…
That's what insurance is for.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,884
Reaction score
48,526
Location
USA
+1 above - and in your friends situation, if true, well that would have been found with daily scrubbing of video.

As @sebastiantombs, I can quickly scrub thru in less than 30 seconds for any overnight activity that warrants saving or future investigation, and a couple of minutes scrubbing key cameras for daytime activity.

You may think having 1.3 million alert files is good, but it is giving you a false sense of security. As you are seeing, at 400,000 files, Windows file manager is already unstable. Wait until it is 3 times that amount.

Here is a test. You have 14 months of alerts you said. Go back 11 months and see how long it takes for you to find a red pickup truck LOL. Or lets go to a two hour window on a specific day - a kid dressed up as a football player for halloween LOL.

Plus in many states a homeowner is not liable for falls due to snow or ice....unless your friend lives in a muni that specially removed it. For example in NY you are not held liable, but in NYC you are...



 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
What ver of BI be are you using?
Reading the help file is helpful.
Hi, and thank you for helping. I update my install every few weeks. I now run 5.6.1.3.
I find the internet much more helpful that the help file. I just searched it for "Database only" and found not results.
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
+1 above - and in your friends situation, if true, well that would have been found with daily scrubbing of video.

As @sebastiantombs, I can quickly scrub thru in less than 30 seconds for any overnight activity that warrants saving or future investigation, and a couple of minutes scrubbing key cameras for daytime activity.

You may think having 1.3 million alert files is good, but it is giving you a false sense of security. As you are seeing, at 400,000 files, Windows file manager is already unstable. Wait until it is 3 times that amount.

Here is a test. You have 14 months of alerts you said. Go back 11 months and see how long it takes for you to find a red pickup truck LOL. Or lets go to a two hour window on a specific day - a kid dressed up as a football player for halloween LOL.

Plus in many states a homeowner is not liable for falls due to snow or ice....unless your friend lives in a muni that specially removed it. For example in NY you are not held liable, but in NYC you are...



I am in NYC unfortunately. I know liability is ridiculous here. It is abused a lot.
I don't have time to check my alerts regularly and it is a huge comfort to know the clips are there.
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
Alerts should be just thumbnail pointers to the BVR file. If you are saving video clips as alerts you are doubling effort and wasting resources.

Then under record tab is cut and combine which takes all the triggers and video and combines them into one file per your setup.
I just checked. I have "Combine or cut video each:" checked with 8 hours and 4GB. I also have the "Also re-triggers" under Alerts unchecked which I believe means that there is only alert file for multiple continuous triggers.

Here is what I don't understand. Many people advice not to create alert clips or thumbnails and save to "Database only". The question is, which database. If it is the continuous 24/7 recording, it is not good because it continuous and therefore huge.

If I could say "Database only" and that database was a separate alerts only database in a different location that the continuous one, then that would be perfect. I am not sure how to test this without messing up my setup but it may be the perfect solution. I will try a test setup on a separate PC. If anyone knows what database is used by "Database only" you will save me a lot of time.

Really, I just want 1 month (14TB) of 24/7 recordings and 1 year (another 14TB) of alerts. Is there another way to do this?
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
You are doing something that isn't typical. Either saving hi rez alert images for every camera and/or saving every AI burned image and/or saving video alert files and/or lots of flagged images that never get deleted and they trigger a lot or double saving files or something else.

Alerts saved to database only are pointers and don't add up in the clip file (clips are any video, pictures).

But if you are not using the default BVR file then EVERY trigger is its own clip and they can add up fast.
If the alerts BVR is separate from the Continuous recording BVR (and a separate drive), then this would be exactly what I want. I will test it over the weekend. Thank you.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,884
Reaction score
48,526
Location
USA
It is saving to the BVR file that is set up for that camera.

What about running a clone camera (s) using just the substreams for that 1 year of data. That would give you alert thumbnails for however long that drive could record 24/7 substream.

But at the end of the day I still think it is giving you a false sense of security.

If you really want that much extra storage, you need to look at a NAS or a more robust VMS system.

I think you would be better served taking those drives and splitting the cameras across them and then daily or weekly scrubs and save anything that looks questionable.

But then again that works for my situation, if you are in NYC where tens of thousands walk past your place every day, that may be a little more problematic.

While having lots of days/months sounds great, the reality of it is unless it was something catastrophic (which you would have known about sooner anyway), most are not going to start scrubbing video for something that may have happened a few weeks ago. I used to save it for years on end and ran out of storage and realized that I will never go back and look at that.

By spending time to dial in the alerts and a frequent peek at what is going on, you would have noticed something around your property within days. I literally every morning in under 30 seconds can scrub what happened the night before and see if anything happened I need to look at further

If a neighbor comes up to me and says "sometime around 2 weeks ago someone backed into my car, can you see if you caught it?" You will find that even with the best scrubbing this is a monumental task. Unless they can narrow down the day/time window, most of us are not going to scour it.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,884
Reaction score
48,526
Location
USA
I am in NYC unfortunately. I know liability is ridiculous here. It is abused a lot.
I don't have time to check my alerts regularly and it is a huge comfort to know the clips are there.
That is unfortunate you live in the town of abused laws...

But in this case, if your friend was sued and lost, how would the video of changed the outcome unless it was clear the fall wasn't on his property? Otherwise it just confirms it was on his property taking away reasonable doubt it wasn't on his property.

Again, a huge comfort knowing the clips are there is immaterial if Windows is unstable with a third of the amount of files you plan to have.

Hopefully one of the options I presented to you above is a more workable solution.
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
That's what insurance is for.
Another can of worms. I can't get umbrella coverage because I am considered too close to the shore - they don't care that I am on a hill, they have their miles distance and that's is all they go by. Complicated. Another problem is they can sue for more than the coverage anyway. Finally, most of these people who sue are crooks. It will be good to show that.

So, I consider BI my friend.
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
But in this case, if your friend was sued and lost, how would the video of changed the outcome unless it was clear the fall wasn't on his property? Otherwise it just confirms it was on his property taking away reasonable doubt it wasn't on his property.
We suspect the person probably staged a fall which may be visible that is is fake. Maybe also had the spot checked out before pretending to fall.
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
It is saving to the BVR file that is set up for that camera.

What about running a clone camera (s) using just the substreams for that 1 year of data. That would give you alert thumbnails for however long that drive could record 24/7 substream.

But at the end of the day I still think it is giving you a false sense of security.

If you really want that much extra storage, you need to look at a NAS or a more robust VMS system.

I think you would be better served taking those drives and splitting the cameras across them and then daily or weekly scrubs and save anything that looks questionable.

But then again that works for my situation, if you are in NYC where tens of thousands walk past your place every day, that may be a little more problematic.

While having lots of days/months sounds great, the reality of it is unless it was something catastrophic (which you would have known about sooner anyway), most are not going to start scrubbing video for something that may have happened a few weeks ago. I used to save it for years on end and ran out of storage and realized that I will never go back and look at that.

By spending time to dial in the alerts and a frequent peek at what is going on, you would have noticed something around your property within days. I literally every morning in under 30 seconds can scrub what happened the night before and see if anything happened I need to look at further

If a neighbor comes up to me and says "sometime around 2 weeks ago someone backed into my car, can you see if you caught it?" You will find that even with the best scrubbing this is a monumental task. Unless they can narrow down the day/time window, most of us are not going to scour it.
Is that alerts BVR in the "New" or the "Alerts" folder?

Cloning is another interesting idea. I could set the clones to record only alerts to their own BVR file. Probably will complicated the interface but guaranteed to work.

It is frustrating that the solutions seems to be so easy: just have BI create a subdirectory per camera or per day.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,884
Reaction score
48,526
Location
USA
Alerts go to alerts folder. What is interesting is that folder will be empty if you look at it in Windows if you are using database only.

If you only have a few cameras you can set up in clips and archiving to save each camera to a different location.

Having a subdirectory for everyday would be a nightmare. I had that on a camera when I was using its free computer VMS and it was horrible having to jump thru all those folders.

BI system works well if using it as intended with BVR files and the way we all scrub videos works well and is quick and efficient.

But if you are doing something like saving mp4 files for every alert I can see that getting problematic fast.

Hopefully the clone option gets you what you need.
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
Having a subdirectory for everyday would be a nightmare. I had that on a camera when I was using its free computer VMS and it was horrible having to jump thru all those folders.
I meant that BI could create an alerts subdirectory for each day automatically. As users of the BI interface we would never notice anything.
 

sebastiantombs

Known around here
Joined
Dec 28, 2019
Messages
11,511
Reaction score
27,692
Location
New Jersey
The bottom line is what you are trying to accomplish taxes the base operating system with the sheer number of files involved. The only way to preserve the video, safely, is to record 24/7 onto disk. That will require, by your numbers, ~150TB of disk and will eliminate millions of alert files. If something happens you will have the date and time available from the law suit. From that information you can locate the incident on those disks. It all comes down to how much you're willing to invest in insuring you have access to that video to protect against a potential loss of hundreds of thousands of dollars. Disk space is relatively inexpensive by comparison.
 

quest100

Pulling my weight
Joined
Aug 10, 2018
Messages
148
Reaction score
223
Location
CA
I have BI ftp alert images plus my front door camera video to a different computer. That computer is backed up via BackBlaze with a one year version history. Around 500 GBs per month. No data base and it would be a real PITA to find something from 10 months ago, but...
 

Thanamite

Young grasshopper
Joined
Jan 7, 2022
Messages
45
Reaction score
17
Location
New York
I have BI ftp alert images plus my front door camera video to a different computer. That computer is backed up via BackBlaze with a one year version history. Around 500 GBs per month. No data base and it would be a real PITA to find something from 10 months ago, but...
Nice. That upload comes to just 1.5Mbps if my calculations are right.
 
Top