Lost almost all my protected clips?

[Schrodinger's Cat]

I periodically protect a clip. I had a couple dozen but I went to find one today, and it was gone. Did a little digging and almost all my protected clips are done. There's 3 remaining from way back in 2020 but all protected clips since then are gone. I usually export the important ones but a few fell through the cracks and it appears they're gone forever despite being protected. I have run maintenance and regenerated database and it hasn't worked. Anything else I can try? The missing raw clips do not appear in the "New" folder where all my clips are, just the usual recent few weeks and the 3 from 2020. Thank you.

 

[IAmATeaf]

Log a support ticket, there may be foibles re protecting clips that need to be fully understood?

In other words I have no idea

 

[looney2ns]

Are you only looking in BI, or have you looked in the folder directly with explorer to see if they are actually still there.

 

[IAmATeaf]

I sort of recall an update which reset or removed the protected clip markers but that was quite a while back.

 

[Schrodinger's Cat]

Are you only looking in BI, or have you looked in the folder directly with explorer to see if they are actually still there.

I checked the folder where all my clips are stored and the only ones outside of the normal history window are the 3 from 2020 that remain protected. None since.

I sort of recall an update which reset or removed the protected clip markers but that was quite a while back.

Interesting that I would still have some but not all. But unfortunate that I lose something very important.

 

[Mike A.]

I don't trust it. I've also lost some over the years. Now anything that I want to make sure that I keep, I export and usually upload/save somewhere else too.

 

[IAmATeaf]

I don't trust it. I've also lost some over the years. Now anything that I want to make sure that I keep, I export and usually upload/save somewhere else too.

Me too, I export and also save the original BVR files to my NAS.

 

[jaydeel]

I've also lost a subset of my protected alerts recently. In my case, I believe I precipitated this by programmatically removing unwanted DAT files from the Alerts folder.

I've subsequently learned that DAT file information is also stored in the database (see the update notes for version 5.5.4 and this post).

I now suspect this action corrupted the database to the point where the protected status for some of my alerts was lost after a database repair/regenerate.

As far as I can tell, only the database knows if an alert and clip is protected. If the database is somehow corrupted, then this knowledge apparently may be lost for good. Before I started digging into this, I sort of expected that a protected Alert JPG might be otherwise 'marked', for example:

• have its file attribute set to read-only
• have its EXIF data modified (like the memo string that was added in version 5.5.4)

But I've found no evidence of this.

PS - I'm still waiting for Ken's reply for a safe way to remove DAT files (which are now consuming a high percentage of the allocated space in my Alerts folder).

 

[mengle007]

Lost all my "protected" files as well. Just updated Blue Iris to "last stable version" 1 week ago - I'm not in front of my machine so I don't have the date. Just noticed the protected file issue this morning. Didn't even think to export these files. Another lesson learned - don't expect software to act like it is supposed to. I keep forgetting that lesson.

 

[jaydeel]

I sort of expected that a protected Alert JPG might be otherwise 'marked', for example:

• have its file attribute set to read-only
• have its EXIF data modified (like the memo string that was added in version 5.5.4)

But I've found no evidence of this.

I’m quoting myself because I suspect this statement may be incorrect. Per the Update Notes for version 5.5.4 Alert JPGs filenames include alert trigger zone and flags. I’ll look into this today.

 

[jaydeel]

I hate when this happens...

I just found a previously forgotten 2-year old (8/4/21) email exchange with Ken where I asked about the DAT filename syntax. Here's his reply:
(Note: back then DAT files had names like 'DW2.20210730_150000.1127538.145.27000.27918.dat')

The filename components are​

camera​

date_time​

offset (msec)​

database flags (to rebuild the db)​

the alert duration (msec)​

Currently my DAT files have filenames like 'FE.20230908_060000.811518.17-0.dat'. So Ken subsequently dropped the extra bits after the offset.
The reason why is described in the Help PDF Update Notes for version 5.6.0 ...


When I inspect* the EXIF data for a current Alert JPG I see the following likely edits by Blue Iris:

• Title
• Subject
• Comments - contains the components mentioned above - and apparently more

** viewable in Windows via Explorer... right-click the filename > Properties > Details tab​

​

Example for an Alert AS-IS (I've not modified the flags or memo)...


Example for an alert for which I have modified the modified both the flags and memo...
Note that my changes are reflected in the alert's database entry (JSON clipstats), but not in the JPG's EXIF data

 

[jaydeel]

I'm still waiting for Ken's reply for a safe way to remove DAT files (which are now consuming a high percentage of the allocated space in my Alerts folder).

Ken has just replied with the following.

"After you delete anything from any managed folder, you must use the Database/Repair & Regenerate option to reconcile the database once again"​

​

"Database/Regenerate has been modified for the next release to look at the read-only status of BOTH the JPG and DAT files, and if either is read-only, the item becomes protected in Blue Iris." YEA!​

I've followed up by asking if Blue Iris will set the JPG/DAT file(s) read-only attributes when the Alert is protected (if the files exist)?

 

[IAmATeaf]

Why do things using read-only attributes of files, why doesn’t he just have a separate DB file that contains details of the protected files, make this file known so that if people need to delete the DB folder they know which file to make a copy off for their protected clips?

The beauty of a separate DB file is it could contain details of the BVR, alert, dat file if applicable etc.

 

[jaydeel]

Why do things using read-only attributes of files, why doesn’t he just have a separate DB file that contains details of the protected files, make this file known so that if people need to delete the DB folder they know which file to make a copy off for their protected clips?

The beauty of a separate DB file is it could contain details of the BVR, alert, dat file if applicable etc.

A more robust solution is certainly highly desirable.
This 'solution' addresses only alerts that have a hi-res or AI mark-up JPG, or DAT file.
EDIT: Just sent a follow-up to Ken.

 

[jaydeel]

I've followed up by asking if Blue Iris will set the JPG/DAT file(s) read-only attributes when the Alert is protected (if the files exist)?

Ken just replied with

"This occurs now on the JPEG. New version will protect/unprotect both."​

I'm definitely not (always) seeing this and passed this on. EDIT: Resolved

Maybe it's just me. Could some of you check if this is the case for you as well?

 

[jaydeel]

A more robust solution is certainly highly desirable.
This 'solution' addresses only alerts that have a hi-res or AI mark-up JPG, or DAT file.
EDIT: Just sent a follow-up to Ken.

Ken just replied

"If the alert has no files and is protected, all we have to go by is the database. If you are not saving files for the alert, you may want to backup the DB folder as well."​

​

Not what were looking for (I did suggest the separate database).

Maybe more requests would move the needle.

 

[wittaj]

Ken just replied

"If the alert has no files and is protected, all we have to go by is the database. If you are not saving files for the alert, you may want to backup the DB folder as well."​

​

Not what were looking for (I did suggest the separate database).

Maybe more requests would move the needle.

@bp2008 (creator of UI3) has been requesting to Ken for years an improvement to the DB - I forget the specifics so won't even attempt to explain it, but basically there are lots of issues with it.

 

[IAmATeaf]

@bp2008 (creator of UI3) has been requesting to Ken for years an improvement to the DB - I forget the specifics so won't even attempt to explain it, but basically there are lots of issues with it.

The biggest seems to be how easily it seems to get corrupted, it also if left just seems to keep growing, well it does on my rig.

 

[jaydeel]

As of version 5.7.9.10, Ken as fixed the issue where Blue Iris was not updating the Alert JPG file's EXIF when the Alert memo was manualy edited.

This means that any edits to the memo should now survive a Database/Regenerate action.
Note that this applies only when the Alert has a JPG!

Example:

Alert JPG "as-created"...
Screenshot shows the Alert's info in the database (via the JSON clipstats cmd)
and the Alert JPG's EXIF data (via the file Properties > Details tab in Windows Explorer)


Alert JPG after manually editing the memo...
Note updated EXIF fields 'Title' and 'Subject' .


EDIT: Astute readers may notice that the JSON clipstats path field changed in the 2nd screenshot. This is the result of a Database/Compact action I performed before taking the 2nd screenshot

 

[mengle007]

Well I apparently did NOT lose my protected files. While viewing a list of confirmed alerts I noticed it was shorter than it should be and led me to the fact that I had at some point selected a particular day on the calendar. When I canceled this and thought to check for my protected files lo and behold there they were. Massive user error.
I'm sure this is not the situation as the OP but thought I would share.