Blue Iris Clips and Files - Disk Usage Utility

[jaydeel]

I've created a PowerShell script for summarizing the contents of the Blue Iris clip directories.

I created this script because I've always found examining the unindexed Blue Iris clips folders using Windows Explorer to be tedious at best.

I'm confident to say that the first time you use this script, you will learn something new or interesting about the contents of your clips directories.

Output and Features:
The following screenshots illustrate typical output - in this case for the 'New' and 'Alerts' directories.
Other directories can be summarized as well (see 'Customizing' below).

• Each table summarizes usage for a single file type, e.g., *.bvr.
• File usage is broken down per camera. The oldest and newest camera file dates are also listed.
  Also, the number of days of files per camera are now listed (see V2 details).
• By default, cameras are listed in descending order by total disk usage.
  The sort order can be now customized (see V3 details).
• Subfolders, if they exist, are recursively examined. The 1st screenshot shows several for illustration.
• The use of non-default filename formats are now supported (see V4 details).

Caveats:
1. The files in the clip directories must use Blue Iris' default naming pattern beginning with the '&CAM' macro. Other naming patterns are NOT supported.
2. Files sizes reported by PowerShell may not match exactly with Windows Explorer.
3. The script can take awhile to gather the clip data. EDIT (1/6/2023) It's only been tested on an 8TB storage drive. YMMV

Download:
To use this script, simply extract the 'BI_clip_usage.ps1' file from the attached zip file to your desktop, or scripts folder.
Run the script from a PowerShell window.
The script can also be executed from cmd.exe using a command like powershell.exe "C:\ps_scripts\BI_clip_usage.ps1"

Customizing:
You can edit the script to include any folders you wish.
Look for the following section in the code.

EDIT (11/05/2023): V6 - Note the new user-settings file. See BREAKING CHANGE in post #24 for more information

 

[jaydeel]

NOTE: If you've never used PowerShell on your Blue Iris PC, you may first to need to change its default security policy.
Open PowerShell in Administrator mode, and type the following, then hit the Enter key Set-ExecutionPolicy RemoteSigned
(click here for a nice beginner's tutorial to using PowerShell).

 

[tech101]

Thank you so much for putting this together this will be helpful. I sometimes go back in BI to see or Drives what are the oldest files there if something has not purged out for what ever reason, Running this now to see.

I am guessing this will take a while to read/output the data ? I have a very large DB.. Its been stuck here or guessing still reading the DB/Files..

 

[jaydeel]

Thank you so much for putting this together this will be helpful. I sometimes go back in BI to see or Drives what are the oldest files there if something has not purged out for what ever reason, Running this now to see.

I am guessing this will take a while to read/output the data ? I have a very large DB.. Its been stuck here or guessing still reading the DB/Files..

Wow. My storage is only 8TB - about 5% of yours (175TB).... I should have added that to my caveats list!!

The routine that reads the folders is finished before the results are written to the terminal. Your screenshot suggests an issue in the table-building routine.
I'll do some reading and see if you can allocate more memory to PowerShell.

 

[tech101]

I think yeah It timed out or need more alocation for the memory.. It never moved beyond that point. Thank you so much for your time and effort for this it seems pretty useful for most of people here the above PS you got will be fine.

 

[jaydeel]

Script updated to V2. see post #1.

Changelog:

1. NEW: added column '#Days' to output
2. FIX: corrected local time handling of dates in 'Oldest' & 'Newest' output columns

Screenshot:
New column '#Days' shows how many days of each cam that are getting recorded, that have alerts, etc.

 

[actran]

@jaydeel Any chance you can make sorting configurable in user settings? Perhaps, if I prefer my output sorted by Camera name?

 

[jaydeel]

@jaydeel Any chance you can make sorting configurable in user settings? Perhaps, if I prefer my output sorted by Camera name?

Yes

 

[jaydeel]

Script updated to V3. see post #1.

Changelog:

1. NEW: added arguments -SortColNo, -Ascending
2. NOTE: if using '-SortColNo 1', the entry 'Total' in the 'Camera' column is tweaked to keep it at top

Screenshots:

 

[actran]

@jaydeel v3 scripts works fine without args. Then I ran it a 2nd time with-SortColNo option and got the following...

 

[jaydeel]

@jaydeel v3 scripts works fine without args. Then I ran it a 2nd time with-SortColNo option and got the following...

View attachment 174936

I rushed and failed to test in PS5. My bad.

 

[jaydeel]

@jaydeel v3 scripts works fine without args. Then I ran it a 2nd time with-SortColNo option and got the following...

I've uploaded a replacement zip file for V3. See post #1.

 

[actran]

Thanks @jaydeel . Confirmed: new sort options are working now.

 

[joshwah]

I've changed my recording path to the following, which makes it easier to manage for me, especially long term recording.

Is it possible to adjust the script for this format?

&CAM.\%d-%m-%y.\&CAM.%Y%m%d_%H%M%S

Which ends up looking like this:
D:\Blueiris\video\frontdoor\18-10-23\
D:\Blueiris\video\frontdoor\17-10-23\
D:\Blueiris\video\frontdoor\16-10-23\
D:\Blueiris\video\frontdoor\15-10-23\

etc.

 

[jaydeel]

Script updated to V4. see post #1 to download the file

this new version may helpful...

• if you do NOT use the default file format (Camera settings > Record tab) ... see changelog items 1-2 below
• if you want to save results by each folder/filetype ... see changelog items 6-7 below)

Changelog:

1. NEW: added support for Record tab non-default file formats, e.g., &CAM\%d-%m-%y\&CAM.%Y%m%d_%H%M%S
   added new user setting $default_filename_format_flg
2. NEW: added new -DefFnameFormat argument (to tempoarily override user setting $default_filename_format_flg)
3. NEW: added new user setting $alerts_folder_path; script no longer assumes the Alerts folder contains the substring 'Alerts'
4. BUG: fixed issue resulting from $sourceN values pointing to drive letters only, e.g., $source3 = 'D:\' or "D:"
5. CHANGE: changed folder used to store the script's temporary CSV files from 'C:\temp\bi_clips_usage' to 'C:\temp\bi_clip_usage'
6. CHANGE: script no longer creates only a single temporary file 'output.csv file', now creates a separate file for each table displayed, like 'output_Alerts_dat.csv'
7. NEW: added display message showing location of the script's temporary CSV files

The following screenshot shows the new USER SETTINGS.

This screenshot shows new features in the Console output.

Finally, thanks to @joshwah for pre-testing this new version.

 

[scoob8000]

I just noticed something. How come sometimes it's going by total GB and others total TB?

Folder: O:

Camera Type Count Total_TB Avg_GB Max_GB #Days Oldest Newest
------ ---- ----- -------- ------ ------ ----- ------ ------
TOTAL bvr 150 1.81 12.34 19.53 2023-10-11T22:00:04 2023-10-30T06:25:52

/////////////////////////////////////////
Folder: D:\BlueIris\New

Camera Type Count Total_GB Avg_MB Max_MB #Days Oldest Newest
------ ---- ----- -------- ------ ------ ----- ------ ------
TOTAL bvr 2168 7430.64 3509.67 20011.35 2023-10-09T05:14:15 2023-10-30T06:25:52

 

[jaydeel]

I just noticed something. How come sometimes it's going by total GB and others total TB?

Folder: O:

Camera Type Count Total_TB Avg_GB Max_GB #Days Oldest Newest
------ ---- ----- -------- ------ ------ ----- ------ ------
TOTAL bvr 150 1.81 12.34 19.53 2023-10-11T22:00:04 2023-10-30T06:25:52

//
Folder: D:\BlueIris\New

Camera Type Count Total_GB Avg_MB Max_MB #Days Oldest Newest
------ ---- ----- -------- ------ ------ ----- ------ ------
TOTAL bvr 2168 7430.64 3509.67 20011.35 2023-10-09T05:14:15 2023-10-30T06:25:52

The column headers and units are automatically determined based on the data in the column.
This is done is to prevent oversized column widths, and a more difficult to read table (my personal opinion).
The only other real option is to use bytes for all columns, which gets ugly.

Just to confirm, can you DM me the CSV files for these 2 tables?

 

[jaydeel]

Script updated to V5. see post #1 to download the file

Please note the BREAKING CHANGE
see changelog item 1 below

This new version may helpful if you want to track changes over time for each folder/filetype
see changelog items 3-5 below

Changelog:

1. BREAKING CHANGE: moved USER SETTINGS to separate script file 'BI_clip_usage_user_settings.ps1'
2. NEW: added new user setting '$app_path' to customize the destination for 'output.csv' files;
   as a result, this script no longer outputs CSV files to 'C:\temp\bi_clip_usage'
3. NEW: added default time stamping of the 'output.csv' filenames, like 'output_Alerts_dat_2023-10-30_125531.csv';
   as a result, you can now track clip usage by folder & file type over time
4. NEW: added new user setting '$timestamp_output_csv_files_flg' to opt out of 'output.csv' file time stamping
5. NEW: added new user setting '$archive_lifetime_days' to auto-prune the time stamped 'output.csv' files
6. NEW: script can be executed in the background (e.g., via Windows Scheduler to save regularly save clip usage stats)
7. CHANGE: changed logic for determining optimum units 'TotalXX', 'AvgXX' &'MaxXX' columns
8. CHANGE: removed underscore characters from 'Total', 'Avg' & 'Max') column headers

The following screenshot shows the new REQUIRED USER SETTING.

The following screenshot shows the new optional USER SETTINGS.

Again, thanks to @joshwah for pre-testing this new version.

 

[jaydeel]

The screenshot below is an example showing how to view/compare clip usage by folder & file type over time (2 days in this case).

The CSV files in this example were automatically created by running the script 'BI_clip_usage.ps1' via Windows Task Scheduler at 5:10a daily.

The CSV files are being viewed using the lightweight PowerShell 'Out-GridView' Cmdlet, as described in this post

CSV file viewers (lightweight)

Many of the utilities and Blue Iris actions I post on ipcamtalk create CSV files. I thought I'd share the 'lightweight' CSV file viewers I routinely use to inspect *.csv files. Of course, you can open CSV files in your favorite text editor, but often these viewers are not ideal for inspecting...

ipcamtalk.com

 

[Philip Gonzales]

Maybe I missed a really simple step here but I downloaded the V5 version of the script, but every time I ran it, I got the following error.

ERR/failed to load user settings file 'BI_clip_usage_user_settings.ps1'
Tip: try executing it directly to confirm it exists and reveal syntax errors

I validated both scripts existed in C:\scripts.

I ended up having to change line 868 to the explicit full path of the settings .ps1 file. Then it worked fine.
$user_settings_filename = 'C:\Scripts\BI_clip_usage_user_settings.ps1'

When line 868 was set as below
$user_settings_filename = $ScriptName.Replace(".ps1", "_user_settings.ps1")

The output of Write-Host $user_settings_filename was
BI_clip_usage_user_settings.ps1