Yet Another Free Extension for Blue Iris Adding AI Object Dectection/Reduction in False Alarms/Enhanced Notification of Activity/On Guard

[Ken98045]

The two cameras I have setup are for my front door and my driveway. I left the cooldown for both of them set to the default value of 300 seconds. I defined the front door cam as "entry door" and the driveway cam as "driveway area". I tried lowering the cooldown to 10 seconds for both cameras and restated the app. When I restarted there were unprocessed files in my aiinput directory and the app processed 1 file for each camera and area of interest and then stopped even though some of the files had timestamps of more than 10 seconds apart. I definitely have something misconfigured somewhere. I'll keep playing around to see if I can figure out what's wrong.

On Guard does not process any unprocessed pictures on startup, it just loads them into the working set. It does send the first image to DeepStack for analysis so that any objects in the first picture can be identified. In addition, as you step through pictures any pictures that have motion in them they will be added to the motion database so that they will be shown when "motion only" is down/green. It does not send them through for notifications. Only pictures that are taken while On Guard is active will cause notifications

It does sound like you have things configured so that at least some of the later images in the log file you provided should have triggered the http notification. I'll look at it tomorrow.

 

[Vettester]

It does sound like you have things configured so that at least some of the later images in the log file you provided should have triggered the http notification.

I added a 3rd camera last and let the app run through the night. The cooldown for all three cameras is set to 30 seconds.

Here is the log file from overnight:

2020.12.29 22:38:53:1658 - On Guard started at: 12/29/2020 10:38:53 PM
2020.12.29 22:43:50:9603 - 0 interesting objects found in file: C:\aiinput\aiedw.20201229_224350491.jpg
2020.12.29 22:43:59:1457 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201229_224358412.jpg
2020.12.29 22:43:59:1637 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201229_224358412.jpg&memo=Someone

is in the driveway
2020.12.29 23:14:00:0452 - 0 interesting objects found in file: C:\aiinput\aifrontdoor.20201229_231359606.jpg
2020.12.29 23:14:05:5294 - 0 interesting objects found in file: C:\aiinput\aiedw.20201229_231405068.jpg
2020.12.29 23:14:08:8556 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201229_231408365.jpg
2020.12.29 23:26:00:8291 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201229_232600347.jpg
2020.12.29 23:35:16:8093 - 0 interesting objects found in file: C:\aiinput\aiedw.20201229_233516356.jpg
2020.12.29 23:35:35:9234 - 0 interesting objects found in file: C:\aiinput\aiedw.20201229_233535446.jpg
2020.12.29 23:35:46:8043 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201229_233546333.jpg
2020.12.30 01:33:19:2857 - 0 interesting objects found in file: C:\aiinput\aifrontdoor.20201230_013318826.jpg
2020.12.30 01:33:21:5169 - 1 interesting objects found in file: C:\aiinput\aiedw.20201230_013321064.jpg
2020.12.30 01:33:25:5547 - An error occurred while sending the request.
2020.12.30 01:33:26:6309 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_013326149.jpg
2020.12.30 01:36:06:6232 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_013606146.jpg
2020.12.30 01:39:00:6399 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_013900141.jpg
2020.12.30 01:42:57:6141 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_014257134.jpg
2020.12.30 02:35:52:5674 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_023552053.jpg
2020.12.30 04:14:18:4334 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_041417899.jpg
2020.12.30 04:57:58:2129 - 0 interesting objects found in file: C:\aiinput\aiedw.20201230_045757742.jpg
2020.12.30 05:00:22:8895 - 1 interesting objects found in file: C:\aiinput\aiedw.20201230_050022412.jpg
2020.12.30 05:00:32:3395 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_050031828.jpg

If I'm reading this correctly I should have received notifications at the following times:

2020.12.29 22:43:59:1457 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201229_224358412.jpg - successful
2020.12.29 23:14:08:8556 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201229_231408365.jpg - no alert
2020.12.30 01:33:21:5169 - 1 interesting objects found in file: C:\aiinput\aiedw.20201230_013321064.jpg - error occurred
2020.12.30 01:33:26:6309 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_013326149.jpg - no alert
2020.12.30 01:36:06:6232 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_013606146.jpg - no alert
2020.12.30 01:39:00:6399 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_013900141.jpg - no alert
2020.12.30 01:42:57:6141 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_014257134.jpg - no alert
2020.12.30 02:35:52:5674 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_023552053.jpg - no alert
2020.12.30 04:14:18:4334 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201230_041417899.jpg - no alert
2020.12.30 05:00:22:8895 - 1 interesting objects found in file: C:\aiinput\aiedw.20201230_050022412.jpg - no alert

Out of the 10 notifications only the first one came through while the rest failed. The error that occurred was from the last camera added, which is configured the same as the others, but it doesn't provide a lot of detail as to what happened.



I've removed all but one of the cameras and will continue to troubleshoot this issue.

 

[Ken98045]

On Guard does not process any unprocessed pictures on startup, it just loads them into the working set. It does send the first image to DeepStack for analysis so that any objects in the first picture can be identified. In addition, as you step through pictures any pictures that have motion in them they will be added to the motion database so that they will be shown when "motion only" is down/green. It does not send them through for notifications. Only pictures that are taken while On Guard is active will cause notifications

It does sound like you have things configured so that at least some of the later images in the log file you provided should have triggered the http notification. I'll look at it tomorrow.

OK, there was very definitely a problem. It should be fixed within a couple of hours of this post. It was pretty nasty to track down.

 

[Ken98045]

The latest release had a bad problem. It should be fixed within a couple of hours. Please don't use it in the mean time.

 

[Ken98045]

OK, the problem I found has been corrected. There is a new release at: Release On Guard Security Assistant Version 1.4.2 · Ken98045/On-Guard

The problem fixed was related to the fact that after the first one motion is recognized subsequent ones weren't reported. This bug should also have fixed the most recently reported bug by VetTester. That one might have been unrelated, but we need to start here.

After downloading the new release please let me know when (not if) you find additional problems.

 

[jz3082]

1.4.2

I am connecting to the BI/On Guard PC with RDP. I cannot see all of the settings on the MQTT config page. Can you add a scroll bar to that page?

 

[Ken98045]

1.4.2

I am connecting to the BI/On Guard PC with RDP. I cannot see all of the settings on the MQTT config page. Can you add a scroll bar to that page?

Yes. I thought it was a little long but I thought it would fit. I have to admit that I didn’t check it out

 

[jz3082]

I thought the time in DeepStack was in sync with the PC but now the time is 6 hours ahead. I installed 1.4.1 and had to uninstall 1.4.1 to install 1.4.2. All of the settings from 1.4.1 were still in place but I had to delete the cameras before images would process. I am getting MQTT messages. So far everything is working accept for the time being off in DeepStack. The processing times below are from deepquestai/Deepstack in MODE=High with WSL enabled on my PC. DeepStack is in a Docker container. The processing time in high mode in Docker is about the same as the beta Windows CPU version in normal mode.

 

[Ken98045]

Yes. I thought it was a little long but I thought it would fit. I have to admit that I didn’t check it out

Release 1.4.3 is now on Github. This includes your request for scrolling on the MQTT dialog. That's pretty much it.

 

[Vettester]

Release 1.4.3 is now on Github. This includes your request for scrolling on the MQTT dialog. That's pretty much it.

I've only been running this for about 30 minutes and it appears to be working now. I did notice a minor bug in the notification. If you have multiple AOIs setup it combines the name of all of them in the memo for the notification.

I'll let the one camera I have setup run through the night and let you know if I see any issues.

-Thanks

 

[Ken98045]

I've only been running this for about 30 minutes and it appears to be working now. I did notice a minor bug in the notification. If you have multiple AOIs setup it combines the name of all of them in the memo for the notification.

I'll let the one camera I have setup run through the night and let you know if I see any issues.

-Thanks

That will be in the next release. The timing on that depends on other bugs and/or features added.

Personally, I have found that the memo field isn't too useful since Blue Iris (iPhone) makes the camera name much too big, and wastes a whole lot of space besides. I really don't need a huge camera name, and extra blank line or two, and to have the duration and time frame on separate lines. If they reduced that they could increase the memo field font substantially. Maybe they figured that nobody would use that field.

 

[Vettester]

That will be in the next release. The timing on that depends on other bugs and/or features added.

Personally, I have found that the memo field isn't too useful since Blue Iris (iPhone) makes the camera name much too big, and wastes a whole lot of space besides. I really don't need a huge camera name, and extra blank line or two, and to have the duration and time frame on separate lines. If they reduced that they could increase the memo field font substantially. Maybe they figured that nobody would use that field.

Agreed... It would be more useful if you would define the object that triggered the notification instead of the name of the AOI.

Here’s a screenshot of how the memo field is being used in AITools:

 

[Vettester]

Release 1.4.3 is now on Github. This includes your request for scrolling on the MQTT dialog. That's pretty much it.

Everything seemed to be working with 1.4.2 so I updated to 1.4.3 and let the app run through the night. I'm pretty sure I had everything configured correctly, however there were not any notifications sent even though there were numerous motion events. I had the AOI set to monitor the street and there were numerous cars that past by. I uninstalled everything and reinstalled 1.4.3 and let it run again for about 30 minutes. There were still no notifications being created. As an example in the screenshot below the car moving in the street should have created a notification, but it didn't. You can also see that out of 52 pictures 0 images were processed.



I uninstalled 1.4.3 and rolled back to 1.4.2 to see if I could get it working again. In 1.4.2 the app processed one motion event and then stopped again.



I believe there are still some major issues with this app. Please let me know if there is anything else I can do to help.

 

[Roger]

Ken, quick UI note. One the Application settings dialog box you have "Port Number fo the AI"

 

[Ken98045]

Everything seemed to be working with 1.4.2 so I updated to 1.4.3 and let the app run through the night. I'm pretty sure I had everything configured correctly, however there were not any notifications sent even though there were numerous motion events. I had the AOI set to monitor the street and there were numerous cars that past by. I uninstalled everything and reinstalled 1.4.3 and let it run again for about 30 minutes. There were still no notifications being created. As an example in the screenshot below the car moving in the street should have created a notification, but it didn't. You can also see that out of 52 pictures 0 images were processed.

View attachment 78242

I uninstalled 1.4.3 and rolled back to 1.4.2 to see if I could get it working again. In 1.4.2 the app processed one motion event and then stopped again.

View attachment 78243

I believe there are still some major issues with this app. Please let me know if there is anything else I can do to help.

Some general notes for others: There is specific code in there to weed out parked cars. The cars in the foreground were obviously parked a while. The car on the street is a different matter.

In this specific case, the Number Processed is the the giveaway. That number is incremented very early on, just when the first new .jpg file is seen in your directory. It is saying that there were no pictures at all being recognized anywhere in the directory you are monitoring that matched your camera name prefix. I am certain that that code works in general.

Please go to the directory you specified for your camera. Make sure that there are pictures there with a recent timestamp. Double check your directory and prefix. Double check the file timestamp against activity in your log. If you find anything there (probably not) copy it and email it to me or post it here. Double check that Monitor Cameras has your camera checked. THEN completely exit On Guard and start it again. See if the Images Processed number goes up.

Note that there is code that temporarily turns off monitoring when you go into Camera Settings. It is possible that things aren't being restarted correctly when you exit Camera Settings. This is why I suggest you exit the app and start it up again. I don't think that that is your problem though.

The way things work internally is that the code sets up a "directory monitor" request through Windows. That monitor looks for new files being dropped in the camera directory. As far as I know the directory monitoring process is fairly foolproof. However, it is possible that my part of it isn't recognizing an error starting that up, or there is some error returned that I don't catch.

BTW, you can test to see if On Guard is recognizing new file without relying on Blue Iris to output pictures. Just copy a .jpg file with the correct prefix (camera name) into your monitored directory. The Images Processed number should go up immediately.

Also, be aware that the setup files from versions before 1.4.1 are not compatible at all with the 1.4.1 version. You should have needed to re-define your areas as well as your cameras. If that didn't happen then something was wrong. However, all the post 1.4.1 files are compatible.

If you don't come up with nothing that explains your problem I will post a version (not a release) on GitHub that has additional log file info. Even though the log files can get very big, they would be bigger still if I logged absolutely everything.

Let me know. I'll keep working on this with you until we get it right.



Also, note that

 

[Ken98045]

Some general notes for others: There is specific code in there to weed out parked cars. The cars in the foreground were obviously parked a while. The car on the street is a different matter.

In this specific case, the Number Processed is the the giveaway. That number is incremented very early on, just when the first new .jpg file is seen in your directory. It is saying that there were no pictures at all being recognized anywhere in the directory you are monitoring that matched your camera name prefix. I am certain that that code works in general.

Please go to the directory you specified for your camera. Make sure that there are pictures there with a recent timestamp. Double check your directory and prefix. Double check the file timestamp against activity in your log. If you find anything there (probably not) copy it and email it to me or post it here. Double check that Monitor Cameras has your camera checked. THEN completely exit On Guard and start it again. See if the Images Processed number goes up.

Note that there is code that temporarily turns off monitoring when you go into Camera Settings. It is possible that things aren't being restarted correctly when you exit Camera Settings. This is why I suggest you exit the app and start it up again. I don't think that that is your problem though.

The way things work internally is that the code sets up a "directory monitor" request through Windows. That monitor looks for new files being dropped in the camera directory. As far as I know the directory monitoring process is fairly foolproof. However, it is possible that my part of it isn't recognizing an error starting that up, or there is some error returned that I don't catch.

BTW, you can test to see if On Guard is recognizing new file without relying on Blue Iris to output pictures. Just copy a .jpg file with the correct prefix (camera name) into your monitored directory. The Images Processed number should go up immediately.

Also, be aware that the setup files from versions before 1.4.1 are not compatible at all with the 1.4.1 version. You should have needed to re-define your areas as well as your cameras. If that didn't happen then something was wrong. However, all the post 1.4.1 files are compatible.

If you don't come up with nothing that explains your problem I will post a version (not a release) on GitHub that has additional log file info. Even though the log files can get very big, they would be bigger still if I logged absolutely everything.

Let me know. I'll keep working on this with you until we get it right.



Also, note that

I did add some additional logging and tried to make it super obvious if directory monitoring didn't startup correctly. That code should have been there to begin with, so I did upload it to GitHub. This is not a general release since I want to make sure we get you going first. So, if you can't figure out what is going on with the suggestions above, please go to GitHub, click on Code then download. Run the setup. You do not need to change your camera/area definitions. You may need to uninstall first.

 

[Ken98045]

OK, I've added yet another Release version 1.4.4 here on Github: Release On Guard Security Assistant Version 1.4.4 · Ken98045/On-Guard

This version is fully compatible with any release from 1.4.1 and up. It adds the capability to send a series of test image (currently 7) to all of your cameras. Between these 7 images there is probably at least one image that will trigger one of your notifications. If not, then you can create a temporary test area that matches one or more of these images. It also has a few minor bug fixes.

I know I'm getting a little Release happy here. However, for those of you having problems getting On Guard to work this release could be an important step in narrowing down your problems. For one thing, it takes Blue Iris out of the equation entirely. I intend to have future versions turn on extended logging that will be in effect only for these test images. That may also help track down any issues without adding yet more spew to the normal log file.

As always, let me know if you have any problems.

 

[Vettester]

Let me know. I'll keep working on this with you until we get it right.

Ken, I'm sure I had everything configured correctly and that BI was sending image files to the correct directory. Also that there was image files that should have been processed. Anyway, I uninstalled On Guard and deleted the OnGuard folder in the C:\Users\Me\AppData\Roaming directory. I reinstalled version 1.4.4 and set it up the same as the other installs and now everything seems to be working.

2020.12.31 17:52:32:2641 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201231_175231860.jpg
2020.12.31 17:52:32:2681 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175231860.jpg&memo=Street

2020.12.31 17:52:34:2751 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175233860.jpg
2020.12.31 17:52:36:2657 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175235860.jpg
2020.12.31 17:52:56:2776 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175255860.jpg
2020.12.31 17:53:08:0558 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_175307630.jpg
2020.12.31 17:53:10:0350 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175309630.jpg
2020.12.31 17:53:10:0710 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175309630.jpg&memo=Street

2020.12.31 17:54:54:2835 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175453858.jpg
2020.12.31 17:54:54:2924 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175453858.jpg&memo=Street

2020.12.31 17:54:56:2688 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_175455858.jpg
2020.12.31 17:55:40:2536 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201231_175539857.jpg
2020.12.31 17:55:40:2566 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175539857.jpg&memo=Street

2020.12.31 17:55:46:2605 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201231_175545857.jpg
2020.12.31 17:56:04:2602 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_175603857.jpg
2020.12.31 17:56:20:2699 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175619857.jpg
2020.12.31 17:56:20:2719 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175619857.jpg&memo=Street

2020.12.31 17:56:28:2596 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_175627857.jpg
2020.12.31 17:56:32:3293 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175631906.jpg
2020.12.31 17:57:28:2613 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175727854.jpg
2020.12.31 17:57:28:2693 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175727854.jpg&memo=Street

2020.12.31 17:58:36:2486 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_175835854.jpg
2020.12.31 17:58:36:2556 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_175835854.jpg&memo=Street

2020.12.31 17:59:12:2423 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_175911853.jpg
2020.12.31 18:07:48:2465 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_180747839.jpg
2020.12.31 18:07:50:2612 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201231_180749839.jpg
2020.12.31 18:07:50:2692 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_180749839.jpg&memo=Street

2020.12.31 18:08:04:2385 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_180803839.jpg
2020.12.31 18:08:06:2512 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_180805839.jpg
2020.12.31 18:08:08:2473 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201231_180807839.jpg
2020.12.31 18:08:36:2730 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_180835839.jpg
2020.12.31 18:08:36:2760 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_180835839.jpg&memo=Street

2020.12.31 18:08:38:2498 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_180837839.jpg
2020.12.31 18:08:42:2421 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_180841839.jpg
2020.12.31 18:08:44:2364 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_180843839.jpg
2020.12.31 18:10:24:2354 - 0 interesting objects found in file: C:\aiinput\aidriveway.20201231_181023827.jpg
2020.12.31 18:10:46:2410 - 2 interesting objects found in file: C:\aiinput\aidriveway.20201231_181045827.jpg
2020.12.31 18:10:46:2480 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=*******&jpeg=C:\aiinput\aidriveway.20201231_181045827.jpg&memo=Street

,
2020.12.31 18:11:30:2447 - 1 interesting objects found in file: C:\aiinput\aidriveway.20201231_181129827.jpg
2020.12.31 18:11:30:2477 - Successfully notified URL:

http://localhost:8080/admin?trigger&camera=driveway&user=aiuser&pw=Vette73&jpeg=C:\aiinput\aidriveway.20201231_181129827.jpg&memo=Street

I'm not sure what the issue was, but I appreciate you working through this with me!

Thanks!

-Vettester

 

[Ken98045]

It was probably related to upgrading from a previous version.

For the next release I'll store the current version in the setup information. Then, when you upgrade I'll do one of two things. Either wipe the settings completely or try to carry over as much information as possible. When I started this project I didn't know if anyone would be interested so I took a few shortcuts. I tried the wipe route for version 1.4.x, but some of you got caught in the transition I think.

NOTE: The OnGuard-Readme.docx file has been updated as an individual file. This adds instructions for the use of IFTTT. It also discusses why On Guard will be using the Blue Iris second/virtual camera approach for the foreseeable future. I would appreciate any feedback regarding anything in the manual that you found particularly confusing. I know that there is a lot to read there, and not everyone (especially me) RTFM (reads the freaking manual). The manual is a work in progress.

 

[jz3082]

1.4-4

Uninstalled 1.4-2 and deleted the OnGuard folder under AppData\Roaming. Installed 1.4-4 and set up everything as if it were a brand new install. I triggered 4 cameras and 10 images were generated and sent to DeepStack. There were 4 images in the OnGuard log from 2 cameras. The other 6 images from the other 2 cameras were not logged. I posted one of the unlogged images at the bottom. It is the last picture taken and the last image processed by DeepStack. The last version that worked without problems for me was 1.2.1.