Blue Iris + Deepstack assistance

[exx1976]

I posted this over on the Blue Iris forum as well, but this place seems to get a lot more traffic, so I'm hoping someone here can help me.

I'm using the Blue Iris eval version right now, I've got like 13 days left on it before I decide if this is the solution for me. Should be plenty of time to iron this stuff out.

I was able to add all my cameras just fine, main streams and substreams, following various threads online. No problem.

Deepstack seems to do ok with the generic object detection - car, truck, person.. But I'm trying to set up the facial recognition. From within Blue Iris, it would appear that I can only add a single face to a name. However, research showed that I could use Postman and make API calls to assign multiple images to the same name - so I did. Then I used Deepstack UI to test that, and the recognition was vastly improved.

HOWEVER - the userids that I added via the API do not get picked up in Blue Iris (where you can configure it to alert based on a face, or to cancel an alert based on a face). Further, when I clicked "Faces" in the Blue Iris UI, I got a message that the userid I just uploaded all the photos for was deleted - immediately upon clicking the button! Then it opened up the faces page, and sure enough, it was blank.

Can anyone point me in the right direction about how to get facial recognition working within this setup? I'd like to be able to send MQTT messages to my home automation system based on who was picked up on camera where, and then take actions based on those messages.

Thanks in advance!

 

[sebastiantombs]

Facial recognition with Blue Iris and DeepStack is a kind of hit and miss proposition. It's a combination of factors, computing power and the size of the model base, IE how many samples of each object that you want to detect are in the model. The built-in model contains thousands of samples. When you start adding facial detection typically one or two are all that get added. To be somewhat effective and reliable I'd guess it takes at least a hundred or two, maybe even more. Then the problem becomes the amount of time needed to analyze a capture against the samples. More sample equal more time and more CPU time.

I tinkered with it when I set up DeepStack and shut it down. To me, it's more of a novelty than a reliable feature. YMMV.

 

[wittaj]

Most of us have found that facial recognition is more gimmicky and novelty than anything else. If you have to put in 5 or 10 or 15 or 35 pictures or more of yourself in the system for it to recognize it is you...then you shouldn't expect much.... My success rate was under 5% so I moved on to other things LOL.

It can work in certain situations like a business that requires everyone to stop in front of the camera and the camera is at head height. Outside of that, the percentage of being accurate is probably not going to be super high.

Someone here posted once how horrible it was inside his house identifying his neighbors and others as him. Another guy his kids and wife were being tagged as him. And then at night, forget it. I wouldn't trust it for any automation.

Unless you spend the big bucks that casinos and airports have LOL.

Heck even in ideal situations like a business with the camera at ideal height and optimal lighting it fails....

What do you think of Dahua's "Uyghur Warnings"?

Good luck. Wish you no ill will. I'm not here to argue with people simply to contribute when and where I can. This is also followed by lending my views on current technology and how it impacts all of us. It's undisputed that splitting of the atom changed the world but once again man decided to...

ipcamtalk.com

 

[exx1976]

While I appreciate both of your attempts at telling me how poorly you think it works, what I am looking for is the answer to the question that I asked. I'm already using ubiquiti cameras with a dream machine pro, which does good enough with person vs not person - meaning that I'm not sure buying blue iris buys me anything except the ability to use non-ubnt cameras. So....

Seeing how facial recognition is one of my goals, and I don't have a problem with providing 30-40 example photos per person I want recognized.. Can anyone tell me how to associate those photos with a single face that I want recognized?

 

[wittaj]

OK then don't say you were not warned when the 2am perp walks up to your house and it thinks it is you and turns off the house alarm and unlocks all the doors LOL.

I tested it with a camera at 5 feet high at the door where it is about as ideal as you can get and half the people that came to the door, it thought it was me LOL. I wouldn't use it for anything other than playing around LOL. Both myself and @sebastiantombs were on the leading edge of trying this out when it became available in BI.

Help file is your friend...not exactly what you are looking for but it is all it has at the moment. But improvements are always added, so maybe one day it can be more functional.

The “to confirm” and “to cancel” boxes may now contain labels ending with the * wildcard.
This is handy if you have multiple faces for a single person—you might use chris* for
example to match defined faces for chris_1, chris_2, chris_side, etc.

Maybe you can improve it making your own models, but enough evidence in the post I linked showing even in ideal situations it misses.

Good luck and keep us posted.

 

[sebastiantombs]

Please read what I said. If you use enough samples and develop your own model that way it may it may provide what you want, but the is no guarantee that it will. If your happy with Ubiquity stick with it.

 

[exx1976]

Well, I am and I'm not. The responsiveness of the camera streams into home assistant is not as good as blue iris, and it doesn't do facial recognition. It does, however, allow me to do 2 way audio with my doorbells..

As for the loading each face and naming them differently... Wow. Tedious and dumb. Can blue iris really not handle using the userids from inside of deepstack - the ones that get created when you train deepstack using the api??

 

[sebastiantombs]

It isn't Blue Iris that makes the ID, DeepStack does.

 

[wittaj]

You would have to create your own model using your own photos and instead of lumping every human as a person, you would provide them their name and then hope that the AI is good enough to tell the difference between two blondes LOL.

That would be the only way I would start to have confidence in facial recognition maybe working.

Keep in mind this is more a Deepstack limitation than Blue Iris.

You are now mixing up two things, or mis-communicated it.

You have two ways to put in faces:

1. Add pictures of faces into the Faces Folder and let DeepStack try to guess them. This requires each face to have the unique names that you then Wildcard. This is the simplest way.
2. Create your own model with your own face images. This will stand the best chance, but is more tedious. But then you could have it by each person's name.

 

[exx1976]

It isn't Blue Iris that makes the ID, DeepStack does.

I'm aware. But when I create the ID by training deepstack by posting images to the api, blue iris is unable to honor those ids, and further, it deletes them if you click the "faces" button from within the settings.

 

[exx1976]

You would have to create your own model using your own photos and instead of lumping every human as a person, you would provide them their name and then hope that the AI is good enough to tell the difference between two blondes LOL.

That would be the only way I would start to have confidence in facial recognition maybe working.

Keep in mind this is more a Deepstack limitation than Blue Iris.

Not really.... Using deepstack ui, I can upload test images once I've trained it. Deepstack does a relatively decent job. Obviously, the more sample images, the better it does. This is relatively easy to test using photos from social media. Just grab a bunch of examples of people, then give it a group photo of those people.

My concern is more why blue iris refuses to honor the user ids that are created in deepstack using a method other than through the blue iris interface.

 

[wittaj]

I'm aware. But when I create the ID by training deepstack by posting images to the api, blue iris is unable to honor those ids, and further, it deletes them if you click the "faces" button from within the settings.

Please elaborate more - I think you are simply calling up images and not actually training a model.

Are you actually training a model with the appropriate programs or putting faces and names in and thinking that is training it? The model training run of the program can run over 12 hours computing and I don't think that is what you are doing.

 

[sebastiantombs]

As I also said, facial recognition on a PC that is running various other program, like Blue Iris, make facial recognition more of a toy than a practical application due to computing limitations. Once again, DeepStack supplies the identity, be it a car, truck, bus, zebra, Joe, Mary, Bobby or anyone else in your facial database. Personally, I'd crate my own model, but even then relying of DS and BI to trigger anything even remotely critical or security related would not be a good idea. No system is perfect and this is "real world", not Hollweird.

 

[wittaj]

Not really.... Using deepstack ui, I can upload test images once I've trained it. Deepstack does a relatively decent job. Obviously, the more sample images, the better it does. This is relatively easy to test using photos from social media. Just grab a bunch of examples of people, then give it a group photo of those people.

My concern is more why blue iris refuses to honor the user ids that are created in deepstack using a method other than through the blue iris interface.

If you want that level of granular control, then you need to use one of the 3rd party DeepStack BI add-ons like AI tools or one of the many variations people have created here.

The Deepstack integration with BI that you are using is more for the casual user that wants ease and just check some boxes. And it has been a great improvement in eliminating false alerts.

Folks that were running the 3rd party DeepStack add-ons have remained with that instead of the BI/DS integration for that very reason.

 

[exx1976]

Please elaborate more - I think you are simply calling up images and not actually training a model.

Are you actually training a model with the appropriate programs or putting faces and names in and thinking that is training it?

Hrmm.. Perhaps my terminology is incorrect? If so, please forgive me, I am a n00b at all this camera stuff.

I found an article somewhere, I forget where, with someone that figured out you can use postman to post multiple images to the same userid in deepstack. I tried it and it worked. Then using deepstack ui, I gave it a group photo that had many of those same people in it, and it did quite a good job.

 

[exx1976]

If you want that level of granular control, then you need to use one of the 3rd party DeepStack BI add-ons like AI tools.

The Deepstack integration with BI that you are using is more for the casual user that wants ease and just check some boxes.

Folks that were running the 3rd party DeepStack add-ons have remained with that instead of the BI/DS integration for that very reason.

Ah, OK - perhaps that's where I've gone astray. Can you provide more information, or a link, or point me in the right direction of how I might go about using one of these add ons, and can I still feed that identification information back to home assistant using Mqtt or something similar?

 

[wittaj]

Ah, OK - perhaps that's where I've gone astray. Can you provide more information, or a link, or point me in the right direction of how I might go about using one of these add ons, and can I still feed that identification information back to home assistant using Mqtt or something similar?

Here are a few of the common ones folks here use:

[tool] [tutorial] Free AI Person Detection for Blue Iris

This program analyzes motion in Blue Iris cameras in real-time using Artificial Intelligence. It notifies you if a human or an object of your choice is detected and can send alert images to Telegram. It is based on DeepQuestAI, a free multipurpose AI that you can install on your own server...

ipcamtalk.com

Last Watch AI - a new platform-independent AI tool with web interface

Introducing Last Watch AI akmolina28/last-watch-ai is yet another open source tool for glueing Blue Iris to Deepstack AI, inspired by gentlepumpkin's AI tools. I created this tool last year to address some of the gaps and frustrations I was experiencing with the other Deepstack tools...

ipcamtalk.com

Yet Another Free Extension for Blue Iris Adding AI Object Dectection/Reduction in False Alarms/Enhanced Notification of Activity/On Guard

I have added a free Windows extension for Blue Iris that uses the DeepStack AI to allow for the identification of objects causing motion alerts. For the lack of anything better I've called it On Guard. This application should dramatically reduce the number of "false alarms". You can detect...

ipcamtalk.com

Keep in mind these are more complicated, but can provide much more control and customization. But it is this level of effort that the BI/DS integration was minimizing with the simple check some boxes. The BI/DS integration will work for the majority of users, but those with more unique or specialized applications at the moment will benefit from the 3rd party add-ons. But with each BI revision, the gap between the two gets smaller.

 

[exx1976]

Here are a few of the common ones folks here use:

[tool] [tutorial] Free AI Person Detection for Blue Iris

This program analyzes motion in Blue Iris cameras in real-time using Artificial Intelligence. It notifies you if a human or an object of your choice is detected and can send alert images to Telegram. It is based on DeepQuestAI, a free multipurpose AI that you can install on your own server...

ipcamtalk.com

Last Watch AI - a new platform-independent AI tool with web interface

Introducing Last Watch AI akmolina28/last-watch-ai is yet another open source tool for glueing Blue Iris to Deepstack AI, inspired by gentlepumpkin's AI tools. I created this tool last year to address some of the gaps and frustrations I was experiencing with the other Deepstack tools...

ipcamtalk.com

Yet Another Free Extension for Blue Iris Adding AI Object Dectection/Reduction in False Alarms/Enhanced Notification of Activity/On Guard

I have added a free Windows extension for Blue Iris that uses the DeepStack AI to allow for the identification of objects causing motion alerts. For the lack of anything better I've called it On Guard. This application should dramatically reduce the number of "false alarms". You can detect...

ipcamtalk.com

Keep in mind these are more complicated, but can provide much more control and customization. But it is this level of effort that the BI/DS integration was minimizing with the simple check some boxes.

OH, trust me - as a new user, I'm all about minimizing complexity. However, when functionality is impacted as part of that effort, well.... I'm not sure that's really a benefit for anyone?

I'll have a read though those and see what I can come up with. Thank you for steering me in the right direction.


As for what I plan to do with this once it's working... Let's just say there will be no 2am perp being automatically let into my house. I do appreciate the warning, and your due diligence. The next 50 people that happen across this thread might need to see it, even if it wasn't completely on the mark for my particular situation.

Have a great night!

 

[fenderman]

While I appreciate both of your attempts at telling me how poorly you think it works, what I am looking for is the answer to the question that I asked. I'm already using ubiquiti cameras with a dream machine pro, which does good enough with person vs not person - meaning that I'm not sure buying blue iris buys me anything except the ability to use non-ubnt cameras. So....

Seeing how facial recognition is one of my goals, and I don't have a problem with providing 30-40 example photos per person I want recognized.. Can anyone tell me how to associate those photos with a single face that I want recognized?

Face recognition with deepstack is less than worthless. But you know better. You have used deepstack and BI for two days. You will not achieve your goals.

 

[wittaj]

OH, trust me - as a new user, I'm all about minimizing complexity. However, when functionality is impacted as part of that effort, well.... I'm not sure that's really a benefit for anyone?

We just wanted you to be aware of the limitations currently with Facial Recognition.

As I said, the primary functionality of the BI/DS integration is to greatly reduce/eliminate the false triggers and being able to get good alerts on people, cars, etc. And for that it has hit the mark. It is much easier to check some boxes and get that functionality (that is sufficient for most) than to go thru all the steps for the 3rd party add-on.

Facial Recognition has been added to the list of functionality but is still very much a work in progress.

And of course there are people desiring additional functionality and that is where the 3rd party add-ons come in. Slowly but surely as BI is able to introduce that additional functionality into BI to be easy for the user, they are doing so. The integration from its first roll-out to now are night and day differences. And the improvements continue to roll out.