Full ALPR Database System for Blue Iris!

I
@algertc

From Earlier Post:

1) The payload should be sent like how Blue Iris sends things. For instance { "plate_number":"&PLATE", "Image":"&ALERT_JPEG", "camera":"&NAME", "timestamp":"&ALERT_TIME" } comes across as multiple payloads under one topic and sorts nicely on my home automation system. As it stands I cant tease out the data I want on the single payload.

This is now working great. I have each data point coming across as a sub topic

View attachment 226660

One point that was mentioned before: The time stamp is wrong.

2) Need to be able to bind a camera to a specific broker. I have three LPR cams and I need to send MQTT messages based on which camera is triggering

I did not see this added to the latest revision

3) Need to be able to send notifications based on tags as well as plates. I have some tags with hundreds of plates. Don't want to manually add those in when a single tag can cover it all

Also did not see this added to the latest revision

Thank you for everything you've done so far.


Working on this part now. I completely see the value and necessity of being able to notify based on tags and other criteria. I've thought through and think I have a good solution for how to create a notification rule builder class, but I can't promise camera-specific topics at the moment. That can definitely come at some point, but it gets a lot more convoluted when you have a bunch of different conditions to notify based on, which also need to go to separate topics.

Would it not work if it were the same topic with the camera name included in the payload? That would be much easier to build around.

I changed code that I thought fixed the TZ conversion, but I guess that's not working properly. Dealing with these timestamps has been an enormous pain throughout the entire app. Is your time zone set to your local region in your docker compose? It seemed to work when I tested it. Are others still seeing wrong times too?


I'll get that aligned at some point, but I'm going to prioritize the other parts mentioned first.
 
I



Working on this part now. I completely see the value and necessity of being able to notify based on tags and other criteria. I've thought through and think I have a good solution for how to create a notification rule builder class, but I can't promise camera-specific topics at the moment. That can definitely come at some point, but it gets a lot more convoluted when you have a bunch of different conditions to notify based on, which also need to go to separate topics.

Would it not work if it were the same topic with the camera name included in the payload? That would be much easier to build around.

I changed code that I thought fixed the TZ conversion, but I guess that's not working properly. Dealing with these timestamps has been an enormous pain throughout the entire app. Is your time zone set to your local region in your docker compose? It seemed to work when I tested it. Are others still seeing wrong times too?


I'll get that aligned at some point, but I'm going to prioritize the other parts mentioned first.
Thank you for spending time on this request. I think it will be helpful for others down the road if there is more granularity in how MQTT can be sent.

I do not think it would work well for the camera name in the payload...which you already have...because any new MQTT message overwrites the old message and would be lost

Here is my situation: I have (now) two LPR cameras on the busy street in front of my house and two LPR cams on the driveway coming up to our house. I am trying to use MQTT to relay the driveway tags/saved plate info to announce who is coming up the driveway. If someone were to come up the driveway and a car would then pass that is in my alerts, then the driveway MQTT message would be overwritten by the car on the street and be incorrect. It is a rare possibility, though, so not a huge deal if it cant be easily implemented.

The flexibility of having a camera specific MQTT message would be ideal as I could also have messages related to the street cams that are different from the driveway cams.

What would work is to have an option for adding new MQTT brokers with a specific camera, or multiple cameras or all cameras. Would definitely need the option to pick more than one camera. Then I could change the topic for each camera, say Blue Iris/Street for both my street LPR cams and Blue Iris/Entry for my driveway LPR cams. That way street LPR cams would not fire when driveway LPR cams pick up the correct plate.

I might be able to configure my home automation to save the last few MQTT messages which could also solve the problem.

Not to muddy the waters, but I am sure there are several folks that have face and/or overview cams on their LPR setups. Was wondering if there was a way to capture those images and store them with the plate data in your database. Not sure how we could accept other cam images and how to get the timing down, but it would be awesome for my situation. I have a face cam set up on the driveway that works well in daytime (not so much at night...yet) and I also have an overview cam that works well in daytime and ok at night. Would be awesome to store those caps with the plate data if available. I think I have seen this feature with the paid versions. Just some food for thought.
 
Thank you for spending time on this request. I think it will be helpful for others down the road if there is more granularity in how MQTT can be sent.

I do not think it would work well for the camera name in the payload...which you already have...because any new MQTT message overwrites the old message and would be lost

Here is my situation: I have (now) two LPR cameras on the busy street in front of my house and two LPR cams on the driveway coming up to our house. I am trying to use MQTT to relay the driveway tags/saved plate info to announce who is coming up the driveway. If someone were to come up the driveway and a car would then pass that is in my alerts, then the driveway MQTT message would be overwritten by the car on the street and be incorrect. It is a rare possibility, though, so not a huge deal if it cant be easily implemented.

The flexibility of having a camera specific MQTT message would be ideal as I could also have messages related to the street cams that are different from the driveway cams.

What would work is to have an option for adding new MQTT brokers with a specific camera, or multiple cameras or all cameras. Would definitely need the option to pick more than one camera. Then I could change the topic for each camera, say Blue Iris/Street for both my street LPR cams and Blue Iris/Entry for my driveway LPR cams. That way street LPR cams would not fire when driveway LPR cams pick up the correct plate.

I might be able to configure my home automation to save the last few MQTT messages which could also solve the problem.

Not to muddy the waters, but I am sure there are several folks that have face and/or overview cams on their LPR setups. Was wondering if there was a way to capture those images and store them with the plate data in your database. Not sure how we could accept other cam images and how to get the timing down, but it would be awesome for my situation. I have a face cam set up on the driveway that works well in daytime (not so much at night...yet) and I also have an overview cam that works well in daytime and ok at night. Would be awesome to store those caps with the plate data if available. I think I have seen this feature with the paid versions. Just some food for thought.
100% I’m interested and have been planning for faces. Trying to nail this down end to end first, but that’s where I’m heading for sure.

If you could investigate the ability to manage the message history and trigger based on items within the payload, that would be great.

To me, it seems like that’s something that should be happening on the HA side, but I’m not much of a HA user, nor an MQTT expert.

I can add the full functionality to select a topic within the notification configuration, but it’s going to get fairly complicated, as far as the logic goes. What I’m thinking about most is what the default values will be for those with less complex setups.


Had I understood this use case before building the MQTT functionality, I probably would have taken the time to separate broker mgmt from topic management. I knew that was a limiting shortcut when I did it.


Honestly, now understanding how this integrates, I might just nuke everyone’s broker tables and go back and do it right to have the proper structure of brokers and topics under each.


Unless anyone has a major objection to that, I think that’s the best approach.


Apologies for the lack of foresight on this one. It’s quite a bit more challenging to develop and build around a system that I don’t ever personally encounter and thus don’t understand the use cases for.


Anyways, faces excite me majorly, and I’d love to cross reference and integrate them with the vehicles. Absolutely coming next. Not even very difficult to build. Once we’re stable enough on the LPR functionality, I’ll be adding that.

I might be able to crank this one out this weekend, but I’d like to hear from some others and get some confirmation that this is how I should proceed, since this will probably require deletion or retirement of some DB columns, and my DB update approach is still extremely haphazard and doesn’t even seem to be working in the first place.


Even when using a proper ORM with migrations, you still run into a lot of the same problems when publishing an app to be self hosted. If anyone has any recommendations for a better way to manage database update without users losing their data, I’d be very eager to hear.
 
Thank you for spending time on this request. I think it will be helpful for others down the road if there is more granularity in how MQTT can be sent.

I do not think it would work well for the camera name in the payload...which you already have...because any new MQTT message overwrites the old message and would be lost

Here is my situation: I have (now) two LPR cameras on the busy street in front of my house and two LPR cams on the driveway coming up to our house. I am trying to use MQTT to relay the driveway tags/saved plate info to announce who is coming up the driveway. If someone were to come up the driveway and a car would then pass that is in my alerts, then the driveway MQTT message would be overwritten by the car on the street and be incorrect. It is a rare possibility, though, so not a huge deal if it cant be easily implemented.

The flexibility of having a camera specific MQTT message would be ideal as I could also have messages related to the street cams that are different from the driveway cams.

What would work is to have an option for adding new MQTT brokers with a specific camera, or multiple cameras or all cameras. Would definitely need the option to pick more than one camera. Then I could change the topic for each camera, say Blue Iris/Street for both my street LPR cams and Blue Iris/Entry for my driveway LPR cams. That way street LPR cams would not fire when driveway LPR cams pick up the correct plate.

I might be able to configure my home automation to save the last few MQTT messages which could also solve the problem.

Not to muddy the waters, but I am sure there are several folks that have face and/or overview cams on their LPR setups. Was wondering if there was a way to capture those images and store them with the plate data in your database. Not sure how we could accept other cam images and how to get the timing down, but it would be awesome for my situation. I have a face cam set up on the driveway that works well in daytime (not so much at night...yet) and I also have an overview cam that works well in daytime and ok at night. Would be awesome to store those caps with the plate data if available. I think I have seen this feature with the paid versions. Just some food for thought.

Have you considered using NodeRed to handle the automation? For example, have NR trigger on the MQTT topic and then filter out the driveway reads to do whatever post processing, including triggering one of your overview cameras to take a snapshot? I think core functionality should remain in ALPR database, and leave the other stuff to other tools. In other words, have the cameras pick up everything and do the filtering and logic elsewhere? I'm not using MQTT with ALPR so maybe I am misunderstanding your need.
 
Working on this part now. I completely see the value and necessity of being able to notify based on tags and other criteria. I've thought through and think I have a good solution for how to create a notification rule builder class, but I can't promise camera-specific topics at the moment. That can definitely come at some point, but it gets a lot more convoluted when you have a bunch of different conditions to notify based on, which also need to go to separate topics.
Not to put a damper on your MQTT development, but the majority of what you are trying to accomplish is already doable via BI and Home Assistant. More than likely you can do the same thing with other home automation systems that support MQTT. For example you can setup an MQTT message with any camera in BI and then have that message trigger anything within your home automation.
 
  • Like
Reactions: PeteJ and looney2ns
Not to put a damper on your MQTT development, but the majority of what you are trying to accomplish is already doable via BI and Home Assistant. More than likely you can do the same thing with other home automation systems that support MQTT. For example you can setup an MQTT message with any camera in BI and then have that message trigger anything within your home automation.
The place where I see the ALPR database having an advantage over just BI (whether it be an MQTT message or a notification) is that ALPR can do some lookup of the plate and add enriched data (such as fixing poor recognition with a fuzzy lookup), adding tags, checking for flagged vehicles, adding number of recent detections, etc.

That enriched data can then be used to decide if a notification is necessary, or if a MQTT message should be broadcast.
 
  • Like
Reactions: prsmith777
ALPR can do some lookup of the plate and add enriched data (such as fixing poor recognition with a fuzzy lookup)
With the vast amount of plate numbers out there I’m not sure how fuzzy lookup could possibly enrich recognition. Poor recognition is typically a camera or model issue with the latter usually being the problem.

With all of the plate data captured and sent anonymously I was hoping for an improved license plate model.
 
With the vast amount of plate numbers out there I’m not sure how fuzzy lookup could possibly enrich recognition. Poor recognition is typically a camera or model issue with the latter usually being the problem.

With all of the plate data captured and sent anonymously I was hoping for an improved license plate model.
I'm using Plate Recognizer for my detections and it is not infallible either. There are times with changing lighting conditions, physical obstructions, etc. where it gets a "close" but slightly wrong plate. Whether it be a database of corrections that have been made previously that it can auto correct, or some fuzzy logic that says "we have 300 sightings of ABC123 and a new sighting of ARC123", I still think ALPR could be helpful in improving the notification process.

As it stands, BI is notifying based on just the detected string of characters (which could be wrong, frequently seen, not important, etc.). But if you could combine that with the knowledge obtained from the ALPR DB you could reduce notifications to those that are most important, needed review, etc.
 
After using this for a while, I think the critical thing missing is auto correction based on "also known as" plates. I am only having approx 50-60% correct read rates right now, and the misreads are usually very similar. For my situation, 90% of the reads are of vehicles I have seen before, so I find myself correcting the same plates over and over again. Auto correct based on "also known as" would be super useful for me, and probably others, too.