OpenALPR Webhook Processor for IP Cameras

[tomas21]

I just turned on logging in my firewall for that port for a month, then grabbed all of the AWS IP's hitting it (Rekor) and made an alias for them. It can of course break, but so far it hasn't

18.206.234.148
54.165.211.244
52.201.255.102
54.221.95.194
35.170.81.143

Is there a known full list... i just checked logging and in 3 minutes these showed up...

 

[IReallyLikePizza2]

There isn't, unless we ask Rekor. Since they are using AWS I doubt they have a full list

Do a whois on all those and see where they come from. It is possible some are generic port scanning from random places

 

[tomas21]

There isn't, unless we ask Rekor. Since they are using AWS I doubt they have a full list

Do a whois on all those and see where they come from. It is possible some are generic port scanning from random places

Yes these are all Amz ip's and looks like various EC2's...

3.234.205.133	Amazon
3.236.172.181	Amazon
3.88.231.80	Amazon
3.90.113.232	Amazon
3.91.62.70	Amazon
34.201.132.123	Amazon
44.202.252.34	Amazon
44.210.148.115	Amazon
54.242.135.108	Amazon
54.83.167.175	Amazon

 

[IReallyLikePizza2]

What I've thought about doing is getting a full list of all the AWS IP blocks and just whitelisting all of them

Would significantly cut down on what can hit your box

 

[djmadfx]

I still have not been able to get the 'Start Scrape' button to work.

All other features work without any issue. I have the following lines added in the conf:

web_server_enabled = 1
websockets_enabled = 1
web_server_debug = 1

I can access "OpenALPR Local Webservice Diagnostics".

I see, "To list keys in the database (requires diagnostic mode to be enabled) use /list?start=[epochms_start]&end=[epochms_end]", but I am not sure what they mean by "diagnostic mode". I tried using that URL with an epoch range and it gave a HTTP ERROR 500.

 

[niutron]

Amaziiiing work, is there a tutorial anywhere on how to install?

 

[djmadfx]

Amaziiiing work, is there a tutorial anywhere on how to install?

Home · mlapaglia/OpenAlprWebhookProcessor Wiki -- See the "First Time Setup" and "Installation" pages on the right side.

 

[Olivier]

...as well as grab the "full size" images from the agent (via proxying through the Rekor web (cloud or on-premises).

I am curious on how you implemented that part? Would you mind expanding on that please? Thanks!

 

[djmadfx]

Are you able to add the ability to send to multiple Pushover users? If you're feeling ambitious you should checkout: GitHub - caronc/apprise: Apprise - Push Notifications that work with just about every platform!

 

[mlapaglia]

Hello everyone, sorry for the lack of updates been busy with work and family.

Version v5.0.0-alpha3 is in alpha right now if anyone wants to give it a try. There are a few changes:

• General package upgrades.
• Dropped preview jpeg columns from db, no longer used.
• Added indexes to speed up filters query.
• Tweaked plate statistics query for faster response.
• Split plate/vehicle images into separate table for faster queries.
• Added optional image compression.
• Upgraded to .net 7, docker dotnet build sdk and runtime images to latest.
• Upgraded to Angular 16.
• Replaced deprecated Angular flex-layout with flexbox.
• Added web request calls being cancelled when navigating away from pages.
• Fixed logging to save to correct folder.
• Fixed issue with JWT not being secure enough.
• Fixed issue with logging out not clearing tokens.
• Improved the responsiveness of displaying the vehicle and plate images on mobile.
• Slight changes to displaying ignored plates.

Depending on database size and the storage medium (ssd, nvme, hdd) the database is on, the first time startup after upgrading is going to take a while, SQLite doesn't support dropping columns without rebuilding the entire database, adding the indexes to speed up multiple queries and moving the plate/vehicle images out to a separate table will take a while. My 40GB db on a nvme took around 1.5 hours to migrate. All the changes are worth it, after the migration larger databases will see improved response times for queries. I saw decreases from 6-10s down to 1-2s for basic queries.

The UI will not be available during the migration. If you see CPU and disk activity you can assume the migration is still in process.

Back up your processor.db before trying the alpha in case something goes wrong! If you try out v5 you will not be able to use the same processor.db file on v4, it is not backwards compatible.

 

[mlapaglia]

v5.0.0-alpha7 has added PWA support, meaning you can install the app on your desktop and phone instead of using the browser.

In chrome on windows the Little download icon will install the app on your desktop like a normal program.


On Android click the menu button in the browser and an option to install will be available. On iOS click the share icon then "add to home screen"

Doing this allows the app to run in the background and you'll get push notifications even if the browser tab isn't open:

 

[mlapaglia]

I reverse engineered the web socket connection between the rekor agent and the cloud and have a basic form of it implemented in the app. v5.1.0-alpha4 adds this websocket support. in your aplrd.conf file make sure websockets_enabled = 1 is set and the upload_address = https://192.168.1.2 has https, this won't work over http

 

[mlapaglia]

v5.1.0-alpha16 adds support for modifying the camera masks the agent uses. You can view the last 10 plates captured and switch to a snapshot view from what the camera is currently seeing to create the mask.

 

[mlapaglia]

v5.1.0 is out.

Added

• Ability to use the websocket feature of the agent, allowing for live agent configuration changes from the app.
• Ability to start/stop the agent from the agent page.
• Showing the agent status report on the agent page.
• Support for configuring the agent's camera masks used for detection.
• Ability to autofocus dahua cameras.
• Ability to get camera snapshots over the websocket.
• Added 4 UI themes to choose from.
• Added an animation to the logo when the backend is connected/disconnected.
• Upgraded to Angular 17.
• All components are standalone now.
• Unifying the accounts/register/login page with the rest of the app.
• Adding scraping agent on interval.

Changed

• Only wait for 5 seconds when loading images through the relay.

 

[IReallyLikePizza2]

Wow massive update! Thanks so much!

I hadn't gotten a notification for the thread for some reason, glad I came back and checked

So if I am reading things right, this will it to work all locally?

 

[IReallyLikePizza2]

Its been about 5 hours and still going, does this look like enough disk activity?



Still seeing 100% of 1 core being used



My database is 135gb

 

[IReallyLikePizza2]

So last time time I reverted to a snapshot I took before the upgrade, as it didn't seem to be working. I think I just didn't wait long enough

Last night at around 10PM I upgraded to the latest container version, and saw very low IO but a lot of CPU, just like before. I monitored the directory with the database in and saw the processor.db-shm and processor.db-wal growing in size, so clearly something was happening

This morning, still no UI so I figured it failed, but checking iotop I now see 250MB/s of disk read, so it must still be working, but taking forever

 

[IReallyLikePizza2]

Well it finally finished! 23 hours

Problem is, search doesn't work at all. Every time I hit search I get this in the log

[02:27:57 ERR] An exception was thrown attempting to execute the error handler.
System.InvalidOperationException: The SPA default page middleware could not return the default page '/index.html' because it was not found, and no other middleware handled the request.

My DB is also double the size now, wondering if something went wrong

 

[mlapaglia]

sqlite doesn't handle drop/create operations very well. while your db size has double, half of it is "unused" sqlite doesn't give up storage space unless you run a vacuum. you can run it yourself if you want to shrink the db
https://stackoverflow.com/questions/18126997/how-to-vacuum-sqlite-database
Try running a browser incognito and see if it works then, you might have some frontend code cached.

 

[mindboggle]

I recently got into LPR cameras and am finding this tool to be a valuable asset, so thank you to @mlapaglia for creating this! Before I'd have to check the snapshots for any activities which was trivial at best. I wanted to add some of my notes here for anyone running MacOS with a M1 chip or better. Specifically, I was having trouble maintaining a stable Docker instance. About every hour or so I'd get a corrupted memory error which I think is the result of incompatibilities between .NET, Docker, and M1, though that's just an educated guess.

My solution was to run this natively on my system until I can get better performance from Docker. So far I've had it running flawlessly for a few days with no crashes. Here are the steps I took:

• Install .NET 7 framework
  • Download .NET 7.0 (Linux, macOS, and Windows) (arm64)
    • Run “dotnet —list-runtimes” and make sure you have both of these:
    • Microsoft.AspNetCore.App 7.0.14 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
    • Microsoft.NETCore.App 7.0.14 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  • Might need to fix dotnet command line: dotnet: command not found in Mac
• Clone or fork the Github repo: GitHub - mlapaglia/OpenAlprWebhookProcessor: An OpenALPR Webhook Processor for IP Camera Overlays
• Make sure you have at least Node 18.3 installed. I’m on 21.4 and it works.
• Copy processor.db and users.db over to the config directory if needed.
• Execute “dotnet build”.
  • If you run into a maximum call stack exceed error see this: Maximum call stack size exceeded on npm install
• Execute “dotnet run”.
  • If you run into port 5000 already in use, change it to something else in launchSettings.json. Make sure you forward that port on your router so cloud.openalpr.com can communicate with it. The port forwarding is unfortunate, but I don't see a way past it unless running an on-premise server.
• To have the app auto start on login:
  • Create a shell script with: “pkill -9 OpenAlprW; pkill -9 OpenAlprWebhookProcessor; cd /yourlocalpath/OpenAlprWebhookProcessor/OpenAlprWebhookProcessor; nohup dotnet run &”
  • The nohup command will allow it to run in the background without the terminal.
  • The need to kill the process is strange. Maybe it wasn’t completely killing the process when I was logging out to test, so this script will make sure it’s done.
  • Add the script as a launch item in MacOS preferences.
• See logs here: “tail -f /yourlocalpath/OpenAlprWebhookProcessor/OpenAlprWebhookProcessor/config/log-[LATEST].txt”

Other notes:

• Replace Magick.NET with this: “<PackageReference Include="Magick.NET-Q8-arm64" Version="13.5.0" />”.
  • Otherwise it errors out with x64.
  • Then rebuild your project and restart it.
• Version 5.0.0 of the tool will produce an error with the Hydration Service. The way around this is to install 4.2.6 and set things up there. Then you can use that database to get the latest version working.
• I’m hitting about 1.5GB a day in data. At some point, I may point the config to an external drive. This can be done by changing configurationDirectory in Startup.cs to point to your external drive, like “/Volumes/EXTERNALDRIVE/OpenAlprWebhookProcessor/config”.

I'm hoping this helps anyone else who wants to run this on MacOS!