OpenALPR Webhook Processor for IP Cameras

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
That's really cool!

i think searching and alerting may be a bit more important for me in the interim but I like the idea of a speed based sensor.
 
Last edited:

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
How do you want to handle purging old plates from the DB other than deleting the db and starting over? It would be nice to have an upper limit set for "X days" that would delete all plates over those dates. It probably shouldn't be a mandatory settings for those wanting to keep plates indefinitely.
 

mlapaglia

Getting comfortable
Joined
Apr 6, 2016
Messages
610
Reaction score
303
the watchman agent stores the images and prunes them automatically. are you worried about disk space? each license entry should be something like 1KB.
 

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
No not worried about size per say. Im just wondering as the DB grows over time if searching functions you will be implementing will slow down over time because of the sheer number of plates to sift through.

I'm averaging about 1200 plates a week or ~62,000 a year. Just wondering how the DB will perform as the size of it grows without a way to purge it.
 
Last edited:

mlapaglia

Getting comfortable
Joined
Apr 6, 2016
Messages
610
Reaction score
303
you'd need a couple million records for you to notice even a slight delay. it can hold the whole thing in memory. i'll look into some configuration options for pruning old plates.

i am calibrating the speed meter currently, looks promising so far!
1610653916268.png
 

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
Not having an update this morning after 2 weeks of alphas to test I almost didn't know what to do with myself! ;)
 

mlapaglia

Getting comfortable
Joined
Apr 6, 2016
Messages
610
Reaction score
303
I'm working on a settings page that will get rid of manually configuring the app.settings file. you'll just need to give it a port to map to and the rest will be done through the UI
 

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
Wow. Replacing that .Json settings file will do wonders to clearing up some questions and errors upon startup.
 

mlapaglia

Getting comfortable
Joined
Apr 6, 2016
Messages
610
Reaction score
303
ok it's ready for a first look, use v3.0.6-settings
you'll need to fill out the agent and camera configuration through the UI to get new plates. they should start adding to your existing plates, you don't need to delete your databases.
 
Last edited:

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
Love the look and its much better than having to mess with a .json file. I also see you removed the hydrator which I think was a good idea since you need a commercial license to pull from their DB anyway.

The only thing that confused me a little is on the OpenALPR Watchman Agent setup page. At the top, the endpoint default line text is http(s):/x.x.x.x/webhook but I don't access my agent via that webhook sub directory. Mine is just x.x.x.x:8355 using their default port number that I had to enable in the openalprd.conf file.

Everything else is great. Man this gets better and better!

Edit: I noticed that dotnet is pegging my VM cpu to 100%. Its a small 1 vCore VM for testing so its running at 100% here. The old program didn't use any CPU at all. I'll look at the logs.

Screenshot from 2021-01-18 07-28-37.png

Logs:

JSON:
{"log":"[13:35:50 WRN] Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.\n","stream":"stdout","time":"2021-01-18T13:35:50.997221854Z"}
{"log":"[13:35:50 INF] User profile is available. Using '/root/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest.\n","stream":"stdout","time":"2021-01-18T13:35:50.997684161Z"}
{"log":"[13:35:51 INF] Entity Framework Core 5.0.2 initialized 'ProcessorContext' using provider 'Microsoft.EntityFrameworkCore.Sqlite' with options: None\n","stream":"stdout","time":"2021-01-18T13:35:51.472759163Z"}
{"log":"[13:35:51 INF] Executed DbCommand (8ms) [Parameters=[], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:35:51.564611542Z"}
{"log":"SELECT COUNT(*) FROM \"sqlite_master\" WHERE \"name\" = '__EFMigrationsHistory' AND \"type\" = 'table';\n","stream":"stdout","time":"2021-01-18T13:35:51.564625559Z"}
{"log":"[13:35:51 INF] Executed DbCommand (1ms) [Parameters=[], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:35:51.582237806Z"}
{"log":"SELECT \"MigrationId\", \"ProductVersion\"\n","stream":"stdout","time":"2021-01-18T13:35:51.58225082Z"}
{"log":"FROM \"__EFMigrationsHistory\"\n","stream":"stdout","time":"2021-01-18T13:35:51.582253515Z"}
{"log":"ORDER BY \"MigrationId\";\n","stream":"stdout","time":"2021-01-18T13:35:51.582255709Z"}
{"log":"[13:35:51 INF] Entity Framework Core 5.0.2 initialized 'UsersContext' using provider 'Microsoft.EntityFrameworkCore.Sqlite' with options: None\n","stream":"stdout","time":"2021-01-18T13:35:51.71410184Z"}
{"log":"[13:35:51 INF] Executed DbCommand (0ms) [Parameters=[], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:35:51.71541434Z"}
{"log":"SELECT COUNT(*) FROM \"sqlite_master\" WHERE \"name\" = '__EFMigrationsHistory' AND \"type\" = 'table';\n","stream":"stdout","time":"2021-01-18T13:35:51.715421043Z"}
{"log":"[13:35:51 INF] Executed DbCommand (0ms) [Parameters=[], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:35:51.717291558Z"}
{"log":"SELECT \"MigrationId\", \"ProductVersion\"\n","stream":"stdout","time":"2021-01-18T13:35:51.717300214Z"}
{"log":"FROM \"__EFMigrationsHistory\"\n","stream":"stdout","time":"2021-01-18T13:35:51.717302639Z"}
{"log":"ORDER BY \"MigrationId\";\n","stream":"stdout","time":"2021-01-18T13:35:51.717304592Z"}
{"log":"[13:35:51 INF] Now listening on: http://[::]:80\n","stream":"stdout","time":"2021-01-18T13:35:51.790339307Z"}
{"log":"[13:35:51 INF] Application started. Press Ctrl+C to shut down.\n","stream":"stdout","time":"2021-01-18T13:35:51.790564839Z"}
{"log":"[13:35:51 INF] Hosting environment: Production\n","stream":"stdout","time":"2021-01-18T13:35:51.790674184Z"}
{"log":"[13:35:51 INF] Content root path: /app\n","stream":"stdout","time":"2021-01-18T13:35:51.790811Z"}
{"log":"[13:35:52 INF] Entity Framework Core 5.0.2 initialized 'ProcessorContext' using provider 'Microsoft.EntityFrameworkCore.Sqlite' with options: None\n","stream":"stdout","time":"2021-01-18T13:35:52.067687792Z"}
{"log":"[13:35:52 INF] Executed DbCommand (2ms) [Parameters=[], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:35:52.203274684Z"}
{"log":"SELECT \"c\".\"Id\", \"c\".\"CameraPassword\", \"c\".\"CameraUsername\", \"c\".\"LatestProcessedPlateUuid\", \"c\".\"Latitude\", \"c\".\"Longitude\", \"c\".\"Manufacturer\", \"c\".\"ModelNumber\", \"c\".\"OpenAlprCameraId\", \"c\".\"OpenAlprName\", \"c\".\"PlatesSeen\", \"c\".\"UpdateOverlayTextUrl\"\n","stream":"stdout","time":"2021-01-18T13:35:52.203323585Z"}
{"log":"FROM \"Cameras\" AS \"c\"\n","stream":"stdout","time":"2021-01-18T13:35:52.203332181Z"}
{"log":"[13:35:52 INF] force clearing overlay for: 211443207\n","stream":"stdout","time":"2021-01-18T13:35:52.261155698Z"}
{"log":"[13:36:09 INF] Request starting HTTP/1.1 POST http://10.200.200.10:3859/users/refresh-token application/json 2\n","stream":"stdout","time":"2021-01-18T13:36:09.529237054Z"}
{"log":"[13:36:09 INF] CORS policy execution successful.\n","stream":"stdout","time":"2021-01-18T13:36:09.607938317Z"}
{"log":"[13:36:09 WRN] attempted login failed\n","stream":"stdout","time":"2021-01-18T13:36:09.823936406Z"}
{"log":"[13:36:09 INF] Successfully validated the token.\n","stream":"stdout","time":"2021-01-18T13:36:09.850064433Z"}
{"log":"[13:36:09 INF] Executing endpoint 'OpenAlprWebhookProcessor.Users.UsersController.RefreshToken (OpenAlprWebhookProcessor)'\n","stream":"stdout","time":"2021-01-18T13:36:09.855920808Z"}
{"log":"[13:36:09 INF] Route matched with {action = \"RefreshToken\", controller = \"Users\"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] RefreshToken(System.Threading.CancellationToken) on controller OpenAlprWebhookProcessor.Users.UsersController (OpenAlprWebhookProcessor).\n","stream":"stdout","time":"2021-01-18T13:36:09.897337657Z"}
{"log":"[13:36:09 INF] Entity Framework Core 5.0.2 initialized 'UsersContext' using provider 'Microsoft.EntityFrameworkCore.Sqlite' with options: None\n","stream":"stdout","time":"2021-01-18T13:36:09.96746909Z"}
{"log":"[13:36:10 INF] Executed DbCommand (3ms) [Parameters=[@__token_0='?' (Size = 88)], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:36:10.239947208Z"}
{"log":"SELECT \"t\".\"Id\", \"t\".\"FirstName\", \"t\".\"LastName\", \"t\".\"PasswordHash\", \"t\".\"PasswordSalt\", \"t\".\"Username\", \"r0\".\"Id\", \"r0\".\"Created\", \"r0\".\"CreatedByIp\", \"r0\".\"Expires\", \"r0\".\"ReplacedByToken\", \"r0\".\"Revoked\", \"r0\".\"RevokedByIp\", \"r0\".\"Token\", \"r0\".\"UserId\"\n","stream":"stdout","time":"2021-01-18T13:36:10.240002291Z"}
{"log":"FROM (\n","stream":"stdout","time":"2021-01-18T13:36:10.240008783Z"}
{"log":"    SELECT \"u\".\"Id\", \"u\".\"FirstName\", \"u\".\"LastName\", \"u\".\"PasswordHash\", \"u\".\"PasswordSalt\", \"u\".\"Username\"\n","stream":"stdout","time":"2021-01-18T13:36:10.240011078Z"}
{"log":"    FROM \"Users\" AS \"u\"\n","stream":"stdout","time":"2021-01-18T13:36:10.240037287Z"}
{"log":"    WHERE EXISTS (\n","stream":"stdout","time":"2021-01-18T13:36:10.240040623Z"}
{"log":"        SELECT 1\n","stream":"stdout","time":"2021-01-18T13:36:10.240042467Z"}
{"log":"        FROM \"RefreshToken\" AS \"r\"\n","stream":"stdout","time":"2021-01-18T13:36:10.24004415Z"}
{"log":"        WHERE (\"u\".\"Id\" = \"r\".\"UserId\") AND (\"r\".\"Token\" = @__token_0))\n","stream":"stdout","time":"2021-01-18T13:36:10.240046084Z"}
{"log":"    LIMIT 2\n","stream":"stdout","time":"2021-01-18T13:36:10.240048268Z"}
{"log":") AS \"t\"\n","stream":"stdout","time":"2021-01-18T13:36:10.240049931Z"}
{"log":"LEFT JOIN \"RefreshToken\" AS \"r0\" ON \"t\".\"Id\" = \"r0\".\"UserId\"\n","stream":"stdout","time":"2021-01-18T13:36:10.240051624Z"}
{"log":"ORDER BY \"t\".\"Id\", \"r0\".\"Id\"\n","stream":"stdout","time":"2021-01-18T13:36:10.240084626Z"}
{"log":"[13:36:10 INF] Executed DbCommand (1ms) [Parameters=[@p3='?', @p0='?' (Size = 88), @p1='?', @p2='?' (Size = 13)], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:36:10.566190366Z"}
{"log":"UPDATE \"RefreshToken\" SET \"ReplacedByToken\" = @p0, \"Revoked\" = @p1, \"RevokedByIp\" = @p2\n","stream":"stdout","time":"2021-01-18T13:36:10.566204914Z"}
{"log":"WHERE \"Id\" = @p3;\n","stream":"stdout","time":"2021-01-18T13:36:10.56625087Z"}
{"log":"SELECT changes();\n","stream":"stdout","time":"2021-01-18T13:36:10.566255068Z"}
{"log":"[13:36:10 INF] Executed DbCommand (0ms) [Parameters=[@p0='?', @p1='?' (Size = 13), @p2='?', @p3='?', @p4='?', @p5='?', @p6='?' (Size = 88), @p7='?'], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:36:10.580714246Z"}
{"log":"INSERT INTO \"RefreshToken\" (\"Created\", \"CreatedByIp\", \"Expires\", \"ReplacedByToken\", \"Revoked\", \"RevokedByIp\", \"Token\", \"UserId\")\n","stream":"stdout","time":"2021-01-18T13:36:10.580750735Z"}
{"log":"VALUES (@p0, @p1, @p2, @p3, @p4, @p5, @p6, @p7);\n","stream":"stdout","time":"2021-01-18T13:36:10.580772235Z"}
{"log":"SELECT \"Id\"\n","stream":"stdout","time":"2021-01-18T13:36:10.580782523Z"}
{"log":"FROM \"RefreshToken\"\n","stream":"stdout","time":"2021-01-18T13:36:10.580784717Z"}
{"log":"WHERE changes() = 1 AND \"rowid\" = last_insert_rowid();\n","stream":"stdout","time":"2021-01-18T13:36:10.580786711Z"}
{"log":"[13:36:10 INF] Executed DbCommand (0ms) [Parameters=[@p5='?', @p0='?' (Size = 6), @p1='?' (Size = 7), @p2='?' (Size = 64), @p3='?' (Size = 128), @p4='?' (Size = 6)], CommandType='Text', CommandTimeout='30']\n","stream":"stdout","time":"2021-01-18T13:36:10.607371134Z"}
{"log":"UPDATE \"Users\" SET \"FirstName\" = @p0, \"LastName\" = @p1, \"PasswordHash\" = @p2, \"PasswordSalt\" = @p3, \"Username\" = @p4\n","stream":"stdout","time":"2021-01-18T13:36:10.607423972Z"}
{"log":"WHERE \"Id\" = @p5;\n","stream":"stdout","time":"2021-01-18T13:36:10.607429041Z"}
{"log":"SELECT changes();\n","stream":"stdout","time":"2021-01-18T13:36:10.607431686Z"}
{"log":"[13:36:10 INF] Executing OkObjectResult, writing value of type 'OpenAlprWebhookProcessor.Users.AuthenticateResponse'.\n","stream":"stdout","time":"2021-01-18T13:36:10.671942556Z"}
{"log":"[13:36:10 INF] Executed action OpenAlprWebhookProcessor.Users.UsersController.RefreshToken (OpenAlprWebhookProcessor) in 813.0363ms\n","stream":"stdout","time":"2021-01-18T13:36:10.723974548Z"}
{"log":"[13:36:10 INF] Executed endpoint 'OpenAlprWebhookProcessor.Users.UsersController.RefreshToken (OpenAlprWebhookProcessor)'\n","stream":"stdout","time":"2021-01-18T13:36:10.724475737Z"}
{"log":"[13:36:10 INF] HTTP POST /users/refresh-token responded 200 in 1193.9018 ms\n","stream":"stdout","time":"2021-01-18T13:36:10.725948177Z"}
{"log":"[13:36:10 INF] Request finished HTTP/1.1 POST http://10.200.200.10:3859/users/refresh-token application/json 2 - 200 - application/json;+charset=utf-8 1205.2275ms\n","stream":"stdout","time":"2021-01-18T13:36:10.727552323Z"}
I've gone back to your last alpha for now until we figure out the high CPU issue.
 

Attachments

Last edited:

mlapaglia

Getting comfortable
Joined
Apr 6, 2016
Messages
610
Reaction score
303
that's fixed in v3.0.10-settings
with my ryzen 3900x i didn't even notice it pegged on my box lol
 

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
Alright! When I get home I'll grab the latest version.

I only noticed it when running commands on the command line seemed sluggish in that VM so when I looked at top I saw the usage.
 

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
Alright. I pulled your latest and the CPU issue is totally gone. Nice job. I also see you have 'alerts' on the left menu for future functionality. Excellent!

Is the appsettings.json file no longer needed since you are storing all the relevant information inside the database now?
 

biggen

Known around here
Joined
May 6, 2018
Messages
1,127
Reaction score
924
From your repo:

CSS:
<mat-form-field fxFlex="80">
        <mat-label>Endpoint Url</mat-label>
        <input matInput [(ngModel)]="agent.endpointUrl" placeholder="http(s):/x.x.x.x:yyyy">
        <mat-icon matTooltip="The URL used to reach the Watchman Agent" style="cursor:default" matSuffix>help_center</mat-icon>
    </mat-form-field>
I know you love doing CSS! :)
 
Top