People can mask USPS AI detections

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
Thank you for your effort on this, it is REALLY appreciated. Is it too much to ask that you place this on github.com so we can track updates?

I'm looking to implement this soon, again, thank you!
The delivery model is now on Github along with exported models for YOLOv5 6.2, YOLOv5 NET, YOLOv8 and YOLOv8 NET
Link:
Custom delivery models on VideoDad's Github
 

ELA

n3wb
Joined
Dec 11, 2021
Messages
5
Reaction score
0
Location
home
The delivery model is now on Github along with exported models for YOLOv5 6.2, YOLOv5 NET, YOLOv8 and YOLOv8 NET
Link:
Custom delivery models on VideoDad's Github
Hello VideoDad,
Thank you for your work in producing these models.

I am still running an older version of Blue IRIS and using Deepstack for AI.
I recently installed your USPS.pt model to be able to detect the mail delivery. It runs on a clone that runs only a single delivery model. It worked ok but with lots of false positives. I kept increasing the confidence level until it worked fairly reliable ay 93%. Today it alerted at 93% on my blue recycle bin :)

The USPS truck often has its door open and that makes it a bit harder to detect.

After reading as many posts as I could find on the subject; I was hopeful that the delivery.pt model had improvements that might help. I have been unable to get the delivery.pt model to work at all on my clone meant for delivery vehicles only.
I had guessed that the YOLOv5 6.2 was what I needed and tried that today without success.
Can you please advise as to which model would be correct for Blue Iris 5.5.9.6 and Deepstack GPU-2021.09.1?
 

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
I recently installed your USPS.pt model to be able to detect the mail delivery.
I didn't create the USPS.pt model; I only used many of the images in the training of my delivery models. And yes, I did improve it with more images especially from actual security cameras and with doors open.
Can you please advise as to which model would be correct for Blue Iris 5.5.9.6 and Deepstack GPU-2021.09.1?
I was using the YOLOv5 6.2 model on Deepstack prior to switching to CPAI. I believe it should still work on DS but haven't officially tested it.

First I assume you've got delivery.pt in the correct folder for DS custom models. In my model, all the labels are lowercase so make sure you're looking for "usps" when using "delivery" as the custom model.

If that's all correct and you're still having problems, post your AI settings for the camera. Also, please post the logs for BI when it should be seeing a delivery vehicle.
 

ELA

n3wb
Joined
Dec 11, 2021
Messages
5
Reaction score
0
Location
home
Thankyou for your reply VideoDad,
My apologies to sstratoti after reviewing that he created the USPS.pt.

Yes I did have the model in the custom models folder ( same one that USPS.pt and all my other previous custom models models are located.)
I did use lower case for the delivery.pt model.
After trying to get it to work without success I found that deepstack stopped working all together. I had to uninstall and re-install to re-establish a working Ai system.

Thankyou for confirmation that the YoloV5 6.2 model is the correct choice. I will try it again.

I had downloaded the delivery.pt model from this site that you had posted and tried that a while back without success. However, DS did not get corrupted during that attempt.

one more question I have:
Now that you have the model on github, is the yolov5 6.2 model the same as the original delivery.pt or different? If they are different I will try both once again.
Thankyou for the work you put into the models. I will give it a more thorough effort now that you have offered your input.
 

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
Thankyou for your reply VideoDad,
My apologies to sstratoti after reviewing that he created the USPS.pt.

Yes I did have the model in the custom models folder ( same one that USPS.pt and all my other previous custom models models are located.)
I did use lower case for the delivery.pt model.
After trying to get it to work without success I found that deepstack stopped working all together. I had to uninstall and re-install to re-establish a working Ai system.

Thankyou for confirmation that the YoloV5 6.2 model is the correct choice. I will try it again.

I had downloaded the delivery.pt model from this site that you had posted and tried that a while back without success. However, DS did not get corrupted during that attempt.

one more question I have:
Now that you have the model on github, is the yolov5 6.2 model the same as the original delivery.pt or different? If they are different I will try both once again.
Thankyou for the work you put into the models. I will give it a more thorough effort now that you have offered your input.
Just to reiterate, when I'm talking about case, I'm talking about the possible objects (not the filename). In the USPS.pt model, the only object was "USPS". In mine, all the objects are lowercase (e.g. amazon, fedex, usps, etc.).

As for updates, I uploaded each of the versions from before as revisions. So if you look at the history of delivery.pt, you'll see the latest plus the prior versions. Version 4 is the one that was previously posted for download.

I have made a couple updates since then to streamline the model, add more training images, etc.
 

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
@ChrisMaunder some new models for you to add to CodeProject.AI install.
@MikeLud1, thanks for proposing this to @ChrisMaunder. I'd love them to get included in the CPAI install. Let me know if there is anything you need from me. Also, if I update the GitHub repository in the future, do I need to notify someone or will the install pull the latest each time?

Mike, thanks for previously hosting a copy of delivery.pt and creating a corresponding delivery.onnx for it. I don't know if you want to update them with the latest in your repository now, or just remove them to avoid confusion?

Finally, are there other modules (Coral TPU?) and sizes (medium, large) you are planning to target with CPAI and your models? If so, let me know and I'll try to do the same.
 

ELA

n3wb
Joined
Dec 11, 2021
Messages
5
Reaction score
0
Location
home
Just to reiterate, when I'm talking about case, I'm talking about the possible objects (not the filename). In the USPS.pt model, the only object was "USPS". In mine, all the objects are lowercase (e.g. amazon, fedex, usps, etc.).

As for updates, I uploaded each of the versions from before as revisions. So if you look at the history of delivery.pt, you'll see the latest plus the prior versions. Version 4 is the one that was previously posted for download.

I have made a couple updates since then to streamline the model, add more training images, etc.
Thanks VideoDad,
I did understand the need for lower case on objects to detect.

I was not familiar enough with Github to understand the release versions. I took a look and saw the original at the bottom and 5 other releases. Please explain what you mean by version 4? I did not see version numbers at the site, but as I said I am not that familiar.

I took the time to do some testing. I was able to get the original at the bottom to work. I was also able to get the 3rd from the bottom to work. (The one where you added the open door images.)
I was unable to get the versions released after that to work. ( higher up on the release page ).

I have no idea whether it is just my install or some thing else. I will stick with the 3rd from the bottom release for now and see how it does going forward.
Just providing some feedback regarding the newer versions.

Thanks again for your help in understanding.
 

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
Thanks VideoDad,
I did understand the need for lower case on objects to detect.

I was not familiar enough with Github to understand the release versions. I took a look and saw the original at the bottom and 5 other releases. Please explain what you mean by version 4? I did not see version numbers at the site, but as I said I am not that familiar.

I took the time to do some testing. I was able to get the original at the bottom to work. I was also able to get the 3rd from the bottom to work. (The one where you added the open door images.)
I was unable to get the versions released after that to work. ( higher up on the release page ).

I have no idea whether it is just my install or some thing else. I will stick with the 3rd from the bottom release for now and see how it does going forward.
Just providing some feedback regarding the newer versions.

Thanks again for your help in understanding.
Thanks for the feedback. I'll have to get a Deepstack instance running and see if I can reproduce your issues. What is it you see when you say it doesn't work?
 

ELA

n3wb
Joined
Dec 11, 2021
Messages
5
Reaction score
0
Location
home
Thanks for the feedback. I'll have to get a Deepstack instance running and see if I can reproduce your issues. What is it you see when you say it doesn't work?
Sorry for being so vague in using the not working comment. I should know better.

I have been using the "Testing and Tuning function- using AI". For the models that work I get a lot of boxes with percentages for what it "sees".
Using the models higher up than the third from the bottom, there are no boxes or percentages displayed. This is using the same alert clips for all testing. I also notice that the computer seems to be struggling or slowed down when running the models that do not present boxes or percentages.

Thanks again, I really appreciate all the work that you and others have put into creating the custom models. This site has been invaluable in helping me get things working.
 

wittaj

IPCT Contributor
Joined
Apr 28, 2019
Messages
24,924
Reaction score
48,629
Location
USA
Sorry for being so vague in using the not working comment. I should know better.

I have been using the "Testing and Tuning function- using AI". For the models that work I get a lot of boxes with percentages for what it "sees".
Using the models after the third from the bottom, there are no boxes or percentages displayed. This is using the same alert clips for all testing. I also notice that the computer seems to be struggling or slowed down when running the models that do not present boxes or percentages.

Thanks again, I really appreciate all the work that you and others have put into creating the custom models. This site has been invaluable in helping me get things working.
Keep in mind that the "Analyze with Deepstack/CodeProject" under "Testing & Tuning" will ALWAYS perform better than live as it is after the fact and should not be used as an analysis tool to try to figure out why it didn't see and trigger for a car or person or when it triggered. It should only be used to see what AI can find in that clip, like "hmm I wonder if AI can find a toothbrush" and then walk around with a toothbrush and have it identify it. I can run this on a camera not using AI and it will show EVERYTHING that AI has in its objects to find that it sees in the clip. This method will show you EVERY ITEM AI searches for.

You need to review the .dat files as that will show you how Deepstack/CodeProject interpreted it.

And as you are seeing, you should really try to limit the number of models a camera will run.
 

ELA

n3wb
Joined
Dec 11, 2021
Messages
5
Reaction score
0
Location
home
Hello wittaj,
Understood. I have always used the .dat files to review, analyze and improve settings. In this particular case I have no new captures using the new delivery models yet, as I have just installed it.
I was only using the Test and Tune on existing clips to confirm the model was or was not working at all. I will tune it further using new .dat files when new deliveries (.dat files) arrive.
 

will40

n3wb
Joined
Aug 16, 2021
Messages
8
Reaction score
2
Location
Boston
Hi VideoDad I updated the delivery model to the most recent on your Github but I am getting this following error in my CP log.

10:39:37:Object Detection (YOLOv5 6.2): [UnpicklingError] : Unable to load model at C:\Program Files\CodeProject\AI\modules\ObjectDetectionYOLOv5-6.2\custom-models\delivery.pt (invalid load key, '\x0a'.)
10:39:37:Object Detection (YOLOv5 6.2): Unable to create YOLO detector for model delivery

Anything else I can try? Thanks
 

will40

n3wb
Joined
Aug 16, 2021
Messages
8
Reaction score
2
Location
Boston
update: I restored a previous version of delivery.pt and the model loads fine. Waiting to test it out
 

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
Hi VideoDad I updated the delivery model to the most recent on your Github but I am getting this following error in my CP log.

10:39:37:Object Detection (YOLOv5 6.2): [UnpicklingError] : Unable to load model at C:\Program Files\CodeProject\AI\modules\ObjectDetectionYOLOv5-6.2\custom-models\delivery.pt (invalid load key, '\x0a'.)
10:39:37:Object Detection (YOLOv5 6.2): Unable to create YOLO detector for model delivery

Anything else I can try? Thanks
Can you first verify the checksum on the most recent version of delivery.pt?

Type this command in a command window:
certutil -hashfile delivery.pt SHA1

b99be438e838b7b811f1477a3532a02aa39821b6
 

will40

n3wb
Joined
Aug 16, 2021
Messages
8
Reaction score
2
Location
Boston
SHA1 hash of delivery.pt:
bb4230457cdb3cce7f5d05b52758eb46c1b3531a
CertUtil: -hashfile command completed successfully

Hope this helps
 

will40

n3wb
Joined
Aug 16, 2021
Messages
8
Reaction score
2
Location
Boston
ok I grabbed another copy now my hash matches. That must have been it. Let me put this model back and try it out. Thanks!
 

VideoDad

Pulling my weight
Joined
Apr 13, 2022
Messages
162
Reaction score
216
Location
USA
SHA1 hash of delivery.pt:
bb4230457cdb3cce7f5d05b52758eb46c1b3531a
CertUtil: -hashfile command completed successfully

Hope this helps
That doesn't match with the hash code for the latest version. What's the file size?
 
Top