Thanks for the update @MikeLud1. I don't know if you saw my question on another thread about incorporating my "delivery" custom model into the CPAI install. I was just wondering if you know when that might be happening. When it does how does one see the updated models? Also, if I make updates in the future, do I need to notify @ChrisMaunder?
CodeProject.AI Version 2.5
- Thread starter MikeLud1
- Start date
I'm observing odd behaviour
YOLOv5 3.1 works with my GTX970
YOLOv5 6.2 stopped working with GTX970 with the last 2 updates
YOLOv8 works with my gtx970 but does not have the custom models yet as far as I see
Haven't figured out why 6.2 has stopped working - only works with the cpu. Does not pick up the gpu at all.
YOLOv5 3.1 works with my GTX970
YOLOv5 6.2 stopped working with GTX970 with the last 2 updates
YOLOv8 works with my gtx970 but does not have the custom models yet as far as I see
Haven't figured out why 6.2 has stopped working - only works with the cpu. Does not pick up the gpu at all.
I have nvcc locally, in the container it's "not found", I don't believe it's part of the docker image: codeproject/ai-server:cuda12_2
There are no errors, the .NET YOLO just will not use the GPU, it is clearly using CPU as inference times are 200ms. With YOLOv5, I get inference times of 25ms.
I have the nvidia-container-toolkit installed.
Version 2.6.1 is coming soon.
One of the new features.
One of the new features.
CodeProject.AI Version 2.6.2 has been released.
www.codeproject.com
CodeProject.AI Server: AI the easy way.
Version 2.6.5. Our fast, free, self-hosted Artificial Intelligence Server for any platform, any language
www.codeproject.com
The model used is the below, it should run on both Nvidia GPU and CPU (CPU will be slow). To run on a GPU you need greater then 6.87 GB of VRAM
TheBloke/Mistral-7B-Instruct-v0.2-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
Code:
/ For loading via llama-cpp.from_pretrained
"CPAI_MODULE_LLAMA_MODEL_REPO": "@TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
"CPAI_MODULE_LLAMA_MODEL_FILEGLOB": "*.Q4_K_M.gguf",
To update to CodeProject.AI Version 2.6.2 you do not need to uninstall if you are on V2.5.6 just disable AI in Blue Iris before installing. After all the module you selected during the install are installed you can enable AI in Blue Iris. If the custom model list is empty restart Blue Iris service and the list should update.
I just did an install on my main BI system and had no issues installing it without uninstall. If you are on the latest version installing an update without uninstalling should work. This might change if there is a major update.
Last edited:
Just upgraded to 2.6.2 and I have selected to use GPU instead of CPU. For some reason it still says CPU (DirectML)... 
what happened between blue iris 5.8.6.x and 5.8.7.x that could cause a massive increase in my codesense AI response time? only new thing i saw in the AI panel was pre-trigger images, but even with that set at 0 i went from 32ms average to 800ms+. I'm also getting CPU spikes 80%-100% when the cameras trigger.
i'm having a hard time finding any discussion about this so, i'm guessing it's user error somewhere.
i'm having a hard time finding any discussion about this so, i'm guessing it's user error somewhere.
I had an issue that was caused by the BI settings AI timeout restarting the CP.AI service. I changed the timeout from roughly 30 seconds to 60 seconds and it resolved my issues.
If using GPU not CPU it should be using YOLOV5 6.2 rather than .NET with DirectML if I remember correctly. Try telling CP.AI to start 6.2 instead and it should change the default to that. Then uncheck GPU in BI settings, hit ok, go back into settings, re-select GPU, and hit OK again. That should make it start using GPU and the correct module.
I don’t have an upgraded GPU and I thought YOLO was for upgraded GPUs. .NET was for non upgraded GPUs.
The .NET module works with all GPUs. Depending on your Nvidia GPU .NET might be faster then CUDA
github.com
GitHub - microsoft/DirectML: DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers, in
DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported ...
github.com