Best hardware accelerator for CP AI (Spring 2022) under $100USD?

[silencery]

Hey ipcamtalk. Sorry if this has been brought up, but things move quickly, so just wanted to get some opinions on the best hardware accelerator options for under $100USD? I was previously looking at coral m.2 since I have one lying around, but it won't work in my current Intel-based system. I can spare 1 PCIe slot in my system, so I'd like to ask what the best options would be right now. Used Nvidia Quadro? USB stick? Something else I'm not aware of?

I don't really have a target inference level, just something better than CPU or 630 iGPU which is currently around 150-200ms. I guess 50ms or faster would be nice?

Thanks!

 

[AlwaysSomething]

Not to change the subject but why won't the Coral M.2 work for you? I have one that I've been using since the beginning of the year. I just bought a Dual TPU with a PCIe adapter so I can use a Medium model size since that worked beest for me. However, I'm having issues with the Dual in my BI PC so went back to the M.2 for now. I think there is a bug in CPAI 2.6.2 when enabling multi tpu.

FYI - I bought one of these adapters for the Coral M.2 to use in a PCIe slot on a different machine and it works:

https://www.amazon.com/gp/product/B07JJTVGZM/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

I didn't have an M.2 slot available and for $10 I figured I'd give it a shot. Been working fine so far. Might be an option until you get some responses here.

 

[silencery]

Oh, great question.

My topic was intended to be a conversation reference for others in the same boat right now. I was looking around and noticed most of the recommended M.2 to PCIe adapters were in the $30 range and Coral Dual TPUs are now back in stock for $40 ($70 total). Got me wondering if a used GPU might be high performing enough to be better for about the same price point to be cheap enough as a toy to play with ($100).

HOWEVER, that adapter you linked is cheap enough to be a bit of a gamechanger. What a great find. If a coral Dual TPU + adapter can be used in most machines (assuming PCIe support is better than M.2 support), the target price point gets pushed down even lower which is certainly a bonus.

 

[mailseth]

There is also this one, but your motherboard has to be compatible with bifurcation.

https://a.co/d/cEtN1G7

 

[silencery]

Awesome. My board does support birfucation. Do you have first-hand experience with that adapter? I'm wondering if the Dual version of the coral works on it.

 

[mailseth]

I do not have experience. It was something I looked into and it seemed interesting until I learned I needed bifurcation and didn’t have it. With that one in particular you may need 4-way bifurcation, which is even more rare, but as I understand it if you only have 2-way that just means that only two of the four slots work.

I’d guess that you would need this adapter with the above linked adapter with the Dual TPU because the Dual TPU itself uses to separate PCIe interfaces.

Dual Edge TPU Adapter - m.2 2280 B+M key

Dual Edge TPU Adapter is designed for Coral m.2 Accelerator with Dual Edge TPU to be used on a system with m.2 2280 B- or M-key slot available.

www.makerfabs.com

But I really don’t know. You’d need to test it out and get back to us.

 

[mailseth]

I’m hoping that the guy finishes this board:

Dual-Edge-TPU-Adapter/PCIex4_2xDualTPU_s.jpeg at main · magic-blue-smoke/Dual-Edge-TPU-Adapter

Dual Edge TPU Adapter to use it on a system with single PCIe port on m.2 A/B/E/M slot - magic-blue-smoke/Dual-Edge-TPU-Adapter

github.com

And then we can drop a few duals into a single slot.

 

[AlwaysSomething]

I bought both versions of the dual adapter, the M.2 and the PCIe. You can use the DUAL TPU M.2 adapter on top of the adapter I posted above and it works. In other words the following will work:

Coral Dual TPU -> M.2 adapter from Makerfabs -> M.2 to PCIe adapter I posted above.

However, if you are going to go the PCIe route you mind as well buy the PCIe adapter meant for the Dual TPU (one less adapter). In my case I tried it out since I already had the hardware and it works.

BTW - The reason I have this all is I bought all my stuff in the middle of the night impulsively (and lacking sleep or brains) so that is why. I should have returned most of it but the cost to ship back wasn't worth the refund so I kept everything and figured I may be able to use on spare PCs later.

 

[silencery]

Thanks for clarifying. Gotta love those late night purchases! We've all been there.
This reminds me all over again that the Dual TPU should be used in dual channel mode in order to use both processing units. An adapter which properly supports the necessary 2x PCIe channels for the Dual TPU to work costs minimum $30, so back to square one:

I wonder if a used quadro for the $100 price range would be better than a coral M.2 right now since a coral is already 5 years old anyway.

 

[mailseth]

The most important part of my calculus ended up being power consumption, which is pretty hard to measure. Running a GPU on a steady stream of video for a year is going to cost a bit. For me, for example, I calculated out that if the card ended up using an extra ~40 watts, I'd spend $100-$200 per year in electricity just to power the thing. The exact power usage of a GPU is going to be very debatable, however, and I don't know of a good way of knowing it in advance. However, we do know that the TPUs are much cheaper and tend to run at under 1 watt each.

 

[silencery]

Yeah, absolutely. power, heat, and noise budget are all part of the calculus. Costs are one thing, but noise could be an even bigger dealbreaker for some.

Too bad benchmarks don't exist for CPAI. Trying to get a handle on what to expect out of various platforms