Raspberry Pi + GPU = Amazing Local AI? Sanctuary Systems Sentinel Core CM5 ITX Review

I’ve already showed you how to set up Home Assistant to locally control your own smart home tech, how to use Home Assistant Voice to run local large language models that control your smart home tech, and now, thanks to the indie hardware makers at Sanctuary Systems, I can show you how to do all of that from a raspberry pi with a graphics card attached. This is their Sentinel Core mini ITX motherboard, which comes preloaded with a Raspberry Pi Compute Module 5 with a custom fork of Home Assistant with GPU drivers already installed. Let me show you around the board, then we’ll get it up and running and see what this puppy can do!

For being an ITX motherboard, it is remarkably empty. Most ITX boards are fighting for every micrometre of space, often turning to vertical PCBs to get enough room to fit the socket, RAM, M.2s and a PCIe slot, but this thing? Well besides the board-to-board connectors for the CM5, you’ve got a breadboard area, Pi GPIO headers, the PCIe slot, and a little bit of rear IO – two USB ports, USB C for programming, two HDMIs and ethernet. That’s about it. The CM5 has WiFi built in, and Sanctuary includes a detachable antenna along with a 3D Printed IO shield. You’ve also got two 4 pin PWM fan headers, and a place to plug in your case’s power button. The unique thing here is that the Pi is powered primarily from the 24 pin ATX power supply connection, and that’s where the 3.3V, 5V and 12V for the breadboard, PCIe slot and fans is pulled from. There are a few quirks – this is a pre-production sample, so the tweaks I’ve had to do to get it working are to be expected, and they have shown a perfect attitude to feedback, learning from mistakes to make a better product when this finally ships to customers. There are a few tweaks I haven’t yet mentioned to them but that I would like to see, stuff like a cleaner way to connect the breadboard section to the Pi that isn’t just jumper wires, more options for storage like the SD card slot and maybe even a PCIe switch to have an M.2 slot AND the GPU (since you won’t be using either at full speed much), and an audio codec built in would all be excellent. 

But if you’re familiar with Pis and compute modules in particular, you might be squinting at this and thinking, gosh, that looks familiar, and you’d be right, because it’s just this, the Compute Module 5 IO Board, but embiggened. They’ve swapped the M.2 slot for the PCIe slot, made it taller to fit the mITX standard, added the 24 pin, and swapped to 4 pin fan headers. There are a couple other changes, like the transistor that latches the power supply on when the Pi is on, but for the most part it’s a modified dupe. You might look at this then and wonder why you’d pay the premium price tag of €219.99 (or about £190 right now), when a CM5 and IO board will set you back just over £100. Well the short answer is that one, the IO board doesn’t let you connect a GPU to the Pi (easily, anyway), and two, you’re buying not just the hardware, but the full package. Sanctuary System’s custom fork of Home Assistant not only comes preinstalled and ready to use, but has graphics drivers built in and a GPU accelerated version of Llama and Whisper ready-to-go. That fork is still being improved, bugs fixed, and I have to assume more features will be added too. That’s where the added value goes. 

So, let’s look at that value, by setting this thing up. With a power supply connected and an RX 7600 in the PCIe slot, the board fires up, and with ethernet connected it’s ready to go. Head to the URL displayed on screen, in my case that’s 192.168.1.41, port 8123, and run through the HASS setup. To run local AI, you’ll need to head to the addon store and install Sanctuary’s version of Llama.cpp and Whisper.cpp, along with Piper, OpenWakeWord, and you’ll need to install HACS too, which once the “add-on” has installed, you’ll need to reboot HASS, then go to Devices & Integrations to configure it, then once you’ve connected your Github to it, you’ll want to find and install Extended OpenAI Conversation. You’ll also need a mic and speakers, to do that locally you’ll want Assist Microphone. Then the Wyoming protocol should pick it all up and once you set up a Voice Assistant, you’ll be ready to go. All of those steps are covered in my existing videos by the way, so if you want more information on the piper voices, whisper settings or (o)llama models, check those out in the cards above. 

The fact this is GPU accelerated means responses are near instant, much faster than my current setup, and the cool thing about using the Extended OpenAI Conversation tool to run the conversation agent is that you can do a whole lot with it. It can do google searches, play youtube or plex or netflix videos on your TV, tell you the weather or give you history on home assistant entities, and even get energy data from Home Assistant. Being tied to an LLM means not only can you ask it the pointless questions about the meaning of life and get long responses back, but it can collate responses and data together and present it however you like. The joy of self-hosting means you can tune everything to how you want it – be that your AI sounding like Jarvis, a slightly robotic Scottish woman, or really anything – and even how it responds by changing the prompt.

If I’m being honest though, that sort of thing isn’t really for me. My particular flavour of autism means I care a lot about practical, useful features, and a lot less about customisations and what appear to me anyway as kinda gimmicky features, and I genuinely think if you added a way to use some proper storage on this, you’re getting awfully close to having the perfect easy to use home server in a tiny box. You can have Home Assistant, Plex or Jellyfin, and a PiHole, all running from here, with GPU acceleration. That’s kind of amazing, and all with fairly low power, and fairly low cost. That would be amazing. But there is a bit of a catch. An AM4 ITX motherboard can be had for around £100, and a Ryzen 5600 – a 6 core desktop CPU – can be had for just £90. Admittedly you’ll have to sink another £30 into 16GB of RAM, but then you’ve got a powerful desktop 6 core, that’s upgradable, with plenty of storage options, a GPU slot, more IO and it still fits in an ITX case, for very similar money. That’s essentially what I did, except I don’t care about size so I have a desktop case with a 10400f and two GPUs, but still. That does mean you’d need to install Home Assistant and all that yourself, but it does also give you more flexibility, so maybe that’s something to consider.

Anyway, this is a really cool board. It should be plug and play, and hell it’s even made in the EU – Aisler, a German PCB maker are the ones building the boards – so that’s really cool too. It is also open source, everything from the HASS fork to the PCB files are all available on their Github repos, and you know how much I enjoy a good open source hardware project… Still I don’t think it’s for me, but if you want a super easy to set up Home Assistant board that can also do GPU accelerated AI stuff, this is a fantastic choice. It is still on preorder, with plans to ship in Q3 this year, so if you’re interested in one, check them out at the link in the description below!