The SECRET WEAPON For Local LLMs – AMD Radeon AI Pro R9700 Showcase

This hidden gem, which both AMD and Overclockers are sponsoring this video so I can help make it less hidden, is AMD’s Radeon R9700 AI Pro workstation graphics card. It’s a 32GB VRAM, 64 compute unit GPU that costs just ONE THIRD of a 5090, yet offers the same VRAM capacity and excellent workstation performance. This, in combination with AMD’s ROCm, will let you skip the Claude or ChatGPT bill AND keep everything entirely private and local. That’s my kind of deal! First, the card.

AMD’s R9700 AI Pro is an RDNA 4 based card – rather similar to the RX 9070 XT in specs actually – but sporting 32GB of GDDR6 VRAM – plenty for most local AI workloads! Of course if that isn’t enough, these Pro cards have support for multi-GPU systems, working in tandem to have 64GB, 96GB or 128GB of VRAM available – all for a third less money than a 5090. That sure sounds great to me! These cards also support ECC (error correcting code) memory, at least on Linux anyway. All of this is available for around £1,500 from OverclockersUK (link in the description), which compared to, say, a VRAM-matching RTX 4500 less than a THIRD the cost, and that one is an awful lot closer in performance. This sure sounds like a steal by comparison.

As for how to start making use of this, it’s remarkably simple. Most LLM tools support AMD GPUs through ROCm already, so you somewhat have your pick of tools. Today I’m going for LM Studio though, or more specifically LM Studio Bionic, which is basically LM Studio, agent edition. Setting this up is as easy as installing it, downloading some models – in my case Gemma 4 31B and 12B QAT, Qwen3.8 27B and Muse Glimmer – then start typing. This agentic flow is pretty clever, so if you ask it to, say, create a calorie tracker app with a react front end, suitable backend and a mariaDB database, it’ll just go and do it all, creating the folders needed and writing files for you – entirely locally. In fact even with just one card and 32GB of VRAM on hand, Qwen3.8 2.7 runs just fine, and decently quickly too. It does use the entire VRAM buffer and it does flood the RAM a fair bit too, but it still rips nicely. Running the same command with Gemma 4 31B QAT took under seven minutes to do the whole thing, although it sure looks like Qwen did a better job of it to me. 

We will come back to the performance and use cases in a second, but I just want to highlight just how good it is to have your AI stuff done locally. The most obvious argument is cost. Sure, if you’re on the £20 a month plan and don’t exceed the limits meaning you don’t end up paying API costs, this card might take a while to pay for itself, but if you’re on the £200 a month plan, or you do end up paying API costs, you’re likely to be saving money by using this thing instead of cloud compute remarkably quickly. Hell, two of these is only a year and three months ROI if you were using the £200 a month plan, and the ongoing costs here are 300 watts of power per card. At the current 26p/kWh rate, that’s around 8p per hour of usage. Even if you used this every hour of every day (which is insane), you’d still be saving £140 per month compared to the subscription. That sounds like a great deal! But the point that’s a lot more important to me is privacy and security. Anything you have these cards do, assuming your interface software is fully local, is entirely private. There’s no data collection going on to leak your company secrets or latest project, there’s no private information being sold to advertisers. It’s entirely yours. Moreover, if the bubble happens to burst, all of the models you currently have are still yours to use. You aren’t reliant on Anthropic or OpenAI to remain solvent, it’s yours to do with as you please, and that’s something I think a lot of people are realising more and more right now. Owning your own hardware and data is essential, and having a card like the R9700 lets you do just that.

Heading back to the performance, it absolutely varies model to model. While Qwen3.8 27B seems to offer the best output, it’s right on the edge for single-card operation. That means it’s a little slow, whereas the Gemma 4 31B QAT model is a fair bit faster. The 12B version is much faster, although the output isn’t quite as effective. I asked Qwen to create an ‘expert’ response time tool (something I’ve been slowly working on for a while now…) and while it took 35 minutes, it did a pretty phenomenal job not only creating the schematic, but giving me a very detailed breakdown of the ADC options and signal pathway. I found it interesting that it not only generated a circuit diagram for me, it also caught its own errors and fixed them. I think that’s the benefit of a more agentic flow, although it does take longer, and the context window keeps growing. It hit 125K by the time it was done! 

One other thing to mention is that while these cards are actively named “AI Pro”, these are workstation cards. If you are a video editor, game developer, software developer, 3D modeller, all of those niches require a workstation system, and a card like the R9700 is a pretty amazing fit. 32GB of VRAM for £1500 makes this one of the best VRAM per pound cards around, with more than enough power to boot. Pair that with a 9950X or hell if you want multiple cards or more outright compute, a Threadripper 9000 series. That’d be one hell of a system, and it’d be leagues cheaper than a Blackwell system instead! Oh, and if you were wondering whether an RX 9070 XT would be a better fit for you – or even a 7900 XTX – beyond the VRAM benefit, the multi-GPU and workstation focus makes this a much better fit for compute. It is still a graphics card though, so if you did want to do a bit of gaming in your off-hours… I’m sure you’d have a pretty good time with it. For me, I’m going to stick this in my own Threadripper server so I can have Qwen3.8 on tap for help programming up a storm, and probably some footage rendering jobs too.