Don't buy a NAS for local AI, get this instead
A NAS is a great addition to any homelab setup. It can back up your photos and other files, run a Jellyfin or Plex server, and replace a few of your subscription services.
It is tempting to think that they'd also be a good way to get into local AI. Unfortunately, their hardware really isn't up to the job. If you need something compact and energy efficient, there are better, most cost-effective options available.
A NAS isn't the answer for local AI
They're convenient but not the best
NAS units are readily available, relatively affordable, and popular, since they can help you cut subscription costs. Increasingly, they're also being used to self-host some AI applications. However, you shouldn't buy one specifically for AI.
Most major NAS manufacturers build their units with low-power CPUs meant for serving files without consuming a ton of electricity, not handling AI workloads. Even if you have a high-end unit with 32GB of RAM or more, you're looking at only a few tokens per second. You'd find yourself waiting on a machine that generates text slower than you can type.
If you already have a NAS, there is certainly no harm in using it for AI—just don't spend money on one with that purpose in mind.
What matters most for AI
NAS hardware is optimized for things like power consumption and number of drive bays—neither of which really help with AI workloads. You shouldn't pay too much attention to NPU marketing at this point either. Most of the important AI tools, like Ollama, llama.cpp, and LM Studio, don't currently send AI workloads to the NPU anyway.
Note: It may be possible for future software to simultaneously take advantage of an integrated GPU and an NPU.
Memory bandwidth is also a significant limitation, which is why the unified memory approach used by modern Macs gives them an edge over other mini PCs that use RAM sticks.
A Mac Mini is a surprisingly good option
Great performance for all tasks
Goran Damnjanovic / How-To Geek
If you need something compact and power efficient to run small to medium-sized AI models, a Mac Mini is a pretty good option. The Mac Mini can be equipped with up to 64GB of unified memory for $2000, which allows it to easily run models with 30 billion (30B) parameters. You could probably squeeze in some quantized models with up to 70B parameters, but you're going to run into some performance bottlenecks.
Since it works out of the box with LM Studio and Ollama, you can easily switch between using it as your day-to-day PC, home server, or dedicated local AI box.
The Mac Mini has one limit
The only real drawback of the Mac Mini is the memory limit. With a 64GB max on the M4 Pro, you won't be able to run models with more than about 70 billion parameters. If you need to run larger models, you have to jump up to a Mac Studio, which is significantly more expensive.
A mini PC with an AMD AI chip is great too
More memory than you know what to do with
Bill Loguidice / How-To Geek
However capable Mac Minis are, they're not the only mini PC that can serve as an at-home AI server. If you're looking for a reasonable option that costs less than the Mac Mini, start with mini PCs that have an AMD Ryzen AI 9 HX 370.
As one example, MINISFORUM produces a mini PC with a Ryzen AI 9 HX 370 and 32GB, 64GB, or 96GB of RAM that starts around $1,100. Multiple manufacturers produce models with up to 96GB of RAM, though those usually cost around $2,000.
If you need even more power, I'd suggest looking at mini PCs with the Ryzen AI Max+ 395 or the AI Max+ 388. They support up to 128GB of unified memory, and you could allocate up to 96GB as VRAM . This allows you to comfortably run 70B+ models that would be impossible to fit on a Mac Mini.
Unfortunately, mini PCs with the AI Max+ 395 processor and 128GB of RAM are pretty pricey, though they tend to be a bit less expensive than the Mac Studio with an equivalent amount of RAM. When the newer AI Max+ 388 becomes widely available, it'll likely be a bit cheaper and may be a good option if you're looking to save some money on an AI PC.
ROCm provides an edge
AMD's ROCm has matured quite a bit, and it now runs llama.cpp, Ollama, and LM Studio without too many issues. It's a reasonable choice if you prefer Linux for headless servers.
AMD claims that you could see up to 12.2x faster time-to-first-token than Intel Lunar Lake on some models, the speed is potentially there. As an added perk, if you get a mini PC with OCuLink ports, you have the ability to add a separate GPU later on if you want the extra performance.
You shouldn't skip on the NAS
Just because a NAS won't be great for running AI models doesn't mean they're not worth anything. They're great for what they are: convenient storage devices. If you don't already have an in-home backup solution of some kind, I'd generally recommend buying one.
If you want a device that is both a NAS and an AI server, the most cost-effective option is to build your own from refurbished or secondhand parts . You can add as much RAM as you want, pick a GPU with enough VRAM to run the models you want, and then continually add new drives as your storage needs grow. A home server like that would also be capable of self-hosting almost any other service you want, giving you the option to cut subscriptions in favor of more privacy-friendly alternatives.
