How to choose a local LLM hardware
Last updated
Choose local LLM hardware by the largest model you'll run, then by how many people use it at once. Up to about 32B and shared use, a 24 GB CUDA machine like Olares One fits. For 70B and larger, pick 128 GB of unified memory.
Olares One → Local LLM hardware → VRAM · quantization · tokens per second → The right machine for your model size
Criteria
- Memory for the model: the largest model you plan to run must fit, with room for context
- Memory bandwidth, which sets how many tokens per second a model of a given size can produce
- Speed per request with several requests at once, if a team, agents or coding tools share the machine
- CUDA support for vLLM and other NVIDIA-first tools
- Complete price, including any PC needed around a bare graphics card
- Noise, power and size, for a machine that stays on all day
Machines compared
| Machine | Price | Memory for models | Memory type | GPU | CUDA |
|---|---|---|---|---|---|
| Olares One | $4,999 · €4,599.95 | 24 GB | VRAM | RTX 5090 Mobile | yes |
| NVIDIA DGX Spark | $6,950 | 128 GB | Unified, 273 GB/s | NVIDIA GB10 | yes |
| ASUS Ascent GX10 | $5,999 | 128 GB | Unified | NVIDIA GB10 | yes |
| Apple Mac Studio (M5 Ultra) | $5,499 · €6,599 | 96 GB | Unified, 1.2 TB/s | M5 Ultra, 64-core GPU | no |
| Apple Mac Studio (M5 Max) | $2,499 · €2,999 | 36 GB | Unified, 460 GB/s | M5 Max, 32-core GPU | no |
| Apple Mac mini (M5 Pro, 64GB) | $3,199 · €3,649 | 64 GB | Unified, 307 GB/s | M5 Pro, 20-core GPU | no |
| Minisforum MS-S1 Max | $3,959 | 128 GB | Shared (unified) | Radeon 8060S | no |
| GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB) | $3,499.99 | 128 GB | Shared (unified) | Radeon 8060S | no |
| Framework Desktop (Ryzen AI Max+ 395, 128GB) | $3,449 | 128 GB | Shared (unified) | Radeon 8060S | no |
| Beelink GTR9 Pro (Ryzen AI Max+ 395, 128GB) | $4,199 | 128 GB | Shared (unified) | Radeon 8060S | no |
| Used RTX 3090 tower build | $3,800 | 24 GB | GDDR6X VRAM, 936 GB/s | RTX 3090 (used) | yes |
| NVIDIA GeForce RTX 5090 (desktop card) | $1,999 | 32 GB | GDDR7 VRAM | RTX 5090 desktop | yes |
Prices and specs read on the makers' pages on 12-13 November 2026; each name links to its source.
How do you choose local LLM hardware?
Choose local LLM hardware from the largest model you'll run, then count the users. At 4-bit, a 32B model needs roughly 19-20 GB and a 70B model roughly 40 GB, by our approximate rule. Up to 32B, a 24 GB or 32 GB GPU is the fast, simple option. Above that, you need unified memory or several GPUs.
One person chatting needs a single fast stream. A team, or a coding agent that fires many calls, needs a batching engine and spare memory for each request's context. See hardware for an inference server.
Which machine fits which need?
Five machines in our data cover the common needs, from a used RTX 3090 tower at about 3,800 USD (one build guide's estimate) to a DGX Spark at 6,950 USD. The split is simple. Up to 32B, pick a CUDA GPU with 24 GB or 32 GB of VRAM. For 70B to 120B, pick 128 GB of unified memory.
| If you need | Look at |
|---|---|
| Up to 32B, shared by several people, CUDA | Olares One (24 GB VRAM, 4,999 USD) |
| Up to 32B on a budget, and you can build a PC | Used RTX 3090 tower (24 GB, about 3,800 USD guide estimate) |
| Up to 32B with more context headroom | RTX 5090 desktop card (32 GB, 1,999 USD card only) |
| 70B to 120B, CUDA | DGX Spark (128 GB, 6,950 USD) or ASUS Ascent GX10 (128 GB, 5,999 USD) |
| Very large models in one box, no CUDA needed | Mac Studio M5 Ultra (96 GB base, configurable up to 512 GB) |
Questions & answers
Should I buy for the model I run today or a bigger one later?
Buy for the largest model you have a concrete use for. You can't add memory to a GPU or a unified-memory machine after purchase.
Does more system RAM help an LLM on a GPU machine?
Only as overflow. A model that spills from VRAM into system RAM still runs, but slower. In Olares' own test, gpt-oss-120b ran at about 36 tokens per second while spilling from 24 GB of VRAM into system memory.
Sources for this entry
- Olares One specifications (Olares docs) (Company-published)
- NVIDIA DGX Spark: maker's page (Maker-published)
- ASUS Ascent GX10: maker's page (Maker-published)
- Apple Mac Studio (M5 Ultra): maker's page (Maker-published)
- Apple Mac mini (M5 Pro, 64GB): maker's page (Maker-published)
- Minisforum MS-S1 Max: maker's page (Maker-published)
- GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- Framework Desktop (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- Beelink GTR9 Pro (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- Used RTX 3090 tower build: maker's page (Maker-published)
- NVIDIA GeForce RTX 5090 (desktop card): maker's page (Maker-published)
Continue this reading path
How the related websites connect
Cite this entry
Best Local LLM Hardware. “How to choose a local LLM hardware.” Content reviewed 13 November 2026.
https://bestlocalllmhardware.com/choosing/