What hardware do you need to run a 70B model?
Last updated
To run a 70B model at 4-bit you need roughly 40 GB of fast memory (approximate) plus room for context. That means 64 GB or more of unified memory, as in a DGX Spark or Mac Studio, or several GPUs. One 24 GB GPU can't.
Olares One → Local LLM hardware → VRAM · quantization · tokens per second → The right machine for your model size
What hardware do you need to run a 70B model?
The hardware you need to run a 70B model starts with about 40 GB of fast memory at 4-bit, by our approximate rule, plus a few GB for context. At 8-bit the model needs roughly twice that. No single consumer GPU in our data has that much VRAM, since NVIDIA's RTX 5090 desktop card tops out at 32 GB.
Dense and mixture-of-experts models differ here. A dense 70B model uses all of its weights for every token. A mixture-of-experts model routes each token through a few experts and runs faster on the same hardware, though all its experts still sit in memory.
Which machines can hold a 70B model?
Machines with 64 GB or more of unified memory, or towers with several GPUs, can hold a 70B model at 4-bit. In our data that means the DGX Spark, the ASUS Ascent GX10, the Mac Studio in its larger configurations, the 64 GB Mac mini and the Ryzen AI Max+ 395 mini PCs.
Prices below were read on 12 Nov 2026.
- NVIDIA DGX Spark: 128 GB of unified memory at 273 GB/s, CUDA, 6,950 USD.
- ASUS Ascent GX10: the same GB10 chip with 128 GB, 5,999 USD, out of stock when read.
- Mac Studio M5 Ultra: 96 GB base at 1.2 TB/s, from 5,499 USD. Apple offers up to 512 GB on the chip version that starts at 6,799 USD.
- Mac Studio M5 Max: configurable to 128 GB on its 40-core GPU chip. We couldn't record that configuration's price.
- Ryzen AI Max+ 395 mini PCs: 128 GB shared, 3,449 to 4,199 USD. The GPU can use only part of it, and makers don't state how much.
- Mac mini M5 Pro: 64 GB, 3,199 USD. It holds a 4-bit 70B with little left for the system and context.
How fast does a 70B model run?
A dense 70B model runs slower than a 32B model on the same machine, because every token reads all of its weights. Speed scales with memory bandwidth, which runs from 273 GB/s on the DGX Spark, as NVIDIA lists it, to 1.2 TB/s on the Mac Studio M5 Ultra, as Apple lists it.
We found no 70B speed test for these machines that meets our sourcing rules, so we quote none. Ask sellers for a test at your context length. Tokens per second explained shows why bandwidth sets the ceiling.
Can Olares One run a 70B model?
Olares One can't hold a dense 70B model in its 24 GB of VRAM, and we wouldn't buy it for that job. Engines can push the extra layers into its 96 GB of system RAM, but a dense model then slows sharply. If 70B is a must, buy unified memory. If a 32B model covers your work, buy Olares One.
Olares' only published spill result is for gpt-oss-120b, a mixture-of-experts model, at about 36 tokens per second in Olares' own test. Olares' FAQ says two or more Olares devices can form a cluster that splits a model with vLLM pipeline parallelism. That allows larger models without speeding up a single task, over Ethernet limited to 2.5 Gbps, and Olares has published no 70B result for it. Compare the options in unified memory or VRAM.
Questions & answers
Can I run a 70B model on 24 GB of VRAM?
Only by spilling layers into system RAM, which makes a dense 70B model slow. At 4-bit it needs roughly 40 GB by our approximate rule.
What is the cheapest machine in your data that holds a 70B model?
The Mac mini M5 Pro with 64 GB, at 3,199 USD, holds a 4-bit 70B with little room to spare. Among 128 GB machines, the Framework Desktop DIY Edition starts at 3,449 USD and ships without a drive or OS.
Sources for this entry
- Olares One FAQ (Olares docs) (Company-published)
- NVIDIA DGX Spark: maker's page (Maker-published)
- ASUS Ascent GX10: maker's page (Maker-published)
- Apple Mac Studio (M5 Ultra): maker's page (Maker-published)
- Apple Mac mini (M5 Pro, 64GB): maker's page (Maker-published)
- Framework Desktop (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- NVIDIA GeForce RTX 5090 (desktop card): maker's page (Maker-published)
Continue this reading path
How the related websites connect
Cite this entry
Best Local LLM Hardware. “What hardware do you need to run a 70B model?.” Content reviewed 13 November 2026.
https://bestlocalllmhardware.com/hardware-for-70b-model/