LLM VRAM calculator: will this model run on my GPU or Mac?

VRAM is the memory on a graphics card, and a local language model only runs at full speed if its weights, its context cache and the runtime overhead all fit in it. Pick a model, quantisation and context length below to see whether it fits your GPU or Mac, roughly how fast it will run, and the cheapest UK hardware that works.

Figures last checked: 11 October 2026. The calculator runs in your browser and sends nothing anywhere.

What is VRAM and why does it decide which models you can run?

VRAM (video random access memory) is the fast memory soldered next to a graphics processor. When a language model generates a token, the processor has to read every active weight in the model from memory. If all of those weights sit in VRAM, a modern GPU reads them hundreds of times a second. If some of them do not fit, runtimes such as llama.cpp and Ollama keep the overflow in ordinary system RAM and process those layers on the CPU, which is far slower because system RAM has a fraction of the bandwidth.

Apple Silicon Macs and AMD's Ryzen AI Max (Strix Halo) machines use unified memory instead: one pool shared by the CPU and GPU. That lets a Mac load much larger models than a consumer graphics card, but macOS only lets the GPU use part of the pool by default (see the limitations below), and the bandwidth is lower than a high-end desktop GPU, so generation is slower.

So the question "will it run?" has two parts. Capacity decides whether the model loads at all. Bandwidth decides how quickly it answers once loaded. The calculator estimates both.

How is the memory a model needs calculated?

Three things have to fit at once, and the calculator adds them up in this order.

1. Weights. The number of parameters multiplied by the bits stored per parameter, divided by eight to get bytes. A model with 8 billion parameters at 16 bits needs 16 GB; at about 4.85 bits (Q4_K_M) it needs about 4.9 GB.
2. Context cache (KV cache). For every token in the conversation the model keeps a key and a value vector in every layer. The size is 2 x layers x KV heads x head dimension x context length x bytes per element. At the default FP16 cache that is 2 bytes per element; the Q8 cache option halves it. Models with grouped-query attention (fewer KV heads than query heads) need far less cache than older designs.
3. Overhead. The GPU driver context, temporary activations and memory fragmentation. The calculator uses the larger of 1 GB or 10 percent of the weights, plus 0.5 GB.

The verdict is "fits" when the total is at or below 92 percent of the memory available, "tight" when it is between 92 and 100 percent, and "offload" when it exceeds the GPU but could spill into up to 64 GB of system RAM. Gigabytes here are decimal (1 GB = 1,000,000,000 bytes), so they look slightly larger than the GiB figures some tools print.

Speed is estimated from bandwidth. Each generated token reads the active weights once, so the ceiling is memory bandwidth divided by the bytes of active weights, and the calculator reports 60 to 70 percent of that ceiling as a realistic range. Mixture-of-experts (MoE) models only read the experts chosen for each token, which is why they can be much faster than a dense model of the same total size.

Worked examples: how much memory do three real models need?

All three use Q4_K_M (4.85 bits per weight in the calculator), an 8,192-token context, one conversation and an FP16 cache. Model shapes come from each model's published card and configuration file, linked in the sources.

Llama 3.1 8B (Meta)

Meta's model card lists an 8B model with grouped-query attention and a 128k context. Its configuration file gives 32 layers, 8 KV heads and a head dimension of 128.

• Weights: 8.03 billion x 4.85 / 8 = about 4.87 GB
• Context cache: 2 x 32 x 8 x 128 x 8,192 x 2 bytes = about 1.07 GB
• Overhead: 1 GB minimum + 0.5 GB = 1.5 GB
• Total: about 7.4 GB

That is "tight" on an 8 GB card and fits comfortably on any 12 GB or 16 GB card. Cut the context to 4,096 tokens and the total drops to about 6.9 GB, which fits an 8 GB card. Raise it to 32,768 tokens and the cache alone grows to about 4.3 GB, for a total near 10.7 GB. Context length is the setting people most often forget when a model that "should fit" runs out of memory. As a cross-check, llama.cpp's own quantisation table lists the Llama 3.1 8B Q4_K_M file at 4.58 GiB (about 4.9 GB), in line with the weights figure above.

Qwen3 32B (Alibaba Qwen)

The Qwen3 32B model card lists 32.8 billion parameters, 64 layers, 64 query heads and 8 KV heads, with a native context of 32,768 tokens (131,072 with YaRN scaling). The head dimension is 128.

• Weights: 32.8 billion x 4.85 / 8 = about 19.9 GB
• Context cache: 2 x 64 x 8 x 128 x 8,192 x 2 bytes = about 2.15 GB
• Overhead: 10 percent of the weights (1.99 GB) + 0.5 GB = about 2.5 GB
• Total: about 24.5 GB

That is just over the 24 GB on an RTX 3090 or RTX 4090, so a little of the model spills to system RAM. Halving the context or switching the cache to Q8 brings the total to about 23.5 GB, which the calculator rates "tight" on a 24 GB card: it will usually load, but with nothing to spare. A 32 GB RTX 5090 holds it with room for a longer context. On a Mac Studio with an M4 Max and 36 GB, the default GPU share leaves about 24.8 GB usable by the calculator's rule, so it fits narrowly, and the bandwidth estimate (410 GB/s) gives roughly 12 to 14 tokens a second.

gpt-oss-120b (OpenAI)

OpenAI's model card lists 117 billion total parameters with 5.1 billion active per token, and says the model fits on a single 80 GB GPU such as an Nvidia H100 or AMD MI300X. Its configuration file gives 36 layers, 8 KV heads, a head dimension of 64, 128 experts with 4 active per token, and sliding-window attention on half of the layers.

• Weights: 117 billion x 4.85 / 8 = about 70.9 GB
• Context cache: 2 x 36 x 8 x 64 x 8,192 x 2 bytes = about 0.6 GB (the real figure is lower because of the sliding-window layers)
• Overhead: 7.1 GB + 0.5 GB = about 7.6 GB
• Total: about 79 GB

OpenAI ships the expert weights in its own MXFP4 4-bit format rather than a llama.cpp K-quant, so Q4_K_M is the nearest setting in the calculator. No consumer GPU holds 79 GB, but a Mac Studio or a Strix Halo mini PC with 128 GB of unified memory does. Because only 5.1 billion parameters are read per token, the speed estimate is high for a model this size: roughly 106 to 124 tokens a second as a bandwidth ceiling estimate on an M4 Max with 546 GB/s. Treat that as an upper bound; prompt processing on long inputs will be much slower than generation.

How much usable memory do common GPUs and Macs have?

The table applies the calculator's own rules: a GPU's full VRAM is available, a Mac or Strix Halo machine gets 75 percent of its unified memory by default, and the "fits" verdict needs the total under 92 percent of that. The last column is the largest dense model from the calculator's list that fits at Q4_K_M with an 8k context.

Two GPUs add capacity but not speed for a single conversation: runtimes split the layers across the cards and each token still passes through both. The Gemma 3 models use sliding-window attention, so their real cache is smaller than the calculator shows.

What is quantisation and which level should you pick?

Models are trained and published in 16-bit formats (BF16 or FP16). Quantisation stores each weight in fewer bits, grouping weights into blocks that share a scale factor so the rounding error stays small. The "K" formats in llama.cpp (Q4_K_M, Q5_K_M and so on) also keep the most sensitive tensors at higher precision, which is why the average bits per weight is not a round number.

llama.cpp publishes both the size and the quality cost of each level. The bits per weight below are from its table for Llama 3.1 8B; the perplexity increase (lower is better, zero means no measurable loss) is from the figures built into its quantize tool for Llama 3 8B.

A common rule of thumb in the local-model community, rather than something these figures prove on their own: a larger model at Q4_K_M usually beats a smaller one at Q8, but a model pushed down to Q2_K often loses to the next size down at Q4_K_M.

Assumptions and data

Every default in the calculator comes from the table below, verified on . Model shapes are read from each model's published configuration, hardware memory and bandwidth from the makers' specification pages, and prices are indicative UK bands.

What are the calculator's limitations?

• Runtime differences. llama.cpp, Ollama, LM Studio, vLLM and MLX allocate the cache and scratch buffers differently. vLLM, for example, reserves a fixed share of GPU memory up front. Expect real use within about 15 percent of the estimate.
• macOS GPU limit. macOS caps how much unified memory the GPU may wire, reported to Metal as recommendedMaxWorkingSetSize. Community measurements put the default at roughly two thirds to three quarters of RAM depending on the machine, which is why the calculator uses 75 percent and lets you change it. On macOS Sonoma and later the cap can be raised with sudo sysctl iogpu.wired_limit_mb=<megabytes>; the change does not normally survive a restart, and leaving too little for macOS makes the system unstable.
• Sliding-window and compressed attention. Gemma 3 and gpt-oss use sliding-window attention on some layers, and DeepSeek V3 uses multi-head latent attention. The calculator flags these; the true cache is smaller than the plain formula.
• Speed is generation only. The tokens-per-second range is single-stream decoding. Processing a long prompt is compute-bound and can take many seconds before the first token appears, especially on Macs.
• Offloading is a cliff, not a slope. Once layers spill into system RAM, speed falls towards what the CPU and RAM can manage. "Offload" means it will run, not that it will be pleasant.
• Prices are bands. The UK prices behind the hardware suggestions are indicative bands checked on the date above, not live quotes.

Frequently asked questions

How much VRAM do I need to run a 70B model?

At Q4_K_M a dense 70B model such as Llama 3.3 70B needs about 43 GB for the weights and roughly 50 GB in total with an 8k context and runtime overhead. That means two 24 GB GPUs with some layers offloaded, or a Mac or Strix Halo machine with 96 GB or more of unified memory. A single 32 GB RTX 5090 cannot hold it without offloading to system RAM.

Can I run a local LLM on 8 GB of VRAM?

Yes, for models up to about 8B parameters. Llama 3.1 8B at Q4_K_M needs about 7.4 GB with an 8k context, which is tight on an 8 GB card; cutting the context to 4k brings it to about 6.9 GB. Larger models need a lower quantisation or partial offloading to system RAM, which is much slower.

Is a Mac or a GPU better for running local models?

A desktop GPU has far more memory bandwidth, so it generates text faster for any model that fits in its VRAM. A Mac with 64 GB or more of unified memory can load models that no single consumer GPU can hold, but generates more slowly because its bandwidth is lower. Pick the GPU for speed on small and mid-sized models, and the Mac for capacity.

What does quantisation do to quality?

It stores each weight in fewer bits. llama.cpp's own figures for Llama 3 8B show Q8_0 adding almost nothing to perplexity, Q4_K_M adding about 0.18, Q3_K_M about 0.66 and Q2_K about 3.5. Q4_K_M is the usual compromise; below Q3 the loss becomes easy to notice, especially on small models.

Why is my real memory use different from the estimate?

Runtimes such as llama.cpp, Ollama, vLLM and MLX allocate the context cache and working buffers in different ways, some models use sliding-window attention that shrinks the cache, and the operating system and other apps hold memory too. Treat the estimate as a planning figure within roughly 15 percent, not a guarantee.

Sources

• Meta Llama 3.1 8B Instruct model card and configuration file (mirror of the gated original)
• Qwen3 32B model card
• OpenAI gpt-oss-120b model card and configuration file
• llama.cpp quantize README (bits per weight and sizes) and quantize.cpp (perplexity figures)
• llama.cpp discussion on the Apple Silicon GPU memory limit
• Nvidia GeForce RTX 50 series specifications
• Apple Mac Studio technical specifications (UK) and Mac mini technical specifications (UK)

Related reading

• How to run a local LLM on a Mac in 2026
• Local LLM VRAM requirements in 2026, by tier
• Which local AI models fit your RAM
• Unified memory explained: why 128 GB matters for local AI
• Self-hosting an LLM versus paying for an API

Changelog

• 11 October 2026: added the formula in words, three worked examples from model cards, a usable-memory table, quantisation figures from llama.cpp, limitations and sources. Calculator logic unchanged.
• 11 October 2026: first release, 21 models, 17 hardware presets.