# VRAM Calculator

> Free, in-browser calculator for the GPU memory an open large language model needs to run or fine-tune. https://vram-calculator.skillsafe.ai/

## What it does

- Pick one of 25 bundled models (Llama 3.x, Mistral, Mixtral, Qwen2.5, Qwen3, Gemma 2 and 3, Phi-3, Phi-4, gpt-oss, DeepSeek V3/R1) or paste a Hugging Face config.json.
- Choose inference, LoRA, QLoRA or a full fine-tune; the weight precision (FP32, BF16, FP16, FP8, INT8, GPTQ/AWQ INT4, NF4, MXFP4, GGUF Q8_0 down to IQ2_XS); the KV cache precision; context length and batch size; the optimizer and gradient checkpointing.
- Get a breakdown of weights, KV cache, activations, logits, gradients, optimizer states and overhead, each with the formula and your numbers substituted.
- See whether it fits on 26 GPUs (RTX 30/40/50, T4, L4, A10, A100, H100, H200, B200, MI300X, Apple Silicon), how many of each you need, the longest context and largest batch that fit, a llama.cpp layer-offload count and a memory-bandwidth decode-speed ceiling.
- Share the setup as a link, copy a Markdown table or download a CSV. Nothing is sent to a server.

## Formulas

- Weights = parameters x bits per weight / 8.
- KV cache = 2 x KV heads x head dim x layers x context x batch x bytes per value. Grouped-query attention uses KV heads. Sliding-window layers cache only their window. DeepSeek multi-head latent attention caches 512 + 64 values per token per layer.
- Full fine-tune with mixed-precision AdamW = 16 bytes per parameter (2 weights, 2 gradients, 4 FP32 master, 8 moments) plus activations.
- LoRA trainable parameters = rank x (input width + output width) for every adapted matrix. Adapters, their gradients and AdamW moments cost 16 bytes per trainable parameter.
- QLoRA stores linear layers in NF4 (about 4.127 bits with double quantisation) and keeps embeddings in BF16.
- Training activations per layer per token, 16-bit: 8 x hidden + 4 x query width + 4 x KV width + 6 x MLP width (Korthikanti et al. 2022, adapted to grouped-query attention and gated MLPs). Gradient checkpointing keeps only each layer's input.
- Decode ceiling = memory bandwidth / (weights read per token + KV cache).

## Worked numbers

- Llama 3.1 8B, BF16, 8K context, batch 1: weights 14.96 GiB, KV cache 1 GiB.
- Llama 3.1 8B KV cache at 128K tokens in FP16: 16 GiB. Llama 3.1 70B: 40 GiB.
- Llama 3.1 8B LoRA rank 16 on all linear layers: 41,943,040 trainable parameters.