KV Cache Calculator

Wed, 20 May 2026 00:00:00 +0000

This calculator estimates how much GPU memory you need to serve a large language model. It accounts for model weights, the KV cache (the part most people under-budget), activation/runtime overhead, and the nominal memory capacity of common current and announced accelerators. GPU presets compare memory only; they do not validate framework, precision, interconnect, or sharding support, and Preview capacities can change.

It’s intended to pair with the KV Cache blog post . All math runs in your browser — nothing is sent to a server. Paste a Hugging Face model id (e.g. Qwen/Qwen2.5-7B-Instruct) to auto-fill the architecture from the model’s config.json — only metadata is fetched, never weights — or pick a curated preset, then tweak any field in Advanced options to override.

Calculator on Melchi

KV Cache Calculator