https://melchi.me/tools/Recent content in Tools on MelchiHugoenWed, 20 May 2026 00:00:00 +0000https://melchi.me/tools/kv-cache-calculator/Wed, 20 May 2026 00:00:00 +0000https://melchi.me/tools/kv-cache-calculator/<p>This calculator estimates how much GPU memory you need to serve a large language model. It accounts for <strong>model weights</strong>, the <strong>KV cache</strong> (the part most people under-budget), activation/runtime overhead, and GPU memory currently available on common accelerators.</p> <p>Pair it with the <a href="https://melchi.me/posts/kv-cache/" rel="">KV Cache blog post</a> . Paste a Hugging Face model id (e.g. <code>Qwen/Qwen2.5-7B-Instruct</code>) to auto-fill the architecture, or pick a curated preset, then tweak any field in <strong>Advanced options</strong> to override.</p>