Tools

A small, growing set of interactive tools that pair with the blog posts.

LLM Inference Performance Estimator

Estimates the speed and memory use of a Hugging Face model on your own hardware: NVIDIA GeForce, RTX workstation and data center GPUs, DGX Spark and DGX Station, AMD Radeon, Instinct and Ryzen AI Max, and Apple Silicon Macs, on one GPU or several joined by NVLink, Infinity Fabric or PCIe.

Open tool →

KV Cache Calculator

Interactive calculator that estimates KV cache and total VRAM for serving an LLM. Pick a model preset (or custom config), batch size, sequence length, KV cache precision, and GPU class — and see whether it fits before you hit OOM.

Open tool →