LLM Inference Performance Estimator
Estimates the speed and memory use of a Hugging Face model on your own hardware: NVIDIA GeForce, RTX workstation and data center GPUs, DGX Spark and DGX Station, AMD Radeon, Instinct and Ryzen AI Max, and Apple Silicon Macs, on one GPU or several joined by NVLink, Infinity Fabric or PCIe.
Open tool →