LLM Inference Performance Estimator

Pick a model to start.

Hugging Face modelSearch, or type a model id such as openai/gpt-oss-20b and press Enter.
Hardware
Machines: Identical copies of this setup on separate machines behind a load balancer. They share nothing, so each serves its own share of the users. 1
GPUs 1
Workload
Target
tokens/s ms
Parallelism
Load a model first
Advanced
Calibration defaults

Pick a model to see how it runs.

Memory per GPU GB

No model loaded yet.

Speed vs users

Table
Estimates, not benchmarks.