Estimating LLM hosting costs
How to compare per-token pricing with GPU hours, and the hidden costs that catch teams out.
Pricing pages are hard to compare because providers bill in different units. Converting everything to cost per million tokens at your real traffic makes the choice much clearer.
Per-token pricing
Managed APIs charge separately for input and output tokens. Estimate your average prompt and response length, multiply by expected requests, and you have a monthly figure.
GPU-hour pricing
For dedicated or self-hosted models, measure how many tokens per second one GPU serves for your workload, then divide the hourly price by tokens per hour.
Costs people forget
- Idle time on dedicated instances
- Cold starts and the extra capacity you keep warm to avoid them
- Engineering time to maintain a serving stack
- Egress and storage for model weights
Rule of thumb: start on a managed API, and revisit when your monthly bill is large enough to pay for the engineering time of self-hosting.