10.6.3 · Skill in Operational economics
AI inference costs
What model calls cost, and how to keep them in check.
AI features have per-request costs that grow with usage. Model choice, prompt length, caching and batching change them a lot. They can turn a profitable feature into a loss, so they need tracking per feature.
Resources
2- docsGo deeperPrompt caching ↗Anthropic docs · 20 min
Reusing the stable part of a prompt to cut the cost and latency of model calls.
- docsGo deeperBatch processing ↗Anthropic docs · 15 min
Running model work that is not urgent in batches at a lower price.
Tools
none yetTools for this topic are being researched.