How much does it cost to build on top of the OpenAI API?

After my first OSS project hit 10k users in 24 hours, the second thing I looked at — right after the analytics — was the OpenAI dashboard. Spoiler: it wasn't free.

What I learned

• Tokens are not words. A "cheap" prompt can balloon once you include system instructions, few-shot examples, and user history.
• Streaming saves nothing on cost. It's only a UX optimisation. You still pay for every token.
• Caching is underrated. For anything deterministic (templated prompts, repeat queries), cache the response. Even a dumb in-memory LRU cut my bill meaningfully.
• Free tier users are the most expensive. The people who never pay you are the ones hammering the endpoint hardest.

How I covered cost

Three lanes, pick whichever fits the product:

1. Usage caps — free tier gets N generations/day, hard stop.
2. BYOK (bring your own key) — power users plug in their own OpenAI key; you pay $0.
3. One-time payment — small flat fee, no subscriptions, no churn math.

I ended up mixing all three across different projects. Subscriptions were the worst fit — too much overhead for tools people use once.