Self-Hosted LLMs vs API: When Ollama Beats OpenAI for Real Workloads
Self-hosted LLMs (Ollama, vLLM, TGI) vs API providers compared at 100K, 1M, 10M, and 100M tokens / month with team-time costs and break-even analysis.
Self-hosted LLM economics modeled across four scale tiers (100K, 1M, 10M, 100M tokens / month) with realistic team-time costs. The break-even point is higher than most assume.
- Break-even vs API at hobby scale (< 1M tokens / month): Never (API is cheaper)
- Break-even vs API at small production (5-10M tokens / month): Borderline (API still wins on TCO)
- Break-even vs API at medium production (50-100M tokens / month): Self-hosted wins on cost; API wins on time
- Break-even vs API at large production (200M+ tokens / month): Self-hosted wins decisively if team has GPU experience
- The verdict: Self-host below 50M tokens / month is usually a hobby project. Above 100M tokens / month is genuinely cheaper if you can spare the team-time.
Self-hosting LLMs is fashionable. The reality is more nuanced than the Twitter discourse. We modeled four scale tiers (100K, 1M, 10M, 100M tokens / month) across the three leading self-hosted runtimes (Ollama for development, vLLM for production, TGI for enterprise) versus the API alternatives (Claude, GPT-5, Together-hosted Llama). Real team-time costs included. Here is when self-hosting actually wins.
01Per-axis comparison
| Scale tier | API cost / mo | Self-host cost / mo | Team-time cost / mo | Verdict |
|---|---|---|---|---|
| 100K tokens / month | $1-3 | Hardware: $0 (laptop) | $200-500 | API wins |
| 1M tokens / month | $10-30 | Hardware: $30-80 (single GPU rental) | $500-1,500 | API wins |
| 10M tokens / month | $100-300 | Hardware: $300-800 (dedicated GPU) | $1,500-3,000 | API wins on TCO |
| 50M tokens / month | $500-1,500 | Hardware: $1,500-3,000 (multi-GPU or H100) | $2,500-5,000 | Borderline |
| 100M tokens / month | $1,000-3,000 | Hardware: $2,500-5,000 (production cluster) | $3,000-6,000 | Self-host wins on cost |
| 500M tokens / month | $5,000-15,000 | Hardware: $8,000-15,000 (multi-cluster) | $5,000-12,000 | Self-host wins decisively |
02Why API wins below 50M tokens / month
Below 50M tokens / month, the team-time cost of running self-hosted LLMs exceeds the per-token savings. Buy from APIs and put your team on product work.
Buy if: not applicable. Skip if: not applicable.
Three reasons API providers win below 50M tokens / month. First, fixed costs of self-hosting (GPU rental + ops overhead) do not scale down. A single A100 40GB costs $1.10-1.80 / hour rented; that is $800-1,300 / month even idle. Second, team-time is the hidden cost. Setting up vLLM with proper batching, monitoring, autoscaling, and reliability typically eats 80-160 engineering hours, plus 20-40 hours / month ongoing. Third, API quality is higher than open-weight models on most workloads. The Llama 3.3 70B / Claude Sonnet 4.5 gap is real (15-25% lower accuracy on code, agentic, and complex reasoning). For most companies below 50M tokens / month, hosting Llama in-house costs more than buying Claude and produces lower-quality outputs.
03Why self-host wins above 100M tokens / month
Above 100M tokens / month, the per-token savings dominate. With dedicated GPU infrastructure and amortized team-time, self-hosted Llama 70B runs at $0.20-0.40 per 1M tokens versus $1.00-3.00 for hosted equivalents.
Buy if: not applicable. Skip if: not applicable.
At 100M+ tokens / month the math inverts. A 4xA100 80GB cluster costs $4,500-5,500 / month and serves Llama 3.3 70B at ~150-250 tokens / second per worker across batched requests. At 100M tokens / month average load, that is roughly $0.045 per 1M tokens of effective infrastructure cost. Compare to Together AI at $0.55 / $0.65 per 1M (input / output) for hosted Llama 70B. The gap is 12-15x. Even after team-time amortization (1 senior engineer at 30% time = $5K-7K / month) the all-in cost lands 3-5x cheaper than API hosting at 100M tokens. Past 500M tokens, self-hosting is the only economically viable option for production teams. The decision rule: own your inference once it pays for a dedicated headcount.
04Workload matters more than headline scale
Workload type changes the break-even point. Long-output generation favors APIs longer; short-classification favors self-hosting earlier.
Buy if: not applicable. Skip if: not applicable.
The break-even points above assume a balanced workload. In practice, workload type changes the math. Long-output generation (creative writing, code generation, long-form summarization) favors API providers longer because self-hosted throughput on long outputs is inefficient (single-stream limitation). Short-classification workloads (categorize tickets, score leads, tag content) favor self-hosting earlier. At 5-10M tokens / month a classification workload can break even on a single GPU with high batch size. For mixed workloads, route by type: send classification to self-hosted, send generation to API. This is the same routing pattern that works for cost optimization across API providers.
05Which option should you pick?
Pick by your situation
- Volume below 10M tokens / month? → API (no exceptions)
- Volume 10-50M tokens / month? → API on TCO; self-host only as a strategic skill investment
- Volume 50-100M tokens / month? → Borderline; self-host wins if your team has GPU experience
- Volume above 100M tokens / month? → Self-host wins; allocate dedicated headcount
- Workload is classification-heavy? → Break-even shifts left; consider self-host at 20M tokens
- Workload is generation-heavy? → Break-even shifts right; stay on API longer
- Privacy / sovereignty is the binding requirement? → Self-host at any scale (cost is the wrong axis)
06FAQ
Is Ollama production-ready?
Ollama is great for development and personal use. For production, Ollama under-uses GPU and lacks production batching. Use vLLM or TGI for production self-hosted serving. Ollama is the development environment; vLLM is the production serving layer.
What about quantization (GGUF, AWQ, GPTQ)?
Quantization (4-bit / 8-bit weights) reduces memory by 50-75% and lets smaller GPUs serve larger models. Quality cost is real (3-7 percentage points lower accuracy on code and reasoning). For non-mission-critical workloads quantization is a real cost-saver. For accuracy-sensitive work, run full precision.
How does this change with H100 / H200 / B200 hardware?
Newer Nvidia generations (H100, H200, Blackwell) push throughput up 2-4x at moderate price premium versus A100. The break-even points shift left. At H200 throughput, self-hosting can win at 30-50M tokens / month versus 100M on A100. Hardware refresh cycles favor the patient self-hoster.
What about renting GPUs from RunPod / Lambda / Vast.ai?
Cheaper than AWS / GCP GPU pricing by 30-60%. Lambda Cloud A100 at $1.10 / hour vs AWS p4d at $32.77 / hour for 8 GPUs ($4.10 / GPU equivalent). RunPod Community at $0.39 / hour for spot RTX 4090 is the budget tier (less reliable). For serious production self-hosting, dedicated bare-metal from Hetzner / Lambda is the right pick.
Should I self-host for privacy?
Privacy is a different axis than cost. If your data cannot leave your network (HIPAA, GDPR with strict residency, defense), self-host at any scale because the cost framing does not apply. For other privacy concerns, Anthropic / OpenAI Enterprise tiers offer Zero Data Retention agreements that cover most use cases without the operational cost of self-hosting.
07WikiWalls verdict
WikiWalls verdict. Self-host LLMs above 100M tokens / month or for hard privacy / sovereignty requirements. Stay on APIs below 50M tokens / month even if the Twitter discourse says otherwise. The team-time cost is the line item self-hosters underestimate. Match infrastructure decisions to where your team should be spending time.
Last reviewed by WikiWalls editorial with current pricing, first-party benchmark data, and tested production reliability. Recommendations are editorially independent.
Last reviewed by WikiWalls editorial. Recommendations are editorially independent. Methodology: /test-methodology/. Editorial standards: /editorial-standards/.
