GPT-5 Review: 90 Days in Production
GPT-5 reviewed after 90 days in production with 30M tokens of mixed workload. Latency, cost, accuracy, and production gotchas analyzed. Per-axis verdict.
GPT-5 reviewed after 90 days in production at WikiWalls (~30M tokens of mixed workload). Per-axis verdict on cost, latency, accuracy, and production reliability.
- Strongest axes: Cost (cheapest credible frontier model) + latency + context window (400K)
- Weakest axes: Code edit accuracy (8 points behind Claude) + tool-use reliability
- Production ranking (out of 10): 8.7. Strong all-rounder, not best on any single high-stakes axis
- Best: High-volume classification, summarization, latency-sensitive chat
- The verdict: GPT-5 is the cost-effective production default for non-code workloads. Route code and agents to Claude.
OpenAI shipped GPT-5; we adopted it in production early 2026 alongside Claude Sonnet 4.5. After 90 days and ~30M tokens of real workload (chat, classification, summarization, RAG), here is the honest verdict on what GPT-5 does well, where Claude pulls ahead, and how to deploy GPT-5 in production.
01At a glance: what we tested
| Axis | GPT-5 score | Sonnet 4.5 score | Notes |
|---|---|---|---|
| Cost per 1M input | $1.25 | $3.00 | GPT-5 58% cheaper |
| Cost per 1M output | $10.00 | $15.00 | GPT-5 33% cheaper |
| Latency P50 (chat) | 410ms | 780ms | GPT-5 47% faster |
| Context window standard | 400K | 200K (1M enterprise) | GPT-5 wider |
| Code edit accuracy (50-PR) | 79% | 87% | Claude leads by 8 points |
| Tool-use reliability (600 calls) | 96.8% | 99.1% | Claude leads by 2.3 points |
| Classification accuracy | 94% | 94% | Tied at the top |
| Summarization quality | Strong | Strong | Tied |
| Streaming reliability | 99.91% | 99.96% | Claude marginally better |
02What GPT-5 does well
Cost, latency, and context window are the strongest axes. GPT-5 is the production default for non-code workloads where cost-per-correct-output matters.
Buy if: your workload is high-volume classification, summarization, or latency-sensitive chat. Skip if: your workload is code or agentic.
On classification tasks (categorize support tickets, label leads, tag content), GPT-5 matched Claude on accuracy at half the input cost. On summarization across long inputs, GPT-5’s 400K context window is the standard tier (vs Claude’s 200K). Meaningfully cheaper for RAG over large corpora. P50 latency at 410ms makes GPT-5 the right pick for real-time UI chat where sub-500ms P50 is the target. The honest framing: GPT-5 is the cost-effective all-rounder. For workloads where accuracy ceiling matters less than cost-and-latency floor, GPT-5 wins.
03Where GPT-5 falls short
Code edit accuracy lags Claude by 8 percentage points; tool-use reliability lags by 2.3 points. Route code and agent workloads to Claude.
Buy if: not applicable. Skip if: not applicable.
On the 50-PR code-edit benchmark, GPT-5 produced edits accepted at 79%, 8 percentage points behind Sonnet 4.5. Tool-use reliability returned 96.8% valid tool calls (vs Claude 99.1%); the 2.3-point gap matters in agent loops where each call carries decision weight. Long-context coherence past 80K tokens degrades faster than Claude on multi-document reasoning. The pattern in production: GPT-5 is the wrong choice when accuracy compounds (one wrong output cascades). For independent classification calls, GPT-5 is fine.
04Production patterns we adopted
Three patterns to make GPT-5 work in production: workload-route to Claude for code, prompt-cache long inputs, fallback chain for reliability.
Buy if: not applicable. Skip if: not applicable.
In our 90-day production run, GPT-5 became the default for ~60% of token volume (classification, summarization, latency-critical chat). The other 40% (code, agents, long-context reasoning) went to Claude. Prompt caching on GPT-5’s 400K context cut RAG-cost 50%. Fallback chain (try GPT-5 first; if 429 or 5xx, fall back to GPT-5 mini, then to GPT-4 Turbo) kept effective uptime at 99.99% despite raw 99.91% reliability. The combination of routing + caching + fallback is the production-grade GPT-5 pattern. Single-provider, no-routing deployments leave money and accuracy on the table.
05Which option should you pick?
Pick by your situation
- Your workload is high-volume classification or summarization? → GPT-5 (cost wins)
- Your workload is real-time UI chat with strict latency? → GPT-5 (latency wins)
- Your workload is code or agentic? → Claude Sonnet 4.5 (accuracy wins)
- You’re building first prototype with no scale? → Either. Pick by familiarity
- You need 1M+ context window? → Gemini 2.5 Pro (2M context); GPT-5 is 400K standard
- Cost is the absolute binding constraint? → DeepSeek V3 or Gemini Flash (cheaper still)
06FAQ
Is GPT-5 actually a meaningful upgrade over GPT-4o?
Yes. Latency is 50% lower, cost is 30% lower, context window is 4x wider, classification accuracy is 3-5 points higher. Code accuracy is similar (within 1-2 points). For greenfield deployment, default to GPT-5 over GPT-4o.
What about o1 and reasoning models?
o1 (and o3, o4 successors) trade latency and cost for stronger reasoning on complex tasks (math, science, multi-step planning). For ~5% of workloads where reasoning depth justifies 5-10x cost and 10-30s latency, reasoning models win. For other 95% of production traffic, GPT-5 / Sonnet are the right defaults.
Is OpenAI’s reliability a concern?
Less than it was. 90-day production uptime hit 99.91% (vs 99.96% Claude. Narrow gap). Status page is honest. Build retry logic anyway. Both providers had 1-2 brief incidents in our test window.
Should I use the OpenAI Assistants API or raw chat completions?
Raw chat completions for production. Assistants API has lower production-readiness, more limitations, and less documentation. The chat completions endpoint is mature, well-documented, and used by every major LangChain / LlamaIndex / framework.
How does GPT-5 enterprise tier compare?
Enterprise adds: Zero Data Retention, custom SOC 2 / HIPAA contracts, dedicated capacity, custom rate limits, 24/7 support. Pricing is custom (typically $50K-500K/year minimum). For enterprises with compliance mandates, the enterprise tier is necessary. For startups and indie operators, the standard API is fine.
07WikiWalls verdict
WikiWalls verdict. GPT-5 is the cost-effective production default for classification, summarization, and latency-sensitive chat. Route code and agents to Claude Sonnet 4.5. Multi-provider routing typically saves 30-50% all-in cost vs single-provider commitment.
Last reviewed by WikiWalls editorial with current pricing, first-party benchmark data, and tested production reliability. Recommendations are editorially independent.
Last reviewed by WikiWalls editorial. Recommendations are editorially independent. Methodology: /test-methodology/. Editorial standards: /editorial-standards/.
