The Innovation Attorney Podcast
The Innovation Attorney Podcast · Oct 15, 2026 · 33:08
Listen in the Podli app 🎧
Follow your favourite podcasts, listen offline and in the car with CarPlay and Android Auto, and always pick up where you left off. Free to try.
A startup on OpenAI’s Agents API hands over the cost lever that decides its margin on GPT 6.1 Sol, priced at $2 input and $10 output per million tokens.
That lever is the cache hit rate. A 30 turn agent run costs $0.77 when every earlier token is read from cache at $0.10 per million, and $4.83 when none is, a spread of 6.25 times. Keep the loop in your own code, rent the model and the sandbox through adapters, and use the hosted service for internal work until it reaches general availability.
OpenAI announced Sol at its September 29, 2026 developer event, along with Dots, always on agents that carry their own cloud computer, and a widened Agents API that has been in public beta since September 10. The Agents API runs the agent loop on OpenAI’s servers: session state, context compaction, tool search, and subagents. It charges no orchestration fee, and you pay for tokens, tools, and container minutes.
Why does the cache hit rate decide gross margin?
Every turn of an agent loop resends the entire conversation, so input tokens grow with the square of the number of turns. A run that adds 5,000 tokens per turn over 30 turns processes 2.325 million input tokens. Double the turns to 60 and the total climbs to 9.15 million, nearly four times as much.
Caching is what tames that curve. Sol lists cached input at $0.10 per million tokens against $2 for fresh input, plus a cache write rate of $2.50. The discount applies only when a prompt starts with exactly the tokens of an earlier prompt. A summary that replaces old turns, or a tool definition loaded midstream, breaks the match and bills everything after it at the full rate. Prompts above 272,000 tokens also carry a surcharge of twice the input rate, so builders compact context, and compaction edits the prefix. Whoever runs compaction decides the margin.
The Agents API runs compaction and tool search inside OpenAI’s service, and OpenAI has not published how they interact with caching. They may do it well, and nobody outside can tell. My own model of a 30 turn run with a 50 percent prefix miss rate costs $2.84. Charge $5 per task and add sandbox minutes, and the owned loop earns an 83 percent gross margin while the hosted case earns 38 percent. Those are my assumptions, and the only way to replace them with facts is to meter your own workload.
What does hosted agent infrastructure cost beyond tokens?
Container time bills by the minute with a five minute minimum, and the rate is $0.006 per minute for a 4 GB sandbox. An agent that holds that sandbox open all day costs $8.64, which is $259.20 over a 30 day month before a single token.
Sell an always on agent at $99 per month and compute alone loses money. Sell at $200 and you compete with the Pro plan, whose first Dot costs the customer nothing extra. The startups that survive that comparison build agents that sleep: event driven wake ups, checkpoints, and billing for active minutes only. That logic lives in the orchestrator, which is the part you would be renting.
How much is the option to switch models worth?
The option is worth the difference between the price cuts you capture and the ones your vendor chooses to pass along. Inside one OpenAI generation, Astra at $10 input and $50 output became Sol at $2 and $10, an 80 percent cut. Opus 5.5 is reported at $4 and $20 after Opus 5 at $5 and $25.
The quality picture is mixed. An independent index scores Sol at 52 and Opus 5.5 at 58, yet running the index costs $1,082 for Sol and $8,708 for Opus 5.5, because Opus 5.5 spent 119,000 output tokens per task against 38,000. Neither wins on every task, which is the argument for routing each task to the cheapest model that clears your quality bar. The Agents API runs only OpenAI models. A router in your own code runs anything.
OpenAI also shut down its Assistants API on August 26, 2026, and everyone who wired product logic into those objects rewrote it on the vendor’s calendar. In thirty years of drafting technology agreements, the clause I see negotiated last and regretted first is termination assistance. Nobody asks for a session export format on the day of signing, and everybody asks for it on the day they want to leave. I could find no published export format for Agents API sessions.
What should a startup use the Agents API for today?
Use it for internal and bounded jobs. The service is in public beta with breaking changes possible, supports data residency only in the United States, and does not support zero data retention, even with a self hosted sandbox. A hospital or a European bank cannot be your customer on that footing. The Sol model API itself lists United States and European Union residency, so an owned loop can serve those customers with the same model today.
Sign in with ChatGPT shifts inference cost to the user’s plan, which looks like free margin. The terms dated September 29, 2026 require that users can use their plan without paying you, restrict use to the connected application, and let OpenAI suspend access for security or abuse risk. Only Plus and Pro subscribers qualify, and the vendor already cut Pro allowances once. Build developer tools on it if the value lives outside the tokens. Do not build a metered product on it.
What belongs in the loop you own?
Five things belong in your code: the cache policy, the cost meter, the permission layer, the state format, and the model router. Everything else can be rented, and the Codex harness is open source. Tools written to the Model Context Protocol, which joined the Linux Foundation’s Agentic AI Foundation on December 10, 2025, work against Sol and against its competitors.
A team of two or three engineers can build a competent loop. The harder work is evaluation, knowing whether a compaction change cut task success, and a hosted harness hides what changed.
What should you check at general availability?
Check five things:
Episodes: The Innovation Attorney Podcast