the tokens survived the deploy, the reasoning didn't
Week ending June 19, 2026.
This was a plumbing week, and I mean that as a compliment. Nobody dropped a model that ends history. Instead people wrote down the unglamorous stuff: how an inference call survives a deploy, what a GPU actually costs per user, whether cranking reasoning effort makes an agent smarter, and how an agent quietly narrates your secrets to anyone watching its outbound traffic.
The through-line is honesty. Half the week was about building durable infrastructure under agents. The other half was about measuring what those agents actually do, rather than what their marketing slide claims. Both halves share an allergy to vibes. Good. The vibes have been writing checks the benchmarks can't cash for a while now.
the token you already paid for
Sunil Pai's never waste a token nails a cost bug that's easy to miss: you're billed for output tokens the moment they're generated, but the streaming HTTP request lives inside your process. Deploy, eviction, OOM — the connection dies, and your only recovery is to call the model again and pay twice. In an agentic loop with multiple tool calls per turn, every interruption throws away the whole turn.
The fix is to move the provider connection out of the thing that crashes: a separate, never-redeployed buffer that drains the stream into SQLite while handing you a tail. The neat part is that resumable streaming (browser reconnects) and crash recovery are the same durable log — the only question is whether a live producer is still attached. The catch: today only OpenAI's background mode does this natively, Anthropic and Gemini make you re-prompt and re-pay, and the clean version is "coming soon" to Cloudflare's gateway. Still, this is exactly where durable-agent thinking should live.
the napkin doesn't lie
If you want to know whether any of this matters at scale, Inference cost at scale with napkin math walks the whole chain from matmul cost to dollars per user. The load-bearing insight: a B200 crunches bytes ~562x faster than it loads them, so without a KV-cache you're wasting memory bandwidth, and with one you want batch sizes around 300 to saturate the chip.
Then reality bites — VRAM. A 32B model plus a 26GB-per-user KV-cache means six concurrent full-context users, which sounds absurd until PagedAttention and idle duty cycles push you to a few hundred. The payoff is a rented cost of roughly a penny per user-hour. Keep this napkin taped to your monitor before anyone quotes you a trillion-dollar buildout.
an agent is a directory
Vercel's eve treats an agent as a folder of typed files: tools, skills, subagents, channels, schedules. Durable sessions, per-agent sandboxes, human-in-the-loop approvals gated on things like "this query scans more than 50GB," scoped MCP connections, and OpenTelemetry spans per turn. This is architecture, not a prompt cosplaying as one. The skepticism to hold: batteries-included runtimes are lovely until the batteries are proprietary. But the shape — typed contracts, permission boundaries, built-in evals — is the right shape.
thinking harder, scoring worse
Parsia's Brain the Size of a Planet is the receipts-first study I keep wishing more people ran: 26 model/effort/context combinations triaging two real CVEs, ~$9,200 in tokens burned. The headlines that matter — higher reasoning effort is usually but not always better, gpt-5.5-med beat high and xhigh, and function-level analysis crushed whole-file (91% vs 1.7% on openbsd-sack). Models handed the entire file just gave up. A four-LLM triage council agreed 86% of the time, which is reassuring and slightly unsettling. The practical read: don't pay for max reasoning by reflex, and chop your inputs down to the function before you ask.
your agent is telling on you
The wildcard, and my favorite, is MosaicLeaks. A research agent mixing private docs with web search leaks secrets not in any single query but across the cumulative query log — the mosaic effect. Worse, training the agent purely for task performance made leakage climb from 34% to 51.7%, because richer queries retrieve better and reveal more. Their PA-DR method rewards how each query is constructed and drops answer/full-info leakage to 9.9% while keeping task success. The lesson is blunt: you can't prompt privacy in, you have to train it in. Telling an agent to be careful barely moved the needle.
what I'd do Monday
- Instrument token spend around deploys and evictions. If you can't see double-billed retries, you're paying for them blind.
- Run the napkin math on your own model and GPU before you trust anyone's capacity plan.
- Stop defaulting to max reasoning effort. Test med, and feed models functions, not whole files.
- Log your agents' outbound queries and look at them as an adversary would. The leak is in the aggregate, not any one line.
vocabulary inflation
Durable inference — keeping the provider stream alive independent of the process that opened it, so a crash doesn't re-bill you. The mosaic effect / mosaic leakage — private facts reconstructed from a series of individually-benign queries. And from Parsia, gifted to us whether we wanted it or not: claudvicular, tokenmaxxing, and triagemogged. Don't click the footnote. I did.