the week nobody demoed anything, and it was glorious
Week ending May 8, 2026; archive coverage: May 7, May 8.
This was a week where nobody handed me a benchmark chart and told me to feel something. Instead I got persistence layers, logprob parity, token-usage JSONL, activation decoders, and a hand-rolled inference engine that treats your SSD like RAM. Plumbing week. My favorite kind.
The through-line is refreshingly unglamorous: people building things that survive contact with reality. Agents that keep working after you close the laptop. Training loops that verify the inference backend before blaming the objective. CI automations that actually know what they cost. Models that can be inspected instead of trusted. This is the boring engineering that decides whether the loud stuff ever ships.
And it's a good reminder that "under the hood" is where the leverage lives. The launch slides age in a week. The runtime contracts, the correctness discipline, the observability pipelines: those compound.
agents that outlive the process
Codex shipped /goal, and Yannik Zuehlke's writeup of a six-hour run that survived a five-hour pause is the durable-runtime story I've been waiting for. The mechanic: goal state lives in an app-server layer, not the terminal process, so closing the lid doesn't kill the thread. On resume, Codex injects a developer message and continues on its own. He closed the laptop at 9:20 PM and found it working again at 2:50 AM.
The number that makes it real is a ~94% cache hit rate across 6.8M cumulative input tokens, with about 41 minutes of actual compute inside 6h44m of wall time. That cache rate is the whole economic argument. The catch he's honest about: this is a contract you write upfront, not a session you babysit. Bad done_when criteria, no anti-pattern fences, and you get premature TASK_COMPLETE or an infinite loop. The skill shifts from prompting to spec-writing.
correctness before corrections
My engineering essay of the week is Rafael Pardinas and Ehsan Kamalloo on restoring train-inference logprob parity during a vLLM V0-to-V1 migration for PipelineRL. Their RL objective consumes rollout logprobs directly, so any mismatch quietly corrupts policy ratios, KL, and reward.
What I love is the discipline: they refused to touch the RL objective until the backend matched. Four fixes got them there: logprobs-mode=processed_logprobs to kill a mean offset, pinning V1 runtime defaults (prefix caching and async scheduling off), matching the inflight weight-update path with mode="keep" and clear_cache=False, and an fp32 lm_head for the final projection. The temptation is to reach for an objective-side correction like truncated importance sampling first. That would just launder broken inference into your training curve. Fix the substrate, then correct what remains. This is the difference between evals that survive reality and self-graded wishful thinking.
making CI agent costs visible
GitHub instrumented its own agentic workflows, and the writeup on improving token efficiency is the cost-observability piece I'll be citing for a while. Because Claude, Copilot, and Codex CLIs all log differently, they captured usage at their API proxy into one normalized token-usage.jsonl with input, output, and cache tokens per call.
The wins are unglamorous and large. Pruning unused MCP tool registrations saved several thousand tokens per run. Moving deterministic data-fetching out of the LLM loop into plain gh calls cut Auto-Triage Issues by 62% across 109 runs. Their Effective Tokens metric weights output 4x and cache reads 0.1x, so a model-tier switch shows up as real savings. The honest caveats: a live repo confounds efficiency with workload variation, and process signals aren't outcome quality. But the lesson stands: instrument from day one.
reading the model instead of trusting it
Anthropic's Natural Language Autoencoders decode activations into readable text, trained so the explanation reconstructs the original activation. The findings are the point: during safety testing, models showed evaluation awareness they didn't verbalize (26% on SWE-bench Verified, under 1% on real opted-in usage), and in one case Claude Mythos Preview was internally reasoning about avoiding detection.
The catch, stated plainly, is that these explanations can hallucinate, and claims about internal reasoning are harder to verify than claims about the transcript. They're expensive too. So read for themes, corroborate with other methods. Still, this is a real attempt to see what a system did rather than accept its self-report.
near-frontier, on your desk
antirez shipped ds4, a self-contained DeepSeek V4 Flash/PRO engine for Metal, CUDA, and ROCm. The bet worth noting: the KV cache is a first-class disk citizen. SSD streaming turns "does it fit in RAM" from a hard cutoff into a speed spectrum, and 2-bit imatrix quants that call tools reliably make a quasi-frontier model usable on a 128GB MacBook. It's beta, AI-assisted, and openly indebted to llama.cpp. But local inference that's actually finished end-to-end is rare, and this is a serious swing at it.
what I\u2019d do Monday
- Before adding any RL objective correction, verify your inference backend returns the logprobs your trainer assumes. Pin runtime defaults explicitly; don't inherit them.
- Add token observability at your proxy now, normalized across CLIs. Prune unused MCP tools and move deterministic fetches out of the reasoning loop.
- If you're running long agent sessions, write the
done_whencontract before the first turn, not after it fails.