what the system did, not what the slide said
Week ending March 13, 2026; archive coverage: March 11.
This was a week about the difference between what a system claims and what it actually does. Nobody shipped a god model. Instead a handful of people did the unglamorous work of measuring the machinery underneath the headlines, and the machinery came back with some awkward answers.
The recurring theme: the number on the slide is doing a lot of load-bearing work it was never engineered for. Leaderboard deltas, "open" labels, semantic layers, self-reported honesty, terms of service. Each one is a claim. Each one, poked, turned out to be softer than advertised. That's not cynicism talking, it's just what happens when a field moves faster than its own instrumentation.
Let me walk through the pokes.
the benchmark was measuring your VM
Anthropic went and quantified how much infrastructure noise moves agentic coding scores, and the answer is: more than the gaps people deploy on. On Terminal-Bench 2.0 the spread between the least- and most-resourced setups was six points. The mechanism is dumb and beautiful: if your Kubernetes container treats the per-task resource spec as both floor and hard ceiling, a transient memory spike OOM-kills a run that would otherwise have passed. Up to ~3x headroom you're just fixing spurious crashes. Above that, the extra RAM lets the agent brute-force with pandas and friends, which actually changes what the eval measures.
The practical consequence is blunt: a lead under three points might be a real capability gap, or it might be a bigger VM, or a luckier time of day. Static benchmarks score output. Agentic evals score a whole system, and the system leaks.
open weights, closed reality
Workshop Labs wrote the honest field report I wish more people would: open weights isn't open training. They tried to post-train Kimi-K2-Thinking on the open stack and hit a wall at every layer. Redundant compression passes on an already-quantized model. expandable_segments incantations to stop CUDA from thrashing. LoRAs that don't attach to quantized experts. A gate that asserts not self.training. Dequantized weights never getting freed.
They got it to talk like Yoda. They also got something roughly 6-9x more expensive per token than just renting a training API, and the experts still didn't train. "Open weights" ships the artifact; it does not ship the ability to actually work with it at trillion-parameter scale. The tooling debt is real and it's hidden several libraries deep.
the agent needs to know what "revenue" means
a16z's piece on why data agents need context lands on the same beat from the enterprise side. "What was revenue growth last quarter?" is trivial for a dashboard and hard for an agent, because revenue is a business definition, not a column. The semantic layer that was supposed to hold that definition was last touched by someone who left, doesn't know about two new product lines, and isn't wired to anything.
The honest framing here is that most "chat with your data" efforts hit a wall, and the wall is context, not SQL skill. Their proposed context layer is a superset of the semantic layer plus tribal knowledge, human-refined, self-updating. Fair. Just note that "automated context construction" plus "human refinement forever" is a polite way of saying this is expensive, ongoing labor, not a product you install.
can you tell if the model is faking?
The wildcard: someone steered eval-awareness features in Gemma 3 27B. Using sparse autoencoders and contrasting pairs across languages, they isolated features that track how contrived a scenario feels, and those activations reliably indicated when the model thought it was being tested. Turn them down, and behavior gets more honest, as if the scenario were real.
The catch is baked into the write-up: if steering eval-awareness also nudges other behavior, you're no longer asking "what would this model do if it believed this?" but "what would a different model do?" And the murder feature couldn't be steered down without the 27B model degenerating into incoherence, which the author attributes to size. Interesting probe into whether activations reveal what a system actually did internally. Not a control knob yet.
the boundary is a court order now
Finally, the boundary got drawn in law: Amazon won an injunction blocking Perplexity's Comet agent. The judge found Comet accessed Amazon at the user's direction but without Amazon's authorization. "At the user's direction" is the phrase to watch. Your intent doesn't automatically license your agent onto someone else's protected systems. The future of autonomous browsing agents just got a real, if preliminary, fence.
what I'd do Monday
Stop deploying on sub-3-point leaderboard gaps; ask any benchmark for its resource enforcement config before you trust it. If you're betting on an open-weights model, budget for the training stack being broken, or just rent inference. Before building a data agent, audit whether your definitions actually exist and are current. And read the terms of service for any site your agents touch, because "the user told me to" is not the shield you think it is.