a decade of agents, financed by next week's data center
Week ending October 24, 2025; archive coverage: October 22.
The vibes shifted this week, and objectively almost nothing happened. Andrej Karpathy went on a podcast, called today's frontier code "slop," and put AGI roughly a decade out. Suddenly half of tech was writing eulogies for the bubble. The other half was building infrastructure that assumes the hype is basically right. Both can't be correct, and honestly the interesting stuff this week lived in the gap between them.
Here's my read: the flashy launches matter less right now than the boring plumbing. The people quietly solving forgetting, building evals that survive contact with real data, and measuring which cheap model actually works are doing the load-bearing work. Meanwhile the capital and power required to run all of it keeps ballooning in ways nobody has fully priced.
the decade-of-agents reality check
Zvi's breakdown of the Karpathy podcast is the anchor. Karpathy's framing is that 2025 isn't the year of the agent, it's the decade of agents, because these things are like interns with real deficits in intelligence, context, and continual learning. He's dismissive of "pure RL" ("terrible, but everything else is worse"), skeptical of vibe coding on novel code, and fond of a small "cognitive core."
I mostly buy his caution and mostly reject his conclusions. Zvi's pushback is the right one: a ten-year timeline that ends in AGI is still terrifyingly short, and Karpathy's insistence that all of this blends into steady 2% GDP growth is having his cake and eating it. You can't call it an intelligence explosion and also normal technology. Pick one. But his core observation, that models remember wrong and keep reverting to internet priors, is real and worth respecting.
continual learning gets actual numbers
Which is exactly why Jessy Lin's post on the continual learning problem is the most hopeful thing I read this week. The pitch: swap some feedforward layers for sparse memory layers, then finetune only the slots that are specific to a new fact, ranked TF-IDF style against pretraining data.
The numbers earn attention. Learning TriviaQA facts, held-out NaturalQuestions performance drops 89% with full finetuning, 71% with LoRA, and only 11% with sparse memory finetuning. The catch: this is a 1.3B model on toy benchmarks, and she's the first to say the field badly needs better evals. But it's a concrete mechanism for "always training" without breaking the model, and it quietly undercuts the everything-in-context crowd. Nice bonus finding: plain SGD beat AdamW here, because our favorite optimizers assume things that break in the continual regime.
evals that survive reality
On the plumbing side, James Mackey's writeup of how Mura built BOLT is the practitioner counterpart. They turn messy HVAC emails, PDFs, and photos into structured fields, and they needed to measure accuracy field-by-field across customers and model versions. Off-the-shelf tools were too opinionated, so they built a CLI on top of Braintrust datasets and experiments.
The payoff is the story I keep telling people: with real evals they attempted a GPT-4 to GPT-5 migration they'd never have risked by hand, and discovered the mini model matched the big one at 80% lower cost. That's the whole game. Without durable eval infrastructure you can't upgrade, can't refactor, can't even tell if a prompt tweak for field A quietly broke field B.
the mini-model bake-off
Speaking of cheap models earning their keep, Kilo's comparison of Haiku 4.5, GLM-4.6, and GPT-5 Mini is exactly the kind of grubby hands-on data I want. GPT-5 Mini was cheapest ($0.05) and the only one that understood SQLite's concurrency limits and built lease-based locking. Haiku was fastest with the most features and zero tool-calling failures. GLM-4.6 had the cleanest architecture but its reasoning mode broke tool calls entirely, forcing them to disable reasoning to make it work.
That GLM detail matters more than the winner. In an agent runtime, a model that can't reliably call tools while reasoning is a landmine, no matter how pretty the code.
the wildcard: who's paying
And then the money. The WSJ reports BlackRock was among the biggest buyers in a $27B private-debt deal financing Meta's Hyperion data center, buying over $3B of bonds. The structure is telling: Blue Owl owns 80%, Meta 20%, Morgan Stanley arranged it. The compute buildout is now being financed like a highway project, off Meta's balance sheet and onto private credit. That's the real bet on the decade of agents, and it comes due whether or not Karpathy's timeline holds.
what I'd do Monday
Build the eval before the feature. Mura's whole advantage is a fast field-level test loop, and it's the difference between confidently swapping to a mini model and being scared to touch anything. Then actually benchmark the cheap tier yourself on your workload, and specifically test tool-calling under reasoning mode. Finally, if you're doing personalization via LoRA, watch the forgetting; the sparse-memory numbers suggest the default approach quietly eats your held-out performance.