thirty cents to fine-tune, a trillion to justify it
Week ending December 5, 2025; archive coverage: December 5.
Every few months the hype cycle promises that agents will change everything, and every few months the actual demo turns out to be a model grading its own homework. This week felt different. Not because anyone declared victory, but because the interesting work was plumbing: agents that submit real jobs, harnesses that survive long horizons, a verifier that catches a genius model lying with confidence.
The throughline is discipline. Less "look what the model can imagine" and more "did the file exist after the command ran." That's the boring, verifiable stuff that actually ships. And hovering over all of it, a sobering question about whether the money underneath ever adds up.
the agent that actually runs the job
Hugging Face gave a coding agent a real skill instead of a party trick. We Got Claude to Fine-Tune an Open Source LLM wraps training know-how—GPU selection, LoRA versus full fine-tune, Hub auth, dataset validation—into a package Claude Code, Codex, or Gemini CLI can load and act on. You say "fine-tune Qwen3-0.6B on this dataset," it picks t4-small, quotes you thirty cents, submits the job, streams the loss curve through Trackio, and pushes the model to the Hub.
What I like is that it isn't a self-grading loop. It supports SFT, DPO, and GRPO, validates your data on CPU for a fraction of a penny before you burn GPU hours, and reports actual cost. The catch: it tops out around 7B (the docs say 7B+ isn't suitable), you need a paid Jobs plan, and dataset format is still where most runs die. But this is the right shape—typed workflow, human confirmation before spend, real artifacts at the end. Fine-tuning stops being a specialist ritual and becomes a conversation with a receipt attached.
getting out of the model's way
Philipp Schmid's Context Engineering for AI Agents: Part 2 is the practitioner counterweight to "just add more scaffolding." It gives you a taxonomy—offloading, reduction, retrieval, isolation—and names the failure modes: context rot, pollution, confusion. The key admission is that a one-million-token window is marketing; the effective window where quality holds is often under 256k. So you define a pre-rot threshold and compact before the model degrades, keeping recent tool calls raw to preserve rhythm.
The Manus lessons are refreshingly humble. They rewrote the harness five times in six months, and the gains came from removing things—hierarchical action spaces over 100 tools, agents-as-tools instead of an org chart of chatty sub-agents, don't RAG your tool definitions because it wrecks the KV cache. Their advice to not train your own model yet, because the Bitter Lesson will make your harness obsolete, is exactly the kind of judgment I trust more than another framework launch.
a genius who sometimes lies
Steve Hsu claims his QFT paper in Physics Letters B is the first theoretical physics result whose main idea came from an AI—GPT-5 in this case. The interesting artifact is the companion method paper. His description of frontier models is the most honest I've seen: a brilliant, unreliable collaborator who makes both simple arithmetic slips and plausible-sounding conceptual leaps that are wrong.
His fix is a Generate-Verify protocol—one model instance proposes a step, another independently checks it—which he says suppresses hallucination far better than single-pass generation. This is the anti-self-grading pattern, grounded in a published result. Structured orchestration beats trusting one confident pass. I'd hold the "first ever" claim loosely, but the method is the part worth stealing.
who pays for the plumbing
Then Kevin Zhang's Power Overwhelming points at the elephant. Data-center capex drove most of US GDP growth in early 2025, yet AI revenues run an order of magnitude behind spend. His conservative model puts the gap at roughly $1.5T by 2028—AI revenue would need to grow ~30x.
The structural argument is sharp: AI clouds aren't traditional clouds. GPUs from 2020 already rent for a fraction of Blackwell, so six-year depreciation schedules spanning three Nvidia generations look heroic. Whether we dodge an "AI winter" comes down to the internal workloads of maybe five companies. Not a bubble call, exactly—a reminder that the plumbing has a landlord.
what I’d do Monday
Wire a generate-verify step into any pipeline where a wrong-but-plausible answer costs you—a second model instance checking the first is cheap insurance. Set a pre-rot threshold well under your advertised window and compact before quality drops, keeping recent turns raw. Prefer computationally verifiable checks—did it compile, does the file exist—over LLM-as-judge scores. And try the HF trainer skill on a $0.30 demo before committing to any real run; catching a format error early is the whole point.
vocabulary inflation
Context rot: quality degrading as the window fills, well before the token limit. Context pollution / confusion: irrelevant data distracting the model, or the model failing to tell instructions from data. Pre-rot threshold: the token count where you compact preemptively. Generate-Verify protocol: one model proposes, another checks. Useful words, because they name things you can measure.