Prompts Aren't All You Need

Most of an agent's architecture is a bet about what the model can't do, and models improve on a schedule that ignores your roadmap. This piece is for people building the loop: harnesses, tools, long-running agents. If your product is a chat box and a good prompt, most of this won't bite you yet.

Anthropic engineers noticed that Claude Sonnet 4.5 rushed as its context window filled, cutting corners to finish before it ran out of room. They called the behavior "context anxiety" and handled it the way you'd expect: they taught the harness to reset context before the model got nervous. It worked.

Then Opus 4.5 arrived, running on the same harness, and the anxiety was gone. The model paced itself, and the reset logic that had been a clean fix the week before was now interrupting a model that didn't need interrupting. (Anthropic, "Scaling Managed Agents")

The fix wasn't bad engineering. It encoded an assumption, that the model couldn't pace itself and wouldn't learn soon, and the assumption expired on the next release. The retry wrapper, the chunking logic, the planning scaffold you wrote because the model couldn't plan: each one carries the same kind of expiration date, and nobody tells you the date. Keep writing them, since they're how anything ships. Just build so that pulling one out is a deletion rather than surgery.

Rich Sutton's Bitter Lesson describes decades of AI research: general methods that scale with compute keep beating clever, hand-built solutions. A smaller version of the same shape hits my code on a release cadence. The last piece I deleted was a wrapper that broke big edits into supervised chunks because the model used to lose the plot partway through; the new model held the plot fine, and the wrapper had quietly turned into pure latency. Writing a better prompt stopped being the hard part of this work a while ago. The hard part is the architecture around the model, where the bets live.

There is no longer "the model"

The stale assumption here is picking one model and building on it. Every frontier lab ships a family now, and the trap is treating the cheap tier as a toy. An agent makes dozens of small decisions per task, and reaching for a frontier model on each of them is renting a moving truck to carry groceries. The cheap model handles the busywork, the expensive one handles the few steps that earn it, and "which model" stops being a question you answer once. You answer it per step, continuously.

Reasoning changed the cost picture. Models can now spend compute thinking before they answer, and you pay for those hidden tokens, often several times the size of the visible response. On hard problems it's worth it. A lot of industry money is riding on thinking longer as the next source of gains, alongside the older bet on scaling bigger.

Stale scaffolding hurts most here. If you broke hard problems into tiny supervised steps because the model couldn't reason through them, a thinking model often fights that scaffolding instead of benefiting from it. Treat thinking as a cost you manage: latency, a token budget you cap on purpose, and a decision about whether users see the reasoning. Some systems now route between fast and thinking variants on their own, which means you're integrating with a system rather than a single artifact.

The stable ground is the orchestration layer. Own that, and let the model choice underneath stay a routing decision you revisit instead of an identity you defend.

Stop baking assumptions into the harness

The harness is the loop wrapping the model: call it, run its tool calls, feed results back, manage context, decide when to stop. You write that loop, and almost all of it is guesses about what the model can't do. Compaction, retries, planning steps: each is a guess about a current limitation. (Anthropic on harnesses; Managed Agents.)

That pile of guesses is easy to forget about, which is why I like the instruction in Anthropic's long-running agents repo: after each model release, comment out harness pieces one at a time and see what's still load-bearing. Audit your own cleverness and delete what the model has outgrown.

Not everything in the loop is a guess, though. Auth boundaries, spend caps, and audit logs aren't bets about model capability. They're policies about your system, and they survive every release. Part of the audit is learning to tell the two kinds of code apart.

Don't build the one right harness. Build interfaces you can swap behind.

Make replacement safe

If the harness is disposable, its knowledge has to live somewhere else. A component that both thinks and remembers is a single point of failure, because losing it costs you both at once. So split the jobs three ways.

Brain: the model plus the loop. Stateless and disposable; if it dies, start another and replay from the record below.
Hands: sandboxes and tools, behind one boring call. Any tool, any container, any MCP server.
Session: an append-only log of what happened, kept outside the brain and outside the context window. That log is the source of truth.

The session is not the context window. The window is a temporary, lossy view the harness assembles each turn, while the durable record lives in the session. Once that's true, running out of context stops being frightening. Compaction no longer destroys history; you're choosing what to show the model this turn while the full log sits on disk.

Anthropic's Managed Agents makes this split explicit, and I keep seeing the same shape elsewhere under different names: sessions and traces in OpenAI's Agents SDK, checkpoints in the graph frameworks, transcripts in the terminal agents. Whatever a platform calls it, the durable log is the part worth protecting. The model and the loop code around it can go.

Memory is two problems

"I need my agent to remember things" hides at least two jobs, and they need different machinery.

Within a run, memory is context, and curating it is the job. Overfill the window and recall degrades, so the question is what earns a seat this turn, not how to store more. Compact when you must, which is safe as long as the log keeps the originals. Trim stale tool results. Push bulky content somewhere queryable and pull slices back, showing the model a menu and loading the full item only when it bites. Keep prompt prefixes stable so caches hit. The window is working memory, the log and files are everything longer, and the loading decisions you want are the ones you can undo.

Across runs, the question becomes when the agent should know something about you, and the products have split on it. Some build a profile and inject it into every chat, which buys fast personalization at the cost of a stale impression that follows you around and a "fresh start" that never quite is. Others give the model tools to search past chats when it decides to look: blank slate by default, every lookup visible, a beat of latency, and a model that has to choose to remember. Neither approach has won so far, and some vendors ship both.

If you're building, don't treat "memory" as one knob. Split it:

• User facts (name, role, stack): a small list, injected when relevant.
• Episodic: what happened in past sessions. Your session logs are already the raw material.
• Procedural: how to work in this repo. A committed file next to the code (AGENTS.md, CLAUDE.md) plus whatever accrues on the machine.
• Semantic: domain docs. RAG and Skills territory.

The storage half of this is easy, since any database will hold notes. The policy half is harder: deciding what to keep, when to surface it, how to keep one project's notes from bleeding into the next, and when to let something expire. The memory systems I've watched disappoint people usually got storage right and got scoping and decay wrong.

Four interfaces

When an agent acts, it reaches through one of four interfaces. People pitch them as competing choices, but they're layers, and a working agent usually uses more than one.

Tool calling is the base layer. You give the model tools with schemas, it returns a structured call, and you run it and feed the result back. The part people underweight is that the tool description is the routing logic, so wording it well beats tweaking the schema. Write it like a prompt, including when not to use the tool. Fire independent calls in parallel, keep huge result blobs out of the window, and pass errors back to the model instead of swallowing them.

MCP answers who writes the tool for every external service. A service exposes tools and data once and every compatible agent can use them, where before, each agent reimplemented each integration. Watch the context tax, though: every connected server spends tokens on its definitions, so don't connect ten just in case.

Skills, the Markdown-folder pattern Anthropic popularized, spread fast because they're cheap. A Skill is a folder with a Markdown file: a name, a one-line description, and instructions the model loads only when the description matches the task. There's no runtime and no handshake, and if you can write Markdown you can write one. Those instructions go straight into context, so treat a Skill like third-party code: pin versions and review it like a dependency.

Structured outputs are tool calling from another angle. Force a typed object with a tool call, or use strict mode so the decoder can't break your schema. Write the schema like a prompt and keep it flat.

You don't pick one of these forever. A single agent might load a Skill for instructions, hit an MCP server for data, call a native tool to act, and force structured output for the summary.

Running it without getting burned

Design gets you halfway. The rest is surviving hours or days of unattended operation without a crash or a compromise, and two constraints do most of that work. Both lean on the split above.

Keep credentials out of the sandbox. A narrowly scoped token still assumes the model can't be talked into misusing it, and prompt injection is a live threat that gets more rewarding for attackers as agents get more capable. The structural fix is to make tokens unreachable from where generated code runs: bundle auth with the resource (clone with a token at setup, then drop the token), or hold secrets in a vault and attach them in a proxy the sandbox can call but never read. The sandbox gets hands, not keys.

Survive crashes on the session. A long-running agent has to outlive restarts, and it can if the brain is disposable and the session is the truth: reboot, replay the log, continue. The same mechanics give you hours-long human-in-the-loop pauses, and if sandbox state is explicit you can fork a session to try two approaches from the same starting point.

The terminal coding agents have mostly converged on the same feature list: Skills, MCP, plan-then-execute, sub-agents, headless mode for CI. From a distance they look interchangeable. The differences that still matter are model quality, price, and how the harness behaves when something goes wrong.

What I keep on the wall

More model alone doesn't save you, because the parts of your system that assumed the old model are exactly the parts the new one breaks. That fact is enough to change how you design the loop.

• Treat every line of harness as a guess that may be wrong next month, and keep interfaces around the guesses so they come out clean.
• Brains, sandboxes, and tools are disposable. The session log is not.
• Route across tiers instead of crowning one model.
• Memory is policy (keep, surface, scope, forget), not a product you can buy.
• Credentials stay out of the sandbox, because trusting any particular model is the wrong control.

The title nods to "Attention Is All You Need." Attention made the modern model possible, and for a while a good prompt was most of the job of using one. For agents, the job has moved into the architecture around the model, and it moves again every time a release turns one of your workarounds into dead weight.

Prompts got us this far. The next stretch belongs to what you refuse to hard-code.