Building a Platform for Agents, Not Just Another Agent

A few product teams into building AI agents at work, I noticed a pattern that should have been obvious sooner: every new agent started from a blank page.

Someone would need a chatbot for a support flow, or an assistant that could answer questions against internal data, or a multi-step process that classified a request, looked something up, and formatted a response. And every time, an engineer would sit down and rebuild the same scaffolding: wire up a model provider, handle streaming so the response feels alive instead of arriving as one big blob, add retrieval so the agent could ground its answers in real content, figure out session handling, figure out who's allowed to run this thing and who isn't. The actual "agent" part, the part that was supposed to be the interesting work, was a small fraction of the code. Most of it was plumbing, and it was slightly different plumbing every time because nobody had written it down as a reusable pattern yet.

That's the problem I want to write about: not "how do we build an agent" but "how do we stop rebuilding the same agent infrastructure every time."

The pain wasn't the AI part

This is the part I didn't expect going in. The hard part of shipping agents at a company with many product lines was never really prompting or model choice. It was everything around the model: making agent behavior configurable without a redeploy, making sure one team's agent couldn't see another team's data, making streaming output consistent so the frontend didn't need a different renderer for every agent, and giving non-trivial workflows (multiple steps, branching logic, retries) a shape that didn't turn into spaghetti the moment someone wanted to add a third step.

Every team that built an agent solved these problems slightly differently, which meant every agent had its own quirks, its own bugs, and its own maintenance burden. Multiply that across a growing number of product surfaces and you get an organization quietly paying an "agent tax" on every new idea.

What we built instead

Rather than keep writing agents by hand, we built a framework: a shared layer that lets a team describe an agent, a team of agents, or a multi-step workflow in a configuration file rather than in bespoke code. The engineering work shifted from "build this agent" to "build the thing that lets anyone build an agent."

At a high level, the framework gives teams:

• Config-first agent definition. An agent, a team of agents, or a workflow is described declaratively: what it's called, what it does, which model powers it, what tools it can use. Standing one up doesn't require understanding the internals of how models are called or how responses are streamed back.
• Multiple model providers, interchangeable. Teams aren't locked into one vendor or one model size. Swapping the model behind an agent is a configuration change, not a rewrite.
• Real multi-step workflows. Sequential steps, parallel branches, conditional routing, loops. The stuff that turns "call a model once" into an actual process, without every team inventing its own orchestration logic.
• Retrieval built in. Agents can ground answers in a knowledge base instead of relying purely on what the model already knows, and that's a capability every agent gets for free rather than something each team has to plumb in themselves.
• Live, structured streaming. Every agent, team, or workflow streams its output the same way, so the responses feel immediate the way modern chat products do, and the frontend only ever needs to know one protocol, not one per agent.
• Multi-tenant by design. Because this serves many product teams at once, access control and isolation between teams' agents is a first-class part of the platform, not something bolted on afterward.
• Update without downtime. Agent behavior can change in production without redeploying the whole service, which matters a lot once agents stop being side projects and start being product features people depend on.

None of these are novel ideas individually. What made them worth building once, well, was that every team that skipped this and built its own version paid for it later, usually right when the agent needed to scale past its first version.

Why this mattered more than any single agent

The lesson I keep coming back to is that "AI features" are, underneath, a platform problem before they're a model problem. The choice of model matters less than most people assume once you've actually shipped a few of these. What matters is whether adding the fifth agent is as easy as adding the first one, or whether it's harder, because you're now maintaining five slightly different implementations of the same idea.

Building the framework felt, at times, like doing less "AI work" and more "infrastructure work." But it's the kind of infrastructure work that compounds: every agent built on top of it afterward gets streaming, retrieval, access control, and live updates for free, and every hour spent hardening the framework saves several hours across every team that builds on it next. If there's one thing I'd tell a team just starting to add AI agents into their product, it's this: build the thing that builds the agent before you build too many agents by hand. You'll build more of them than you think, and you'll be glad the tenth one is as cheap as the first.