AI SDK Advanced: Production Controls and Durable Agents

1. The difference the user actually feels

Start with the thing your user can see, because that is where everything else hangs off.

generateText waits until the full answer is ready, then hands it over in one piece.

streamText starts updating the interface while the answer is still being produced.

That is the whole product difference in two lines. Generate waits. Stream updates the app as the work happens. Everything in this post is about keeping that same experience under control: deciding when a call should run at all, what path it should take, and what happens while the stream is still moving.

2. When is an agent actually worth it?

This is a question I had to answer for myself early on, because the word agent gets thrown around a lot.

streamText is a primitive. You can absolutely build a tool loop with it. But if you do, you own that loop. You write the part that reads the tool call, runs the tool, feeds the result back, and decides whether to go again.

ToolLoopAgent is for the cases where that loop is the product. It can call tools, observe the results, take another step, and keep going until a stop condition is met. So here is the line I keep in my head: streamText is a primitive, ToolLoopAgent is a managed loop.

That does not make it magic. It just moves the iteration into a reusable agent boundary so you are not rewriting the same loop in every route.

3. Give the agent narrow tools

Here is something I had to learn the slightly harder way: an agent is only as safe as the tools you hand it. The loop is only as good as the choices it can make.

So I keep each tool small and obvious. The search tool gathers source material. The analyze tool extracts the signal. The report tool writes the final output. Each one has a description, an input schema, and a single focused execute function. Nothing clever.

My rule of thumb: if I cannot explain a tool in one sentence, it is probably too broad for the first version. Narrow it.

4. Declare what a tool needs instead of closing over it

The first version of this tool quietly reached for an API key from the module scope. That works right up until you want two tenants, or a test, or a per-request key. Then the tool is welded to whatever happened to be in scope when it was defined.

So a tool now declares its dependencies the same way it declares its input. inputSchema is what the model provides. contextSchema is what your application provides. The callback receives only the fields it asked for, already validated.

The thing I like about this is that the tool stays honest. Read its definition and you can see everything it touches. Nothing arrives by accident.

5. Pass run-wide and per-tool context at the call site

There are two kinds of context, and it is worth keeping them apart.

runtimeContext is for the whole run. A request id, a tenant, a user. Anything the entire generation should be able to see.

toolsContext is scoped per tool, keyed by tool name. The search tool gets the search key. Nothing else does.

That split is the whole point. A secret that only one tool needs should not be ambient across every tool the agent can reach for. This is least privilege, applied to a tool set.

6. Approval is a property of the request, not the tool

This one is a genuine change in how I structure things, and I think it is the right call.

Approval used to live on the tool definition. But whether a delete needs a human depends on the situation, not on the function. The same tool might be fine unattended in a scratch directory and absolutely require a person anywhere else. Baking that into the tool forces one answer forever.

So the gate moves to where the request is configured. Return undefined and the call proceeds. Return "user-approval" and it pauses for a human.

The tool goes back to doing one thing, which also makes it far easier to test.

7. Split MCP tools the model can call from tools the app must draw

Some tools do not want to be a line of text in a chat log. A seat picker, a date range, a diff view. The result is not really prose, and flattening it into prose loses the point.

splitMCPAppTools separates what the model can call from what your app should render. Model-visible tools go to the model as usual. App-visible tools keep their metadata attached to the tool UI part, so your renderer can decide when a tool has earned its own interface.

This is the bridge between "the model can call a tool" and "the tool needs a frontend." You do not need it on day one. You will know the moment you do.

8. Define the agent once, use it everywhere

Now we put it together in one place. The agent holds the model, the instructions, and the tool set as a single unit.

I like this boundary a lot. Once it exists, the rest of the app just calls researchAgent.generate() or researchAgent.stream(). Nobody downstream has to rebuild the tool loop, or remember which tools the agent is allowed to touch. It is defined once.

One small thing I always do: keep the model configurable. That way you can benchmark it, or swap providers, without rewriting the agent itself.

9. Picture the agent as a loop

It helps to actually picture what the agent is doing here. Receive the task. Choose a tool. Use the result. Choose the next action. And eventually produce the final answer.

The word I want you to sit on is eventually. An agent needs the freedom to take multiple steps, because that is the entire point of it. But it also needs a reason to stop. Hold that thought, because the next step is all about giving it one.

10. Run it with generate or stream

The nice part is that the same agent answers both of our product needs.

Use generate() when the user can wait for the finished result. Use stream() when they need feedback while the work is happening. Output can still arrive as text chunks, and structured output can still arrive as partial objects. So you are not choosing the agent based on the UI. You are choosing the call style.

The API shape stays familiar, which is the point. The only real difference now is that the agent owns the loop between model calls, instead of you.

11. Give the loop a reason to stop

This is the reason to stop I promised you. AI SDK agents do ship with a default step cap, but I would not lean on it. In production I think you should choose an explicit stop policy for the workflow, on purpose.

There are a few shapes this takes:

• A hard step cap, such as isStepCount(8).
• A completion tool, such as hasToolCall("generateReport").
• A custom predicate, such as “stop once confidence is high enough.”

And when you pass an array of them, the rule is simple: whichever stop condition fires first wins. You are not ranking them. You are saying any of these is a good enough reason to be done.

12. Stop conditions are also a UI

Here is the bit people miss. Stop conditions are not only backend safety. They are also some of the most honest things you can put in front of a user.

Because once you know the step count and which condition fired, the UI has real things to say. How many steps have run. Which tool finished the job. Whether the agent stopped because it was actually done, or because it hit a safety cap and you cut it off.

And honestly, that is the whole gap between “the agent is thinking” and “the agent is on step three, using the report tool.” One is a spinner. The other is trust.

13. Structure lives in the same flow

When I first hit structured output I assumed it was a separate world with its own rules. It is not. It lives in the same flow as everything else.

The agent can stream text when the user wants readable progress. And it can use Output.object when the product needs a typed result it can actually rely on. Same agent, same loop.

The part I really like is partial structured output. For a dashboard, a report, or a workflow summary, the interface can fill itself in field by field as the model works, instead of sitting empty until the whole object lands.

14. Let the request shape the agent

In a real product, not every user should get the same agent. That sounds obvious, but it is easy to forget when you are testing with one account.

Free users might only get documentation search. Enterprise users might be allowed to create a ticket. An urgent request might deserve a stronger model, or different instructions. Same agent, different behavior per request.

Two pieces make this clean. callOptionsSchema validates the runtime options coming in, so you are not trusting arbitrary input. prepareCall then turns those validated options into the actual agent settings for that run.

15. One path: schema, options, prepareCall

It helps to see those three pieces as a single path rather than three separate features.

The schema defines what is allowed. The options carry the context for this particular request. And prepareCall converts that context into something concrete: instructions, which tools are active, the model choice, provider options, or metadata.

What I like about this is where the personalization ends up living. It sits right next to the agent, in one place, instead of being smeared across every route handler that happens to call it.

16. Reasoning is one option, not a provider dialect

While we are on options, there is one worth calling out because it used to be a mess.

Reasoning effort was provider-specific. Every model family had its own name for it, buried in provider options, so the moment you swapped providers you rewrote that part too. It was exactly the adapter tax the SDK exists to remove, except it survived inside the SDK.

Now reasoning is a normal top-level option, and it means the same thing everywhere.

One trap worth knowing. If you set reasoning at the top level and through providerOptions, the provider-specific value wins. That is a reasonable rule, but it produces a genuinely confusing afternoon when you change the top-level option and nothing happens. Pick one. I use the top-level one and only drop into providerOptions for something with no portable equivalent.

17. Reuse the expensive work

Now we move from the decision layer into making things cheaper and faster. Caching asks one blunt question: can we reuse this expensive result instead of paying for it again?

For non-streaming calls, middleware can check a key-value store before it ever calls doGenerate(). On a hit, you return the stored result. On a miss, the call goes to the model and the result is saved with a TTL. The way I say it to myself: a cache means you run the expensive path once.

One honest gotcha. Dates and provider metadata sometimes need a little normalization after they have been serialized and read back. So keep the cache boundary boring and explicit. This is not the place to be clever.

18. See where the cache pays off

It is worth watching what this actually buys you, because the win is real.

The first call pays the full latency and the full token cost. The second identical call comes back almost immediately, with no new model tokens spent. Lower latency, lower cost, more predictable performance. You ran the expensive path once and everyone after that rides on it.

The tradeoff is freshness, and there is no way around that. So I choose TTLs per use case rather than treating every prompt the same. A pricing summary and a live status answer should not share the same expiry.

19. Caching a stream takes more care

Caching a streamed response is trickier, and this is where people get it subtly wrong. The UI still expects chunks, so you cannot just cache one big blob.

The shape that works: pass the live chunks straight through to the user while you also quietly collect them. When the stream finishes, store the array of parts. On the next hit, you replay those parts with simulateReadableStream.

The trap to avoid is collecting the whole stream first and only then sending it on. Do that and you have turned your stream back into a wait. That defeats the entire point.

20. Replay it so it still feels live

The goal here is that a cached stream still feels alive to the user, even though you already know every word of it.

You do know the full content. But replaying it as chunks means it travels the exact same path as a real response: the same rendering, the same progress behavior, the same message-part contract.

And that is what I am really after. The cache stays completely invisible to the React tree. The UI has no idea whether it just hit the model or hit Redis, and it should not have to care.

21. Attach it all at the model boundary

So where does the caching middleware actually go? wrapLanguageModel is the clean place for it, and for anything else cross-cutting.

Caching, tracing, guardrails, logging. They all get easier the moment the route just calls one wrapped model, instead of the route having to remember every middleware step by hand on every call.

The test I use is simple. The route should read like product logic, not like infrastructure plumbing. If it reads like plumbing, the behavior probably belongs at the model boundary instead.

22. Stop the request before it costs you

Rate limiting is the cheapest protection you have, because it acts before the model call ever starts. You are spending nothing yet, and that is exactly when you want to make the call.

Use the authenticated user id as the key when you can. IP limits have their place, but shared networks, VPNs, and mobile carriers make them noisy, and you end up punishing people who share an exit node with a heavy user.

When someone does hit the limit, return a 429 with a retry time attached. That gives the frontend something concrete to turn into a human message instead of a generic error.

23. The limiter decides who gets through

It is worth being clear about what that gate is really doing, because it is more than blocking bad actors.

The limiter decides who gets through. That caps spend per user. It keeps access fair. And it stops a single noisy client, automated or just overenthusiastic, from quietly eating your entire AI budget while everyone else waits.

Which leads to the line I keep coming back to in this whole layer: the cheapest request is the one that never reaches the model.

24. Let people stop a bad answer

Sometimes the model just starts heading in the wrong direction. You can see it in the first line. The user can too.

If all the UI does is wait, that user pays in time and in tokens for output they already know they are going to throw away. So give them an out. useChat hands you stop(), and I keep the button active only while a response is submitted or streaming, so it is never sitting there pretending it can do something.

That handles the visible stream. But to be honest, it is only half the story, and the missing half is where people get caught out.

25. The server has to hear the abort

Here is the half people miss. Stopping the client does not stop the provider. The button went quiet, but the model call is still running on your server, still billing.

So you forward req.signal into streamText, and now the server can actually cancel the model call when the user bails. Then use onAbort to do the cleanup that matters: persist the partial progress, log that it happened, release whatever you were holding. Use onEnd if you need to save the final message history once the stream completes. Stopping the client is not enough unless the server hears it.

One honest tradeoff to flag. Abort behavior and resumable streams do not sit together cleanly. You will likely have to decide which product behavior matters more for your case, rather than getting both for free.

26. Treat the stop as one honest state

So the stop button is not really a button. It is a small flow, and I think it is worth treating it as one product state from end to end, not just a frontend animation that makes the user feel better.

A cancellation that actually holds together has four moments you can see: the user stops the stream, the client stops rendering new chunks, the server receives the abort signal, and the app then saves or discards the partial run on purpose.

Do that and the user gets real feedback, without the app quietly pretending a cancelled response finished normally. Those are two very different things, and users can tell.

27. Backpressure, and why it matters

Backpressure sounds like a scary word, but the idea is calm: it is what stops a fast producer from overwhelming a slower consumer.

Picture chunks arriving faster than the UI can actually deal with them. The buffer fills, memory climbs, and eventually data gets dropped. Pull mode flips that relationship around. Instead of the producer firehosing chunks, the consumer asks for the next one when it is genuinely ready for it.

The good news is that AI SDK streams handle this for you. The bad news is you can quietly undo it. If you eagerly buffer everything in the middle, you have stepped between the producer and the consumer and broken the very thing that was protecting you.

28. Wrap it in DevTools while you build

AI failures are genuinely hard to debug from the final text alone. You get a wrong answer and almost no clue about which step produced it. I have stared at plenty of those.

DevTools wraps your model and records local traces for generateText, streamText, and agent runs. To be clear, this is a development tool, not a production one. Those traces can contain prompts, outputs, and raw provider data, so it is not something you leave switched on in front of real users.

In v7, telemetry graduates to @ai-sdk/otel: once you call registerTelemetry(new OpenTelemetry()), every AI SDK call is traced by default and you opt out per call with telemetry: { isEnabled: false }.

But while you are building? Reach for it the moment you need to see what actually happened, instead of guessing.

29. Read the run, not just the answer

With agents especially, I have come to value the trace more than the final response. The answer tells you what happened. The trace tells you why.

A trace turns agent behavior into something you can actually inspect: the prompt, each step, the tool parameters, the tool outputs, token usage, timing, the raw request and response payloads, and the stop reason that ended the run.

So when an agent goes sideways, and at some point one will, this is where you go. Not to the output, but to the exact step where it quietly drifted off course.

30. Register telemetry once and every call is traced

DevTools is for you, on your machine, while you build. Production needs the other thing: traces you can read after the fact, from a run you did not watch happen.

Telemetry is no longer experimental and no longer per-call opt-in. You register an integration once, at startup, and every AI SDK call is traced from then on. No wrapping, no remembering to pass a flag on the one route that turns out to matter.

The flag flips the other way now: opt a specific call out with telemetry: { isEnabled: false }. That is the right default. The calls you forget about are exactly the ones you end up needing traces for.

If you are coming from the experimental version, the rename is mechanical: experimental_telemetry is just telemetry.

31. Hang logging off the lifecycle, not the response

The last piece is knowing where your own logging goes.

It is tempting to log around the call, after the promise resolves. That works for one-shot generation and falls apart the moment there is a loop, because the interesting parts happen in the middle and you only ever see the end.

The lifecycle callbacks give you those middle moments as named hooks: run start, step start, tool start, tool end, step end, run end. They are stable now, they work the same on generateText, streamText, and agents, and they are the right home for logging, metrics, and anything you want to attach to a specific step rather than the whole run.

Put your logging here and a long agent run stops being one opaque event. It becomes a timeline you can actually read.

32. An in-memory loop only lives as long as the process

Here is an agent much like the one we built at the top of this post. Nothing about it is wrong.

But look at where its state lives. Each step of the loop, each tool result, each accumulated message sits in this process's memory. That is what makes it fast, and it is also the whole limitation.

If the process dies at step seven of nine, you do not lose one step. You lose all seven. There is no record of the work, and nothing to resume from, so the only honest thing the UI can do is start the whole thing again.

For an eight-second chat turn, who cares. For a four-minute booking flow that already charged someone's card on step five, that is a genuine product problem.

33. Mark the loop as a workflow and each step becomes durable

WorkflowAgent is the same idea as ToolLoopAgent, with one difference that changes everything: each step is a durable workflow step rather than a stack frame.

The "use workflow" directive is what does it. Above that line you are writing an async function. Below it, the runtime is persisting each step as it completes, so the run has an existence outside the process that started it.

I want to be precise about what that buys you, because it is easy to oversell. It does not make the agent smarter, faster, or cheaper. It makes the run recoverable. Completed steps stay completed.

34. Start the run and hand back a resumable stream

The route changes shape slightly, and the change is worth understanding.

You no longer call the agent and stream its response. You start a run, and you get back a handle. The stream you return to the client is a view onto that run, not the run itself.

That distinction is the whole feature. The response can end, the connection can drop, the container can be replaced, and the run carries on regardless, because the run was never the response.

The run id goes out as a header so the client has something to come back with.

35. The run id is the handle you reconnect with

Given a run id, you can attach to a run that is already in progress.

This is the part that makes the previous step useful rather than merely interesting. getRun hands you a handle to a run that already exists, and getReadable({ startIndex }) opens a fresh view onto it from a given chunk. No replay of the model calls, no double-charging, no starting again.

The path matters here. WorkflowChatTransport reconnects to {api}/{runId}/stream and passes the chunk it got to as a startIndex query param, so the route has to live at that exact path and honour that param. Ignore startIndex and you will re-send chunks the client already drew.

It is worth noticing what this quietly gives you for free: a run becomes something you can watch from more than one place. A second tab, a support dashboard, a mobile client that reconnected on a different network.

36. The client reconnects instead of starting over

On the client the whole thing collapses into a transport swap.

useChat does not change. Your message rendering does not change. You hand it WorkflowChatTransport instead of the default, and reconnection becomes the transport's problem rather than yours.

I like this a lot, and not only because it is little code. It means durability is a decision you can make late. You build the feature the normal way, discover it needs to survive a deploy, and change the transport, not the UI.

37. A harness is a whole agent runtime, not just a model

The second thing on the far side of that line is a different shape entirely.

So far, every agent we have built was assembled by us: our tools, our instructions, our loop. A harness inverts that. It is a complete agent runtime, and the SDK gives you adapters for the ones that already exist, Claude Code and Codex among them, each running inside a sandbox.

The harness owns the workspace, its own built-in tools, and its permission flow. You are not constructing a coding agent out of tool definitions any more. You are renting one that already works, and pointing it at a sandbox.

The sandbox is not a detail. An agent that can edit files and run commands needs somewhere that is not your machine and not your production box.

38. Sessions are stateful, so always destroy them

A harness session is a live workspace: a filesystem, a running process, sometimes an open port.

That makes it genuinely different from every other call in this post. generateText has no cleanup. A session absolutely does, and a leaked one is a sandbox you are still paying for.

So createSession and session.destroy() belong in a try/finally from the very first time you write it, not as a hardening pass later. The one time it matters is the time something threw halfway through, which is exactly when you are least likely to be thinking about cleanup.

39. Skills are reusable instruction bundles you attach

Skills are the answer to a problem you feel almost immediately once agents touch a real codebase: the same long preamble gets pasted into every agent, and then drifts.

A skill packages instructions, tools, and behavior under one name, and you attach it to an agent. Update the skill and every agent using it follows.

They show up most naturally around harnesses, because that is where the instructions get long and specific. "How we do migrations here" is a paragraph nobody wants to maintain in four places.

40. Watch the run in a terminal before you build any UI

This one is small and I use it constantly.

@ai-sdk/tui renders a harness or agent stream in the terminal, including tool calls, reasoning sections, and approval prompts.

The reason it matters is sequencing. When you are working out whether an agent is behaving sensibly, building a web UI first means debugging two unfinished things at once. Watch the run in a terminal, get the behavior right, then decide what the interface should be.

41. All of it speaks the same stream you already render

The point I want to land on is that none of this is a separate universe.

An in-memory run, a durable run, and a sandboxed harness run all produce the same message parts, the same tool parts, and the same stream chunks. The response boundary is identical. The useChat code you wrote in the crash course renders all three.

Which means the choice between them is not architectural lock-in. It is one question, asked per feature: how long is this loop allowed to live, and how much of the world is it allowed to touch?

Start in memory. Reach for durability when losing the work becomes expensive. Reach for a harness when the agent needs a real workspace. Nothing above the route has to know which one you picked.