What Actually Happens When You Send a Message to One of Our AI Agents

From a chat window, sending a message to an AI agent looks like one simple round trip: type, send, watch the reply appear. Underneath, on a properly built system, that single exchange involves deciding what the agent's allowed to do this turn, streaming a response token by token as it's generated, persisting every step incrementally in case something crashes mid-way, and making sure a conversation resumed later picks up exactly where it left off, tool calls and all. None of that is visible from the chat window, which is exactly why it's worth tracing through once, because it's where almost all of an AI agent's actual reliability lives.

Before anything: which agent is this, and what can it do

The message arrives at the harness tagged with which Skill is active for that product, we've written separately about how one backend runs several genuinely different AI agents this way. The harness resolves that Skill's system prompt and its specific, scoped list of tools before doing anything else, so from the very first step, the model is only ever handed the capabilities that specific product is supposed to have, not some global list of everything the harness could theoretically do.

Streaming begins before the model has finished thinking

Rather than waiting for a complete response and sending it all at once, the harness streams events back as the model generates them, following an open protocol, the same UI Message Stream Protocol the AI SDK's chat hook expects, sent as server-sent events over a single HTTP response. That's what produces the live, token-by-token typing effect users expect from a modern chat product, and it's a deliberate compatibility choice: because the harness speaks the protocol directly, any frontend built on that same open standard can point straight at it with no translation layer or custom proxy in between.

A subtle bug worth naming: the client that wouldn't stop resubmitting itself

Here's a concrete example of how much detail actually sits inside "streaming works correctly." A chat client following this protocol tracks whether the model's last message is "complete with tool calls," meaning finished and awaiting results, so it knows when to automatically resubmit. Early on, a turn that used a tool early in a longer response, say an automatic memory lookup, could leave that tool call looking permanently unresolved from the client's point of view, even once the model had gone on to produce a normal finished reply afterward. The consequence was a client that kept treating the conversation as still owing a tool result and auto-resubmitting, recursively, forever, a loop that looked like nothing was wrong from a glance at the UI and would have quietly run up model calls indefinitely if left alone.

The fix was a small, specific protocol detail: explicitly marking the boundary of each step in the stream, so the client scopes its "still waiting on a tool result" check to only the current step, not every tool call the conversation has ever made. It's the kind of bug that's completely invisible in casual testing, a normal short conversation never triggers it, and only shows up once a real conversation runs long enough and uses enough tools for the mismatch to surface. That gap between what a quick test catches and what real, sustained use eventually surfaces is exactly why we've written elsewhere about the difference between a demo and something built for production.

Why an open protocol instead of a custom one

It would have been simpler, in one narrow sense, to invent our own wire format for streaming responses back to a frontend, something shaped exactly around our own needs with none of the general-purpose complexity of an existing spec. We didn't, because the cost of a custom protocol shows up later, not upfront: every frontend that wants to talk to the harness would need custom client code written specifically for that format, and any tooling built around the open standard, debugging utilities, client libraries, documentation, becomes unusable the moment you step outside it. Speaking the same protocol a widely used chat framework already expects means any frontend built against that framework can point at the harness directly, no translation layer, no custom transport code, which matters a great deal the moment there's more than one product to build a frontend for.

Every step saved as it happens, not reconstructed afterward

As the model streams its response and, if it needs to, calls a tool, each meaningful step gets written to the database incrementally rather than only at the very end of the turn. That matters for a very specific failure case: if the backend process crashes, the connection drops, or anything else interrupts a turn mid-stream, whatever content already reached the client is already durably saved, not lost because the turn never technically "finished." A conversation that gets cut off mid-response is recoverable rather than corrupted.

Picking a conversation back up exactly where it left off

Every session is listable and resumable: a past conversation's messages come back already shaped the way the frontend's chat interface expects, including any tool calls and their results, so reopening an old conversation and continuing it works exactly like continuing one that never closed. That reconstruction step used to have a real bug worth being honest about: history was being rebuilt from only the plain-text parts of each past message, silently dropping any tool call a model had made. The practical effect was an agent that lost all memory of its own actions the moment a session was resumed, or even on the very next turn within the same session, confidently continuing a conversation while having genuinely forgotten what it had already done. Fixing it meant reconstructing the full shape of a past turn, the model's tool call and the tool's actual result, not just the words the model said around it, and verifying directly that a second request in the same session now correctly includes the first turn's tool activity in what the model sees.

What happens if a tool call needs a human first

Not every step in this trace runs straight through automatically, and it's worth tracing that branch too, since it's where a lot of the actual trust in the system lives. If the model asks to use a tool that requires approval, the stream pauses there rather than continuing, the request is recorded as pending, and the harness waits for a human to say yes or no before anything runs. We've written separately about why that pause is structured to be idempotent, so approving it twice, say after a flaky connection, never causes the action to execute twice. Once it's resolved, the conversation resumes from exactly that point, the model sees the real outcome of what was or wasn't approved, and streaming picks back up as if the pause had never interrupted the underlying turn at all.

What you can actually see: cost, not just output

Token usage is captured at the same granularity as everything else in the loop, once per model call, not once per entire conversation, which is what actually makes it useful. A single turn that involves a tool call is really two model calls stitched together, one that decides to use the tool, one that responds once it has the tool's result, and seeing the token cost of each separately shows you something a single combined number never could: roughly what that specific tool round trip actually cost, in the context it added, versus the cost of the reply itself. That number streams to the frontend live and gets attached the same way when a session is resumed later, so historical and live conversations show cost identically, not as an estimate reconstructed after the fact.

The pattern behind every fix described here

It's worth naming what these two bugs, the infinite-resubmit loop and the dropped tool-call history, actually have in common, because it's not a coincidence. Both were invisible in a short, simple conversation and only surfaced once a real interaction ran long enough or used enough tools for the gap between "looks fine" and "is actually correct" to show up. That's the general shape of almost every reliability bug worth caring about in a system like this: it isn't caught by the first test that runs the happy path once, it's caught by deliberately running the unglamorous, repetitive, long-running case that a demo never bothers to simulate. Building the discipline to keep testing that case, not just the clean first pass, is a bigger part of shipping something dependable than any single fix on this list.

Why this is worth knowing even though you'll never see it

None of this shows up in a chat window, and that's precisely the point of writing it up. A demo doesn't reveal whether a client's protocol implementation quietly leaks into an infinite resubmit loop after a long conversation, whether a crash mid-response loses anything, or whether resuming a session actually restores an agent's memory of its own actions or just its words. Those are the specific, concrete details that separate an AI chat feature that feels solid in a five-minute trial from one that actually holds up across a real, long-running relationship with a real user, and they're exactly the kind of detail we think is worth being transparent about instead of leaving as an implementation detail nobody outside the team ever hears about.