Under the Hood: How an AI Agent Actually Turns a Sentence Into a Real Action

It's easy to watch an AI agent create a file, send a message, or update a record and assume the model did it, the same way a person would: opened a terminal, typed a command, pressed enter. It didn't, because it can't. An LLM is a function that predicts the next token. It has no hands, no terminal, no awareness that an operating system exists underneath it. Every action an AI agent appears to take is the output of a pipeline sitting around the model, deciding what a stream of predicted text is allowed to trigger in the real world. We've written before about that pipeline in the abstract, as the harness half of AI Agent = LLM + Harness. This is the concrete version: the six steps between a model deciding it wants to do something and that thing actually, safely, happening.

Step one: the model can't ask for something it's never been told exists

Before a model can request an action, it has to be told the action is available at all, and told in a very specific shape: a name, a plain-language description of when to use it, and a validated structure for its input. That registration step is also where scope gets decided, and we treat it as the single most consequential decision in the whole pipeline. Every product on our harness runs as a Skill, and a Skill's tool list is exactly and only the actions that product's job requires. A tutoring agent gets a curriculum lookup and a worked-solution lookup. It does not get a generic, do-anything tool that happens to be capable of far more than tutoring ever needs, because a capability nobody asked for is a capability nobody's watching.

Step two: what the model actually outputs isn't prose, it's a request

When a user's message matches something a registered tool can do, the model doesn't write "sure, I'll take care of that" and stop there. It shifts into a distinct output mode and emits a structured block naming the tool and the arguments it wants to call it with, sitting alongside or instead of ordinary reply text. This is still just predicted text, the model has no more special access to the world in this mode than in any other, but it's text shaped precisely enough that software sitting downstream can read it as an instruction rather than a sentence.

Step three: catching the request before it reaches anyone

This is the part that actually deserves the word "agent." As the model's response streams back, the harness is watching that stream, not just relaying it, and the moment it sees a tool call take shape, it breaks the direct path from model to chat window. Nothing about this is visible to whoever's waiting on a reply; from their side it just looks like a brief pause. Underneath, control has shifted from "display this text to a person" to "hand this payload to an internal routine," and if the tool being requested is one that needs a human's sign-off before it runs, that's exactly where the harness parks it and waits, a pattern we've written about in more depth as part of tracing a message end to end.

Step four: turning a string back into something worth trusting

Whatever arrives over the network is just bytes, and even a perfectly formed tool call is, to the software receiving it, one long string until something deliberately turns it back into a structure. That happens in two passes. First, plain deserialization: extracting the tool call block and decoding it into a real object with a name and a set of arguments. Second, and more important, structural validation against the schema that tool was registered with in step one, checking the tool name is one this Skill is actually allowed to use, checking every argument is present and the right type, rejecting anything that doesn't match rather than guessing at what the model probably meant.

Models occasionally produce malformed output, a missing bracket, a badly escaped quote, especially under load or on a smaller model. When that happens we'd rather the call fail loudly and get rejected than have something downstream try to charitably interpret broken input, the same fail-safe instinct behind how we handle a bad memory compaction: a rejected action that didn't happen is recoverable. A malformed action that partially happened usually isn't.

Why we don't hand our agent a generic shell

This is the step where a lot of AI tooling takes a shortcut worth naming directly, because it's the one that turns a promising demo into a real liability. The easiest possible tool to build is an open one, something like a single run_terminal_command tool whose entire input schema is one free-text string, piped straight into a subprocess. It's genuinely useful for a demo, because it can do anything. That's also exactly the problem: its schema promises nothing, so every bit of safety work has to happen after the model has already decided what string to send, which is the latest and worst possible point to catch a mistake.

We build the other way on purpose. Rather than one wide tool with a free-text argument, a Skill gets several narrow tools, each with an input shape that structurally cannot express more than the one thing it's meant to do. Our agent's own database access is a concrete example: not one tool that runs arbitrary SQL, but a read tool that structurally rejects anything except a SELECT before it ever reaches the database, and a separate write tool that goes through approval every time. Neither tool can be talked into doing the other's job by a cleverly worded input, because the schema itself, not a rule the model is trusted to follow, is what makes that impossible.

Step five: executing under limits, not under trust

Once a call has been validated, it's resolved against a registry that maps its name to real, callable code and handed its arguments. What happens next is scoped tightly on purpose. Anything that writes, sends, creates, or deletes waits for a human to approve it first, the default on every tool we build unless it's structurally incapable of changing anything. Anything that runs, runs against infrastructure the agent owns and nothing else, a sandboxed database of its own rather than a real filesystem or a live production system, so the blast radius of a mistake is a table the agent made for itself, not something that matters. And approval, once given, is checked against a record of what's already been completed before anything actually executes, specifically so a retried approval, from a flaky connection or a double click, returns the stored result instead of running the action a second time.

Step six: closing the loop so the model actually learns what happened

Execution isn't the end of the pipeline. The model is still sitting mid-turn, waiting to find out whether its request succeeded, and the outcome, success or failure, gets written back into the conversation as its own message, tagged with the specific call it belongs to, so the model can tell which result answers which request if more than one was in flight. Only once that result is back in context does the model generate the reply a person actually sees. Every step of this, the call, the pause, the result, gets persisted as it happens rather than reconstructed afterward, which is what makes a conversation resumable later without the agent quietly losing track of its own actions.

What this pipeline is actually buying you

None of these six steps show up in a quick demo, because a demo runs once, on a clean input, with nobody sending a malformed request or double-clicking approve. They matter on the hundredth run, the thousandth, the one where a connection drops mid-call or a model hallucinates an argument that almost fits the schema. If you're evaluating an AI vendor and their entire pitch is "the model can call this tool," it's worth asking the boring question directly: what validates the call before it runs, what's it allowed to touch, and what happens if the same approval arrives twice. A model that can request an action is the easy half of the equation. A pipeline that only lets the right actions actually happen, safely, exactly once, is the half that decides whether you can leave it running unattended.