Building an AI Tutor That Refuses to Teach From Its Own Memory

The single scariest failure mode for an AI tutor isn't that it gets something wrong, it's that it gets something wrong fluently. A student has no reliable way to tell a confident, well-explained, completely incorrect method from a correct one; that's precisely why they need a tutor in the first place. So before we'd let anything resembling an AI tutor near a real student, we built it around one non-negotiable rule: it is not allowed to teach or grade from its own memory. Every formula, every worked example, every judgment about whether an answer is right has to trace back to something real that was actually looked up, not generated from the model's general training.

The problem with a tutor that just "knows" the syllabus

A general-purpose model has seen an enormous amount of physics and math content during training, which makes it dangerously plausible-sounding on exactly the subjects where precision matters most. Ask it to explain a syllabus topic and it will, fluently, using whichever version of the method it happens to have absorbed most strongly from its training data, which may not match the specific curriculum, exam board phrasing, or worked-solution method a specific school actually teaches. A student following along has no way to catch the mismatch. They're relying entirely on the tutor being right, and "probably right, drawing on a general sense of the subject" isn't the same guarantee as "grounded in the actual syllabus unit assigned."

What we built: teaching and grading gated behind real lookups

The Skill's instructions are direct about this and structured around two separate tools that have to be used in sequence. Before teaching or quizzing any topic, it has to call a curriculum lookup tool, which returns that syllabus unit's actual teaching notes and a list of available practice questions, prompts only, deliberately no answers included at this stage, so it's safe for the tutor to browse a whole unit's worth of questions without accidentally seeing solutions ahead of when a student needs to attempt them. Once a student actually answers a question, the tutor has to call a second, separate tool to fetch that specific question's real worked solution, step by step, before it's allowed to grade the student's answer. The model still makes the actual grading judgment, comparing the student's answer against the real solution, but it's never allowed to grade from a solution it invented or half-remembered.

The instructions are explicit that the reasoning steps in the fetched solution are the actual teaching content, not just a way to check the final number. The tutor is told to walk through the same steps the real worked solution does, first find this, then apply that, rather than jumping straight to a verdict, and if a student gets something wrong, to point at the specific step where their reasoning diverged instead of just restating the correct answer. That mirrors how a good human tutor actually teaches: the method matters more than the final number, and pointing at exactly where the reasoning went sideways is what actually helps a student improve.

Grading stays with the model, grounding comes from the tool

It's worth being precise about the division of labor here, since it's a design choice we'd defend directly. The solution-lookup tool doesn't grade the answer itself, it returns the real solution and lets the model make the actual judgment call by comparing the student's answer against it. That's deliberate: grading a free-form answer against a worked solution genuinely needs language understanding, recognizing that a student expressed the right method in different words, or made a specific, nameable error partway through, which is exactly the kind of judgment a model is good at and a rigid string-match wouldn't capture. What the tool guarantees isn't the grading judgment itself, it's that the judgment is being made against something real, not something invented.

Progress that survives across sessions, and across a separate assessment page

After grading any attempt, the tutor records it through a dedicated tool, tagged to a specific student by name, so a student's history persists across separate conversations rather than resetting every time they open a new chat. That record is also designed to be shared with a separate, non-chat assessment surface tracking the same students, meaning a student's attempts, whether made while chatting with the tutor or working through a standalone quiz page, add up to one continuous history rather than two disconnected ones. Getting a student's name right early in a conversation matters enough that the Skill's instructions call it out directly: ask if it isn't already known, or check memory first, rather than guessing or leaving it blank, since every recorded attempt depends on being tied to the right student.

Remembering how a student actually struggles, not just whether they got it right

The most interesting piece, and the one that turns this from a grading tool into something closer to an actual tutor, is what happens when the same kind of mistake shows up more than once. The Skill is instructed to notice a recurring pattern, consistently reaching for the wrong formula in a certain type of problem, for instance, and save that specific pattern using the harness's general-purpose memory tool, the same remember/recall system every product on the harness shares. At the start of a future session on a related topic, it's instructed to check that memory for a known weak spot before diving in, so a student doesn't have to re-explain what they struggle with every single session, and the tutoring can actually start from where a real tutor with a good memory would start: not from zero, but from what it already knows about this specific student.

Why the read/write split still applies here

The same approval logic that runs through every product on our harness applies to a tutoring session too, just tuned to what a normal quiz actually requires. Looking up curriculum content and worked solutions is read-only and needs no approval, the same reasoning as any other lookup tool elsewhere in the harness. Recording a quiz attempt is technically a write, but it's exempted from approval for a specific, deliberate reason: it fires once per answered question, and pausing a live quiz session to ask a human to approve every single graded attempt would make an ordinary tutoring session unusable. That's the same judgment call behind exempting the memory tools themselves, some writes happen so routinely, and carry so little risk, that gating every one of them defeats the purpose of the tool that's writing them.

Why grounding matters more in education than almost anywhere else

A generic AI answer in most contexts is a minor inconvenience, you notice it's off and move on. In education specifically, the cost of an ungrounded answer is different in kind, not just degree: a student being taught the wrong method, or graded against a solution the tutor half-remembered instead of one that was actually verified, doesn't just get bad output, they walk away having practiced the wrong thing and possibly graded themselves confident in a mistake. That asymmetry, a wrong answer costing more here than in almost any other AI use case, is exactly why we treated grounding as the one requirement that couldn't be negotiated on, before anything else about this Skill got built.

What a real session actually looks like

Walk through a realistic exchange. A student says they're stuck on a physics unit about circuits. Before saying anything about the topic, the tutor calls the curriculum lookup tool for that exact subject, level and unit, gets back the real teaching notes and a list of available practice questions, and teaches from that content, not from its own general sense of circuit theory. It offers a practice question. The student attempts it, gets the method partly right but applies the wrong formula partway through. Before saying whether the student was right or wrong, the tutor calls the solution lookup tool for that specific question, gets back the real step-by-step answer, and only then compares the student's work against it, pointing at the exact step where the reasoning diverged rather than just marking it wrong.

The attempt gets recorded against the student's name. If this is the second or third time the same student has reached for the wrong formula in a similar problem, that pattern gets saved to memory. Next week, a new session on a related topic starts with the tutor already aware of that specific weak spot, checked from memory before the lesson even begins, rather than the student needing to re-explain what they struggle with from scratch. Nothing in that sequence depended on the model's own general knowledge of physics being reliable. Every substantive fact came from a real lookup.

What's built, and what's honestly still ahead

The Skill and its three tools, curriculum lookup, solution lookup, and attempt recording, are real, working code running against real stored curriculum data today, not a mockup or a plan. What doesn't exist yet is a dedicated, polished student-facing frontend for it; today it's exercised the same way every Skill on the harness can be, through the same general chat interface the rest of the harness uses, formatting math in LaTeX as the instructions require. Building a purpose-built tutoring interface on top is genuinely the easier half of this project. Getting the tutor to actually refuse to teach from its own memory, reliably, was the part worth building first and getting right before anything else.