How to Stop Your AI Agent Making Things Up

Difficulty: Advanced · Estimated time: 20 min

What You'll Learn

Three concrete techniques to catch hallucination and drift before they reach something that matters: an uncertainty rule, a human review checkpoint, and periodic testing.

Why This Matters

A specialised agent that's wrong confidently is more dangerous than a generalist chat that's obviously guessing, because you've started trusting it. The more you reuse an agent, the more a single unnoticed bad habit gets repeated. This guide is about catching that early.

The Concept

Three layers of protection, used together:

An uncertainty rule in the instructions — the agent says it needs more information rather than filling the gap with a guess.

A human review checkpoint — for anything that matters, a person looks at the output before it goes anywhere, built into the process rather than left to memory.

Periodic testing against known-correct answers — every so often, run the agent on a task you already know the right answer to, and check it still gets there.

None of these is a one-off fix. They're habits you build into how the agent is used, not lines you write once and forget.

Step-by-Step

Add an explicit uncertainty rule to every agent's constraints: when a required fact is missing or unclear, it must say so rather than invent a plausible-sounding answer.

Decide which outputs need human sign-off — anything that goes to a client, costs money, or can't easily be undone — and build a review step into the workflow, not just a hope that someone checks.

Build a small test set of three to five inputs you already know the correct output for.

Run that test set periodically — after any instruction change, and on a regular schedule even when nothing's changed.

Log drift when you find it — note what went wrong and fix the instructions or context pack, not just that one output.

Example

For the weekly report writer from Guides 16 and 17:

Uncertainty rule: "If my notes don't mention progress on something you'd expect to see, say 'no update given' rather than assuming it's done or dropped."

Review checkpoint: the draft report always comes to you before it's forwarded to your manager — never auto-sent.

Known-answer test: keep one saved set of notes with a known-good report already written. Run it through the agent every month; if the new output drifts from the saved standard, check what changed in the instructions.

Try This

For one agent you've built in this level, write its uncertainty rule now, and identify exactly which of its outputs need a human to look at them before they're used.

Copy This

VERIFICATION RULES FOR: [AGENT NAME]

UNCERTAINTY RULE:
If [TYPE OF REQUIRED INFORMATION] is missing or unclear, say
"[EXACT PHRASE, e.g. 'Information required: ...']" instead of guessing.

HUMAN REVIEW CHECKPOINT:
Output requiring sign-off before use: [WHICH OUTPUTS]
Reviewed by: [WHO]
Before: [WHAT HAPPENS NEXT, e.g. sending, publishing, filing]

KNOWN-ANSWER TEST LOG
Test case: [INPUT]           Expected output: [KNOWN-CORRECT RESULT]
Last run: [DATE]             Result: [MATCH / DRIFTED]
Action taken if drifted: [WHAT WAS CHANGED]

Common Mistakes

Treating one confident-sounding answer as proof the agent is reliable.

Skipping human review for "low-stakes" output that turns out to matter more than expected.

Never re-testing after updating the instructions or context pack.

Fixing a bad output once without updating the instructions, so the same mistake recurs.

Level-Up Tip

Keep the known-answer test set itself under version control — when you update it, note why, so you can tell a genuine improvement in the agent apart from a test that just got easier.

Action Checklist

○Uncertainty rule added to the agent's constraints

○Outputs needing human review identified and checkpoint built in

○Known-answer test set created with three to five cases

○Test set run at least once and result logged

○Process set for reviewing and fixing drift when it's found

Next Guide

21, continuing the roadmap into Level 5. Head there when you're ready.

LEVEL 4 CHECKPOINT

You should now be able to design a specialised Claude-powered role for a recurring job, rather than rebuilding the same prompt from scratch every time.

Premium Content

Levels 2–6 require a mygyaan membership to access the full guides.

₹1,000/month — full mygyaan.ai access (all modules)

₹3,000 one-time — this course only

What you will learn