AI-Assisted Customer Email: Verify Before You Send
“Teams don't fail because they lack AI policy — they fail when a confident draft bypasses the 40 seconds of checking that would have caught it.”
An interactive lesson that makes the learner feel the failure first, then gives them the checking behavior: a 3-of-5 risk-hotspot hunt with safe decoys, the 4-rung Verify ladder, and a branching send decision where the lazy path produces a customer-consequence email.
The Audience
Support agents using an AI draft-assistant for customer replies — plus their team lead, who runs the same lesson as a huddle with the included facilitator note.
Responsibilities
Lesson design · hallucination scenario writing · interaction logic (hotspot hunt, ladder, branching) · team guideline authoring · facilitator enablement — the moment-of-send behavior is the deliverable.
The Build
Branching scenario design · hotspot interaction · Verify-ladder framework · guideline writing with war-story citations · AI-assisted drafting (disclosed)
How I used AI in this case
AI drafted hallucination-failure email variants for the send decision; the Verify ladder, hotspot design and gating were hand-built. Drafts were checked and rewritten — the lesson itself teaches checking AI output, so the kit practices what it preaches (disclosed).
The build itself, not a video about it — open it, make the calls, and read what each one costs.
Train the AI, Check the AI — Primary Build
The full lesson: hotspot hunt, Verify ladder, branching send decision
3D lesson (the build itself)
The interactive lesson is a real-time 3D scene: the AI draft hovers over the desk, each real risk plants a pin, and the Verify ladder glows rung by rung. The separate 3D page now redirects here.
Floor-Ready AI Verification One-Pager
The one-page guideline with the four rungs, plus the facilitator note for running the lesson in a huddle.
Why conventional training failed the operational reality
Every AI-literacy artifact reviewed before building this was either a policy PDF or a prompt-engineering demo. Neither touches the actual failure moment. In the anchor incident (ticket NW-4821):
The ticket reopened as an escalation. The agent wasn't careless — the draft read confident, warm, and professional. Confidence is not correctness.
Core diagnostic finding
In this fictional brief, support agents had an AI policy and still sent confident drafts with an invented clause, an invented date and no ticket reference. The gap was the forty seconds of checking at the moment of send, not knowledge of the policy.
Feel the failure first
Illustrative premise: invented policy clause — "Rate Protection Guarantee 4.2" exists nowhere
Design Constraint → Build Rule: Telling agents to check AI drafts changes nothing. The lesson had to let a lazy send produce a visible customer consequence before the checking behavior is taught.
Decoys that do not teach paranoia
Illustrative premise: unverified figures — "€412 by Friday" broke on day 4
Design Constraint → Build Rule: The hotspot hunt needed safe passages — plausible text that is actually fine — or agents would learn to flag everything and trust nothing, which is its own failure.
One lesson, two settings
Illustrative premise: ticket references in the reply — Billing couldn't trace the promise
Design Constraint → Build Rule: Self-paced for agents, huddle-led for team leads. The same scenes had to work alone at a desk and guided by a facilitator with a note.
Check at the moment of send
Performance problem
In this fictional software company, three escalations in a quarter traced back to AI-drafted replies that read well and were wrong: a policy clause that does not exist, a date that came from nowhere, an amount nobody in Billing had confirmed.
Learners and context
Support agents using an AI draft assistant, and the team lead who runs the lesson as an 8-minute huddle with the facilitator note.
Is training the answer? Not on its own
- KB search. If searching the KB takes minutes, rung 1 will be skipped; that is a KB fix.
- Assistant settings. A draft assistant that cites KB articles by ID makes rung 1 a click.
- Ticket template. A reply template with the ticket ID pre-filled removes one of the three risks.
Alignment: every objective has practice, evidence, feedback and a route back to the job
| Objective | Practice | Assessment | Feedback | Transfer |
|---|---|---|---|---|
| LO1 Find the invented and unverified claims in a confident AI draft, and leave the safe lines alone | Scene B: 5 marked passages, 3 real risks | Gate: all 3 risks before the ladder (not scored) | Why each passage is a risk or is safe | The guideline’s rule 1 at the desk |
| LO2 Run the four rungs in order on every draft | Scene C: one rung at a time, options shuffled | Gate: each rung cleared with the verifiable fix | Why the wrong fix fails; the rung glows when cleared | One-page guideline with the four rungs |
| LO3 Send only the verified reply | Scene D: verified vs original | The send decision | The customer’s reopened ticket four days later | #ai-catch tags in real reviews (proposed) |
Evidence register: what is real, what is invented
- Fictional premise Nordwind Software, Vermeulen Transport, ticket NW-4821 and the escalations.
- Design decision The four rungs and their names. The earlier 3D page used other rung names (“Run → Verify → Reflect → Send”) and is retired; it redirects to the lesson.
- Not measured Escalations or review tags after rollout. The earlier page described “scores” the lesson never computed.
Measurable Performance Objectives
Three objectives, each tied to one scene of the lesson.
Find the invented clause, the invented date and amount, and the missing ticket reference in a confident AI draft, and recognise the two safe lines.
Evidence in the build: Scene B gate (all three risks).
Run the four rungs in order: KB match, ticket math, brand voice, owner for money.
Evidence in the build: Scene C, one rung at a time.
Send only the verified reply.
Evidence in the build: Scene D send decision, with the consequence of the original.
Why This Modality Won (Trade-Off Analysis)
Before committing to the final architecture, we evaluated and rejected lower-transfer alternatives against the workplace constraints:
Fast to deploy and audit-friendly. Rejected because it produces exactly the artifact the incidents already disproved: signed awareness. Policy states the rules; it does not build the reflex of checking a confident draft — and confident drafts are the whole threat.
Simple messaging. Rejected because it drives usage underground where no checking happens at all — the opposite of the goal — and forfeits the productivity gains the organization actually wants. The lesson teaches safe use, not abstinence.
An interactive lesson that makes the learner feel the failure first, then gives them the checking behavior: a 3-of-5 risk-hotspot hunt with safe decoys, the 4-rung Verify ladder, and a branching send decision where the lazy path produces a customer-consequence email.
Why consequence first
Feel it, then learn it
The draft is shown confident and polished before anything is flagged. The learner feels how easy it is to send, which is the habit the ladder interrupts.
Why safe decoys
Not everything is a risk
Two of the five marked passages are fine. Flagging empathy or a signature as a risk is the over-correction that makes checking feel pointless.
Why one 3D lesson
One scene, not two
The lesson itself is the 3D scene: an agent’s workstation at dusk where the draft, the knowledge base, the ticket, the brand card and the team lead each sit in their own place, and the ladder visits them in order. A second, separate 3D page of the same ladder was retired.
Production Sequence & Traceable Artifacts
From the moment-of-send behavior backward — each stage links the working document produced at that step.
What would tell us it worked
A proposed plan. Nothing below has been measured.
| Question | Measure / instrument | When | What we would do with the answer |
|---|---|---|---|
| Do agents find the real risks? | Huddle debrief: which risk each agent would have caught before the lesson | First huddle | If the missing ticket reference is rarely caught, add it to the reply template |
| Which rung is slow in practice? | Facilitator question 2: the slowest rung on real tickets | First huddle | A slow rung 1 is a KB gap, not an agent problem |
| Does the habit show in real work? | #ai-catch tags in reviewed replies; AI-invented claims in escalations | Two weeks / one quarter | Target zero invented-claim escalations (proposed) |
Instructional Design Takeaway
“Why this form wins: the problem is a skipped habit, not missing knowledge, so the solution makes the learner feel the failure first and practice the verification ladder until 'check before send' survives a deadline.”
The incidents shared one anatomy: a confident, polished AI draft that skipped the forty seconds of checking that would have caught the fabricated citation or the leaked clause. Policy training was not the gap — the same people who breached the policy had signed it. The proposal was a four-scene interactive lesson that walks the actual arc: feel a failure happen, hunt the risks in a real-shaped draft, learn the verification ladder, then make the send decision under deadline pressure.
Transparent Portfolio Scope
Honest limitation: a fictional company and customer. The lesson gates progress but scores nothing; it is designed to be run in a huddle.