Decide when AI reply drafts go out without review and when a person checks them, and watch wrong replies and review time trade off.
90%+
3 min
A confidence score says how sure the tool is, not whether it's right. Tap any ticket to read it and its draft.
Example data. 24 tickets and drafts written for this lab, with made-up confidence scores. The review time is your own estimate.
Sent without review
10 of 24
Wrong drafts sent
3
Wrong policy, promise, tone or feature
Needed a person, got the bot
2
Review time
42 min
14 drafts × 3 min
Support inbox: today's 24 tickets
Help desk
Sent by AI10
The customer reads the AI draft as written.
Agent review14
An agent runs the five-point check, fixes or escalates.
Draft has a problem Needs a person% is the tool's confidence in its draft
Ticket #3
Sent by AI
C
Customer
What's your return window?
AI draft replyConfidence 95%
Good news: you can return any item within 60 days for a full refund!
Sent automatically. No one read it first.
What an agent's check finds
The policy says 30 days, not 60.
Problem: wrong policy
The tradeoff across every threshold
Harmful replies sent unchecked4
Tickets a person checks14
Raising the bar cuts harm only by sending more work to people, and below 100% something still slips through: a draft can be confidently wrong, and a grieving or angry customer can get a high-confidence answer. Turn on the escalation rule and watch the red line drop at every threshold.