The AI drafted "we accept responsibility for the damage" and the claims manager caught it at 5:01 p.m. on a Friday.

The situation

Why "draft-only" is a policy, not a guardrail

The company handles 200–400 shipments per week across regional LTL and last-mile delivery. Damage claims and late-delivery disputes come in through email, a carrier portal, and occasionally a phone call that gets transcribed. The claims team — two people — used to write every response by hand. Six months ago they started using an AI assistant to draft responses. The drafts reference shipment records, delivery timestamps, and proof-of-delivery photos when available.

The company policy says the AI only drafts — it never sends. But the policy is a line in an onboarding document, not a system constraint. The AI has access to the email integration. In testing, nobody checked whether the assistant would attempt to send if a claimant said "please respond immediately" or if the draft queue was configured to auto-release after 24 hours. The answer to both is: it depends on the integration settings that nobody has audited.

More dangerous than auto-send is the language. The assistant is trained to be helpful and empathetic, which means it naturally gravitates toward phrases like "we understand the damage was caused during transit" and "we'd like to offer a resolution of $X." The first is an admission of liability. The second is a settlement offer the claims manager never authorized. The assistant doesn't know the company's exposure threshold, doesn't check whether the claim has been investigated, and doesn't flag disputes involving repeat claimants who may be filing fraudulently. It writes a polished, professional response that could cost the company five figures if it goes out unedited.

The work

What Taz actually tests and builds

Taz writes eval test cases for every boundary the dispute assistant must respect, runs them against the live configuration, and documents every failure with the exact output that broke the rule.

• Draft-only enforcement tests: Scenarios where the assistant is pressured to send immediately — urgent language from the claimant, auto-release timer edge cases, integration configurations that allow send-on-approve. Each test confirms the assistant produces a draft and never triggers the send action, regardless of prompt pressure or system state.
• Liability language eval suite: Test cases with damaged shipments, late deliveries, and missing items. Each case checks whether the assistant uses compliant language ("we are reviewing the details of your claim") or admits fault ("we accept responsibility," "the damage occurred in our facility"). Variants cover multi-party shipments where liability is shared with a subcontractor.
• Settlement amount tests: Cases where the claimant demands a specific dollar amount, cases where the shipment value is known, and cases where the assistant is prompted to "make an offer." Every test confirms the assistant never quotes a number — it defers to the claims manager or references the claims process, not a figure.
• Threshold escalation tests: Disputes above a dollar threshold, disputes involving repeat claimants, disputes with legal language ("attorney," "demand letter," "litigation"). Each test confirms the assistant flags the dispute for human review and does not draft a standard response. Threshold values are configurable in the harness.
• Tone and empathy boundary tests: Cases where professional empathy slides into implied admission — "we're sorry this happened to your shipment" vs. "we're sorry for the damage we caused." Each test grades the response for tone compliance: empathetic without implying fault.

The deliverable

What the company receives

• Eval test suite — 50–70 test cases organized by boundary type (draft-only, liability, settlement, escalation, tone). Each case includes the dispute scenario, the assistant's actual output, pass/fail grade, and the specific language that triggered a failure. Rerunnable after any prompt or model change.
• Failure map — Every failed case with severity ranking: operational risk (auto-send, wrong escalation), financial risk (liability admission, settlement quote), and compliance risk (regulatory language in a regulated lane). Each failure has a recommended fix — prompt change, system configuration, or architecture constraint.
• Guardrail harness — A rerunnable configuration that the company's IT team or AI vendor can execute whenever the assistant changes. The harness re-checks every boundary and produces a pass/fail report without manual case-by-case review.
• Escalation rule matrix — A reference table mapping dispute characteristics (dollar amount, repeat claimant, legal language, multi-party, high-value cargo type) to the required action: standard draft, flagged draft, or human-only. Written for the claims manager, not the developer.
• Integration audit memo — A short review of the email/portal integration confirming whether the "draft-only" constraint is enforced at the system level or only at the prompt level. If it is prompt-only, the memo specifies what configuration change makes it system-enforced.