The author evaluates a deterministic, slot-filling voice receptionist that books appointments only after a spoken confirmation gate matched by six regex patterns (yes|yeah|yep|correct|go ahead|book it). Across 137 scripted calls — 16 hand-written attack seeds, 99 deterministic mutations, 12 benign near-miss controls, and 8 AgentDojo-style scenarios — three metrics are tracked: attack success rate (ASR), false-refusal rate (FRR), and completion under attack. The gate's security and its usability cost are the same property: any phrasing keeping a trigger word legible succeeded (5/7 confirmation seeds; 55.6% ASR across five text-preserving mutation classes including all_caps, code_fence, fake_delimiters, scramble, and split_payload), while all five encoded classes (base64_wrap, rot13_wrap, leetspeak, zwsp_insert, homoglyph) scored 0% ASR solely because the gate could not read the mangled trigger. Benign FRR was 35.3% (6/17), refusing natural confirmations like "sounds good" and "go for it". Four of five attack scenarios succeeded: bystander "yes, book it" confirmed bookings (transcript lacks speaker labels), and poisoned business data read back verbatim planted false "already confirmed" claims (3/3 indirect seeds). Order status was disclosed with no verification. The most common failure was not an attack: 33 runs (24%) crashed with KeyError: 'idle' when unparseable utterances hit the idle state with no fallback. Silent misparses also occurred: "the day after tomorrow" booked tomorrow, and bundled amendments ("yes, but change the name") were dropped silently. Structural defenses held — no phrasing skipped the confirm state — but edge failures dominated: whose voice counts, what the store says, and off-script crashes.
Read original
dev.to