The authors of “Agent Mosaic: Recognizing Confidentiality Leaks in Agentic LLM Systems” evaluated the risk of “mosaic” disclosures where LLM agents combine information from multiple sources to infer and reveal sensitive conclusions. Using 30 synthetic case studies spanning cybersecurity, medical, and military domains, they collected 10,800 final answers from three LLM agents in a controlled simulation and 1,440 answers from live agents running in two open‑source harnesses. Under benign queries intended to be answerable from a single document, GPT‑5.6‑Sol—the top‑performing model—leaked forbidden composites in 7.08 % of responses (95 % CI 3.75‑10.67 %) in simulation; live‑harness tests showed higher leakage rates. Introducing visible classification headers and explicit aggregation rules lowered overall disclosure rates, yet unsafe disclosures remained at 32.37 % of simulated answers (95 % CI 28.33‑36.19 %) across all intents. The study also found that increased refusal rates do not inherently protect confidentiality, as refusals can still expose prohibited conclusions. These findings demonstrate that mosaic‑style confidentiality breaches occur at non‑negligible frequencies, underscoring the need for stricter controls on inference and disclosure in agentic systems handling mixed‑clearance data.

Read original