Researchers Pingyu Wu et al. investigate the fundamental limitations of Large Language Model (LLM) safeguards that rely on copyable context to determine safety. The authors argue that because current safeguards decide whether to fulfill a request before observing its downstream application, they face an inherent dilemma regarding dual-use tasks where benign and malicious intents appear identical. By mathematically separating model capability from the evidence of intent, the study derives a theoretical worst-case floor on attacker assistance that persists when intent evidence is easily replicable. This leads to the identification of a "safety trilemma," asserting that useful capability, reliable safety, and open access cannot simultaneously be achieved. To mitigate these risks, the paper proposes integrating trusted credentials into the interaction loop; such credentials provide hard-to-copy metadata that predicts actual downstream use, thereby addressing the limitations of context-based evidence. The study validates these findings through dual-use evaluations, adaptive attack simulations, and observations from deployed trusted-access programs, concluding that adding non-copyable information is a necessary condition to overcome the theoretical floor of attacker assistance and improve the reliability of LLM safety mechanisms.
Read original
huggingface/daily-papers