Researchers have introduced a novel framework for "narrow-boundary safety," addressing the challenge of fine-tuning Large Language Models (LLMs) to exhibit context-specific refusal behaviors. Unlike traditional topic-level alignment, this method utilizes an offline self-generated distillation framework designed to distinguish between harmful content and benign queries within the same subject matter. The approach employs controlled topic generation, coverage repair, in-distribution compensation data, and the creation of harmful-benign pairs. To mitigate data scarcity, the authors utilized an "Escalate" method; while single-shot generation left 19.88% of prompts without refusal traces, escalating retries reduced this gap to 0.20%. Experimental results on the Qwen3-8B model demonstrated that training on Escalate-derived data increased target-domain refusal from 9.47% to 84.75% and slashed unsafe-response rates across three benchmarks from 26.26% to 0.14%. However, this came at the cost of increasing XSTest over-refusal from 2.00% to 74.00%. The study highlights that replacing external responses with verified target-model responses can reduce over-refusal from 15.20% to 5.20%, while utilizing boundary-pair data significantly improves usability by reducing comply-side over-refusal on held-out pairs from 32.94% to 4.16% without substantially impacting harmful-side refusal rates (decreasing only from 91.88% to 87.72%).
Read original
huggingface/daily-papers