Diffusion Language Models (DLMs) face distinct quantization challenges due to their iterative denoising process. This work demonstrates that quantization sensitivity varies across both downstream tasks and denoising timesteps, with each task requiring different precision levels at different steps. Crucially, these sensitivity patterns are predictable without ground-truth labels or task-specific metrics. Leveraging this insight, the authors propose T²Quant, a task- and time-aware mixed-precision quantization framework that employs a label-free sensitivity signal to allocate activation bit-widths optimally across denoising steps under a fixed total bit budget. Evaluated on three DLMs across six benchmarks, T²Quant consistently boosts the average accuracy of baseline quantization methods, delivering improvements of up to 6.7 percentage points while maintaining the same bit-width budget. The approach eliminates the need for task-specific calibration data, enabling practical deployment of quantized DLMs across diverse tasks.

Read original