Researchers have introduced Fairness Pruning, a lightweight structural intervention designed to localize demographic bias within the Gated Linear Unit (GLU) MLP layers of large language models. By employing minimally contrastive prompt pairs and capturing inference-time activations at the down_proj input, the method identifies specific neurons that exhibit differential responses to demographic attributes. The study validated this approach on models up to 3 billion parameters, including the Llama-3.2 and Salamandra-2B families. Results indicate that zeroing these identified neurons can alter a model's response to demographic variables, though the authors note that using an unsigned BiasScore can lead to bidirectional bias destabilization by mixing neurons that reinforce or counteract stereotypes. Crucially, the intervention is highly surgical; for instance, zeroing fewer than 40 neurons in Llama-3.2-1B—representing less than 0.031% of the total MLP width—retains 99.49% of the model's reasoning and general knowledge capabilities. These findings provide empirical evidence that demographic bias processing and general model capabilities are located within dissociable circuits, establishing a foundation for future research to move from simple neuron zeroing toward more sophisticated, directional modulation of model behavior.

Read original