Microsoft Research demonstrated a backdoor attack on an AI agent by directly modifying fewer than 10 dormant neurons. This technique creates "sleeper agents" that behave normally until triggered by specific inputs. The research raises concerns about supply chain risks in quantized or fine-tuned models, as such backdoors could be embedded in publicly distributed weights. If reproducible, this represents a significant attack vector for model poisoning.
Read original
reddit/r/LocalLLaMA