This paper addresses misalignment in reinforcement learning for non-verifiable instruction following by proposing policy-aware prompt adaptation, moving beyond static prompt corpora to dynamically adjust prompt difficulty relative to evolving policies. By integrating policy-specific rubrics into reward signals, the method ensures discriminative feedback even when prompts fail. The approach reduces reliance on fixed, potentially mismatched prompts that hinder reward signal recovery. The work is published on arXiv under the identifier 2607.04412