To address the challenges of tactical coordination and ambiguous task allocation in complex air combat simulations, researchers have introduced DRG-MAPPO, a Hierarchical Dynamic Role-Graph Multi-Agent Proximal Policy Optimization framework. This approach mitigates the limitations of conventional flat MARL architectures by integrating graph-based relational modeling with a hierarchical decision-making structure. The system utilizes graph attention mechanisms to construct a relational representation of battlefield entities, effectively extracting critical features from interactions between allies, enemies, and various threats. A high-level policy manages dynamic role assignment, designating tactical responsibilities such as "leader" or "supporter" to ensure structured coordination. These roles, combined with the graph-encoded relational features, condition a low-level policy that executes discrete maneuver actions. To further enhance collaborative behavior, the authors implemented a target-priority auxiliary task designed to encourage emergent tactics like focus-fire. Experimental evaluations indicate that DRG-MAPPO achieves a state-of-the-art win rate of 87% in cooperative air combat scenarios. By combining structured relational modeling with explicit role assignment, the framework successfully improves optimization stability and interpretability, facilitating more sophisticated multi-agent tactical execution compared to standard reinforcement learning methods.

Read original