Multi-Agent Reinforcement Learning (MARL) has become a key paradigm for decision‑making in autonomous systems and air combat. Although MARL shows promise in aerial engagements, achieving sophisticated tactical coordination remains difficult. Two main limitations are identified: (1) the lack of structured relational modeling prevents agents from capturing time‑varying interactions among battlefield entities; (2) conventional flat architectures do not explicitly model tactical roles, leading to ambiguous task allocation in highly dynamic environments. To overcome these issues, we introduce Hierarchical Dynamic Role‑Graph Multi‑Agent Proximal Policy Optimization (DRG‑MAPPO), which bridges graph‑based relational modeling and dynamic role assignment. First, a battlefield interaction graph is built where allies, enemies and threats are nodes; a graph attention network extracts critical relational features. Next, a high‑level policy employs a dynamic role‑assignment mechanism to allocate tactical responsibilities such as "leader" and "supporter" to each agent. Conditioned on the assigned role and the encoded graph features, a low‑level policy outputs discrete maneuver actions, enabling joint optimization of tactical strategy and collaborative execution. An auxiliary target‑priority task is also designed to encourage behaviors like focus‑fire. Experiments demonstrate that DRG‑MAPPO achieves an 87% win rate in cooperative air combat, balancing relational modeling, interpretability and optimization stability.
Review: This approach leverages graph attention and hierarchical role assignment to markedly improve coordinated air combat performance, highlighting the importance of structured relational modeling in MARL.