Distributed Pursuit–Evasion Game Decision-Making Based on Multi-Agent Deep Reinforcement Learning

TY - JOUR

T1 - Distributed Pursuit–Evasion Game Decision-Making Based on Multi-Agent Deep Reinforcement Learning

AU - Lin, Yanghui

AU - Gao, Han

AU - Xia, Yuanqing

N1 - Publisher Copyright: © 2025 by the authors.

PY - 2025/6

Y1 - 2025/6

N2 - Pursuit–evasion games are a fundamental framework for advancing autonomous decision-making and cooperative control in multi-UAV systems. However, the application of reinforcement learning to pursuit–evasion games involving fixed-wing UAVs remains challenging due to constraints, such as minimum velocity, limited turning radius, and high-dimensional continuous action spaces. To address these issues, this paper proposes a method that integrates automatic curriculum learning with multi-agent proximal policy optimization. A self-play mechanism is introduced to simultaneously train both pursuers and evaders, enabling dynamic and adaptive encirclement strategies. In addition, a reward structure specifically tailored for the encirclement task was designed to guide the pursuers in gradually achieving the encirclement of the evader while ensuring their own safety. To further improve training efficiency and convergence, this paper develops a subgame curriculum learning framework that progressively exposes agents to increasingly complex scenarios, facilitating experience accumulation and skill transfer. The simulation results demonstrate that the proposed approach improves learning efficiency and cooperative pursuit performance under realistic fixed-wing UAV dynamics. This work provides a practical and scalable solution for multiple fixed-wing UAV pursuit–evasion missions in complex environments.

AB - Pursuit–evasion games are a fundamental framework for advancing autonomous decision-making and cooperative control in multi-UAV systems. However, the application of reinforcement learning to pursuit–evasion games involving fixed-wing UAVs remains challenging due to constraints, such as minimum velocity, limited turning radius, and high-dimensional continuous action spaces. To address these issues, this paper proposes a method that integrates automatic curriculum learning with multi-agent proximal policy optimization. A self-play mechanism is introduced to simultaneously train both pursuers and evaders, enabling dynamic and adaptive encirclement strategies. In addition, a reward structure specifically tailored for the encirclement task was designed to guide the pursuers in gradually achieving the encirclement of the evader while ensuring their own safety. To further improve training efficiency and convergence, this paper develops a subgame curriculum learning framework that progressively exposes agents to increasingly complex scenarios, facilitating experience accumulation and skill transfer. The simulation results demonstrate that the proposed approach improves learning efficiency and cooperative pursuit performance under realistic fixed-wing UAV dynamics. This work provides a practical and scalable solution for multiple fixed-wing UAV pursuit–evasion missions in complex environments.

KW - curriculum learning

KW - deep reinforcement learning

KW - pursuit–evasion game

KW - unmanned aerial vehicle

UR - https://www.scopus.com/pages/publications/105007667991

U2 - 10.3390/electronics14112141

DO - 10.3390/electronics14112141

M3 - Article

AN - SCOPUS:105007667991

SN - 2079-9292

VL - 14

JO - Electronics (Switzerland)

JF - Electronics (Switzerland)

IS - 11

M1 - 2141

ER -