Asynchronous cooperative multi-agent deep reinforcement learning for joint RMSA and spectrum defragmentation in optical fiber communication networks

TY - JOUR

T1 - Asynchronous cooperative multi-agent deep reinforcement learning for joint RMSA and spectrum defragmentation in optical fiber communication networks

AU - Zhang, Xiao

AU - Tian, Qinghua

AU - Xin, Xiangjun

AU - Pan, Yiqun

AU - Yao, Haipeng

AU - Wang, Fu

AU - Dong, Ze

AU - Pan, Xiaolong

AU - Zhou, Sitong

AU - Tian, Feng

AU - Gao, Ran

N1 - Publisher Copyright: © 2009-2012 Optica Publishing Group.

PY - 2026/3

Y1 - 2026/3

N2 - The rapid escalation of Internet traffic and increasingly heterogeneous service demands impose stringent requirements on optical networks for dynamic resource scheduling, efficient spectrum utilization, and automated operation. Elastic optical networks (EONs) are regarded as a promising solution, yet their performance remains constrained by two critical challenges: routing, modulation format, and spectrum allocation (RMSA) and spectrum defragmentation (SD). Existing approaches predominantly focus on optimizing one of these tasks, which may lead to limited adaptability and suboptimal network efficiency. To address this gap, we propose an asynchronous cooperative multi-agent deep reinforcement learning framework, termed MADRL-JRASD, for the joint optimization of RMSA and proactive SD. The framework incorporates an RMSA agent with dynamic allocation capability and an SD agent with autonomous decision-making ability, coordinated through an asynchronous architecture that enables adaptive responses to environmental changes. Invalid-action masking and carefully designed reward functions are further integrated to enhance training stability and convergence. Comprehensive evaluations over three representative topologies demonstrate that MADRL-JRASD reduces the blocking probability by up to 81% compared with RMSA heuristics without SD and achieves an 85% reduction in overhead relative to heuristic algorithms combining RMSA and SD that attain similar blocking performance. Moreover, the sensitivity analysis shows that the SD agent improves spectrum utilization and that multi-agent cooperation enhances global decision coordination, while action masking and reward design jointly strengthen the convergence and efficiency of MADRL-JRASD.

AB - The rapid escalation of Internet traffic and increasingly heterogeneous service demands impose stringent requirements on optical networks for dynamic resource scheduling, efficient spectrum utilization, and automated operation. Elastic optical networks (EONs) are regarded as a promising solution, yet their performance remains constrained by two critical challenges: routing, modulation format, and spectrum allocation (RMSA) and spectrum defragmentation (SD). Existing approaches predominantly focus on optimizing one of these tasks, which may lead to limited adaptability and suboptimal network efficiency. To address this gap, we propose an asynchronous cooperative multi-agent deep reinforcement learning framework, termed MADRL-JRASD, for the joint optimization of RMSA and proactive SD. The framework incorporates an RMSA agent with dynamic allocation capability and an SD agent with autonomous decision-making ability, coordinated through an asynchronous architecture that enables adaptive responses to environmental changes. Invalid-action masking and carefully designed reward functions are further integrated to enhance training stability and convergence. Comprehensive evaluations over three representative topologies demonstrate that MADRL-JRASD reduces the blocking probability by up to 81% compared with RMSA heuristics without SD and achieves an 85% reduction in overhead relative to heuristic algorithms combining RMSA and SD that attain similar blocking performance. Moreover, the sensitivity analysis shows that the SD agent improves spectrum utilization and that multi-agent cooperation enhances global decision coordination, while action masking and reward design jointly strengthen the convergence and efficiency of MADRL-JRASD.

UR - https://www.scopus.com/pages/publications/105029969197

U2 - 10.1364/JOCN.579970

DO - 10.1364/JOCN.579970

M3 - Article

AN - SCOPUS:105029969197

SN - 1943-0620

VL - 18

SP - 180

EP - 194

JO - Journal of Optical Communications and Networking

JF - Journal of Optical Communications and Networking

IS - 3

ER -