Deployment Design for Multi-UAV-Assisted IoT Networks: A Digital Twin-Driven Deep Reinforcement Learning Approach

TY - JOUR

T1 - Deployment Design for Multi-UAV-Assisted IoT Networks

T2 - A Digital Twin-Driven Deep Reinforcement Learning Approach

AU - Zhao, Le

AU - Fei, Zesong

AU - Huang, Jingxuan

AU - Wang, Xinyi

AU - Li, Bin

AU - Yuan, Weijie

N1 - Publisher Copyright: © 2002-2012 IEEE.

PY - 2026

Y1 - 2026

N2 - In this paper, we address the multi-uncrewed aerial vehicles (UAVs)-assisted internet of things network (UAIoTN) in a partially-known 3-D urban environment. With randomly distributed and stationary ground nodes (GNs), the UAIoTN deployment is decoupled into two successive markov decision processes: the mission transfer stage (MTS) and the mission maintaining stage (MMS), forming a Bi-stage deployment (BiSD). We optimize UAV trajectories for data collection via a digital twin (DT)-empowered deep reinforcement learning approach. To achieve this, we propose a DT-driven double deep Q-learning network (DT-DDQN) and construct a novel DT-assisted training framework (DTTF) to enable model pre-training and updating. During mission execution, we enable obstacle avoidance and model the channel deterministically based on link blockage conditions. Furthermore, we streamline multi-UAV deployment through K-means-based mission division, assigning mission sub-regions and their GNs to UAVs, and integrating MTS and MMS for safe and efficient maneuvering and network coverage. Numerical results demonstrate that the proposed DT-DDQN-BiSD, operating in partially-known environments, outperforms baseline methods evaluated under ideal fully-known conditions, in terms of mission execution time, DRL model convergence, and generalization.

AB - In this paper, we address the multi-uncrewed aerial vehicles (UAVs)-assisted internet of things network (UAIoTN) in a partially-known 3-D urban environment. With randomly distributed and stationary ground nodes (GNs), the UAIoTN deployment is decoupled into two successive markov decision processes: the mission transfer stage (MTS) and the mission maintaining stage (MMS), forming a Bi-stage deployment (BiSD). We optimize UAV trajectories for data collection via a digital twin (DT)-empowered deep reinforcement learning approach. To achieve this, we propose a DT-driven double deep Q-learning network (DT-DDQN) and construct a novel DT-assisted training framework (DTTF) to enable model pre-training and updating. During mission execution, we enable obstacle avoidance and model the channel deterministically based on link blockage conditions. Furthermore, we streamline multi-UAV deployment through K-means-based mission division, assigning mission sub-regions and their GNs to UAVs, and integrating MTS and MMS for safe and efficient maneuvering and network coverage. Numerical results demonstrate that the proposed DT-DDQN-BiSD, operating in partially-known environments, outperforms baseline methods evaluated under ideal fully-known conditions, in terms of mission execution time, DRL model convergence, and generalization.

KW - Uncrewed aerial vehicles (UAVs)-assisted network

KW - deep reinforcement learning

KW - digital twin

KW - internet of things

KW - trajectory design

UR - https://www.scopus.com/pages/publications/105013316991

U2 - 10.1109/TWC.2025.3596864

DO - 10.1109/TWC.2025.3596864

M3 - Article

AN - SCOPUS:105013316991

SN - 1536-1276

VL - 25

SP - 2468

EP - 2483

JO - IEEE Transactions on Wireless Communications

JF - IEEE Transactions on Wireless Communications

ER -