Data Detection for Oversampled OTSM System With Low-Bit ADCs

TY - JOUR

T1 - Data Detection for Oversampled OTSM System With Low-Bit ADCs

AU - Yu, Xindi

AU - Shen, Wenqian

AU - Chen, Jiaxin

AU - An, Jianping

N1 - Publisher Copyright: © 2025 IEEE.

PY - 2026

Y1 - 2026

N2 - Orthogonal time sequency multiplexing (OTSM) has emerged as a promising modulation scheme for wireless communication in high-mobility scenarios. It achieves a bit error ratio (BER) performance comparable to orthogonal time frequency space (OTFS) modulation at a lower complexity, as the Walsh-Hadamard transform (WHT) avoids multiplication operations. However, existing system models typically assume ideal infinite-precision analog-to-digital converters (ADCs), which entail high power consumption and complex hardware implementation. Therefore, we investigate an oversampled OTSM system with low-bit ADCs to achieve an attractive trade-off between data detection performance and power consumption. We first derive the input-output relationship of the new system under oversampling and analyze the sparsity of the channel matrix in the time domain. Next, we introduce a three-layer generative stochastic network (3L-GSN) structure to reformulate the data detection problem. The first two layers are dedicated to implementing noise whitening operation for solving the noise correlation problem caused by oversampling, while the last layer characterizes the low-bit quantization in a more precise probabilistic manner. To solve the data detection problem within the 3L-GSN structure, we propose a data detector based on the modified time-domain multi-layer vector approximate message passing (MTD-ML-VAMP) algorithm, leveraging the discrete characteristic of the OTSM signal to improve detection performance. Our simulation results demonstrate the superior BER performance of our proposed data detector compared to traditional detectors.

AB - Orthogonal time sequency multiplexing (OTSM) has emerged as a promising modulation scheme for wireless communication in high-mobility scenarios. It achieves a bit error ratio (BER) performance comparable to orthogonal time frequency space (OTFS) modulation at a lower complexity, as the Walsh-Hadamard transform (WHT) avoids multiplication operations. However, existing system models typically assume ideal infinite-precision analog-to-digital converters (ADCs), which entail high power consumption and complex hardware implementation. Therefore, we investigate an oversampled OTSM system with low-bit ADCs to achieve an attractive trade-off between data detection performance and power consumption. We first derive the input-output relationship of the new system under oversampling and analyze the sparsity of the channel matrix in the time domain. Next, we introduce a three-layer generative stochastic network (3L-GSN) structure to reformulate the data detection problem. The first two layers are dedicated to implementing noise whitening operation for solving the noise correlation problem caused by oversampling, while the last layer characterizes the low-bit quantization in a more precise probabilistic manner. To solve the data detection problem within the 3L-GSN structure, we propose a data detector based on the modified time-domain multi-layer vector approximate message passing (MTD-ML-VAMP) algorithm, leveraging the discrete characteristic of the OTSM signal to improve detection performance. Our simulation results demonstrate the superior BER performance of our proposed data detector compared to traditional detectors.

KW - Data detection

KW - low-bit ADCs

KW - noise whitening

KW - orthogonal time sequency multiplexing (OTSM)

KW - oversampling

UR - https://www.scopus.com/pages/publications/105014589906

U2 - 10.1109/TSP.2025.3603612

DO - 10.1109/TSP.2025.3603612

M3 - Article

AN - SCOPUS:105014589906

SN - 1053-587X

VL - 74

SP - 309

EP - 322

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

ER -