Conditional diffusion model for infrared and visible image fusion in open environments with few denoising steps

TY - JOUR

T1 - Conditional diffusion model for infrared and visible image fusion in open environments with few denoising steps

AU - Yang, Luojie

AU - Li, Chunming

AU - Yu, Meng

AU - Chen, Guangyan

AU - Yue, Yufeng

N1 - Publisher Copyright: © 2025 Elsevier B.V.

PY - 2026/6

Y1 - 2026/6

N2 - Multi-modality fusion improves perceptual robustness and accuracy by leveraging multi-source sensor data. Current RGB-T fusion methods still falter with adverse illumination and weather. Recent advances in generative methods especially diffusion models have shown the ability to enhance visible images under adverse conditions. However, the fusion of RGB-T still suffer from cross-modal feature loss, sensitivity to environmental interference, and prolonged generation times. These limitations arise due to: (1) difficulties in sufficiently extracting modality-specific information only within shared forward networks; (2) neglecting the interference from adverse weather conditions; (3) the multi-step denoising process in diffusion-based models, which increases temporal cost. To overcome these challenges, we propose a novel conditional diffusion model for RGB-T image fusion, named CDMFusion, which incorporates: (1) a three-branch network designed for fusion to more fully preserve information; (2) a multi-scene adaptive feature enhancer that dynamically enhances valuable features while mitigating interference; (3) a novel skip patrol mechanism enabling high-quality generation via two-step denoising without extra training. Additionally, a new multi-scene RGB-T image dataset and a dataset with multi-interference are released for comprehensive evaluation. Experiments demonstrate our method achieves superior performance across 7 datasets compared to 14 state-of-the-art methods. Code and datasets are at https://github.com/yangluojie/CDM.

AB - Multi-modality fusion improves perceptual robustness and accuracy by leveraging multi-source sensor data. Current RGB-T fusion methods still falter with adverse illumination and weather. Recent advances in generative methods especially diffusion models have shown the ability to enhance visible images under adverse conditions. However, the fusion of RGB-T still suffer from cross-modal feature loss, sensitivity to environmental interference, and prolonged generation times. These limitations arise due to: (1) difficulties in sufficiently extracting modality-specific information only within shared forward networks; (2) neglecting the interference from adverse weather conditions; (3) the multi-step denoising process in diffusion-based models, which increases temporal cost. To overcome these challenges, we propose a novel conditional diffusion model for RGB-T image fusion, named CDMFusion, which incorporates: (1) a three-branch network designed for fusion to more fully preserve information; (2) a multi-scene adaptive feature enhancer that dynamically enhances valuable features while mitigating interference; (3) a novel skip patrol mechanism enabling high-quality generation via two-step denoising without extra training. Additionally, a new multi-scene RGB-T image dataset and a dataset with multi-interference are released for comprehensive evaluation. Experiments demonstrate our method achieves superior performance across 7 datasets compared to 14 state-of-the-art methods. Code and datasets are at https://github.com/yangluojie/CDM.

KW - Deep learning

KW - Diffusion model

KW - Enhancement

KW - Image fusion

KW - Image processing

UR - https://www.scopus.com/pages/publications/105027590541

U2 - 10.1016/j.sigpro.2025.110437

DO - 10.1016/j.sigpro.2025.110437

M3 - Article

AN - SCOPUS:105027590541

SN - 0165-1684

VL - 243

JO - Signal Processing

JF - Signal Processing

M1 - 110437

ER -