ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs

Boyu Chang; Qi Wang; Xi Guo; Zhixiong Nan; Yazhou Yao; Tianfei Zhou

doi:10.1609/aaai.v40i4.37258

ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs

Boyu Chang
, Qi Wang
, Xi Guo
, Zhixiong Nan
, Yazhou Yao
, Tianfei Zhou^*

^*此作品的通讯作者

人工智能学院

Beijing Institute of Technology
Chongqing University
Nanjing University of Science and Technology
State Key Laboratory of Environment Characteristics and Effects for Near-space

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

Visual abductive reasoning (VAR) is a challenging task that requires AI systems to infer the most likely explanation for incomplete visual observations. While recent MLLMs develop strong general-purpose multimodal reasoning capabilities, they fall short in abductive inference, as compared to human beings. To bridge this gap, we draw inspiration from the interplay between verbal and pictorial abduction in human cognition, and propose to strengthen abduction of MLLMs by mimicking such dual-mode behavior. Concretely, we introduce AbductiveMLLM comprising of two synergistic components: REASONER and IMAGINER. The REASONER operates in the verbal domain. It first explores a broad space of possible explanations using a blind LLM and then prunes visually incongruent hypotheses based on cross-modal causal alignment. The remaining hypotheses are introduced into the MLLM as targeted priors, steering its reasoning toward causally coherent explanations. The IMAGINER, on the other hand, further guides MLLMs by emulating human-like pictorial thinking. It conditions a text-to-image diffusion model on both the input video and the REASONER ’s output embeddings to “imagine” plausible visual scenes that correspond to verbal explanation, thereby enriching MLLMs’ contextual grounding. The two components are trained jointly in an end-to-end manner. Experiments on standard VAR benchmarks show that AbductiveMLLM achieves state-of-the-art performance, consistently outperforming traditional solutions and advanced MLLMs.

源语言	英语
主期刊名	Proceedings of the AAAI Conference on Artificial Intelligence
编辑	Sven Koenig, Chad Jenkins, Matthew E. Taylor
出版商	Association for the Advancement of Artificial Intelligence
页	2698-2706
页数	9
版本	4
ISBN（印刷版）	9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067, 9781577359067
DOI	https://doi.org/10.1609/aaai.v40i4.37258
出版状态	已出版 - 2026
活动	40th AAAI Conference on Artificial Intelligence, AAAI 2026 - Singapore, 新加坡期限: 20 1月 2026 → 27 1月 2026

出版系列

姓名	Proceedings of the AAAI Conference on Artificial Intelligence
编号	4
卷	40
ISSN（印刷版）	2159-5399
ISSN（电子版）	2374-3468

会议

会议	40th AAAI Conference on Artificial Intelligence, AAAI 2026
国家/地区	新加坡
市	Singapore
时期	20/01/26 → 27/01/26

访问文件

10.1609/aaai.v40i4.37258

其它文件与链接

链接到 Scopus 的出版物

引用此

Chang, B., Wang, Q., Guo, X., Nan, Z., Yao, Y., & Zhou, T. (2026). ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs. 在 S. Koenig, C. Jenkins, & M. E. Taylor (编辑), Proceedings of the AAAI Conference on Artificial Intelligence (4 编辑, 页码 2698-2706). (Proceedings of the AAAI Conference on Artificial Intelligence; 卷 40, 号码 4). Association for the Advancement of Artificial Intelligence. https://doi.org/10.1609/aaai.v40i4.37258

Chang, Boyu ; Wang, Qi ; Guo, Xi 等. / ABDUCTIVEMLLM : Boosting Visual Abductive Reasoning Within MLLMs. Proceedings of the AAAI Conference on Artificial Intelligence. 编辑 / Sven Koenig ; Chad Jenkins ; Matthew E. Taylor. 4. 编辑 Association for the Advancement of Artificial Intelligence, 2026. 页码 2698-2706 (Proceedings of the AAAI Conference on Artificial Intelligence; 4).

@inproceedings{32e2474c15134d6e849d2dd88ba42b13,

title = "ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs",

abstract = "Visual abductive reasoning (VAR) is a challenging task that requires AI systems to infer the most likely explanation for incomplete visual observations. While recent MLLMs develop strong general-purpose multimodal reasoning capabilities, they fall short in abductive inference, as compared to human beings. To bridge this gap, we draw inspiration from the interplay between verbal and pictorial abduction in human cognition, and propose to strengthen abduction of MLLMs by mimicking such dual-mode behavior. Concretely, we introduce AbductiveMLLM comprising of two synergistic components: REASONER and IMAGINER. The REASONER operates in the verbal domain. It first explores a broad space of possible explanations using a blind LLM and then prunes visually incongruent hypotheses based on cross-modal causal alignment. The remaining hypotheses are introduced into the MLLM as targeted priors, steering its reasoning toward causally coherent explanations. The IMAGINER, on the other hand, further guides MLLMs by emulating human-like pictorial thinking. It conditions a text-to-image diffusion model on both the input video and the REASONER {\textquoteright}s output embeddings to “imagine” plausible visual scenes that correspond to verbal explanation, thereby enriching MLLMs{\textquoteright} contextual grounding. The two components are trained jointly in an end-to-end manner. Experiments on standard VAR benchmarks show that AbductiveMLLM achieves state-of-the-art performance, consistently outperforming traditional solutions and advanced MLLMs.",

author = "Boyu Chang and Qi Wang and Xi Guo and Zhixiong Nan and Yazhou Yao and Tianfei Zhou",

note = "Publisher Copyright: {\textcopyright} 2026, Association for the Advancement of Artificial Intelligence. All rights reserved.; 40th AAAI Conference on Artificial Intelligence, AAAI 2026 ; Conference date: 20-01-2026 Through 27-01-2026",

year = "2026",

doi = "10.1609/aaai.v40i4.37258",

language = "English",

isbn = "9781577359067",

series = "Proceedings of the AAAI Conference on Artificial Intelligence",

publisher = "Association for the Advancement of Artificial Intelligence",

number = "4",

pages = "2698--2706",

editor = "Sven Koenig and Chad Jenkins and Taylor, \{Matthew E.\}",

booktitle = "Proceedings of the AAAI Conference on Artificial Intelligence",

edition = "4",

}

Chang, B, Wang, Q, Guo, X, Nan, Z, Yao, Y & Zhou, T 2026, ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs. 在 S Koenig, C Jenkins & ME Taylor (编辑), Proceedings of the AAAI Conference on Artificial Intelligence. 4 编辑, Proceedings of the AAAI Conference on Artificial Intelligence, 号码 4, 卷 40, Association for the Advancement of Artificial Intelligence, 页码 2698-2706, 40th AAAI Conference on Artificial Intelligence, AAAI 2026, Singapore, 新加坡, 20/01/26. https://doi.org/10.1609/aaai.v40i4.37258

ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs. / Chang, Boyu; Wang, Qi; Guo, Xi 等.
Proceedings of the AAAI Conference on Artificial Intelligence. 编辑 / Sven Koenig; Chad Jenkins; Matthew E. Taylor. 4. 编辑 Association for the Advancement of Artificial Intelligence, 2026. 页码 2698-2706 (Proceedings of the AAAI Conference on Artificial Intelligence; 卷 40, 号码 4).

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

TY - GEN

T1 - ABDUCTIVEMLLM

T2 - 40th AAAI Conference on Artificial Intelligence, AAAI 2026

AU - Chang, Boyu

AU - Wang, Qi

AU - Guo, Xi

AU - Nan, Zhixiong

AU - Yao, Yazhou

AU - Zhou, Tianfei

PY - 2026

Y1 - 2026

N2 - Visual abductive reasoning (VAR) is a challenging task that requires AI systems to infer the most likely explanation for incomplete visual observations. While recent MLLMs develop strong general-purpose multimodal reasoning capabilities, they fall short in abductive inference, as compared to human beings. To bridge this gap, we draw inspiration from the interplay between verbal and pictorial abduction in human cognition, and propose to strengthen abduction of MLLMs by mimicking such dual-mode behavior. Concretely, we introduce AbductiveMLLM comprising of two synergistic components: REASONER and IMAGINER. The REASONER operates in the verbal domain. It first explores a broad space of possible explanations using a blind LLM and then prunes visually incongruent hypotheses based on cross-modal causal alignment. The remaining hypotheses are introduced into the MLLM as targeted priors, steering its reasoning toward causally coherent explanations. The IMAGINER, on the other hand, further guides MLLMs by emulating human-like pictorial thinking. It conditions a text-to-image diffusion model on both the input video and the REASONER ’s output embeddings to “imagine” plausible visual scenes that correspond to verbal explanation, thereby enriching MLLMs’ contextual grounding. The two components are trained jointly in an end-to-end manner. Experiments on standard VAR benchmarks show that AbductiveMLLM achieves state-of-the-art performance, consistently outperforming traditional solutions and advanced MLLMs.

AB - Visual abductive reasoning (VAR) is a challenging task that requires AI systems to infer the most likely explanation for incomplete visual observations. While recent MLLMs develop strong general-purpose multimodal reasoning capabilities, they fall short in abductive inference, as compared to human beings. To bridge this gap, we draw inspiration from the interplay between verbal and pictorial abduction in human cognition, and propose to strengthen abduction of MLLMs by mimicking such dual-mode behavior. Concretely, we introduce AbductiveMLLM comprising of two synergistic components: REASONER and IMAGINER. The REASONER operates in the verbal domain. It first explores a broad space of possible explanations using a blind LLM and then prunes visually incongruent hypotheses based on cross-modal causal alignment. The remaining hypotheses are introduced into the MLLM as targeted priors, steering its reasoning toward causally coherent explanations. The IMAGINER, on the other hand, further guides MLLMs by emulating human-like pictorial thinking. It conditions a text-to-image diffusion model on both the input video and the REASONER ’s output embeddings to “imagine” plausible visual scenes that correspond to verbal explanation, thereby enriching MLLMs’ contextual grounding. The two components are trained jointly in an end-to-end manner. Experiments on standard VAR benchmarks show that AbductiveMLLM achieves state-of-the-art performance, consistently outperforming traditional solutions and advanced MLLMs.

UR - https://www.scopus.com/pages/publications/105035020676

U2 - 10.1609/aaai.v40i4.37258

DO - 10.1609/aaai.v40i4.37258

M3 - Conference contribution

AN - SCOPUS:105035020676

SN - 9781577359067

T3 - Proceedings of the AAAI Conference on Artificial Intelligence

SP - 2698

EP - 2706

BT - Proceedings of the AAAI Conference on Artificial Intelligence

A2 - Koenig, Sven

A2 - Jenkins, Chad

A2 - Taylor, Matthew E.

PB - Association for the Advancement of Artificial Intelligence

Y2 - 20 January 2026 through 27 January 2026

ER -

Chang B, Wang Q, Guo X, Nan Z, Yao Y, Zhou T. ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs. 在 Koenig S, Jenkins C, Taylor ME, 编辑, Proceedings of the AAAI Conference on Artificial Intelligence. 4 编辑 Association for the Advancement of Artificial Intelligence. 2026. 页码 2698-2706. (Proceedings of the AAAI Conference on Artificial Intelligence; 4). doi: 10.1609/aaai.v40i4.37258

ABDUCTIVEMLLM: Boosting Visual Abductive Reasoning Within MLLMs

摘要

出版系列

会议

访问文件

其它文件与链接

学术指纹

引用此