EMKG: Embodied Memory Knowledge Graphs for Object-Goal Navigation in Dynamic Open Worlds

Mingyi Li; Hui Liu; Ying Li; Shubo Zhang; Chunle Gao; Xiaokang Ma; Hanqing Hu; Weixin Mao

doi:10.1109/LRA.2026.3655297

EMKG: Embodied Memory Knowledge Graphs for Object-Goal Navigation in Dynamic Open Worlds

Mingyi Li
, Hui Liu
, Ying Li^*
, Shubo Zhang
, Chunle Gao
, Xiaokang Ma
, Hanqing Hu
, Weixin Mao

^*此作品的通讯作者

Beijing Institute of Technology
Beijing University of Posts and Telecommunications
Beijing LimX Dynamics Technology

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Object-Goal Navigation (OGN) in complex domestic environments remains challenging due to spatial memory and semantic uncertainties. To address this, we introduce EMKG, an embodied multimodal memory knowledge graph framework that enables openworld navigation. In contrast to conventional vision-language navigation (VLN) methods that depend on explicit step-by-step instructions, EMKG utilizes a multimodal memory mechanism for object navigation based on semantic retrieval, which supports dynamic perception, semantic reasoning, and adaptive control. Specifically, to bridge quadruped robot memory and cognition, EMKG’s core component, the Dynamic Memory Knowledge Graph (DMKG), integrates visual inputs, 3D spatial coordinates, and target captions to generalize cross-modal feature correspondences. To ensure the knowledge base remains accurate and actionable over time, we design a visual sampling-based retrieval-augmented generation (RAG) updating evaluation strategy that continuously assesses target reachability and incrementally updates the multimodal vector memory. EMKG further employs visual PPO-CLIP adaptive planning, visual pattern recognition, and modality-switching locomotion control to unify multimodal mapping and memory update processes within a single end-to-end pipeline. EMKG establishes stable semantic associations across different modalities and is deployed end-to-end on a quadruped robot platform, supporting low-latency autonomous decision-making and obstacle avoidance. Extensive experiments show that EMKG outperforms baseline methods, achieving an average 6.7–12.6% higher Success Rate (SR) and 7.3–15.6% improved Success-weighted Path Length (SPL) on the Habitat simulator and in physical environments. These results validate EMKG’s efficacy in memory-enhanced semantic reasoning and embodied ObjectNav in domestic open-world environments.

源语言	英语
页（从-至）	4537-4544
页数	8
期刊	IEEE Robotics and Automation Letters
卷	11
期	4
DOI	https://doi.org/10.1109/LRA.2026.3655297
出版状态	已出版 - 2026
已对外发布	是

访问文件

10.1109/LRA.2026.3655297

其它文件与链接

链接到 Scopus 的出版物

引用此

@article{8ef6ad73197c4338ac2ba7b2dac2d574,

title = "EMKG: Embodied Memory Knowledge Graphs for Object-Goal Navigation in Dynamic Open Worlds",

abstract = "Object-Goal Navigation (OGN) in complex domestic environments remains challenging due to spatial memory and semantic uncertainties. To address this, we introduce EMKG, an embodied multimodal memory knowledge graph framework that enables openworld navigation. In contrast to conventional vision-language navigation (VLN) methods that depend on explicit step-by-step instructions, EMKG utilizes a multimodal memory mechanism for object navigation based on semantic retrieval, which supports dynamic perception, semantic reasoning, and adaptive control. Specifically, to bridge quadruped robot memory and cognition, EMKG{\textquoteright}s core component, the Dynamic Memory Knowledge Graph (DMKG), integrates visual inputs, 3D spatial coordinates, and target captions to generalize cross-modal feature correspondences. To ensure the knowledge base remains accurate and actionable over time, we design a visual sampling-based retrieval-augmented generation (RAG) updating evaluation strategy that continuously assesses target reachability and incrementally updates the multimodal vector memory. EMKG further employs visual PPO-CLIP adaptive planning, visual pattern recognition, and modality-switching locomotion control to unify multimodal mapping and memory update processes within a single end-to-end pipeline. EMKG establishes stable semantic associations across different modalities and is deployed end-to-end on a quadruped robot platform, supporting low-latency autonomous decision-making and obstacle avoidance. Extensive experiments show that EMKG outperforms baseline methods, achieving an average 6.7–12.6\% higher Success Rate (SR) and 7.3–15.6\% improved Success-weighted Path Length (SPL) on the Habitat simulator and in physical environments. These results validate EMKG{\textquoteright}s efficacy in memory-enhanced semantic reasoning and embodied ObjectNav in domestic open-world environments.",

keywords = "Retrieval-augmented generation, object-goal navigation, quadruped robot",

author = "Mingyi Li and Hui Liu and Ying Li and Shubo Zhang and Chunle Gao and Xiaokang Ma and Hanqing Hu and Weixin Mao",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.",

year = "2026",

doi = "10.1109/LRA.2026.3655297",

language = "English",

volume = "11",

pages = "4537--4544",

journal = "IEEE Robotics and Automation Letters",

issn = "2377-3766",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "4",

}

TY - JOUR

T1 - EMKG

T2 - Embodied Memory Knowledge Graphs for Object-Goal Navigation in Dynamic Open Worlds

AU - Li, Mingyi

AU - Liu, Hui

AU - Li, Ying

AU - Zhang, Shubo

AU - Gao, Chunle

AU - Ma, Xiaokang

AU - Hu, Hanqing

AU - Mao, Weixin

PY - 2026

Y1 - 2026

N2 - Object-Goal Navigation (OGN) in complex domestic environments remains challenging due to spatial memory and semantic uncertainties. To address this, we introduce EMKG, an embodied multimodal memory knowledge graph framework that enables openworld navigation. In contrast to conventional vision-language navigation (VLN) methods that depend on explicit step-by-step instructions, EMKG utilizes a multimodal memory mechanism for object navigation based on semantic retrieval, which supports dynamic perception, semantic reasoning, and adaptive control. Specifically, to bridge quadruped robot memory and cognition, EMKG’s core component, the Dynamic Memory Knowledge Graph (DMKG), integrates visual inputs, 3D spatial coordinates, and target captions to generalize cross-modal feature correspondences. To ensure the knowledge base remains accurate and actionable over time, we design a visual sampling-based retrieval-augmented generation (RAG) updating evaluation strategy that continuously assesses target reachability and incrementally updates the multimodal vector memory. EMKG further employs visual PPO-CLIP adaptive planning, visual pattern recognition, and modality-switching locomotion control to unify multimodal mapping and memory update processes within a single end-to-end pipeline. EMKG establishes stable semantic associations across different modalities and is deployed end-to-end on a quadruped robot platform, supporting low-latency autonomous decision-making and obstacle avoidance. Extensive experiments show that EMKG outperforms baseline methods, achieving an average 6.7–12.6% higher Success Rate (SR) and 7.3–15.6% improved Success-weighted Path Length (SPL) on the Habitat simulator and in physical environments. These results validate EMKG’s efficacy in memory-enhanced semantic reasoning and embodied ObjectNav in domestic open-world environments.

AB - Object-Goal Navigation (OGN) in complex domestic environments remains challenging due to spatial memory and semantic uncertainties. To address this, we introduce EMKG, an embodied multimodal memory knowledge graph framework that enables openworld navigation. In contrast to conventional vision-language navigation (VLN) methods that depend on explicit step-by-step instructions, EMKG utilizes a multimodal memory mechanism for object navigation based on semantic retrieval, which supports dynamic perception, semantic reasoning, and adaptive control. Specifically, to bridge quadruped robot memory and cognition, EMKG’s core component, the Dynamic Memory Knowledge Graph (DMKG), integrates visual inputs, 3D spatial coordinates, and target captions to generalize cross-modal feature correspondences. To ensure the knowledge base remains accurate and actionable over time, we design a visual sampling-based retrieval-augmented generation (RAG) updating evaluation strategy that continuously assesses target reachability and incrementally updates the multimodal vector memory. EMKG further employs visual PPO-CLIP adaptive planning, visual pattern recognition, and modality-switching locomotion control to unify multimodal mapping and memory update processes within a single end-to-end pipeline. EMKG establishes stable semantic associations across different modalities and is deployed end-to-end on a quadruped robot platform, supporting low-latency autonomous decision-making and obstacle avoidance. Extensive experiments show that EMKG outperforms baseline methods, achieving an average 6.7–12.6% higher Success Rate (SR) and 7.3–15.6% improved Success-weighted Path Length (SPL) on the Habitat simulator and in physical environments. These results validate EMKG’s efficacy in memory-enhanced semantic reasoning and embodied ObjectNav in domestic open-world environments.

KW - Retrieval-augmented generation

KW - object-goal navigation

KW - quadruped robot

UR - https://www.scopus.com/pages/publications/105028209557

U2 - 10.1109/LRA.2026.3655297

DO - 10.1109/LRA.2026.3655297

M3 - Article

AN - SCOPUS:105028209557

SN - 2377-3766

VL - 11

SP - 4537

EP - 4544

JO - IEEE Robotics and Automation Letters

JF - IEEE Robotics and Automation Letters

IS - 4

ER -

EMKG: Embodied Memory Knowledge Graphs for Object-Goal Navigation in Dynamic Open Worlds

摘要

访问文件

其它文件与链接

学术指纹

引用此