ReEXplore: Improving MLLMs for Embodied Exploration With Contextualized Retrospective Experience Replay

MCML Authors

Gengyuan Zhang

→ Group Volker Tresp
 Database Systems, Data Mining and AI

Mingcong Ding

→ Group Enkelejda Kasneci
 Human-Centered Technologies for Learning

Ruotong Liao

→ Group Volker Tresp
 Database Systems, Data Mining and AI

Volker Tresp

Prof. Dr.

Principal Investigator

Database Systems, Data Mining and AI

Abstract

Embodied exploration is a target-driven process that requires embodied agents to possess fine-grained perception and knowledge-enhanced decision making. While recent attempts leverage MLLMs for exploration due to their strong perceptual and reasoning abilities, we find that MLLM-based embodied agents remain suboptimal in exploring new environments: (i) they rely on profound but stale pre-trained knowledge, (ii) training-based approaches such as imitation learning or reinforcement learning are expensive for long-horizon tasks with sparse outcome rewards, and (iii) frontier-based exploration yields a large, visually nuanced action space that is difficult for MLLMs to make reliable decisions. We address these challenges with ReEXplore, a training-free framework that performs retrospective experience replay to inject distilled, abstract experience at inference time, and hierarchical frontier selection to decompose frontier ranking into coarse-to-fine decisions. Our approach enables robust, traceable, and efficient exploration. Across multiple embodied exploration benchmarks, ReEXplore yields great improvements over strong MLLM baselines, up to 3x higher performance in both success rate and in navigation efficiency under open-source backbones.

misc ZDW+25

---

Preprint

Nov. 2025

---

Authors

G. Zhang • M. Ding • J. Wu • R. Liao • V. Tresp

---

Links

arXiv

---

Research Areas

 A3 | Computational Models

 B3 | Multimodal Perception