Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation

Published in ICML 2026.

Zhixuan Shen

Jiawei Du

Ziyu Guo

Han Luo

University of Leeds / Southwest Jiaotong University

Lilan Peng

Joey Tianyi Zhou

Haonan Luo

Tianrui Li

Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation

Abstract

This work introduces SAGE, a generative experience-driven framework for embodied navigation that trains agents through physics-grounded semantic abstractions rather than relying only on photorealistic simulation. The framework synthesizes sandbox tasks, distills structured embodied experience through reinforcement learning, and transfers learned priors into open-world navigation, improving navigation performance and real-world generalization.

The project studies how embodied agents can plan in simplified, physics-grounded semantic environments and transfer those abstracted experiences to open-world navigation. It was accepted to ICML 2026 on May 1, 2026.

SAGE agent overview

SAGE agent overview. The framework uses a physics-grounded sandbox for self-evolving data generation and policy optimization, then transfers learned priors to open-world embodied navigation.

SAGE framework

SAGE framework. The system operates through Genesis, Evolution, and Navigation phases, connecting sandbox task synthesis, hybrid prompt-augmented policy optimization, and real-world planning and actuation.

Materials

PDF 
@inproceedings{shen2026plan,
  title = {Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation},
  author = {Zhixuan Shen and Jiawei Du and Ziyu Guo and Han Luo and Lilan Peng and Joey Tianyi Zhou and Haonan Luo and Tianrui Li},
  booktitle = {Proceedings of the International Conference on Machine Learning},
  year = {2026}
}