Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning
Modern robot navigation systems encounter difficulties in diverse and complex indoor environments. Traditional approaches rely on multiple modules with small models or rule-based systems and thus lack adaptability to new environments. To address this, we developed Astra, a comprehensive dual-model architecture, Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a multimodal LLM, processes vision and language inputs to perform self and goal localization using a hybrid topological-semantic graph as the global map, and outperforms traditional visual place recognition methods. Astra-Local, a multitask network, handles local path planning and odometry estimation. Its 4D spatial-temporal encoder, trained through self-supervised learning, generates robust 4D features for downstream tasks. The planning head utilizes flow matching and a novel masked ESDF loss to minimize collision risks for generating local trajectories, and the odometry head integrates multi-sensor inputs via a transformer encoder to predict the relative pose of the robot. Deployed on real in-house mobile robots, Astra achieves high end-to-end mission success rate across diverse indoor environments.
View on arXiv@article{chen2025_2506.06205,
title={ Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning },
author={ Sheng Chen and Peiyu He and Jiaxin Hu and Ziyang Liu and Yansheng Wang and Tao Xu and Chi Zhang and Chongchong Zhang and Chao An and Shiyu Cai and Duo Cao and Kangping Chen and Shuai Chu and Tianwei Chu and Mingdi Dan and Min Du and Weiwei Fang and Pengyou Fu and Junkai Hu and Xiaowei Jiang and Zhaodi Jiang and Fuxuan Li and Jun Li and Minghui Li and Mingyao Li and Yanchang Li and Zhibin Li and Guangming Liu and Kairui Liu and Lihao Liu and Weizhi Liu and Xiaoshun Liu and Yufei Liu and Yunfei Liu and Qiang Lu and Yuanfei Luo and Xiang Lv and Hongying Ma and Sai Ma and Lingxian Mi and Sha Sa and Hongxiang Shu and Lei Tian and Chengzhi Wang and Jiayu Wang and Kaijie Wang and Qingyi Wang and Renwen Wang and Tao Wang and Wei Wang and Xirui Wang and Chao Wei and Xuguang Wei and Zijun Xia and Zhaohao Xiao and Tingshuai Yan and Liyan Yang and Yifan Yang and Zhikai Yang and Zhong Yin and Li Yuan and Liuchun Yuan and Chi Zhang and Jinyang Zhang and Junhui Zhang and Linge Zhang and Zhenyi Zhang and Zheyu Zhang and Dongjie Zhu and Hang Li and Yangang Zhang },
journal={arXiv preprint arXiv:2506.06205},
year={ 2025 }
}