LMDrive: Closed-Loop End-to-End Driving with Large Language Models

12 December 2023

Steven L. Waslander

Papers citing "LMDrive: Closed-Loop End-to-End Driving with Large Language Models"

50 / 56 papers shown

Title
Sky-Drive: A Distributed Multi-Agent Simulation Platform for Human-AI Collaborative and Socially-Aware Future Transportation Zilin Huang Zihao Sheng Zhengyang Wan Yansong Qu Yuhao Luo ... Jiancong Chen Keke Long Jiayi Meng Yue Leng Sikai Chen 99 0 0 25 Apr 2025
Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios Jingzheng Li Xianglong Liu Shikui Wei Zhijun Chen Yangqiu Song Qing Guo Xianqi Yang Yanjun Pu Jiakai Wang AAML ELM 113 0 0 31 Mar 2025
VisRL: Intention-Driven Visual Perception via Reinforced Reasoning Zhangquan Chen Xufang Luo Dongsheng Li OffRL LRM 104 3 0 10 Mar 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta Lawrence Yunliang Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 171 195 0 17 Jan 2025
Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning Ji Hyeok Jung Eun Tae Kim S. Kim Joo Ho Lee Bumsoo Kim Buru Chang VLM 413 1 0 24 Nov 2024
CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM Jingwei Xu Chenyu Wang Zibo Zhao Wen Liu Yi-An Ma Shenghua Gao 79 17 0 07 Nov 2024
SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Prediction Yang Zhou Hao Shao Letian Wang Steven Waslander Hongsheng Li Yu Liu 60 2 0 11 Oct 2024
SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Prediction Yang Zhou Hao Shao Letian Wang Steven L. Waslander Hongsheng Li Yu Liu 61 16 0 18 Mar 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 168 112 0 08 Feb 2024
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 108 2,672 0 05 Oct 2023
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 225 173 0 04 Oct 2023
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving Long Chen Oleg Sinavski Jan Hünermann Alice Karnsund Andrew James Willmott Danny Birch Daniel Maund Jamie Shotton MLLM 77 200 0 03 Oct 2023
GPT-Driver: Learning to Drive with GPT Jiageng Mao Yuxi Qian Junjie Ye Hang Zhao Yue Wang LRM 51 232 0 02 Oct 2023
DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model Zhenhua Xu Yujia Zhang Enze Xie Zhen Zhao Yong Guo Kwan-Yee. K. Wong Zhenguo Li Hengshuang Zhao MLLM 67 290 0 02 Oct 2023
DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving Xiaosong Jia Yulu Gao Li Chen Junchi Yan Patrick Langechuan Liu Hongyang Li 51 72 0 01 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 267 11,791 0 18 Jul 2023
End-to-end Autonomous Driving: Challenges and Frontiers Li Chen Peng Wu Kashyap Chitta Bernhard Jaeger Andreas Geiger Hongyang Li 3DV 97 296 0 29 Jun 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 78 631 0 27 Jun 2023
ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling Quanyi Li Zhenghao Peng Lan Feng Zhizheng Liu Chenda Duan Wen-An Mo Bolei Zhou 52 48 0 21 Jun 2023
Hidden Biases of End-to-End Driving Models Bernhard Jaeger Kashyap Chitta Andreas Geiger 55 66 0 13 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 314 4,288 0 09 Jun 2023
ReasonNet: End-to-End Driving with Temporal and Global Reasoning Hao Shao Letian Wang Ruobing Chen Steven L. Waslander Hongsheng Li Y. Liu LRM 69 78 0 17 May 2023
Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving Xiaosong Jia Peng Wu Li Chen Jiangwei Xie Conghui He Junchi Yan Hongyang Li 72 103 0 10 May 2023
Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors Letian Wang Jie Liu Hao Shao Wenshuo Wang Ruobing Chen Y. Liu Steven L. Waslander 25 21 0 08 May 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 126 2,007 0 20 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 437 4,715 0 17 Apr 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.3K 13,100 0 27 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 401 4,527 0 30 Jan 2023
Planning-oriented Autonomous Driving Yi Hu Jiazhi Yang Li Chen Keyu Li Chonghao Sima ... Xiaosong Jia Qiang Liu Jifeng Dai Yu Qiao Hongyang Li 73 631 0 20 Dec 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 167 3,110 0 20 Oct 2022
TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios Lan Feng Quanyi Li Zhenghao Peng Shuhan Tan Bolei Zhou 65 87 0 12 Oct 2022
Motion Transformer with Global Intention Localization and Local Movement Refinement Shaoshuai Shi Li Jiang Dengxin Dai Bernt Schiele 81 236 0 27 Sep 2022
Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills Tong Zhou Letian Wang Ruobing Chen Wenshuo Wang Y. Liu 48 12 0 24 Sep 2022
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer Hao Shao Letian Wang Ruobing Chen Hongsheng Li Y. Liu 82 204 0 28 Jul 2022
Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline Peng Wu Xiaosong Jia Li Chen Junchi Yan Hongyang Li Yu Qiao 73 195 0 16 Jun 2022
TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving Kashyap Chitta Aditya Prakash Bernhard Jaeger Zehao Yu Katrin Renz Andreas Geiger ViT 148 317 0 31 May 2022
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers Zhiqi Li Wenhai Wang Hongyang Li Enze Xie Chonghao Sima Tong Lu Qiao Yu Jifeng Dai 118 1,286 0 31 Mar 2022
Learning from All Vehicles Dian Chen Philipp Krahenbuhl 90 182 0 22 Mar 2022
End-to-End Urban Driving by Imitating a Reinforcement Learning Coach Zhejun Zhang Alexander Liniger Dengxin Dai Feng Yu Luc Van Gool 78 207 0 18 Aug 2021
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving Aditya Prakash Kashyap Chitta Andreas Geiger ViT 103 525 0 19 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 826 29,341 0 26 Feb 2021
Implicit Latent Variable Model for Scene-Consistent Motion Forecasting Sergio Casas Cole Gulino Simon Suo Katie Z Luo Renjie Liao R. Urtasun 192 158 0 23 Jul 2020
Center-based 3D Object Detection and Tracking Tianwei Yin Xingyi Zhou Philipp Krahenbuhl 3DPC 95 1,602 0 19 Jun 2020
Temporal Interlacing Network Hao Shao Shengju Qian Yu Liu 56 93 0 17 Jan 2020
Learning by Cheating Dian Chen Brady Zhou V. Koltun Philipp Krahenbuhl SSL 88 515 0 27 Dec 2019
End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances Marin Toromanoff É. Wirbel Fabien Moutarde OffRL 113 208 0 25 Nov 2019
PointPillars: Fast Encoders for Object Detection from Point Clouds Alex H. Lang Sourabh Vora Holger Caesar Lubing Zhou Jiong Yang Oscar Beijbom 3DPC 71 3,441 0 14 Dec 2018
SlowFast Networks for Video Recognition Christoph Feichtenhofer Haoqi Fan Jitendra Malik Kaiming He 162 3,262 0 10 Dec 2018
Frustum PointNets for 3D Object Detection from RGB-D Data C. Qi Wen Liu Chenxia Wu Hao Su Leonidas Guibas 3DPC 147 2,259 0 22 Nov 2017
CARLA: An Open Urban Driving Simulator Alexey Dosovitskiy G. Ros Felipe Codevilla Antonio M. López V. Koltun VLM 133 5,146 0 10 Nov 2017