Title
Vision-Language-Action Models: Concepts, Progress, Applications and Challenges Ranjan Sapkota Yang Cao Konstantinos I. Roumeliotis Manoj Karkee LM&Ro 388 2 0 07 May 2025
An Empirical Study of Autoregressive Pre-training from Videos Jathushan Rajasegaran Ilija Radosavovic Rahul Ravishankar Yossi Gandelsman Christoph Feichtenhofer Jitendra Malik 60 9 0 10 Jan 2025
DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT Xiaotao Hu Wei Yin Mingkai Jia Junyuan Deng Xiaoyang Guo Qian Zhang Xiaoxiao Long Ping Tan VGen 148 14 0 31 Dec 2024
GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control Mariam Hassan Sebastian Stapf Ahmad Rahimi Pedro M B Rezende Yasaman Haghighi ... Mathieu Salzmann Davide Scaramuzza Marc Pollefeys Paolo Favaro Alexandre Alahi VLM VGen 137 12 0 15 Dec 2024
InfinityDrive: Breaking Time Limits in Driving World Models Xi Guo C. Ding Haoxuan Dou Xin Zhang Weixuan Tang Wei Wu VGen 137 5 0 02 Dec 2024
WorldSimBench: Towards Video Generation Models as World Simulators Yiran Qin Zhelun Shi Jiwen Yu Xijun Wang Enshen Zhou ... Lu Sheng Jing Shao Junlin Wu Wanli Ouyang Ruimao Zhang EGVM VGen 197 474 0 23 Oct 2024
Pyramidal Flow Matching for Efficient Video Generative Modeling Yang Jin Zhicheng Sun Ningyuan Li Kun Xu K. Xu ... Nan Zhuang Quzhe Huang Yang Song Yadong Mu Zhouchen Lin VGen 156 86 0 08 Oct 2024
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking D. Dauner Marcel Hallgarten Tianyu Li Xinshuo Weng Zhiyu Huang ... Igor Gilitschenski Boris Ivanovic Marco Pavone Andreas Geiger Kashyap Chitta 102 63 0 21 Jun 2024
CarLLaVA: Vision language models for camera-only closed-loop driving Katrin Renz Long Chen Ana-Maria Marcu Jan Hünermann Benoît Hanotte Alice Karnsund Jamie Shotton Elahe Arani Oleg Sinavski VLM 131 26 0 14 Jun 2024
OpenVLA: An Open-Source Vision-Language-Action Model Moo Jin Kim Karl Pertsch Siddharth Karamcheti Ted Xiao Ashwin Balakrishna ... Russ Tedrake Dorsa Sadigh Sergey Levine Percy Liang Chelsea Finn LM&Ro VLM 255 533 0 13 Jun 2024
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Peize Sun Yi Jiang Shoufa Chen Shilong Zhang Bingyue Peng Ping Luo Zehuan Yuan VLM 121 300 0 10 Jun 2024
Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability Shenyuan Gao Jiazhi Yang Li Chen Kashyap Chitta Yihang Qiu Andreas Geiger Jun Zhang Hongyang Li 142 102 0 27 May 2024
Diffusion for World Modeling: Visual Details Matter in Atari Eloi Alonso Adam Jelley Vincent Micheli Anssi Kanervisto Amos Storkey Tim Pearce Franccois Fleuret 107 69 0 20 May 2024
NeuroNCAP: Photorealistic Closed-loop Safety Testing for Autonomous Driving William Ljungbergh Adam Tonderski Joakim Johnander Holger Caesar Kalle Åström Michael Felsberg Christoffer Petersson 67 25 0 11 Apr 2024
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies Shengding Hu Yuge Tu Xu Han Chaoqun He Ganqu Cui ... Chaochao Jia Guoyang Zeng Dahai Li Zhiyuan Liu Maosong Sun MoE 107 345 0 09 Apr 2024
Generalized Predictive Model for Autonomous Driving Jiazhi Yang Shenyuan Gao Yihang Qiu Li Chen Tianyu Li ... Ping Luo Jun Zhang Andreas Geiger Yu Qiao Hongyang Li VGen 119 76 0 14 Mar 2024
Behavior Generation with Latent Actions Seungjae Lee Yibin Wang Haritheja Etukuru H. J. Kim Mahi Shafiullah Lerrel Pinto VGen OffRL 106 80 0 05 Mar 2024
LMDrive: Closed-Loop End-to-End Driving with Large Language Models Hao Shao Yuxuan Hu Letian Wang Steven L. Waslander Yu Liu Hongsheng Li ELM 100 138 0 12 Dec 2023
Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research Cole Gulino Justin Fu Wenjie Luo George Tucker Eli Bronstein ... Brandyn White Aleksandra Faust R. McAllister Drago Anguelov Benjamin Sapp 86 103 0 12 Oct 2023
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation Lijun Yu José Lezama N. B. Gundavarapu Luca Versari Kihyuk Sohn ... Boqing Gong Ming-Hsuan Yang Irfan Essa David A. Ross Lu Jiang 112 325 0 09 Oct 2023
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 256 179 0 04 Oct 2023
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving Long Chen Oleg Sinavski Jan Hünermann Alice Karnsund Andrew James Willmott Danny Birch Daniel Maund Jamie Shotton MLLM 108 207 0 03 Oct 2023
GPT-Driver: Learning to Drive with GPT Jiageng Mao Yuxi Qian Junjie Ye Hang Zhao Yue Wang LRM 64 239 0 02 Oct 2023
DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model Zhenhua Xu Yujia Zhang Enze Xie Zhen Zhao Yong Guo Kwan-Yee. K. Wong Zhenguo Li Hengshuang Zhao MLLM 110 305 0 02 Oct 2023
GAIA-1: A Generative World Model for Autonomous Driving Masane Fuchi Lloyd Russell Hudson Yeo Zak Murez Hiroto Minami Alex Kendall Tomohiro Takagi Gianluca Corrado VGen 116 251 0 29 Sep 2023
Finite Scalar Quantization: VQ-VAE Made Simple Fabian Mentzer David C. Minnen E. Agustsson Michael Tschannen 98 190 0 27 Sep 2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Xi Chen ... Ted Xiao Peng Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro LRM 198 1,291 0 28 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 413 12,076 0 18 Jul 2023
End-to-end Autonomous Driving: Challenges and Frontiers Li Chen Peng Wu Kashyap Chitta Bernhard Jaeger Andreas Geiger Hongyang Li 3DV 166 309 0 29 Jun 2023
Parting with Misconceptions about Learning-based Vehicle Motion Planning D. Dauner Marcel Hallgarten Andreas Geiger Kashyap Chitta 76 125 0 13 Jun 2023
Towards In-context Scene Understanding Ivana Balazevic David Steiner Nikhil Parthasarathy Relja Arandjelović Olivier J. Hénaff 76 31 0 02 Jun 2023
Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes Jiang-Tian Zhai Ze Feng Jinhao Du Yongqiang Mao Enchao Gong Zichang Tan Yifu Zhang Xiaoqing Ye Jingdong Wang 37 86 0 17 May 2023
Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models A. Blattmann Robin Rombach Huan Ling Tim Dockhorn Seung Wook Kim Sanja Fidler Karsten Kreis 3DGS VGen 219 1,104 0 18 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 571 4,925 0 17 Apr 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 392 3,506 0 14 Apr 2023
Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster Nolan Dey Gurpreet Gosal Zhiming Chen Chen Hemant Khachane William Marshall Ribhu Pathria Marvin Tom Joel Hestness MoE LRM 116 108 0 06 Apr 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 257 1,200 0 27 Mar 2023
VAD: Vectorized Scene Representation for Efficient Autonomous Driving Bo Jiang Shaoyu Chen Qing Xu Bencheng Liao Jiajie Chen Helong Zhou Qian Zhang Wenyu Liu Chang Huang Xinggang Wang 175 241 0 21 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,472 0 27 Feb 2023
Planning-oriented Autonomous Driving Yi Hu Jiazhi Yang Li Chen Keyu Li Chonghao Sima ... Xiaosong Jia Qiang Liu Jifeng Dai Yu Qiao Hongyang Li 92 659 0 20 Dec 2022
MAGVIT: Masked Generative Video Transformer Lijun Yu Yong Cheng Kihyuk Sohn José Lezama Han Zhang ... Alexander G. Hauptmann Ming-Hsuan Yang Yuan Hao Irfan Essa Lu Jiang DiffM VGen 82 248 0 10 Dec 2022
Model-Based Imitation Learning for Urban Driving Anthony Hu Gianluca Corrado Nicolas Griffiths Zak Murez Corina Gurau Hudson Yeo Alex Kendall R. Cipolla Jamie Shotton 174 142 0 14 Oct 2022
Flow Matching for Generative Modeling Y. Lipman Ricky T. Q. Chen Heli Ben-Hamu Maximilian Nickel Matt Le OOD 213 1,389 0 06 Oct 2022
Phenaki: Variable Length Video Generation From Open Domain Textual Description Ruben Villegas Mohammad Babaeizadeh Pieter-Jan Kindermans Hernan Moraldo Han Zhang M. Saffar Santiago Castro Julius Kunze D. Erhan DiffM VGen 139 395 0 05 Oct 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 85 1,434 0 29 Sep 2022
HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator Younggyo Seo Kimin Lee Fangchen Liu Stephen James Pieter Abbeel VGen 59 29 0 15 Sep 2022
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos Bowen Baker Ilge Akkaya Peter Zhokhov Joost Huizinga Jie Tang Adrien Ecoffet Brandon Houghton Raul Sampedro Jeff Clune OffRL 140 303 0 23 Jun 2022
TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving Kashyap Chitta Aditya Prakash Bernhard Jaeger Zehao Yu Katrin Renz Andreas Geiger ViT 176 332 0 31 May 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 227 1,642 0 07 Apr 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 208 1,987 0 29 Mar 2022