v1v2 (latest)

Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving

3 October 2023

Andrew James Willmott

Jamie Shotton

Papers citing "Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving"

50 / 59 papers shown

Title
AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving Kangan Qian Sicong Jiang Yang Zhong Ziang Luo Zilin Huang ... Yifei Hu Guang Li Guang Chen Hao Ye Lijun Sun LRM 73 1 0 21 May 2025
HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving Zhiwen Chen Bo Leng Zhuoren Li Hanming Deng Guizhe Jin Ran Yu Huanxi Wen 223 0 0 21 May 2025
VERDI: VLM-Embedded Reasoning for Autonomous Driving Bowen Feng Zhiting Mei Baiang Li Julian Ost Roger Girgis Anirudha Majumdar Felix Heide VLM LRM 225 0 0 21 May 2025
The Mirage of Performance Gains: Why Contrastive Decoding Fails to Address Multimodal Hallucination Hao Yin Gunagzong Si Zilei Wang 457 0 0 14 Apr 2025
ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models Hao Yin Guangzong Si Zilei Wang 392 1 0 17 Mar 2025
Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Model Bo-Kai Ruan Hao-Tang Tsui Yung-Hui Li Hong-Han Shuai LM&Ro 146 10 0 20 Feb 2025
Episodic memory in AI agents poses risks that should be studied and mitigated Chad DeChant 129 4 0 20 Jan 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta Lawrence Yunliang Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 234 206 0 17 Jan 2025
Large Language Model-based Decision-making for COLREGs and the Control of Autonomous Surface Vehicles Klinsmann Agyei Pouria Sarhadi W. Naeem 164 0 0 25 Nov 2024
LaVida Drive: Vision-Text Interaction VLM for Autonomous Driving with Token Selection, Recovery and Enhancement Siwen Jiao Yangyi Fang Baoyun Peng Wangqun Chen Bharadwaj Veeravalli 158 4 0 20 Nov 2024
Driving by the Rules: A Benchmark for Integrating Traffic Sign Regulations into Vectorized HD Map Xinyuan Chang Maixuan Xue Xinran Liu Zheng Pan Xing Wei 191 2 0 31 Oct 2024
Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles Zhengming Wang Junli Wang Pengfei Li Zhaohan Li Peng Li Yilun Chen Peng Li Yilun Chen 388 0 0 21 Oct 2024
Relevance-driven Decision Making for Safer and More Efficient Human Robot Collaboration Xiaotong Zhang Dingcheng Huang Kamal Youcef-Toumi 63 2 0 21 Sep 2024
Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Art Neeloy Chakraborty Melkior Ornik Katherine Driggs-Campbell LRM 200 12 0 25 Mar 2024
Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Driving Shahin Atakishiyev Mohammad Salameh Randy Goebel 213 6 0 18 Mar 2024
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 248 178 0 04 Oct 2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Xi Chen ... Ted Xiao Peng Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro LRM 175 1,285 0 28 Jul 2023
ImageBind: One Embedding Space To Bind Them All Rohit Girdhar Alaaeldin El-Nouby Zhuang Liu Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra VLM 158 940 0 09 May 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 569 4,910 0 17 Apr 2023
Textual Explanations for Automated Commentary Driving Marc Alexander Kühn Daniel Omeiza Lars Kunze 52 6 0 12 Apr 2023
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment Yang Liu Dan Iter Yichong Xu Shuohang Wang Ruochen Xu Chenguang Zhu ELM ALM LM&MA 181 1,208 0 29 Mar 2023
ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks Fabrizio Gilardi Meysam Alizadeh M. Kubli AI4MH 118 923 0 27 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,699 0 15 Mar 2023
Is ChatGPT a Good NLG Evaluator? A Preliminary Study Jiaan Wang Yunlong Liang Fandong Meng Zengkui Sun Haoxiang Shi Zhixu Li Jinan Xu Jianfeng Qu Jie Zhou LM&MA ELM ALM AI4MH 131 470 0 07 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,437 0 27 Feb 2023
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey Tianlin Li Guangyao Chen Guangwu Qian Pengcheng Gao Xiaoyong Wei Yaowei Wang Yonghong Tian Wen Gao AI4CE VLM 117 213 0 20 Feb 2023
GPTScore: Evaluate as You Desire Jinlan Fu See-Kiong Ng Zhengbao Jiang Pengfei Liu LM&MA ALM ELM 157 289 0 08 Feb 2023
ADAPT: Action-aware Driving Caption Transformer Bu Jin Xinyi Liu Yupeng Zheng Pengfei Li Hao Zhao Tong Zhang Yuhang Zheng Guyue Zhou Jingjing Liu 124 73 0 01 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 429 4,642 0 30 Jan 2023
Self-Instruct: Aligning Language Models with Self-Generated Instructions Yizhong Wang Yeganeh Kordi Swaroop Mishra Alisa Liu Noah A. Smith Daniel Khashabi Hannaneh Hajishirzi ALM SyDa LRM 142 2,247 0 20 Dec 2022
PlanT: Explainable Planning Transformers via Object-Level Representations Katrin Renz Kashyap Chitta Otniel-Bogdan Mercea A. Sophia Koepke Zeynep Akata Andreas Geiger ViT 94 99 0 25 Oct 2022
Model-Based Imitation Learning for Urban Driving Anthony Hu Gianluca Corrado Nicolas Griffiths Zak Murez Corina Gurau Hudson Yeo Alex Kendall R. Cipolla Jamie Shotton 161 142 0 14 Oct 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,602 0 29 Apr 2022
Reimagining an autonomous vehicle Jeffrey Hawke E. Haibo Vijay Badrinarayanan Alex Kendall 70 12 0 12 Aug 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs Andrew Jaegle Sebastian Borgeaud Jean-Baptiste Alayrac Carl Doersch Catalin Ionescu ... Olivier J. Hénaff M. Botvinick Andrew Zisserman Oriol Vinyals João Carreira MLLM VLM GNN 86 584 0 30 Jul 2021
LanguageRefer: Spatial-Language Model for 3D Visual Grounding Junha Roh Karthik Desingh Ali Farhadi Dieter Fox 70 95 0 07 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 490 10,496 0 17 Jun 2021
What data do we need for training an AV motion planner? Long Chen Lukas Platinsky Stefanie Speichert B. Osinski Oliver Scheel Yawei Ye Hugo Grimmett Luca Del Pero Peter Ondruska 41 13 0 26 May 2021
Explanations in Autonomous Driving: A Survey Daniel Omeiza Helena Webb Marina Jirotka Lars Kunze 63 219 0 09 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021
VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation Jiyang Gao Chen Sun Hang Zhao Yi Shen Dragomir Anguelov Congcong Li Cordelia Schmid 128 814 0 08 May 2020
End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances Marin Toromanoff É. Wirbel Fabien Moutarde OffRL 140 209 0 25 Nov 2019
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI Alejandro Barredo Arrieta Natalia Díaz Rodríguez Javier Del Ser Adrien Bennetot Siham Tabik ... S. Gil-Lopez Daniel Molina Richard Benjamins Raja Chatila Francisco Herrera XAI 130 6,293 0 22 Oct 2019
A Survey of Deep Learning Techniques for Autonomous Driving Sorin Grigorescu Bogdan Trasnea Tiberiu T. Cocias G. Macesanu 3DPC 89 1,401 0 17 Oct 2019
Conditional Driving from Natural Language Instructions Junha Roh Chris Paxton Andrzej Pronobis Ali Farhadi Dieter Fox LM&Ro 56 34 0 16 Oct 2019
Explain Yourself! Leveraging Language Models for Commonsense Reasoning Nazneen Rajani Bryan McCann Caiming Xiong R. Socher ReLM LRM 89 566 0 06 Jun 2019
Attention is not Explanation Sarthak Jain Byron C. Wallace FAtt 148 1,328 0 26 Feb 2019
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst Mayank Bansal A. Krizhevsky A. Ogale OOD 89 741 0 07 Dec 2018
Robot Representation and Reasoning with Knowledge from Reinforcement Learning Keting Lu Shiqi Zhang Peter Stone Xiaoping Chen OffRL 136 18 0 28 Sep 2018
On Offline Evaluation of Vision-based Driving Models Felipe Codevilla Antonio M. López V. Koltun Alexey Dosovitskiy OffRL 71 103 0 13 Sep 2018