Improved Baselines with Visual Instruction Tuning

5 October 2023

Papers citing "Improved Baselines with Visual Instruction Tuning"

50 / 474 papers shown

Title
Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving Zongchuang Zhao Haoyu Fu Dingkang Liang Xin Zhou Dingyuan Zhang Hongwei Xie Bing Wang Xiang Bai MLLM VLM 49 0 0 13 May 2025
From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation Yifu Yuan Haiqin Cui Yibin Chen Zibin Dong Fei Ni Longxin Kou Jinyi Liu Pengyi Li Yan Zheng Jianye Hao 28 0 0 13 May 2025
Visually Interpretable Subtask Reasoning for Visual Question Answering Yu Cheng A. Goel Hakan Bilen LRM 31 0 0 12 May 2025
Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning Zexian Yang Dian Li Dayan Wu Gang Liu Weiping Wang MLLM LRM 41 0 0 12 May 2025
Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models Bidur Khanal Sandesh Pokhrel Sanjay Bhandari Ramesh Rana Nikesh Shrestha Ram Bahadur Gurung Cristian A. Linte Angus Watson Y. Shrestha Binod Bhattarai VLM 36 0 0 11 May 2025
Visual Instruction Tuning with Chain of Region-of-Interest Yixin Chen Shuai Zhang Boran Han Bernie Wang 26 0 0 11 May 2025
DiffLocks: Generating 3D Hair from a Single Image using Diffusion Models Radu Alexandru Rosu Keyu Wu Yao Feng Youyi Zheng M. Black DiffM 3DH 49 0 0 09 May 2025
Understanding and Mitigating Toxicity in Image-Text Pretraining Datasets: A Case Study on LLaVA Karthik Reddy Kanjula Surya Guthikonda Nahid Alam Shayekh Bin Islam 26 0 0 09 May 2025
UniVLA: Learning to Act Anywhere with Task-centric Latent Actions Qingwen Bu Y. Yang Jisong Cai Shenyuan Gao Guanghui Ren Maoqing Yao Ping Luo Hongyang Li 116 0 0 09 May 2025
The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization Jae-Won Chung Jiachen Liu Jeff J. Ma Ruofan Wu Oh Jun Kweon Yuxuan Xia Zhiyu Wu Mosharaf Chowdhury 28 0 0 09 May 2025
X-Driver: Explainable Autonomous Driving with Vision-Language Models Wei Liu J. A. Zhang Binxiong Zheng Yufeng Hu Yingzhan Lin Zengfeng Zeng VLM LRM 60 0 0 08 May 2025
TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation Haokun Lin Teng Wang Yixiao Ge Yuying Ge Zhichao Lu Ying Wei Qingfu Zhang Zhenan Sun Ying Shan MLLM VLM 70 0 0 08 May 2025
CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation Viacheslav Vasilev V. Arkhipkin Julia Agafonova Tatiana Nikulina Evelina Mironova Alisa Shichanina Nikolai Gerasimenko Mikhail Shoytov Denis Dimitrov 46 0 0 07 May 2025
VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making Jake Grigsby Yuke Zhu Michael S Ryoo Juan Carlos Niebles OffRL VLM 41 0 0 06 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 74 0 0 05 May 2025
MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation Mingcheng Li Xiaolu Hou Ziyang Liu Dingkang Yang Ziyun Qian Jiawei Chen Jinjie Wei Y. Jiang Qingyao Xu L. Zhang DiffM 148 0 0 05 May 2025
Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks Baoxia Du H. Du Dusit Niyato Ruidong Li 58 0 0 05 May 2025
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 83 0 0 05 May 2025
A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models Liqiang Jing Guiming Hardy Chen Ehsan Aghazadeh Xin Eric Wang Xinya Du 50 0 0 04 May 2025
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 56 0 0 03 May 2025
Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos Markos Stamatakis Joshua Berger Christian Wartena Ralph Ewerth Anett Hoppe AI4Ed 41 0 0 03 May 2025
Efficient Vocabulary-Free Fine-Grained Visual Recognition in the Age of Multimodal LLMs Hari Chandana Kuchibhotla Sai Srinivas Kancheti Abbavaram Gowtham Reddy Vineeth N. Balasubramanian 45 0 0 02 May 2025
Dynamic Robot Tool Use with Vision Language Models Noah Trupin Zixing Wang A. H. Qureshi 37 0 0 02 May 2025
ScaleTrack: Scaling and back-tracking Automated GUI Agents Jing Huang Zhixiong Zeng WenKang Han Yufeng Zhong Liming Zheng Shuai Fu Jingyuan Chen Lin Ma 129 0 0 01 May 2025
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models Wufei Ma Luoxin Ye Nessa McWeeney Celso M de Melo A. Yuille Jieneng Chen LRM 65 1 0 01 May 2025
A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI Lik Hang Kenny Wong Xueyang Kang Kaixin Bai Jianwei Zhang 56 0 0 01 May 2025
Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models Sangmin Woo Kang Zhou Yun Zhou Shuai Wang Sheng Guan Haibo Ding Lin Lee Cheong VPVLM 83 0 0 30 Apr 2025
GarmentDiffusion: 3D Garment Sewing Pattern Generation with Multimodal Diffusion Transformers Xinyu Li Qi Yao Y. Wang DiffM 48 0 0 30 Apr 2025
Rethinking Visual Layer Selection in Multimodal LLMs H. Chen Junyan Lin Xinhao Chen Yue Fan Xin Jin Hui Su Jianfeng Dong Jinlan Fu Xiaoyu Shen VLM 95 0 0 30 Apr 2025
Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding Trilok Padhi R. Kaur Adam D. Cobb Manoj Acharya Anirban Roy Colin Samplawski Brian Matejek Alexander M. Berenbeim Nathaniel D. Bastian Susmit Jha 28 0 0 30 Apr 2025
Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception Yuanchen Wu Lu Zhang Hang Yao Junlong Du Ke Yan Shouhong Ding Yunsheng Wu X. Li MLLM 71 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 77 0 0 29 Apr 2025
UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities Woongyeong Yeo Kangsan Kim Soyeong Jeong Jinheon Baek S. Hwang 54 1 0 29 Apr 2025
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers Quentin Guimard Moreno DÍncà Massimiliano Mancini Elisa Ricci SSL 72 0 0 29 Apr 2025
YoChameleon: Personalized Vision and Language Generation Thao Nguyen Krishna Kumar Singh Jing Shi Trung H. Bui Yong Jae Lee Yuheng Li MLLM 82 0 0 29 Apr 2025
CompleteMe: Reference-based Human Image Completion Yu-Ju Tsai Brian L. Price Qing Liu Luis Figueroa D. Pakhomov Zhihong Ding Scott D. Cohen Ming Yang 3DH 50 0 0 28 Apr 2025
Learning Streaming Video Representation via Multitask Training Yibin Yan Jilan Xu Shangzhe Di Yikun Liu Yudi Shi Qirui Chen Zeqian Li Yifei Huang Weidi Xie CLL 84 0 0 28 Apr 2025
Platonic Grounding for Efficient Multimodal Language Models Moulik Choraria Xinbo Wu Akhil Bhimaraju Nitesh Sekhar Yue Wu Xu Zhang Prateek Singhal L. Varshney 54 0 0 27 Apr 2025
CapsFake: A Multimodal Capsule Network for Detecting Instruction-Guided Deepfakes Tuan Nguyen Naseem Khan Issa Khalil AAML 64 0 0 27 Apr 2025
Anyprefer: An Agentic Framework for Preference Data Synthesis Yiyang Zhou Z. Wang Tianle Wang Shangyu Xing Peng Xia ... Chetan Bansal Weitong Zhang Ying Wei Mohit Bansal Huaxiu Yao 61 0 0 27 Apr 2025
Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation Peiyuan Jing Kinhei Lee Zhenxuan Zhang Huichi Zhou Zhengqing Yuan Zhifan Gao Lei Zhu G. Papanastasiou Yingying Fang Guang Yang MedIm OffRL LRM 65 0 0 25 Apr 2025
Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization Kesen Zhao B. Zhu Qianru Sun Hanwang Zhang MLLM LRM 86 0 0 25 Apr 2025
DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models J. Liu Hangyu Guo Ranjie Duan Xingyuan Bu Yancheng He ... Yingshui Tan Yanan Wu Jihao Gu Y. Li J. Zhu MLLM 148 0 0 25 Apr 2025
Token Sequence Compression for Efficient Multimodal Computing Yasmine Omri Parth Shroff Thierry Tambe 53 0 0 24 Apr 2025
DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs Z. Wang Senthil Purushwalkam Caiming Xiong S. Heng Ji R. Xu 38 0 0 23 Apr 2025
FaceInsight: A Multimodal Large Language Model for Face Perception Jingzhi Li Changjiang Luo Ruoyu Chen Hua Zhang Wenqi Ren Jianhou Gan Xiaochun Cao CVBM LRM 65 0 0 22 Apr 2025
Manifold Induced Biases for Zero-shot and Few-shot Detection of Generated Images Jonathan Brokman Amit Giloni Omer Hofman Roman Vainshtein Hisashi Kojima Guy Gilboa 28 2 0 21 Apr 2025
LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation Jiachen Li Qing Xie Xiaohan Yu Hongyun Wang Jinyu Xu Yongjian Liu ObjD 78 0 0 20 Apr 2025
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens Kaihang Pan Wang Lin Zhongqi Yue Tenglong Ao Liyu Jia Wei Zhao Juncheng Billy Li Siliang Tang Hanwang Zhang 49 2 0 20 Apr 2025
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark Enxin Song Wenhao Chai Weili Xu Jianwen Xie Yuxuan Liu Gaoang Wang 62 0 0 20 Apr 2025