Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,253 papers shown

Title
Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions Cunxin Fan Xiaosong Jia Yihang Sun Yixiao Wang Jianglan Wei ... Xiangyu Zhao Masayoshi Tomizuka Xue Yang Junchi Yan Mingyu Ding LM&Ro VLM 69 3 0 04 May 2025
R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation Meng-Hao Guo Jiajun Xu Yi Zhang Jiaxi Song Haoyang Peng ... Yongming Rao Houwen Peng Han Hu Gordon Wetzstein Shi-Min Hu ELM LRM 60 2 0 04 May 2025
RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation Run Ling Wei Wang Yuting Liu G. Guo Linying Jiang Xingwei Wang DiffM 57 0 0 03 May 2025
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 70 0 0 03 May 2025
Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos Markos Stamatakis Joshua Berger Christian Wartena Ralph Ewerth Anett Hoppe AI4Ed 43 0 0 03 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu Lefei Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 72 1 0 02 May 2025
Improving Editability in Image Generation with Layer-wise Memory Daneul Kim Jaeah Lee Jaesik Park DiffM KELM 60 0 0 02 May 2025
Dynamic Robot Tool Use with Vision Language Models Noah Trupin Zixing Wang A. H. Qureshi 42 0 0 02 May 2025
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng Vivian Wang Huayu Wang Huapeng Zhou Yi-Hao Peng ... Wenhao Chai Yi-Ling Chen Vibhav Vineet Qin Cai Lei Li AI4TS 196 0 0 02 May 2025
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding Zongxia Li Xiyang Wu Guangyao Shi Yubin Qin Hongyang Du Tianyi Zhou Dinesh Manocha Jordan Lee Boyd-Graber MLLM 57 0 0 02 May 2025
Robotic Visual Instruction Y. Li Ziyang Gong Yiming Li Xiaoqi Huang Haolan Kang Guangping Bai Xianzheng Ma LM&Ro 76 0 0 01 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Jen-tse Huang Joey Tianyi Zhou AAML MU 99 3 0 01 May 2025
JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers Kwon Byung-Ki Qi Dai Lee Hyoseok Chong Luo Tae-Hyun Oh 73 0 0 01 May 2025
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models Wufei Ma Luoxin Ye Nessa McWeeney Celso M de Melo A. Yuille Jieneng Chen LRM 65 1 0 01 May 2025
ScaleTrack: Scaling and back-tracking Automated GUI Agents Jing Huang Zhixiong Zeng Wenkang Han Yufeng Zhong Liming Zheng Shuai Fu Jingyuan Chen Lin Ma 188 0 0 01 May 2025
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu Kaipeng Zhang Lizhuang Ma Yufei Guo Jun Wang Wenbo Zhang MQ 57 0 0 01 May 2025
Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learning Lang Feng Weihao Tan Zhiyi Lyu Longtao Zheng Haiyang Xu Ming Yan Fei Huang Jingyi Wang 29 0 0 01 May 2025
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT D. Jiang Ziyu Guo Renrui Zhang Zhuofan Zong Hao Li Le Zhuo Shilin Yan Pheng-Ann Heng Yiming Li LRM 72 3 0 01 May 2025
MINERVA: Evaluating Complex Video Reasoning Arsha Nagrani Sachit Menon Ahmet Iscen Shyamal Buch Ramin Mehran ... Yukun Zhu Carl Vondrick Mikhail Sirotenko Cordelia Schmid Tobias Weyand 60 0 0 01 May 2025
Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs Dung Tien Nguyen Minh Khoi Ho Huy Ta T. Nguyen Qi Chen ... Zhibin Liao Minh Nguyen Nhat To Johan W. Verjans Phi Le Nguyen Vu Minh Hieu Phan 34 0 0 30 Apr 2025
Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarks Chaoyi Wang Junjie Zheng Zihao Chen Shiyu Xia Chaofan Ding Xiaohao Zhang Xi Tao Xiaoming He Xinhan Di AuLLM 168 0 0 30 Apr 2025
Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design Vasudev Sharma Ahmed Alagha Abdelhakim Khellaf Vincent Quoc-Huy Trinh Mahdi S. Hosseini 38 0 0 30 Apr 2025
Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models Minh-Hao Van Xintao Wu VLM 88 0 0 30 Apr 2025
COMPACT: COMPositional Atomic-to-Complex Visual Capability Tuning Xindi Wu Hee Seung Hwang Polina Kirichenko Olga Russakovsky VLM CoGe 73 0 0 30 Apr 2025
UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation Linshan Wu Yuxiang Nie Sunan He Jiaxin Zhuang Hao Chen LM&MA MedIm 75 0 0 30 Apr 2025
Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selection Daniel Bogdoll Rajanikant Ananta Abeyankar Giridharan Isabel Moore Gregory Stevens Henry X. Liu VLM 56 0 0 30 Apr 2025
Rethinking Visual Layer Selection in Multimodal LLMs H. Chen Junyan Lin Xinhao Chen Yue Fan Xin Jin Hui Su Jianfeng Dong Jinlan Fu Xiaoyu Shen VLM 95 0 0 30 Apr 2025
Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning Pengxiang Li Zhi Gao Bofei Zhang Yapeng Mi Xiaojian Ma ... Tao Yuan Yuwei Wu Yunde Jia Song-Chun Zhu Qing Li LLMAG 75 0 0 30 Apr 2025
VividListener: Expressive and Controllable Listener Dynamics Modeling for Multi-Modal Responsive Interaction Shiying Li Xingqun Qi Bingkun Yang Chen Weile Zezhao Tian Muyi Sun Qifeng Liu Man Zhang Zhenan Sun 64 0 0 30 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 91 0 0 29 Apr 2025
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers Quentin Guimard Moreno DÍncà Massimiliano Mancini Elisa Ricci SSL 72 0 0 29 Apr 2025
YoChameleon: Personalized Vision and Language Generation Thao Nguyen Krishna Kumar Singh Jing Shi Trung H. Bui Yong Jae Lee Yuheng Li MLLM 82 0 0 29 Apr 2025
CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation Jianyu Wu Yizhou Wang Xiangyu Yue Xinzhu Ma J. Guo Dongzhan Zhou Wanli Ouyang Shixiang Tang 75 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 77 0 0 29 Apr 2025
Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception Yuanchen Wu Lu Zhang Hang Yao Junlong Du Ke Yan Shouhong Ding Yunsheng Wu Xuzhao Li MLLM 73 0 0 29 Apr 2025
CompleteMe: Reference-based Human Image Completion Yu-Ju Tsai Brian L. Price Qing Liu Luis Figueroa D. Pakhomov Zhihong Ding Scott D. Cohen Ming Yang 3DH 52 0 0 28 Apr 2025
VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning Run Luo Renke Shan Longze Chen Ziqiang Liu Lu Wang Min Yang Xiaobo Xia MLLM VLM 99 0 0 28 Apr 2025
Learning Streaming Video Representation via Multitask Training Yibin Yan Jilan Xu Shangzhe Di Yikun Liu Yudi Shi Qirui Chen Zeqian Li Yifei Huang Weidi Xie CLL 84 0 0 28 Apr 2025
PhenoAssistant: A Conversational Multi-Agent AI System for Automated Plant Phenotyping Feng Chen Ilias Stogiannidis Andrew Wood Danilo Bueno Dominic Williams ... Stephen A. Rolfe Tracy Lawson Tony Pridmore M. Giuffrida Sotirios A. Tsaftaris 62 0 0 28 Apr 2025
$RadioFormer: A Multiple-Granularity Radio Map Estimation Transformer with 1\textpertenthousand Spatial Sampling$ RadioFormer: A Multiple-Granularity Radio Map Estimation Transformer with 1\textpertenthousand Spatial Sampling Zheng Fang Kangjun Liu Ke Chen Qingyu Liu J. Zhang Lingyang Song Yaowei Wang 46 0 0 27 Apr 2025
VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs? Mohamed Gado Towhid Taliee Muhammad Memon D. Ignatov Radu Timofte 72 0 0 27 Apr 2025
Platonic Grounding for Efficient Multimodal Language Models Moulik Choraria Xinbo Wu Akhil Bhimaraju Nitesh Sekhar Yue Wu Xu Zhang Prateek Singhal L. Varshney 59 0 0 27 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong Xuzhao Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Tianwei Zhang ALM ELM 96 2 0 26 Apr 2025
Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability Zishen Wan Jiayi Qian Yuhang Du Jason J. Jabbour Yilun Du Yang Katie Zhao A. Raychowdhury Tushar Krishna Vijay Janapa Reddi LM&Ro 93 0 0 26 Apr 2025
HRScene: How Far Are VLMs from Effective High-Resolution Image Understanding? Yusen Zhang Wenliang Zheng Aashrith Madasu Peng Shi Ryo Kamoi ... Ranran Haoran Zhang Avitej Iyer Renze Lou Wenpeng Yin Rui Zhang 68 0 0 25 Apr 2025
Unsupervised Visual Chain-of-Thought Reasoning via Preference Optimization Kesen Zhao B. Zhu Qianru Sun Hanwang Zhang MLLM LRM 86 0 0 25 Apr 2025
A Large Vision-Language Model based Environment Perception System for Visually Impaired People Zezhou Chen Zhaoxiang Liu Ning Wang Kohou Wang Shiguo Lian 55 0 0 25 Apr 2025
Dual Prompting Image Restoration with Diffusion Transformers Dehong Kong Fan Li Zhixin Wang Jiaqi Xu Renjing Pei W. J. Li Wenqi Ren DiffM 69 0 0 24 Apr 2025
Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation Phillip Y. Lee Jihyeon Je Chanho Park Mikaela Angelina Uy Leonidas J. Guibas Minhyuk Sung LRM 46 0 0 24 Apr 2025
FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding De-An Huang Subhashree Radhakrishnan Zhiding Yu Jan Kautz VGen VLM 96 0 0 24 Apr 2025