Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,253 papers shown

Title
Mavors: Multi-granularity Video Representation for Multimodal Large Language Model Yang Shi Jiaheng Liu Yushuo Guan Zhikai Wu Yujie Zhang ... Bohan Zeng Wei Zhang Fuzheng Zhang Wenjing Yang Di Zhang VGen VLM 73 0 0 14 Apr 2025
ReasonDrive: Efficient Visual Question Answering for Autonomous Vehicles with Reasoning-Enhanced Small Vision-Language Models Amirhosein Chahe Lifeng Zhou LRM 42 0 0 14 Apr 2025
MIEB: Massive Image Embedding Benchmark Chenghao Xiao Isaac Chung Imene Kerboua Jamie Stirling Xin Zhang Márton Kardos Roman Solomatin Noura Al Moubayed Kenneth Enevoldsen Niklas Muennighoff VLM 42 0 0 14 Apr 2025
VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents Ryota Tanaka Taichi Iki Taku Hasegawa Kyosuke Nishida Kuniko Saito Jun Suzuki VLM 52 2 0 14 Apr 2025
The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer Weixian Lei Jiacong Wang Haochen Wang Xuelong Li Jun Hao Liew Jiashi Feng Zilong Huang 28 2 0 14 Apr 2025
Improving Multimodal Hateful Meme Detection Exploiting LMM-Generated Knowledge Maria Tzelepi Vasileios Mezaris 34 0 0 14 Apr 2025
SilVar-Med: A Speech-Driven Visual Language Model for Explainable Abnormality Detection in Medical Imaging Tan-Hanh Pham Chris Ngo Trong-Duong Bui Minh Luu Quang Tan-Huong Pham Truong-Son Hy 29 1 0 14 Apr 2025
AgMMU: A Comprehensive Agricultural Multimodal Understanding and Reasoning Benchmark Aruna Gauba Irene Pi Yunze Man Ziqi Pang Vikram S. Adve Yu-Xiong Wang 178 0 0 14 Apr 2025
Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? Yanbo Wang Jiyang Guan Jian Liang Ran He 56 0 0 14 Apr 2025
The Mirage of Performance Gains: Why Contrastive Decoding Fails to Address Multimodal Hallucination Hao Yin Gunagzong Si Zilei Wang 197 0 0 14 Apr 2025
LangPert: Detecting and Handling Task-level Perturbations for Robust Object Rearrangement Xu Yin Min-Sung Yoon Yuchi Huo Kang Zhang Sung-eui Yoon 34 0 0 14 Apr 2025
SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model Kaiyu Li Zepeng Xin Li Pang Chao Pang Yupeng Deng Jing Yao Guisong Xia Deyu Meng Zhi Wang Xiangyong Cao VLM LRM 37 0 0 13 Apr 2025
GenEDA: Unleashing Generative Reasoning on Netlist via Multimodal Encoder-Decoder Aligned Foundation Model Wenji Fang Jing Wang Yao Lu Shang Liu Zhiyao Xie AI4CE 42 1 0 13 Apr 2025
BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learning Shengao Wang Arjun Chandra Aoming Liu Venkatesh Saligrama Boqing Gong MLLM VLM 47 0 0 13 Apr 2025
AirVista-II: An Agentic System for Embodied UAVs Toward Dynamic Scene Semantic Understanding Fei Lin Yonglin Tian Tengchao Zhang Jun Huang Sangtian Guan Fei-Yue Wang 43 2 0 13 Apr 2025
SDIGLM: Leveraging Large Language Models and Multi-Modal Chain of Thought for Structural Damage Identification Wenjie Qu Shiyin Wei Yong Huang Yawu Su Shanshan Lu Hui Li AI4CE 31 0 0 12 Apr 2025
Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Models Haotian Ye Himanshu Jain Chong You A. Suresh Haowei Lin James Zou Felix X. Yu 36 0 0 12 Apr 2025
Evolved Hierarchical Masking for Self-Supervised Learning Zhanzhou Feng Shiliang Zhang 49 0 0 12 Apr 2025
PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks Junfei Wu Hao Yang Xinhua Zeng Guibing He Zhengzhang Chen Zhu Li Xinming Zhang Yangyang Ma Run Fang Yang Liu LRM 151 0 0 12 Apr 2025
Mimic In-Context Learning for Multimodal Tasks Yuchu Jiang Jiale Fu Chenduo Hao Xinting Hu Yingzhe Peng Xin Geng Xu Yang 32 0 0 11 Apr 2025
VLMT: Vision-Language Multimodal Transformer for Multimodal Multi-hop Question Answering Qi Zhi Lim C. Lee K. Lim Kalaiarasi Sonai Muthu Anbananthen 31 0 0 11 Apr 2025
AstroLLaVA: towards the unification of astronomical data and natural language Sharaf Zaman Michael J. Smith P. Khetarpal Rishabh Chakrabarty Michele Ginolfi ... Maja Jabłońska Sandor Kruk Matthieu Le Lain Sergio J. Rodríguez Méndez Dimitrios Tanoglidis 31 0 0 11 Apr 2025
PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models M. Dhouib Davide Buscaldi Sonia Vanier A. Shabou VLM 44 1 0 11 Apr 2025
Spatial Audio Processing with Large Language Model on Wearable Devices Ayushi Mishra Yang Bai Priyadarshan Narayanasamy Nakul Garg Nirupam Roy 30 0 0 11 Apr 2025
FindAnything: Open-Vocabulary and Object-Centric Mapping for Robot Exploration in Any Environment Sebastián Barbas Laina Simon Boche Sotiris Papatheodorou Simon Schaefer Jaehyung Jung Stefan Leutenegger 57 0 0 11 Apr 2025
FocalLens: Instruction Tuning Enables Zero-Shot Conditional Image Representations Cheng-Yu Hsieh Pavan Kumar Anasosalu Vasu Fartash Faghri Raviteja Vemulapalli Chun-Liang Li Ranjay Krishna Oncel Tuzel Hadi Pouransari VLM 207 0 0 11 Apr 2025
Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input Jian Wang Rishabh Dabral D. Luvizon Zhe Cao Lingjie Liu Thabo Beeler Christian Theobalt EgoV 50 0 0 11 Apr 2025
Steering CLIP's vision transformer with sparse autoencoders Sonia Joseph Praneet Suresh Ethan Goldfarb Lorenz Hufe Yossi Gandelsman Robert Graham Danilo Bzdok Wojciech Samek Blake A. Richards 56 2 0 11 Apr 2025
F $^3$ Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos Zhaoyu Liu Kan Jiang Murong Ma Zhé Hóu Yun Lin Jin Song Dong 37 0 0 11 Apr 2025
VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning Haozhe Wang C. Qu Zuming Huang Wei Chu Fangzhen Lin Wenhu Chen OffRL ReLM SyDa LRM VLM 80 1 0 10 Apr 2025
Data Metabolism: An Efficient Data Design Schema For Vision Language Model Jingyuan Zhang Hongzhi Zhang Zhou Haonan Chenxi Sun Xingguang Ji Jiakang Wang Fanheng Kong Yong-Jin Liu Qi Wang Fuzheng Zhang VLM 63 1 0 10 Apr 2025
Marmot: Multi-Agent Reasoning for Multi-Object Self-Correcting in Improving Image-Text Alignment Jiayang Sun H. Wang Jie Cao Huaibo Huang Ran He DiffM 76 0 0 10 Apr 2025
Capybara-OMNI: An Efficient Paradigm for Building Omni-Modal Language Models Xingguang Ji Jiakang Wang Hongzhi Zhang Jingyuan Zhang Haonan Zhou Chenxi Sun Yong-Jin Liu Qi Wang Fuzheng Zhang MLLM VLM 65 0 0 10 Apr 2025
SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement Qing Guo Zhiyong Yang Chao Feng Hongjin Lu Linjie Li Chung-Ching Lin Kevin Qinghong Lin Furong Huang Lijuan Wang OODD ReLM VLM LRM 69 1 0 10 Apr 2025
TokenFocus-VQA: Enhancing Text-to-Image Alignment with Position-Aware Focus and Multi-Perspective Aggregations on LVLMs Zijian Zhang Xuhui Zheng X. Wu Chong Peng Xuezhi Cao 37 0 0 10 Apr 2025
SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding Yangliu Hu Zikai Song Na Feng Yawei Luo Junqing Yu Yi-Ping Phoebe Chen Wei Yang 33 0 0 10 Apr 2025
MM-IFEngine: Towards Multimodal Instruction Following Shengyuan Ding Shenxi Wu Xiangyu Zhao Yuhang Zang Haodong Duan Xiaoyi Dong Pan Zhang Yuhang Cao Dahua Lin Jiaqi Wang OffRL 60 2 0 10 Apr 2025
Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understanding Dibyadip Chatterjee Edoardo Remelli Yale Song Bugra Tekin Abhay Mittal ... Shreyas Hampali Eric Sauser Shugao Ma Angela Yao Fadime Sener VLM 51 0 0 10 Apr 2025
How Can Objects Help Video-Language Understanding? Zitian Tang Shijie Wang Junho Cho Jaewook Yoo Chen Sun 45 0 0 10 Apr 2025
VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model Haozhan Shen Peng Liu Jianxin Li Chunxin Fang Yibo Ma ... Zilun Zhang Kangjia Zhao Qianqian Zhang Ruochen Xu Tiancheng Zhao VLM LRM 76 35 0 10 Apr 2025
Perception-R1: Pioneering Perception Policy with Reinforcement Learning En Yu Kangheng Lin Liang Zhao Jisheng Yin Yana Wei ... Zheng Ge Xiangyu Zhang Daxin Jiang Jingyu Wang Wenbing Tao VLM OffRL LRM 40 3 0 10 Apr 2025
Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models Yuxiang Lin Jingdong Sun Zhi-Qi Cheng Jue Wang Haomin Liang Zebang Cheng Yifei Dong Jun-Yan He Xiaojiang Peng Xian-Sheng Hua 52 0 0 10 Apr 2025
VideoExpert: Augmented LLM for Temporal-Sensitive Video Understanding Henghao Zhao Ge-Peng Ji Rui Yan Huan Xiong Zechao Li 29 0 0 10 Apr 2025
LVC: A Lightweight Compression Framework for Enhancing VLMs in Long Video Understanding Ziyi Wang Haoran Wu Yiming Rong Deyang Jiang Yixin Zhang Yue Zhao Shuang Xu Bo Xu VLM 49 0 0 09 Apr 2025
ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Models Seonghwan Park Jaehyeon Jeong Yongjun Kim Jaeho Lee Namhoon Lee VLM 50 1 0 09 Apr 2025
OmniCaptioner: One Captioner to Rule Them All Yiting Lu Jiakang Yuan Zhen Li Jike Zhong Qi Qin ... Lei Bai Zhibo Chen Peng Gao Bo Zhang Peng Gao MLLM 81 0 0 09 Apr 2025
Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuning Ashutosh Chaubey Xulang Guan Mohammad Soleymani CVBM MLLM VLM 77 0 0 09 Apr 2025
Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models Wei Chen Xin Yan Bin Wen Fan Yang Tingting Gao Di Zhang Long Chen MLLM 97 0 0 09 Apr 2025
Perception in Reflection Yana Wei Liang Zhao Kangheng Lin En Yu Yuang Peng ... Jianjian Sun Haoran Wei Zheng Ge Xiangyu Zhang Vishal M. Patel 31 0 0 09 Apr 2025
Are We Done with Object-Centric Learning? Alexander Rubinstein Ameya Prabhu Matthias Bethge Seong Joon Oh OCL 683 0 0 09 Apr 2025