Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,278 papers shown

Title
VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity Jing Bi Junjia Guo Susan Liang Guangyu Sun Luchuan Song ... Jinxi He Jiarui Wu Ali Vosoughi Chong Chen Chenliang Xu LRM 79 3 0 14 Mar 2025
Direction-Aware Diagonal Autoregressive Image Generation Yijia Xu Jianzhong Ju Jian Luan J. Cui 57 0 0 14 Mar 2025
FastVID: Dynamic Density Pruning for Fast Video Large Language Models Leqi Shen Guoqiang Gong Tao He Yifeng Zhang Pengzhang Liu Sicheng Zhao Guiguang Ding VLM 77 0 0 14 Mar 2025
OmniDiff: A Comprehensive Benchmark for Fine-grained Image Difference Captioning Yong-Jin Liu Saihui Hou Saijie Hou Jiabao Du Shibei Meng Yongzhen Huang VLM 63 0 0 14 Mar 2025
DeskVision: Large Scale Desktop Region Captioning for Advanced GUI Agents Yibin Xu Liang Yang Hao Chen Hua Wang Zhi Chen Yaohua Tang 3DV 65 0 0 14 Mar 2025
Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers Weiming Ren Wentao Ma Huan Yang Cong Wei Ge Zhang Wenhu Chen Mamba 65 4 0 14 Mar 2025
EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks Yi Zhang Qiang Zhang Xiaozhu Ju Ziqiang Liu Jilei Mao ... Jiaxu Wang Yiqun Duan Jiahang Cao Renjing Xu Jian Tang LM&Ro LRM 67 0 0 14 Mar 2025
Safe Vision-Language Models via Unsafe Weights Manipulation Moreno DÍncà E. Peruzzo Xingqian Xu Humphrey Shi N. Sebe Massimiliano Mancini MU 68 0 0 14 Mar 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 66 1 0 14 Mar 2025
Semantic-Clipping: Efficient Vision-Language Modeling with Semantic-Guidedd Visual Selection Bangzheng Li Fei Wang Wenxuan Zhou Nan Xu Ben Zhou Sheng Zhang Hoifung Poon Mengzhao Chen MLLM VLM 89 0 0 14 Mar 2025
Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection Chuhan Zhang Chaoyang Zhu Pingcheng Dong Long Chen Dong Zhang ObjD VLM 251 0 0 14 Mar 2025
SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion A. Nassar Andres Marafioti Matteo Omenetti Maksym Lysak Nikolaos Livathinos ... Yusik Kim A. Said Gurbuz Michele Dolfi Miquel Farré Peter W. J. Staar 61 4 0 14 Mar 2025
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing Rongyao Fang Chengqi Duan Kun Wang Linjiang Huang Hao Li ... Xingyu Zeng R. Zhao Jifeng Dai Xihui Liu Hongsheng Li MLLM ReLM LRM 112 8 0 13 Mar 2025
VisualPRM: An Effective Process Reward Model for Multimodal Reasoning Weiyun Wang Zhangwei Gao L. Chen Zhe Chen Jinguo Zhu ... Lewei Lu Haodong Duan Yu Qiao Jifeng Dai Wenhai Wang LRM 73 12 0 13 Mar 2025
Piece it Together: Part-Based Concepting with IP-Priors Elad Richardson Kfir Goldberg Yuval Alaluf Daniel Cohen-Or DiffM 71 0 0 13 Mar 2025
VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search Yiming Jia Junlong Li Xiang Yue Bo Li Ping Nie Kai Zou Wenhu Chen LRM 79 2 0 13 Mar 2025
UVE: Are MLLMs Unified Evaluators for AI-Generated Videos? Yuanxin Liu Rui Zhu Shuhuai Ren Jiacong Wang Haoyuan Guo Xu Sun Lu Jiang 211 1 0 13 Mar 2025
AudioX: Diffusion Transformer for Anything-to-Audio Generation Zeyue Tian Yizhu Jin Zhaoyang Liu Ruibin Yuan Xu Tan Qifeng Chen Wei Xue Yu Guo 67 3 0 13 Mar 2025
Hybrid Agents for Image Restoration Bingchen Li Xiaochen Li Yiting Lu Zhibo Chen 88 1 0 13 Mar 2025
CINEMA: Coherent Multi-Subject Video Generation via MLLM-Based Guidance Yufan Deng Xun Guo Yuanda Wang Jacob Zhiyuan Fang Angtian Wang Shenghai Yuan Yiding Yang Bo Liu Haibin Huang Chongyang Ma DiffM VGen 77 0 0 13 Mar 2025
Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs Chongjun Tu Peng Ye Dongzhan Zhou Lei Bai Gang Yu Tao Chen Wanli Ouyang 66 0 0 13 Mar 2025
A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1 Zhaoyi Li Xiaohan Zhao Dong-Dong Wu Jiacheng Cui Zhiqiang Shen AAML VLM 77 1 0 13 Mar 2025
Towards Understanding Graphical Perception in Large Multimodal Models Kai Zhang Jianwei Yang J. Inala Chandan Singh Jianfeng Gao Yu Su Chenglong Wang 55 1 0 13 Mar 2025
CalliReader: Contextualizing Chinese Calligraphy via an Embedding-Aligned Vision-Language Model Yuxuan Luo Jiaqi Tang Chenyi Huang Feiyang Hao Zhouhui Lian VLM 63 0 0 13 Mar 2025
The Power of One: A Single Example is All it Takes for Segmentation in VLMs Mir Rayat Imtiaz Hossain Mennatullah Siam Leonid Sigal James J. Little MLLM VLM Presented at ResearchTrend Connect \| VLM on 21 May 2025 102 0 0 13 Mar 2025
Hoi2Anomaly: An Explainable Anomaly Detection Approach Guided by Human-Object Interaction Yuhan Wang Cheng Liu Daou Zhang Weichao Wu 41 0 0 13 Mar 2025
Towards Fast, Memory-based and Data-Efficient Vision-Language Policy Haoxuan Li Sixu Yan Yongqian Li Xinggang Wang LM&Ro 66 0 0 13 Mar 2025
TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention Jinhao Duan Fei Kong Hao-Ran Cheng James Diffenderfer B. Kailkhura Lichao Sun Xiaofeng Zhu Xiaoshuang Shi Kaidi Xu 254 0 0 13 Mar 2025
Traffic Regulation-aware Path Planning with Regulation Databases and Vision-Language Models Xu Han Zhiwen Wu Xin Xia Jiaqi Ma 36 0 0 13 Mar 2025
TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models Xudong Tan Peng Ye Chongjun Tu Jianjian Cao Yaoxin Yang Lin Zhang Dongzhan Zhou Tao Chen VLM 64 0 0 13 Mar 2025
TacticExpert: Spatial-Temporal Graph Language Model for Basketball Tactics Xu Lingrui Liu Mandi Zhang Lei 51 0 0 13 Mar 2025
GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding R. Hu Lianghui Zhu Yuxuan Zhang Tianheng Cheng Lei Liu Heng Liu Longjin Ran Xiaoxin Chen Wenyu Liu Xinggang Wang ObjD 66 0 0 13 Mar 2025
Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding Shunqi Mao Chaoyi Zhang Weidong Cai MLLM 238 0 0 13 Mar 2025
Unveiling the Invisible: Reasoning Complex Occlusions Amodally with AURA Zhixuan Li Hyunse Yoon Sanghoon Lee Weisi Lin 65 0 0 13 Mar 2025
Learning to Inference Adaptively for Multimodal Large Language Models Zhuoyan Xu Khoi Duc Nguyen Preeti Mukherjee Saurabh Bagchi Somali Chaterji Yingyu Liang Yin Li LRM 54 1 0 13 Mar 2025
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Jiaming Liu Hao Chen Pengju An Zhuoyang Liu Renrui Zhang ... Chengkai Hou Mengdi Zhao KC alex Zhou Pheng-Ann Heng Shanghang Zhang 77 9 0 13 Mar 2025
Long-Video Audio Synthesis with Multi-Agent Collaboration Yehang Zhang Xinli Xu Xiaojie Xu L. Liu Yuxiao Chen DiffM VGen 58 0 0 13 Mar 2025
VLog: Video-Language Models by Generative Retrieval of Narration Vocabulary Kevin Qinghong Lin Mike Zheng Shou VGen 255 1 0 12 Mar 2025
Training Data Provenance Verification: Did Your Model Use Synthetic Data from My Generative Model for Training? Yuechen Xie Jie Song Huiqiong Wang Mingli Song 57 0 0 12 Mar 2025
2HandedAfforder: Learning Precise Actionable Bimanual Affordances from Human Videos Marvin Heidinger Snehal Jauhri V. Prasad Georgia Chalvatzaki 73 0 0 12 Mar 2025
Teaching LMMs for Image Quality Scoring and Interpreting Zicheng Zhang H. Wu Ziheng Jia Weisi Lin Guangtao Zhai 73 1 0 12 Mar 2025
Cockatiel: Ensembling Synthetic and Human Preferenced Training for Detailed Video Caption Luozheng Qin Zhiyu Tan Mengping Yang Xiaomeng Yang Hao Li 90 0 0 12 Mar 2025
Revisiting semi-supervised learning in the era of foundation models Ping Zhang Zheda Mai Quang-Huy Nguyen Wei-Lun Chao 52 0 0 12 Mar 2025
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Marianne Arriola Aaron Gokaslan Justin T Chiu Zhihan Yang Zhixuan Qi Jiaqi Han Subham Sekhar Sahoo Volodymyr Kuleshov DiffM 80 6 0 12 Mar 2025
Vi-LAD: Vision-Language Attention Distillation for Socially-Aware Robot Navigation in Dynamic Environments Mohamed Bashir Elnoor K. Weerakoon Gershom Seneviratne Jing Liang Vignesh Rajagopal Dinesh Manocha 60 0 0 12 Mar 2025
Florenz: Scaling Laws for Systematic Generalization in Vision-Language Models Julian Spravil Sebastian Houben Sven Behnke VLM 83 0 0 12 Mar 2025
Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding Haoyu Zhang Qiaohui Chu Meng Liu Yunxiao Wang Bin Wen Fan Yang Yan Li Di Zhang Yaowei Wang Liqiang Nie EgoV 75 0 0 12 Mar 2025
Memory-enhanced Retrieval Augmentation for Long Video Understanding Huaying Yuan Zhengyang Liang Minhao Qin Hongjin Qian Yan Shu Zhicheng Dou Zhicheng Dou VOS RALM VLM 89 1 0 12 Mar 2025
VideoScan: Enabling Efficient Streaming Video Understanding via Frame-level Semantic Carriers Ruanjun Li Yuedong Tan Yuanming Shi Jiawei Shao VLM 213 0 0 12 Mar 2025
SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment Katrin Renz Long Chen Elahe Arani Oleg Sinavski MLLM 73 1 0 12 Mar 2025