Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,278 papers shown

Title
MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency Dongzhi Jiang Renrui Zhang Ziyu Guo Yanwei Li Yu Qi ... Shen Yan Bo Zhang Chaoyou Fu Peng Gao Hongsheng Li MLLM LRM 98 23 0 13 Feb 2025
Human-Centric Foundation Models: Perception, Generation and Agentic Modeling Shixiang Tang Yunhong Wang Lu Chen Yuan Wang Sida Peng Dan Xu W. Ouyang VGen 143 2 0 12 Feb 2025
Handwritten Text Recognition: A Survey Carlos Garrido-Munoz Antonio Ríos-Vila Jorge Calvo-Zaragoza 106 0 0 12 Feb 2025
I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models Zhenxing Mi Kuan-Chieh Wang Guocheng Qian Hanrong Ye Runtao Liu Sergey Tulyakov Kfir Aberman Dan Xu LRM 54 0 0 12 Feb 2025
DeepSeek on a Trip: Inducing Targeted Visual Hallucinations via Representation Vulnerabilities Chashi Mahiul Islam Samuel Jacob Chacko Preston Horne Xiuwen Liu 112 1 0 11 Feb 2025
Imit Diff: Semantics Guided Diffusion Transformer with Dual Resolution Fusion for Imitation Learning Yuhang Dong Haizhou Ge Yupei Zeng Jingyang Zhang Beiwen Tian ... Yufei Jia Ruixiang Wang Ran Yi Guyue Zhou Longhua Ma 61 0 0 11 Feb 2025
Sparse Autoencoders for Scientifically Rigorous Interpretation of Vision Models Samuel Stevens Wei-Lun Chao T. Berger-Wolf Yu-Chuan Su VLM 79 2 0 10 Feb 2025
CoS: Chain-of-Shot Prompting for Long Video Understanding Jian Hu Zixu Cheng Chenyang Si Wei Li Shaogang Gong 57 4 0 10 Feb 2025
Deciphering Functions of Neurons in Vision-Language Models Jiaqi Xu Cuiling Lan Xuejin Chen Yan Lu VLM 107 0 0 10 Feb 2025
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths Weijia Mao Zheng Yang Mike Zheng Shou MoE 85 0 0 10 Feb 2025
Dual Caption Preference Optimization for Diffusion Models Amir Saeidi Yiran Luo Agneet Chatterjee Shamanthak Hegde Bimsara Pathiraja Yezhou Yang Chitta Baral DiffM 65 0 0 09 Feb 2025
DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control Junjie Wen Bo Li Jinming Li Zhibin Tang Yaxin Peng Feifei Feng VLM 66 14 0 09 Feb 2025
Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails Yijun Yang L. Wang Xiao Yang Lanqing Hong Jun Zhu AAML 66 0 0 09 Feb 2025
HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation Yi Li Yuquan Deng Jingyang Zhang Joel Jang Marius Memme ... Fabio Ramos Dieter Fox Anqi Li Abhishek Gupta Ankit Goyal LM&Ro 102 10 0 08 Feb 2025
Survey on AI-Generated Media Detection: From Non-MLLM to MLLM Yueying Zou Peipei Li Zekun Li Huaibo Huang Xing Cui Xuannan Liu Chenghanyu Zhang Ran He DeLMO 132 3 0 07 Feb 2025
Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More Feng Wang Yaodong Yu Guoyizhe Wei Wei Shao Yuyin Zhou Alan Yuille Cihang Xie ViT 101 4 0 06 Feb 2025
PixFoundation: Are We Heading in the Right Direction with Pixel-level Vision Foundation Models? Mennatullah Siam VLM 89 1 0 06 Feb 2025
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion Marco Mistretta Alberto Baldrati Lorenzo Agnolucci Marco Bertini Andrew D. Bagdanov CLIP VLM 109 2 0 06 Feb 2025
Controllable Satellite-to-Street-View Synthesis with Precise Pose Alignment and Zero-Shot Environmental Control Xianghui Ze Zhenbo Song Qiwei Wang Jianfeng Lu Yujiao Shi 70 0 0 05 Feb 2025
Rethinking Homogeneity of Vision and Text Tokens in Large Vision-and-Language Models Chia-Wen Kuo Sijie Zhu Fan Chen Xiaohui Shen Longyin Wen VLM 65 1 0 04 Feb 2025
Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models Mingi Jung Saehuyng Lee Eunji Kim Sungroh Yoon 73 0 0 03 Feb 2025
Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models H. Malik Fahad Shamshad Muzammal Naseer Karthik Nandakumar Fahad Shahbaz Khan Salman Khan AAML MLLM VLM 75 0 0 03 Feb 2025
Al-Khwarizmi: Discovering Physical Laws with Foundation Models Christopher E. Mower Haitham Bou-Ammar AI4CE 84 1 0 03 Feb 2025
Hypo3D: Exploring Hypothetical Reasoning in 3D Ye Mao Weixun Luo Junpeng Jing Anlan Qiu K. Mikolajczyk 90 0 0 02 Feb 2025
VLM-Assisted Continual learning for Visual Question Answering in Self-Driving Yuxin Lin Mengshi Qi Liang Liu Huadong Ma CLL 51 1 0 02 Feb 2025
Beyond Token Compression: A Training-Free Reduction Framework for Efficient Visual Processing in MLLMs Hongliang Li Jiaxin Zhang Wenhui Liao Dezhi Peng Kai Ding Lianwen Jin OffRL MQ 80 0 0 31 Jan 2025
Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation Bin Zhu Hui yan Qi Yinxuan Gui Jingjing Chen Chong-Wah Ngo Ee-Peng Lim 214 1 0 31 Jan 2025
A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approaches Luca Ciampi Ali Azmoudeh Elif Ecem Akbaba Erdi Sarıtaş Ziya Ata Yazıcı H. K. Ekenel Giuseppe Amato Fabrizio Falchi 105 0 0 31 Jan 2025
Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation Lin Chen Qi Yang Kun Ding Zhu Li Gang Shen Fei Li Qiyuan Cao Shiming Xiang VLM 64 0 0 29 Jan 2025
Mobile Manipulation Instruction Generation from Multiple Images with Automatic Metric Enhancement Kei Katsumata Motonari Kambara Daichi Yashima Ryosuke Korekata Komei Sugiura 70 0 0 28 Jan 2025
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Min Zhang LM&MA AILaw 107 157 0 28 Jan 2025
TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data Jeremy Irvin Emily Ruoyu Liu Joyce Chuyi Chen Ines Dormoy Jinyoung Kim Samar Khanna Zhuo Zheng Stefano Ermon MLLM VLM 65 6 0 28 Jan 2025
Audio-Language Models for Audio-Centric Tasks: A survey Yi Su Jisheng Bai Qisheng Xu Kele Xu Yong Dou AuLLM 99 2 0 28 Jan 2025
Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference Zhihang Lin Mingbao Lin Luxi Lin Rongrong Ji 61 17 0 28 Jan 2025
Scaling Large Vision-Language Models for Enhanced Multimodal Comprehension In Biomedical Image Analysis Robinson Umeike N. Getty Fangfang Xia Rick L. Stevens 45 2 0 28 Jan 2025
Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data Jiajie Li Brian R Quaranto Chenhui Xu Ishan Mishra Ruiyang Qin Dancheng Liu Peter C W Kim Jinjun Xiong 99 0 0 25 Jan 2025
MPG-SAM 2: Adapting SAM 2 with Mask Priors and Global Context for Referring Video Object Segmentation Fu Rong Meng Lan Qian Zhang Lefei Zhang VOS VGen 78 1 0 23 Jan 2025
DynamicEarth: How Far are We from Open-Vocabulary Change Detection? Kaiyu Li Xiangyong Cao Yupeng Deng Chao Pang Zepeng Xin Deyu Meng Zhi Wang ObjD 86 1 0 22 Jan 2025
PSGSL: A Probabilistic Framework Integrating Semantic Scene Understanding and Gas Sensing for Gas Source Localization Pepe Ojeda J. Monroy Javier González Jiménez 39 0 0 22 Jan 2025
Patent Figure Classification using Large Vision-language Models Sushil Awale Eric Müller-Budack Ralph Ewerth 43 0 0 22 Jan 2025
ViDDAR: Vision Language Model-Based Task-Detrimental Content Detection for Augmented Reality Yanming Xiu T. Scargill M. Gorlatova 77 2 0 22 Jan 2025
InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling Yi Wang Xinhao Li Ziang Yan Yinan He Jiashuo Yu ... Kai Chen Wenhai Wang Yu Qiao Yali Wang Limin Wang 93 26 0 21 Jan 2025
VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model Xianwei Zhuang Yuxin Xie Yufan Deng Liming Liang Jinghan Ru Yuguo Yin Yuexian Zou MLLM VLM LRM 117 8 0 21 Jan 2025
CBVLM: Training-free Explainable Concept-based Large Vision Language Models for Medical Image Classification Cristiano Patrício Isabel Rio-Torto J. S. Cardoso Luís F. Teixeira João C. Neves VLM 322 1 0 21 Jan 2025
PAINT: Paying Attention to INformed Tokens to Mitigate Hallucination in Large Vision-Language Model Kazi Hasan Ibn Arif Sajib Acharjee Dip Khizar Hussain Lang Zhang Chris Thomas 76 0 0 21 Jan 2025
Dynamic Scene Understanding from Vision-Language Representations Shahaf Pruss Morris Alper Hadar Averbuch-Elor OCL 286 0 0 20 Jan 2025
Know "No'' Better: A Data-Driven Approach for Enhancing Negation Awareness in CLIP J. Park Jungbeom Lee Jongyoon Song Sangwon Yu Dahuin Jung Sungroh Yoon 54 0 0 19 Jan 2025
MedFILIP: Medical Fine-grained Language-Image Pre-training Xinjie Liang Xiangyu Li Fanding Li Jie Jiang Qing Dong Wei Wang Kaidi Wang Suyu Dong Gongning Luo Shuo Li LM&MA VLM MedIm 77 4 0 18 Jan 2025
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words Junyi Ao Yuancheng Wang Xiaohai Tian Dekun Chen Jingyang Zhang Lu Lu Yansen Wang Haizhou Li Zhikai Wu AuLLM 90 19 0 17 Jan 2025
Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social Spaces Amirreza Payandeh Daeun Song Mohammad Nazeri Jing Liang Praneel Mukherjee Amir Hossain Raj Yangzhe Kong Dinesh Manocha Xuesu Xiao LM&Ro LRM 79 5 0 17 Jan 2025