Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

2 December 2016

Devi Parikh

Papers citing "Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering"

50 / 1,968 papers shown

Title
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin Xinyu Wei Ruichuan An Peng Gao Bocheng Zou Yulin Luo Siyuan Huang Shanghang Zhang Hongsheng Li VLM 71 33 0 29 Mar 2024
LocCa: Visual Pretraining with Location-aware Captioners Bo Wan Michael Tschannen Yongqin Xian Filip Pavetić Ibrahim M. Alabdulmohsin Xiao Wang André Susano Pinto Andreas Steiner Lucas Beyer Xiao-Qi Zhai VLM 56 6 0 28 Mar 2024
Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models Yanwei Li Yuechen Zhang Chengyao Wang Zhisheng Zhong Yixin Chen Ruihang Chu Shaoteng Liu Jiaya Jia VLM MLLM MoE 45 215 0 27 Mar 2024
Quantifying and Mitigating Unimodal Biases in Multimodal Large Language Models: A Causal Perspective Meiqi Chen Yixin Cao Yan Zhang Chaochao Lu 37 13 0 27 Mar 2024
Toward Interactive Regional Understanding in Vision-Large Language Models Jungbeom Lee Sanghyuk Chun Sangdoo Yun VLM 35 1 0 27 Mar 2024
Beyond Embeddings: The Promise of Visual Table in Visual Reasoning Yiwu Zhong Zi-Yuan Hu Michael R. Lyu Liwei Wang 31 1 0 27 Mar 2024
A Gaze-grounded Visual Question Answering Dataset for Clarifying Ambiguous Japanese Questions Shun Inadumi Seiya Kawano Akishige Yuguchi Yasutomo Kawanishi Koichiro Yoshino 36 1 0 26 Mar 2024
IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models Haz Sameen Shahgir Khondker Salman Sayeed Abhik Bhattacharjee Wasi Uddin Ahmad Yue Dong Rifat Shahriyar VLM MLLM 50 10 0 23 Mar 2024
LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models Yuzhang Shang Mu Cai Bingxin Xu Yong Jae Lee Yan Yan VLM 55 107 0 22 Mar 2024
InternVideo2: Scaling Video Foundation Models for Multimodal Video Understanding Yi Wang Kunchang Li Xinhao Li Jiashuo Yu Yinan He ... Hongjie Zhang Yifei Huang Yu Qiao Yali Wang Limin Wang 42 50 0 22 Mar 2024
Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Models Qiong Wu Weihao Ye Yiyi Zhou Xiaoshuai Sun Rongrong Ji MoE 52 1 0 22 Mar 2024
Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering Bowen Jiang Zhijun Zhuang Shreyas S. Shivakumar Dan Roth Camillo J Taylor LLMAG 34 3 0 21 Mar 2024
Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference Han Zhao Min Zhang Wei Zhao Pengxiang Ding Siteng Huang Donglin Wang Mamba 57 69 0 21 Mar 2024
VL-Mamba: Exploring State Space Models for Multimodal Learning Yanyuan Qiao Zheng Yu Longteng Guo Sihan Chen Zijia Zhao Mingzhen Sun Qi Wu Jing Liu Mamba 43 66 0 20 Mar 2024
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs Théophane Vallaeys Mustafa Shukor Matthieu Cord Jakob Verbeek 59 12 0 20 Mar 2024
HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models Wenqiao Zhang Tianwei Lin Jiang Liu Fangxun Shu Haoyuan Li ... Zheqi Lv Hao Jiang Juncheng Li Siliang Tang Yueting Zhuang VLM MLLM 41 4 0 20 Mar 2024
SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Models Tongtian Yue Jie Cheng Longteng Guo Xingyuan Dai Zijia Zhao Xingjian He Gang Xiong Yisheng Lv Jing Liu 48 9 0 20 Mar 2024
Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models Zuyan Liu Yuhao Dong Yongming Rao Jie Zhou Jiwen Lu LRM 27 13 0 19 Mar 2024
When Do We Not Need Larger Vision Models? Baifeng Shi Ziyang Wu Maolin Mao Xin Wang Trevor Darrell VLM LRM 59 42 0 19 Mar 2024
As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks? Anjun Hu Jindong Gu Francesco Pinto Konstantinos Kamnitsas Philip Torr AAML SILM 45 5 0 19 Mar 2024
X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment Dongjae Shin Hyunseok Lim Inho Won Changsu Choi Minjun Kim Seungwoo Song Hangyeol Yoo Sangmin Kim Kyungtae Lim 23 5 0 18 Mar 2024
Few-Shot VQA with Frozen LLMs: A Tale of Two Approaches Igor Sterner Weizhe Lin Jinghong Chen Bill Byrne 30 3 0 17 Mar 2024
SQ-LLaVA: Self-Questioning for Large Vision-Language Assistant Guohao Sun Can Qin Jiamian Wang Zeyuan Chen Ran Xu Zhiqiang Tao MLLM VLM LRM 37 9 0 17 Mar 2024
Mitigating Dialogue Hallucination for Large Vision Language Models via Adversarial Instruction Tuning Dongmin Park Zhaofang Qian Guangxing Han Ser-Nam Lim MLLM 48 0 0 15 Mar 2024
Knowledge Condensation and Reasoning for Knowledge-based VQA Dongze Hao Jian Jia Longteng Guo Qunbo Wang Te Yang ... Yanhua Cheng Bo Wang Quan Chen Han Li Jing Liu 44 1 0 15 Mar 2024
An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models Haochen Luo Jindong Gu Fengyuan Liu Philip Torr VLM VPVLM AAML 60 21 0 14 Mar 2024
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training Brandon McKinzie Zhe Gan J. Fauconnier Sam Dodge Bowen Zhang ... Zirui Wang Ruoming Pang Peter Grasch Alexander Toshev Yinfei Yang MLLM 43 189 0 14 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 46 2 0 14 Mar 2024
UniCode: Learning a Unified Codebook for Multimodal Large Language Models Sipeng Zheng Bohan Zhou Yicheng Feng Ye Wang Zongqing Lu VLM MLLM 46 7 0 14 Mar 2024
DAM: Dynamic Adapter Merging for Continual Video QA Learning Feng Cheng Ziyang Wang Yi-Lin Sung Yan-Bo Lin Mohit Bansal Gedas Bertasius CLL MoMe 44 10 0 13 Mar 2024
An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model Yuxin Tian Mouxing Yang Yunfan Li Dayiheng Liu Xingzhang Ren Xiaocui Peng Jiancheng Lv VLM 45 0 0 13 Mar 2024
CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model Cheng Chen Sitong Su Xu Luo Hengtao Shen Lianli Gao Jingkuan Song CLL 42 13 0 13 Mar 2024
Fine-tuning Large Language Models with Sequential Instructions Hanxu Hu Simon Yu Pinzhen Chen Edoardo Ponti ALM LRM 84 15 0 12 Mar 2024
Synth $^2$ : Boosting Visual-Language Models with Synthetic Captions and Image Embeddings Sahand Sharifzadeh Christos Kaplanis Shreya Pathak D. Kumaran Anastasija Ilić Jovana Mitrović Charles Blundell Andrea Banino VLM 51 9 0 12 Mar 2024
Multi-modal Auto-regressive Modeling via Visual Words Tianshuo Peng Zuchao Li Lefei Zhang Hai Zhao Ping Wang Bo Du OffRL 35 5 0 12 Mar 2024
Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Models Minjie Zhu Yichen Zhu Xin Liu Ning Liu Zhiyuan Xu Yaxin Peng Chaomin Shen Zhicai Ou Feifei Feng Jian Tang VLM 57 20 0 10 Mar 2024
SnapNTell: Enhancing Entity-Centric Visual Question Answering with Retrieval Augmented Multimodal LLM Jielin Qiu Andrea Madotto Zhaojiang Lin Paul A. Crook Yongjun Xu Xin Luna Dong Christos Faloutsos Lei Li Babak Damavandi Seungwhan Moon 45 9 0 07 Mar 2024
Yi: Open Foundation Models by 01.AI 01. AI Alex Young 01.AI Alex Young Bei Chen Chao Li ... Yue Wang Yuxuan Cai Zhenyu Gu Zhiyuan Liu Zonghong Dai OSLM LRM 150 512 0 07 Mar 2024
CoTBal: Comprehensive Task Balancing for Multi-Task Visual Instruction Tuning Yanqi Dai Dong Jing Nanyi Fei Zhiwu Lu Nanyi Fei Guoxing Yang Zhiwu Lu 61 3 0 07 Mar 2024
Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoning Deepanway Ghosal Vernon Toh Yan Han Chia Yew Ken Soujanya Poria ReLM LRM 122 5 0 06 Mar 2024
Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models Gen Luo Yiyi Zhou Yuxin Zhang Xiawu Zheng Xiaoshuai Sun Rongrong Ji VLM 36 57 0 05 Mar 2024
MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer Jianjian Cao Peng Ye Shengze Li Chong Yu Yansong Tang Jiwen Lu Tao Chen 38 17 0 05 Mar 2024
Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters Weizhi Wang Khalil Mrini Linjie Yang Sateesh Kumar Yu Tian Xifeng Yan Heng Wang 46 16 0 05 Mar 2024
Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use Imad Eddine Toubal Aditya Avinash N. Alldrin Jan Dlabal Wenlei Zhou ... Chun-Ta Lu Howard Zhou Ranjay Krishna Ariel Fuxman Tom Duerig VLM 85 7 0 05 Mar 2024
VEglue: Testing Visual Entailment Systems via Object-Aligned Joint Erasing Zhiyuan Chang Mingyang Li Junjie Wang Cheng Li Qing Wang 27 0 0 05 Mar 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 69 12 0 05 Mar 2024
Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training David Wan Jaemin Cho Elias Stengel-Eskin Mohit Bansal VLM ObjD 53 30 0 04 Mar 2024
An Improved Traditional Chinese Evaluation Suite for Foundation Model Zhi Rui Tam Ya-Ting Pai Yen-Wei Lee Jun-Da Chen Wei-Min Chu Sega Cheng Hong-Han Shuai ELM 45 11 0 04 Mar 2024
NPHardEval4V: A Dynamic Reasoning Benchmark of Multimodal Large Language Models Lizhou Fan Wenyue Hua Xiang Li Kaijie Zhu Mingyu Jin ... Haoyang Ling Jinkui Chi Jindong Wang Xin Ma Yongfeng Zhang LRM 50 14 0 04 Mar 2024
Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks Fakhraddin Alwajih El Moatez Billah Nagoudi Gagan Bhatia Abdelrahman Mohamed Muhammad Abdul-Mageed VLM LRM 35 11 0 01 Mar 2024