v1v2 (latest)

Unifying Vision-and-Language Tasks via Text Generation

4 February 2021

ArXiv (abs)PDF HTML Github (372★)

Papers citing "Unifying Vision-and-Language Tasks via Text Generation"

50 / 170 papers shown

Title
Manager: Aggregating Insights from Unimodal Experts in Two-Tower VLMs and MLLMs Xiao Xu L. Qin Wanxiang Che Min-Yen Kan MoE VLM 30 0 0 13 Jun 2025
Vision Generalist Model: A Survey Ziyi Wang Yongming Rao Shuofeng Sun Xinrun Liu Yi Wei ... Zuyan Liu Yanbo Wang Hongmin Liu Jie Zhou Jiwen Lu 65 0 0 11 Jun 2025
Generating Vision-Language Navigation Instructions Incorporated Fine-Grained Alignment Annotations Yibo Cui Liang Xie Yu Zhao Jiawei Sun Erwei Yin 17 0 0 10 Jun 2025
CoCoA-Mix: Confusion-and-Confidence-Aware Mixture Model for Context Optimization Dasol Hong Wooju Lee Hyun Myung 17 0 0 09 Jun 2025
FREE: Fast and Robust Vision Language Models with Early Exits Divya J. Bajpai M. Hanawal VLM 15 0 0 07 Jun 2025
A Large Vision-Language Model based Environment Perception System for Visually Impaired People Zezhou Chen Zhaoxiang Liu Ning Wang Kohou Wang Shiguo Lian 228 0 0 25 Apr 2025
FLIP Reasoning Challenge Andreas Plesner Turlan Kuzhagaliyev Roger Wattenhofer AAML VLM LRM 187 0 0 16 Apr 2025
AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection Peng Wu Wanshun Su Guansong Pang Yujia Sun Qingsen Yan Peng Wang Yize Zhang VLM 106 1 0 06 Apr 2025
Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval Haoqiang Lin Haokun Wen Xuemeng Song Meng Liu Yupeng Hu Liqiang Nie 185 16 0 25 Mar 2025
FlowTok: Flowing Seamlessly Across Text and Image Tokens Ju He Qihang Yu Qihao Liu Liang-Chieh Chen 148 1 0 13 Mar 2025
UFO: A Unified Approach to Fine-grained Visual Perception via Open-ended Language Interface Hao Tang Chenwei Xie Haiyang Wang Xiaoyi Bao Tingyu Weng Pandeng Li Yun Zheng Liwei Wang ObjD VLM 134 1 0 03 Mar 2025
MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs Jiarui Zhang Mahyar Khayatkhoei P. Chhikara Filip Ilievski LRM 110 16 0 24 Feb 2025
Natural Language Supervision for Low-light Image Enhancement Jiahui Tang Kaihua Zhou Zhijian Luo Yueen Hou 136 0 0 11 Jan 2025
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 176 7 0 24 Oct 2024
Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approach Rory Young Nicolas Pugeault AAML 136 5 0 14 Oct 2024
Generalizable Prompt Tuning for Vision-Language Models Qian Zhang VLM VPVLM 149 0 0 04 Oct 2024
One missing piece in Vision and Language: A Survey on Comics Understanding Emanuele Vivoli Andrey Barsky Mohamed Ali Souibgui Artemis LLabres Marco Bertini Dimosthenis Karatzas 124 5 0 14 Sep 2024
Toward Interactive Regional Understanding in Vision-Large Language Models Jungbeom Lee Sanghyuk Chun Sangdoo Yun VLM 82 3 0 27 Mar 2024
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 137 3 0 04 Mar 2024
SIMPLOT: Enhancing Chart Question Answering by Distilling Essentials Wonjoong Kim S. Park Yeonjun In Seokwon Han Chanyoung Park LRM ReLM 88 4 0 22 Feb 2024
ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training Rongsheng Wang Qingsong Yao Zihang Jiang Zhiyang He Xiaodong Tao Zihang Jiang S.Kevin Zhou MedIm VLM 108 6 0 20 Dec 2023
MAFA: Managing False Negatives for Vision-Language Pre-training Jaeseok Byun Dohoon Kim Taesup Moon VLM 81 6 0 11 Dec 2023
Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Prompts Yunshi Lan Xiang Li Xin Liu Yang Li Wei Qin Weining Qian LRM ReLM 154 29 0 15 Nov 2023
Location-Aware Visual Question Generation with Lightweight Models Nicholas Collin Suwono Justin Chih-Yao Chen Tun-Min Hung T. Huang I-Bin Liao Yung-Hui Li Lun-Wei Ku Shao-Hua Sun 53 4 0 23 Oct 2023
Multiscale Superpixel Structured Difference Graph Convolutional Network for VL Representation Siyu Zhang Ye-Ting Chen Fang Wang Yaoru Sun Jun Yang Lizhi Bai SSL 61 0 0 20 Oct 2023
Beyond Segmentation: Road Network Generation with Multi-Modal LLMs Sumedh Rasal Sanjay K. Boddhu 66 5 0 15 Oct 2023
Uni-NLX: Unifying Textual Explanations for Vision and Vision-Language Tasks Fawaz Sammani Nikos Deligiannis 39 5 0 17 Aug 2023
LOIS: Looking Out of Instance Semantics for Visual Question Answering Siyu Zhang Ye Chen Yaoru Sun Fang Wang Haibo Shi Haoran Wang 57 5 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming-Hsuan Yang Fahad Shahbaz Khan VLM 146 127 0 25 Jul 2023
Emu: Generative Pretraining in Multimodality Quan-Sen Sun Qiying Yu Yufeng Cui Fan Zhang Xiaosong Zhang Yueze Wang Hongcheng Gao Jingjing Liu Tiejun Huang Xinlong Wang MLLM 119 138 0 11 Jul 2023
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels Bang-ju Yang Fenglin Liu Zheng Li Qingyu Yin Chenyu You Bing Yin Yuexian Zou VLM 104 5 0 05 Jul 2023
Align, Adapt and Inject: Sound-guided Unified Image Generation Yue Yang Kaipeng Zhang Yuying Ge Wenqi Shao Zeyue Xue Yu Qiao Ping Luo DiffM 101 6 0 20 Jun 2023
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models Shuo Chen Jindong Gu Zhen Han Yunpu Ma Philip Torr Volker Tresp VPVLM VLM 118 21 0 03 Jun 2023
"Let's not Quote out of Context": Unified Vision-Language Pretraining for Context Assisted Image Captioning Abisek Rajakumar Kalarani P. Bhattacharyya Niyati Chhaya Sumit Shekhar CoGe VLM 106 9 0 01 Jun 2023
Benchmarking Diverse-Modal Entity Linking with Generative Models Sijia Wang Alexander Hanbo Li He Zhu Shenmin Zhang Chung-Wei Hang ... William Wang Zhiguo Wang Vittorio Castelli Bing Xiang Patrick Ng VLM 102 9 0 27 May 2023
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts Yunshui Li Binyuan Hui Zhichao Yin Min Yang Fei Huang Yongbin Li MoE 87 21 0 24 May 2023
UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning Ahmed Masry P. Kavehzadeh Do Xuan Long Enamul Hoque Shafiq Joty LRM 95 113 0 24 May 2023
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions Woojeong Jin Subhabrata Mukherjee Yu Cheng Yelong Shen Weizhu Chen Ahmed Hassan Awadallah Damien Jose Xiang Ren ObjD VLM 116 8 0 24 May 2023
Preconditioned Visual Language Inference with Weak Supervision Ehsan Qasemi Amani Maina-Kilaas Devadutta Dash Khalid Alsaggaf Muhao Chen 83 0 0 22 May 2023
Album Storytelling with Iterative Story-aware Captioning and Large Language Models Munan Ning Yujia Xie Dongdong Chen Zeyin Song Lu Yuan Yonghong Tian QiXiang Ye Liuliang Yuan 71 8 0 22 May 2023
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data Ziyi Yang Mahmoud Khademi Yichong Xu Reid Pryzant Yuwei Fang ... Yu Shi Lu Yuan Takuya Yoshioka Michael Zeng Xuedong Huang 63 2 0 21 May 2023
Sequence-to-Sequence Pre-training with Unified Modality Masking for Visual Document Understanding ShuWei Feng Tianyang Zhan Zhanming Jie Trung Quoc Luong Xiaoran Jin 41 1 0 16 May 2023
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding Le Xue Ning Yu Shu Zhen Zhang Artemis Panagopoulou Junnan Li ... Jiajun Wu Caiming Xiong Ran Xu Juan Carlos Niebles Silvio Savarese 121 129 0 14 May 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Joey Tianyi Zhou 147 142 0 11 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 193 2,101 0 11 May 2023
VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation Xilun Chen L. Yu Wenhan Xiong Barlas Ouguz Yashar Mehdad Wen-tau Yih VGen 58 3 0 04 May 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 133 112 0 17 Apr 2023
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks Weicheng Kuo A. Piergiovanni Dahun Kim Xiyang Luo Benjamin Caine ... Luowei Zhou Andrew M. Dai Zhifeng Chen Claire Cui A. Angelova MLLM VLM 117 25 0 29 Mar 2023
Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models Xinyang Liu Dongsheng Wang Bowei Fang Miaoge Li Zhibin Duan Yishi Xu Bo Chen Mingyuan Zhou VLM VPVLM 125 6 0 16 Mar 2023
Contextually-rich human affect perception using multimodal scene information Digbalay Bose Rajat Hebbar Krishna Somandepalli Shrikanth Narayanan 77 3 0 13 Mar 2023