Unified Vision-Language Pre-Training for Image Captioning and VQA

24 September 2019

Lei Zhang

Papers citing "Unified Vision-Language Pre-Training for Image Captioning and VQA"

50 / 238 papers shown

Title
Hadamard product in deep learning: Introduction, Advances and Challenges Grigorios G. Chrysos Yongtao Wu Razvan Pascanu Philip Torr V. Cevher AAML 98 1 0 17 Apr 2025
Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Tasks Mohammad Saleha Azadeh Tabatabaeib 52 0 0 14 Apr 2025
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Xia Hu Bo Yuan VLM 64 0 0 24 Mar 2025
Feature Calibration enhanced Parameter Synthesis for CLIP-based Class-incremental Learning Jiaxin Guo Xiaoguang Zhu Xiaoguang Zhu Lianlong Sun Liangyu Teng Yang Liu Di Li Wei Zhou Liang Song CLL VLM 62 1 0 24 Mar 2025
Quantifying Memorization and Retriever Performance in Retrieval-Augmented Vision-Language Models Peter Carragher Abhinand Jha R Raghav Kathleen M. Carley RALM 75 0 0 20 Feb 2025
CMAL: A Novel Cross-Modal Associative Learning Framework for Vision-Language Pre-Training Zhiyuan Ma Jianjun Li Guohui Li Kaiyan Huang VLM 56 9 0 16 Oct 2024
Visual-O1: Understanding Ambiguous Instructions via Multi-modal Multi-turn Chain-of-thoughts Reasoning Minheng Ni Yutao Fan Lei Zhang Wangmeng Zuo LRM AI4CE 33 6 0 04 Oct 2024
GeoBiked: A Dataset with Geometric Features and Automated Labeling Techniques to Enable Deep Generative Models in Engineering Design Phillip Mueller Sebastian Mueller Lars Mikelsons 33 1 0 25 Sep 2024
QTG-VQA: Question-Type-Guided Architectural for VideoQA Systems Zhixian He Pengcheng Zhao Fuwei Zhang Shujin Lin 46 0 0 14 Sep 2024
Exploiting the Semantic Knowledge of Pre-trained Text-Encoders for Continual Learning Lu Yu Hesong Li Ying Fu Joost van de Weijer Changsheng Xu CLL 60 1 0 02 Aug 2024
Learning Visual Grounding from Generative Vision and Language Model Shijie Wang Dahun Kim A. Taalimi Chen Sun Weicheng Kuo ObjD 38 6 0 18 Jul 2024
Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tags Daiqing Qi Handong Zhao Zijun Wei Sheng Li 51 2 0 16 Jun 2024
Synthesizing Efficient Data with Diffusion Models for Person Re-Identification Pre-Training Ke Niu Haiyang Yu X. Qian Teng Fu Bin Li Xiangyang Xue 47 2 0 10 Jun 2024
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models Hao Fang Jiawei Kong Wenbo Yu Bin Chen Jiawei Li Hao Wu Ke Xu Ke Xu AAML VLM 40 13 0 08 Jun 2024
Multi-Modal Generative Embedding Model Feipeng Ma Hongwei Xue Guangting Wang Yizhou Zhou Fengyun Rao Shilin Yan Yueyi Zhang Siying Wu Mike Zheng Shou Xiaoyan Sun VLM 39 4 0 29 May 2024
OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All Yuanhuiyi Lyu Xueye Zheng Dahun Kim Lin Wang 59 14 0 25 May 2024
MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing Siddhant Agarwal Shivam Sharma Preslav Nakov Tanmoy Chakraborty 24 4 0 18 May 2024
A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes Ting Yu Xiaojun Lin Shuhui Wang Weiguo Sheng Qingming Huang Jun-chen Yu 3DV 59 10 0 12 Mar 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 67 12 0 05 Mar 2024
Acquiring Linguistic Knowledge from Multimodal Input Theodor Amariucai Alexander Scott Warstadt CLL 42 2 0 27 Feb 2024
SIMPLOT: Enhancing Chart Question Answering by Distilling Essentials Wonjoong Kim S. Park Yeonjun In Seokwon Han Chanyoung Park LRM ReLM 32 3 0 22 Feb 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 44 1 0 06 Feb 2024
Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking Yanlei Gao Haojun Xu Nannan Wang Jie Li Xinbo Gao VOT 50 4 0 14 Dec 2023
CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observations Xuzhe Dang Stefan Edelkamp 37 4 0 06 Nov 2023
BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioning Ching-Yu Chiang I-Hua Chang Shih-Wei Liao 55 1 0 26 Sep 2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 34 2 0 06 Sep 2023
DeViL: Decoding Vision features into Language Meghal Dani Isabel Rio-Torto Stephan Alaniz Zeynep Akata VLM 42 7 0 04 Sep 2023
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning Manuele Barraco Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara VLM 60 19 0 23 Aug 2023
Explore and Tell: Embodied Visual Captioning in 3D Environments Anwen Hu Shizhe Chen Liang Zhang Qin Jin LM&Ro 43 2 0 21 Aug 2023
Lip Reading for Low-resource Languages by Learning and Combining General Speech Knowledge and Language-specific Knowledge Minsu Kim Jeong Hun Yeo J. Choi Y. Ro 34 16 0 18 Aug 2023
Recommender Systems in the Era of Large Language Models (LLMs) Zihuai Zhao Wenqi Fan Jiatong Li Yunqing Liu Xiaowei Mei ... Zhen Wen Fei Wang Xiangyu Zhao Jiliang Tang Qing Li KELM 63 311 0 05 Jul 2023
Seeing in Words: Learning to Classify through Language Bottlenecks Khalid Saifullah Yuxin Wen Jonas Geiping Micah Goldblum Tom Goldstein VLM 21 2 0 29 Jun 2023
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models Lingxi Xie Longhui Wei Xiaopeng Zhang Kaifeng Bi Xiaotao Gu Jianlong Chang Qi Tian 41 7 0 14 Jun 2023
Putting Humans in the Image Captioning Loop Aliki Anagnostopoulou Mareike Hartmann Daniel Sonntag VLM 35 1 0 06 Jun 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 44 21 0 25 May 2023
ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst Zijia Zhao Longteng Guo Tongtian Yue Si-Qing Chen Shuai Shao Xinxin Zhu Zehuan Yuan Jing Liu MLLM 40 53 0 25 May 2023
Evaluating Pragmatic Abilities of Image Captioners on A3DS Polina Tsvilodub Michael Franke EGVM 25 3 0 22 May 2023
A request for clarity over the End of Sequence token in the Self-Critical Sequence Training J. Hu Roberto Cavicchioli Alessandro Capotondi 37 6 0 20 May 2023
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding Le Xue Ning Yu Shu Zhen Zhang Artemis Panagopoulou Junnan Li ... Jiajun Wu Caiming Xiong Ran Xu Juan Carlos Niebles Silvio Savarese 29 115 0 14 May 2023
IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of Images Varuna Krishna S. Suryavardan Shreyash Mishra Sathyanarayanan Ramamoorthy Parth Patwa Megha Chakraborty Aman Chadha Amitava Das Amit P. Sheth VLM 33 3 0 12 May 2023
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning Yu Yang Besmira Nushi Hamid Palangi Baharan Mirzasoleiman 44 36 0 08 Apr 2023
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks Weicheng Kuo A. Piergiovanni Dahun Kim Xiyang Luo Benjamin Caine ... Luowei Zhou Andrew M. Dai Zhifeng Chen Claire Cui A. Angelova MLLM VLM 37 23 0 29 Mar 2023
CoBIT: A Contrastive Bi-directional Image-Text Generation Model Haoxuan You Mandy Guo Zhecan Wang Kai-Wei Chang Jason Baldridge Jiahui Yu DiffM 54 13 0 23 Mar 2023
Text with Knowledge Graph Augmented Transformer for Video Captioning Xin Gu G. Chen Yufei Wang Libo Zhang Tiejian Luo Longyin Wen 32 47 0 22 Mar 2023
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Bo He Jun Wang Jielin Qiu Trung Bui Abhinav Shrivastava Zhaowen Wang 22 66 0 13 Mar 2023
TQ-Net: Mixed Contrastive Representation Learning For Heterogeneous Test Questions He Zhu Xihua Li Xuemin Zhao Yunbo Cao Shan Yu 23 0 0 09 Mar 2023
Toward Unsupervised Realistic Visual Question Answering Yuwei Zhang Chih-Hui Ho Nuno Vasconcelos CoGe 22 2 0 09 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 38 509 0 07 Mar 2023
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks Xiaoping Han Xiatian Zhu Licheng Yu Li Zhang Yi-Zhe Song Tao Xiang VLM 24 38 0 04 Mar 2023
ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing Zequn Zeng Hao Zhang Zhengjue Wang Ruiying Lu Dongsheng Wang Bo Chen BDL DiffM 24 33 0 04 Mar 2023