Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts

11 May 2023

Yantao Shen

Papers citing "Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts"

41 / 41 papers shown

Title
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 109 12 0 05 Mar 2024
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 85 3 0 04 Mar 2024
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks Hao Li Jinguo Zhu Xiaohu Jiang Xizhou Zhu Hongsheng Li ... Xiaohua Wang Yu Qiao Xiaogang Wang Wenhai Wang Jifeng Dai MLLM 52 56 0 17 Nov 2022
A Unified Sequence Interface for Vision Tasks Ting-Li Chen Saurabh Saxena Lala Li Nayeon Lee David J. Fleet Geoffrey E. Hinton VLM MLLM 56 151 0 15 Jun 2022
ProQA: Structural Prompt-based Pre-training for Unified Question Answering Wanjun Zhong Yifan Gao Ning Ding Yujia Qin Zhiyuan Liu Ming Zhou Jiahai Wang Jian Yin Nan Duan 46 34 0 09 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 356 3,532 0 29 Apr 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 139 873 0 07 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 508 4,340 0 28 Jan 2022
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 88 706 0 08 Dec 2021
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks Xizhou Zhu Jinguo Zhu Hao Li Xiaoshi Wu Xiaogang Wang Hongsheng Li Xiaohua Wang Jifeng Dai 91 133 0 02 Dec 2021
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 79 116 0 23 Nov 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 119 904 0 22 Nov 2021
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 77 215 0 22 Nov 2021
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning Jianfeng Wang Xiaowei Hu Zhe Gan Zhengyuan Yang Xiyang Dai Zicheng Liu Yumao Lu Lijuan Wang ViT 64 57 0 19 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 274 1,040 0 09 Oct 2021
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 262 347 0 22 Sep 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 150 3,742 0 03 Sep 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 118 796 0 24 Aug 2021
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification Shengding Hu Ning Ding Huadong Wang Zhiyuan Liu Jingang Wang Juan-Zi Li Wei Wu Maosong Sun VLM 82 367 0 04 Aug 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs Andrew Jaegle Sebastian Borgeaud Jean-Baptiste Alayrac Carl Doersch Catalin Ionescu ... Olivier J. Hénaff M. Botvinick Andrew Zisserman Oriol Vinyals João Carreira MLLM VLM GNN 59 579 0 30 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 155 778 0 25 Jun 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan ViT 104 1,201 0 09 Jun 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 173 1,014 0 04 Mar 2021
UniT: Multimodal Multitask Learning with a Unified Transformer Ronghang Hu Amanpreet Singh ViT 76 300 0 22 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 438 3,839 0 11 Feb 2021
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers Lisa Anne Hendricks John F. J. Mellor R. Schneider Jean-Baptiste Alayrac Aida Nematzadeh 108 114 0 31 Jan 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 219 4,247 0 01 Jan 2021
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 113 972 0 15 Sep 2020
Self-Supervised MultiModal Versatile Networks Jean-Baptiste Alayrac Adrià Recasens R. Schneider Relja Arandjelović Jason Ramapuram J. Fauw Lucas Smaira Sander Dieleman Andrew Zisserman SSL 117 374 0 29 Jun 2020
Rethinking Positional Encoding in Language Pre-training Guolin Ke Di He Tie-Yan Liu 55 295 0 28 Jun 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 334 1,612 0 21 Jan 2020
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 130 1,403 0 28 Nov 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 233 2,477 0 20 Aug 2019
Visual Entailment: A Novel Task for Fine-Grained Image Understanding Ning Xie Farley Lai Derek Doran Asim Kadav CoGe 95 322 0 20 Jan 2019
The Natural Language Decathlon: Multitask Learning as Question Answering Bryan McCann N. Keskar Caiming Xiong R. Socher AIMat MLLM BDL 135 644 0 20 Jun 2018
One Model To Learn Them All Lukasz Kaiser Aidan Gomez Noam M. Shazeer Ashish Vaswani Niki Parmar Llion Jones Jakob Uszkoreit VLM ViT 77 334 0 16 Jun 2017
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering Yash Goyal Tejas Khot D. Summers-Stay Dhruv Batra Devi Parikh CoGe 326 3,235 0 02 Dec 2016
A Neural Attention Model for Abstractive Sentence Summarization Alexander M. Rush S. Chopra Jason Weston CVBM 182 2,700 0 02 Sep 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 210 7,734 0 31 Aug 2015
Microsoft COCO Captions: Data Collection and Evaluation Server Xinlei Chen Hao Fang Nayeon Lee Ramakrishna Vedantam Saurabh Gupta Piotr Dollar C. L. Zitnick 207 2,475 0 01 Apr 2015
Generating Sequences With Recurrent Neural Networks Alex Graves GAN 146 4,033 0 04 Aug 2013