Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

16 July 2021

Junnan Li

Ramprasaath R. Selvaraju

Akhilesh Deepak Gotmare

Papers citing "Align before Fuse: Vision and Language Representation Learning with Momentum Distillation"

45 / 1,195 papers shown

Title
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 30 111 0 03 Feb 2022
MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learning Zejun Li Zhihao Fan Huaixiao Tou Jingjing Chen Zhongyu Wei Xuanjing Huang 25 16 0 29 Jan 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 392 4,154 0 28 Jan 2022
Multimodal data matters: language model pre-training over structured and unstructured electronic health records Sicen Liu Xiaolong Wang Yongshuai Hou Ge Li Hui Wang Huiqin Xu Yang Xiang Buzhou Tang 52 30 0 25 Jan 2022
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks Zhecan Wang Noel Codella Yen-Chun Chen Luowei Zhou Jianwei Yang Xiyang Dai Bin Xiao Haoxuan You Shih-Fu Chang Lu Yuan CLIP VLM 22 39 0 15 Jan 2022
A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision Ajinkya Tejankar Maziar Sanjabi Bichen Wu Saining Xie Madian Khabsa Hamed Pirsiavash Hamed Firooz VLM 29 17 0 27 Dec 2021
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 32 100 0 23 Dec 2021
Align and Prompt: Video-and-Language Pre-training with Entity Prompts Dongxu Li Junnan Li Hongdong Li Juan Carlos Niebles Guosheng Lin 28 191 0 17 Dec 2021
Contrastive Vision-Language Pre-training with Limited Resources Quan Cui Boyan Zhou Yu Guo Weidong Yin Hao Wu Osamu Yoshie Yubo Chen VLM CLIP 19 33 0 17 Dec 2021
Distilled Dual-Encoder Model for Vision-Language Understanding Zekun Wang Wenhui Wang Haichao Zhu Ming Liu Bing Qin Furu Wei VLM FedML 29 30 0 16 Dec 2021
DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings Chaochen Gao Xing Wu Peng Wang Jue Wang Liangjun Zang Zhongyuan Wang Songlin Hu 20 3 0 10 Dec 2021
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 40 690 0 08 Dec 2021
MLP Architectures for Vision-and-Language Modeling: An Empirical Study Yi-Liang Nie Linjie Li Zhe Gan Shuohang Wang Chenguang Zhu Michael Zeng Zicheng Liu Joey Tianyi Zhou Lijuan Wang 27 6 0 08 Dec 2021
General Facial Representation Learning in a Visual-Linguistic Manner Yinglin Zheng Hao Yang Ting Zhang Jianmin Bao Dongdong Chen Yangyu Huang Lu Yuan Dong Chen Ming Zeng Fang Wen CVBM 146 163 0 06 Dec 2021
VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning Qibin Chen Jeremy Lacomis Edward J. Schwartz Graham Neubig Bogdan Vasilescu Claire Le Goues VLM 18 34 0 05 Dec 2021
LAFITE: Towards Language-Free Training for Text-to-Image Generation Yufan Zhou Ruiyi Zhang Changyou Chen Chunyuan Li Chris Tensmeyer Tong Yu Jiuxiang Gu Jinhui Xu Tong Sun VLM 35 163 0 27 Nov 2021
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 34 246 0 24 Nov 2021
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 27 111 0 23 Nov 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 36 879 0 22 Nov 2021
TraVLR: Now You See It, Now You Don't! A Bimodal Dataset for Evaluating Visio-Linguistic Reasoning Keng Ji Chow Samson Tan MingSung Kan LRM 26 4 0 21 Nov 2021
MaIL: A Unified Mask-Image-Language Trimodal Network for Referring Image Segmentation Zizhang Li Mengmeng Wang Jianbiao Mei Yong Liu 20 18 0 21 Nov 2021
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 31 189 0 19 Nov 2021
UFO: A UniFied TransfOrmer for Vision-Language Representation Learning Jianfeng Wang Xiaowei Hu Zhe Gan Zhengyuan Yang Xiyang Dai Zicheng Liu Yumao Lu Lijuan Wang ViT 29 57 0 19 Nov 2021
Open Vocabulary Object Detection with Pseudo Bounding-Box Labels M. Gao Chen Xing Juan Carlos Niebles Junnan Li Ran Xu Wenhao Liu Caiming Xiong VLM ObjD 9 86 0 18 Nov 2021
Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts Yan Zeng Xinsong Zhang Hang Li VLM CLIP 18 298 0 16 Nov 2021
CLIP2TV: Align, Match and Distill for Video-Text Retrieval Zijian Gao Jiaheng Liu Weiqi Sun S. Chen Dedan Chang Lili Zhao VLM CLIP 28 17 0 10 Nov 2021
FILIP: Fine-grained Interactive Language-Image Pre-Training Lewei Yao Runhu Huang Lu Hou Guansong Lu Minzhe Niu Hang Xu Xiaodan Liang Zhenguo Li Xin Jiang Chunjing Xu VLM CLIP 30 616 0 09 Nov 2021
An Empirical Study of Training End-to-End Vision-and-Language Transformers Zi-Yi Dou Yichong Xu Zhe Gan Jianfeng Wang Shuohang Wang ... Pengchuan Zhang Lu Yuan Nanyun Peng Zicheng Liu Michael Zeng VLM 32 369 0 03 Nov 2021
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Hangbo Bao Wenhui Wang Li Dong Qiang Liu Owais Khan Mohammed Kriti Aggarwal Subhojit Som Furu Wei VLM MLLM MoE 20 533 0 03 Nov 2021
VLDeformer: Vision-Language Decomposed Transformer for Fast Cross-Modal Retrieval Lisai Zhang Hongfa Wu Qingcai Chen Yimeng Deng Zhonghua Li Dejiang Kong Bo Zhao Joanna Siebert Yunpeng Han ViT VLM 27 20 0 20 Oct 2021
SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training Ankur Bapna Yu-An Chung Na Wu Anmol Gulati Ye Jia J. Clark Melvin Johnson Jason Riesa Alexis Conneau Yu Zhang VLM 61 94 0 20 Oct 2021
Cascaded Fast and Slow Models for Efficient Semantic Code Search Akhilesh Deepak Gotmare Junnan Li Chenyu You Guosheng Lin 33 10 0 15 Oct 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 101 984 0 09 Oct 2021
ActionCLIP: A New Paradigm for Video Action Recognition Mengmeng Wang Jiazheng Xing Yong Liu VLM 152 362 0 17 Sep 2021
Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers Stella Frank Emanuele Bugliarello Desmond Elliott 32 81 0 09 Sep 2021
Image2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Models Chenfeng Xu Shijia Yang Tomer Galanti Bichen Wu Xiangyu Yue Bohan Zhai Wei Zhan Peter Vajda Kurt Keutzer Masayoshi Tomizuka 3DPC 39 53 0 08 Jun 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 109 54 0 23 Apr 2021
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval Gregor Geigle Jonas Pfeiffer Nils Reimers Ivan Vulić Iryna Gurevych 35 59 0 22 Mar 2021
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives Nils Rethmeier Isabelle Augenstein SSL VLM 90 90 0 25 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 299 1,084 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 334 3,708 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Joey Tianyi Zhou MLLM 277 525 0 04 Feb 2021
VinVL: Revisiting Visual Representations in Vision-Language Models Pengchuan Zhang Xiujun Li Xiaowei Hu Jianwei Yang Lei Zhang Lijuan Wang Yejin Choi Jianfeng Gao ObjD VLM 260 157 0 02 Jan 2021
A Mutual Information Maximization Perspective of Language Representation Learning Lingpeng Kong Cyprien de Masson dÁutume Wang Ling Lei Yu Zihang Dai Dani Yogatama SSL 226 165 0 18 Oct 2019
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018