Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

16 July 2021

Junnan Li

Ramprasaath R. Selvaraju

Akhilesh Deepak Gotmare

Papers citing "Align before Fuse: Vision and Language Representation Learning with Momentum Distillation"

50 / 1,195 papers shown

Title
Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision T. Wang Jorma T. Laaksonen T. Langer Heikki Arponen Tom E. Bishop VLM 16 6 0 24 Oct 2022
Multilingual Multimodal Learning with Machine Translated Text Chen Qiu Dan Oneaţă Emanuele Bugliarello Stella Frank Desmond Elliott 48 13 0 24 Oct 2022
Towards Unifying Reference Expression Generation and Comprehension Duo Zheng Tao Kong Ya Jing Jiaan Wang Xiaojie Wang ObjD 35 6 0 24 Oct 2022
LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling Dongsheng Chen Chaofan Tao Lu Hou Lifeng Shang Xin Jiang Qun Liu VLM 29 18 0 21 Oct 2022
Dissecting Deep Metric Learning Losses for Image-Text Retrieval Hong Xuan Xi Chen 23 2 0 21 Oct 2022
Image-Text Retrieval with Binary and Continuous Label Supervision Zheng Li Caili Guo Zerun Feng Lei Li Ying Jin Yufeng Zhang VLM 34 4 0 20 Oct 2022
CLIP-Driven Fine-grained Text-Image Person Re-identification Shuanglin Yan Neng Dong Liyan Zhang Jinhui Tang 45 87 0 19 Oct 2022
MMGA: Multimodal Learning with Graph Alignment Xuanqi Yang Quanjin Tao Xiaojuan Feng Donghong Cai Xiang Ren Yang Yang 24 0 0 18 Oct 2022
Probing Cross-modal Semantics Alignment Capability from the Textual Perspective Zheng Ma Shi Zong Mianzhi Pan Jianbing Zhang Shujian Huang Xinyu Dai Jiajun Chen 30 4 0 18 Oct 2022
Non-Contrastive Learning Meets Language-Image Pre-Training Jinghao Zhou Li Dong Zhe Gan Lijuan Wang Furu Wei VLM CLIP 25 26 0 17 Oct 2022
Contrastive Language-Image Pre-Training with Knowledge Graphs Xuran Pan Tianzhu Ye Dongchen Han S. Song Gao Huang VLM CLIP 30 43 0 17 Oct 2022
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training A. M. H. Tiong Junnan Li Boyang Albert Li Silvio Savarese Guosheng Lin MLLM 37 101 0 17 Oct 2022
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning Tiannan Wang Wangchunshu Zhou Yan Zeng Xinsong Zhang VLM 36 37 0 14 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 32 62 0 14 Oct 2022
FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learning Rui Ye Zhenyang Ni Chenxin Xu Jianyu Wang Siheng Chen Yonina C. Eldar FedML 27 31 0 14 Oct 2022
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting Oscar Manas Pau Rodríguez López Saba Ahmadi Aida Nematzadeh Yash Goyal Aishwarya Agrawal VLM VPVLM 18 48 0 13 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 28 1 0 12 Oct 2022
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning Yuchong Sun Hongwei Xue Ruihua Song Bei Liu Huan Yang Jianlong Fu AI4TS VLM 20 68 0 12 Oct 2022
Stochastic Constrained DRO with a Complexity Independent of Sample Size Q. Qi Jiameng Lyu Kung-Sik Chan E. Bai Tianbao Yang 50 15 0 11 Oct 2022
MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model Yatai Ji Junjie Wang Yuan Gong Lin Zhang Yan Zhu Hongfa Wang Jiaxing Zhang Tetsuya Sakai Yujiu Yang MLLM 27 29 0 11 Oct 2022
Contrastive Video-Language Learning with Fine-grained Frame Sampling Zixu Wang Yujie Zhong Yishu Miao Lin Ma Lucia Specia 52 11 0 10 Oct 2022
MAMO: Masked Multimodal Modeling for Fine-Grained Vision-Language Representation Learning Zijia Zhao Longteng Guo Xingjian He Shuai Shao Zehuan Yuan Jing Liu 21 8 0 09 Oct 2022
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment Siyu Lai Zhen Yang Fandong Meng Jinan Xu Jinan Xu Jie Zhou 33 6 0 09 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 32 21 0 09 Oct 2022
Learning Fine-Grained Visual Understanding for Video Question Answering via Decoupling Spatial-Temporal Modeling Hsin-Ying Lee Hung-Ting Su Bing-Chen Tsai Tsung-Han Wu Jia-Fong Yeh Winston H. Hsu 27 2 0 08 Oct 2022
CLIP model is an Efficient Continual Learner Vishal Thengane Salman Khan Munawar Hayat Fahad Shahbaz Khan BDL VLM CLL 112 46 0 06 Oct 2022
Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture M. Jung He Zhao Joanna Dipnall Belinda Gabbe Lan Du UQCV EDL 64 12 0 06 Oct 2022
Progressive Text-to-Image Generation Zhengcong Fei Mingyuan Fan Li Zhu Junshi Huang 89 4 0 05 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Zou VLM CoGe 30 362 0 04 Oct 2022
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models Guangyi Chen Weiran Yao Xiangchen Song Xinyue Li Yongming Rao Kun Zhang VPVLM VLM 8 62 0 03 Oct 2022
Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approach Georgios Tziafas H. Kasaei LM&Ro 20 3 0 03 Oct 2022
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models Weicheng Kuo Huayu Chen Xiuye Gu A. Piergiovanni A. Angelova MLLM VLM ObjD 51 134 0 30 Sep 2022
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training Bin Shan Weichong Yin Yu Sun Hao Tian Hua Wu Haifeng Wang VLM 27 19 0 30 Sep 2022
Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding Fengyuan Shi Ruopeng Gao Weilin Huang Limin Wang 30 23 0 28 Sep 2022
Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval Zheng Li Caili Guo Xin Wang Zerun Feng Lei Li Zhongtian Du VLM 28 2 0 28 Sep 2022
TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval Xiaohan Zou Changqiao Wu Lele Cheng Zhongyuan Wang 94 6 0 28 Sep 2022
Revisiting Few-Shot Learning from a Causal Perspective Guoliang Lin Yongheng Xu Hanjiang Lai Jian Yin CML 32 3 0 28 Sep 2022
Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors Vignav Ramesh Nathan Chi Pranav Rajpurkar MedIm 36 49 0 27 Sep 2022
Unsupervised Hashing with Semantic Concept Mining Rong-Cheng Tu Xian-Ling Mao Kevin Qinghong Lin Chengfei Cai Weize Qin Hongfa Wang Wei Wei Heyan Huang 62 10 0 23 Sep 2022
LGDN: Language-Guided Denoising Network for Video-Language Modeling Haoyu Lu Mingyu Ding Nanyi Fei Yuqi Huo Zhiwu Lu VLM 88 16 0 23 Sep 2022
DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection Lewei Yao Jianhua Han Youpeng Wen Xiaodan Liang Dan Xu Wei Zhang Zhenguo Li Chunjing Xu Hang Xu CLIP VLM 115 153 0 20 Sep 2022
LAVIS: A Library for Language-Vision Intelligence Dongxu Li Junnan Li Hung Le Guangsen Wang Silvio Savarese Guosheng Lin VLM 131 52 0 15 Sep 2022
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks Junke Wang Dongdong Chen Zuxuan Wu Chong Luo Luowei Zhou Yucheng Zhao Yujia Xie Ce Liu Yu-Gang Jiang Lu Yuan MLLM VLM 38 149 0 15 Sep 2022
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge Zhihong Chen Guanbin Li Xiang Wan 127 65 0 15 Sep 2022
Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering Jingjing Jiang Zi-yi Liu Nanning Zheng 26 8 0 14 Sep 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 25 68 0 14 Sep 2022
OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network Tiancheng Zhao Peng Liu Kyusong Lee VLM MLLM ObjD 27 7 0 10 Sep 2022
FETA: Towards Specializing Foundation Models for Expert Task Applications Amit Alfassy Assaf Arbelle Oshri Halimi Sivan Harary Roei Herzig ... Christoph Auer Kate Saenko Peter W. J. Staar Rogerio Feris Leonid Karlinsky 23 19 0 08 Sep 2022
Design of the topology for contrastive visual-textual alignment Zhun Sun 32 1 0 05 Sep 2022
RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection Hangjie Yuan Jianwen Jiang Samuel Albanie Tao Feng Ziyuan Huang Dong Ni Mingqian Tang VLM 37 51 0 05 Sep 2022