v1v2 (latest)

Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

16 July 2021

Junnan Li

Ramprasaath R. Selvaraju

Akhilesh Deepak Gotmare

ArXiv (abs)PDF HTML Github (1658★)

Papers citing "Align before Fuse: Vision and Language Representation Learning with Momentum Distillation"

50 / 1,231 papers shown

Title
Zero-Shot Embeddings Inform Learning and Forgetting with Vision-Language Encoders Laura Niss Kevin Vogt-Lowell Theodoros Tsiligkaridis VLM 95 1 0 22 Jul 2024
Spatial-Temporal Cross-View Contrastive Pre-training for Check-in Sequence Representation Learning Letian Gong Huaiyu Wan Shengnan Guo Xiucheng Li Yan Lin Erwen Zheng Tianyi Wang Zeyu Zhou Youfang Lin AI4TS 84 1 0 22 Jul 2024
Rethinking Domain Adaptation and Generalization in the Era of CLIP Ruoyu Feng Tao Yu Xin Jin Xiaoyuan Yu Lei Xiao Zhibo Chen VLM 104 2 0 21 Jul 2024
Braille-to-Speech Generator: Audio Generation Based on Joint Fine-Tuning of CLIP and Fastspeech2 Chun Xu En-Wei Sun 75 0 0 19 Jul 2024
HOTS3D: Hyper-Spherical Optimal Transport for Semantic Alignment of Text-to-3D Generation Zezeng Li Weimin Wang WenHai Li Na Lei Na Lei Xianfeng Gu OT DiffM 84 0 0 19 Jul 2024
Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Models Xiaoyu Zhu Hao Zhou Pengfei Xing Long Zhao Hao Xu Junwei Liang Alex Hauptmann Ting Liu Andrew C. Gallagher DiffM 123 4 0 18 Jul 2024
Multimodal Label Relevance Ranking via Reinforcement Learning Taian Guo Taolin Zhang Haoqian Wu Hanjun Li Ruizhi Qiao Xing Sun OffRL 45 0 0 18 Jul 2024
ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inference Mengcheng Lan Chaofeng Chen Yiping Ke Xinjiang Wang Xue Jiang Wayne Zhang VLM 117 29 0 17 Jul 2024
Object-Aware Query Perturbation for Cross-Modal Image-Text Retrieval Naoya Sogi Takashi Shibata Makoto Terao VLM 87 2 0 17 Jul 2024
Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challenge Kang Shen Xuxiong Liu Boyan Wang Jun Yao Xin Liu Yujie Guan Yu Wang Gengchen Li Xiao Sun CVBM 91 2 0 17 Jul 2024
Cross-Modal Augmentation for Few-Shot Multimodal Fake News Detection Ye Jiang Taihang Wang Xiaoman Xu Yimin Wang Xingyi Song Diana Maynard 94 2 0 16 Jul 2024
PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigation Renjie Lu Jingke Meng Wei-Shi Zheng 81 3 0 16 Jul 2024
How and where does CLIP process negation? Vincent Quantmeyer Pablo Mosteiro Albert Gatt CoGe 73 9 0 15 Jul 2024
Open Vocabulary Multi-Label Video Classification Rohit Gupta Mamshad Nayeem Rizve Jayakrishnan Unnikrishnan Ashish Tawari Son Tran Mubarak Shah Benjamin Z. Yao Trishul Chilimbi VLM 95 1 0 12 Jul 2024
15M Multimodal Facial Image-Text Dataset Dawei Dai Yutang Li Yingge Liu Mingming Jia Zhang YuanHui Guoyin Wang VLM 103 7 0 11 Jul 2024
Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement Zijie Yue Miaojing Shi Hanli Wang Shuai Ding Qijun Chen Shanlin Yang 107 0 0 11 Jul 2024
TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data Siyi Du Shaoming Zheng Yinsong Wang Wenjia Bai D. O’Regan Chen Qin LMTD 97 5 0 10 Jul 2024
How to Make Cross Encoder a Good Teacher for Efficient Image-Text Retrieval? Yuxin Chen Zongyang Ma Ziqi Zhang Zhongang Qi Chunfeng Yuan Bing Li Junfu Pu Ying Shan Xiaojuan Qi Weiming Hu 62 2 0 10 Jul 2024
Resolving Sentiment Discrepancy for Multimodal Sentiment Detection via Semantics Completion and Decomposition Daiqing Wu Dongbao Yang Huawen Shen Can Ma Yu Zhou 66 4 0 09 Jul 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 150 5 0 09 Jul 2024
Learning to Adapt Category Consistent Meta-Feature of CLIP for Few-Shot Classification Jiaying Shi Xuetong Xue Shenghui Xu VLM 143 0 0 08 Jul 2024
Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition Zirun Guo Tao Jin Zhou Zhao 98 14 0 07 Jul 2024
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? Zhaorun Chen Yichao Du Zichen Wen Yiyang Zhou Chenhang Cui ... Jiawei Zhou Zhuokai Zhao Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM MLLM 117 35 0 05 Jul 2024
MARS: Paying more attention to visual attributes for text-based person search Alex Ergasti Tomaso Fontanini Claudio Ferrari Massimo Bertozzi Andrea Prati 86 10 0 05 Jul 2024
Visual Grounding with Attention-Driven Constraint Balancing Weitai Kang Luowei Zhou Junyi Wu Changchang Sun Yan Yan 74 4 0 03 Jul 2024
SafaRi:Adaptive Sequence Transformer for Weakly Supervised Referring Expression Segmentation Sayan Nag Koustava Goswami Srikrishna Karanam 107 4 0 02 Jul 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 98 15 0 01 Jul 2024
CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation Yuxuan Wang Yijun Liu Fei Yu Chen Huang Kexin Li Zhiguo Wan Wanxiang Che VLM CoGe 77 5 0 01 Jul 2024
From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning Nan Xu Fei Wang Sheng Zhang Hoifung Poon Muhao Chen 139 7 0 01 Jul 2024
From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models Mehar Bhatia Sahithya Ravi Aditya Chinchure EunJeong Hwang Vered Shwartz VLM 117 4 0 28 Jun 2024
Structure-aware World Model for Probe Guidance via Large-scale Self-supervised Pre-train Haojun Jiang Meng Li Zhenguo Sun Ning Jia Yu Sun Shaqi Luo Shiji Song Gao Huang 81 2 0 28 Jun 2024
Enhancing Continual Learning in Visual Question Answering with Modality-Aware Feature Distillation Malvina Nikandrou Georgios Pantazopoulos Ioannis Konstas Alessandro Suglia 77 2 0 27 Jun 2024
Advancing Cross-domain Discriminability in Continual Learning of Vison-Language Models Yicheng Xu Yuxin Chen Jiahao Nie Yusong Wang Huiping Zhuang Manabu Okumura VLM CLL 79 10 0 27 Jun 2024
Foundational Models for Pathology and Endoscopy Images: Application for Gastric Inflammation H. Kerdegari Kyle Higgins Dennis Veselkov I. Laponogov I. Poļaka ... Junior Andrea Pescino M. Leja M. Dinis-Ribeiro T. F. Kanonnikoff Kirill Veselkov 106 5 0 26 Jun 2024
MAGIC: Meta-Ability Guided Interactive Chain-of-Distillation for Effective-and-Efficient Vision-and-Language Navigation Liuyi Wang Zongtao He Mengjiao Shen Jingwei Yang Chengju Liu Qijun Chen VLM 119 2 0 25 Jun 2024
DN-CL: Deep Symbolic Regression against Noise via Contrastive Learning Jingyi Liu Yanjie Li Lina Yu Min Wu Weijun Li Wenqiang Li Meilan Hao Yusong Deng Shu Wei 79 0 0 21 Jun 2024
Revealing Vision-Language Integration in the Brain with Multimodal Networks Vighnesh Subramaniam C. Conwell Christopher Wang Gabriel Kreiman Boris Katz Ignacio Cases Andrei Barbu 100 12 0 20 Jun 2024
LARP: Language Audio Relational Pre-training for Cold-Start Playlist Continuation Rebecca Salganik Xiaohao Liu Yunshan Ma Jian Kang Tat-Seng Chua CLL 92 2 0 20 Jun 2024
Towards a multimodal framework for remote sensing image change retrieval and captioning Roger Ferrod Luigi Di Caro Dino Ienco 52 2 0 19 Jun 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 120 7 0 18 Jun 2024
SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterations Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad CoGe 94 12 0 17 Jun 2024
LAIP: Learning Local Alignment from Image-Phrase Modeling for Text-based Person Search Haiguang Wang Yu Wu Mengxia Wu Cao Min Min Zhang 78 2 0 16 Jun 2024
Candidate Pseudolabel Learning: Enhancing Vision-Language Models by Prompt Tuning with Unlabeled Data Jiahan Zhang Qinglai Wei Feng Liu Lei Feng VLM 86 12 0 15 Jun 2024
Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesis Zongyue Qin Yunsheng Bai Atefeh Sohrabizadeh Zijian Ding Ziniu Hu Yizhou Sun Jason Cong 84 2 0 13 Jun 2024
Explore the Limits of Omni-modal Pretraining at Scale Yiyuan Zhang Handong Li Jing Liu Xiangyu Yue VLM LRM 82 1 0 13 Jun 2024
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms Miaosen Zhang Yixuan Wei Zhen Xing Yifei Ma Zuxuan Wu ... Zheng Zhang Qi Dai Chong Luo Xin Geng Baining Guo VLM 84 1 0 13 Jun 2024
Language-driven Grasp Detection An Dinh Vuong Minh Nhat Vu Baoru Huang Nghia Nguyen Hieu Le T. Vo Anh Nguyen VLM 116 19 0 13 Jun 2024
MirrorCheck: Efficient Adversarial Defense for Vision-Language Models Samar Fares Klea Ziu Toluwani Aremu Nikita Durasov Martin Takáč Pascal Fua Karthik Nandakumar Ivan Laptev VLM AAML 99 5 0 13 Jun 2024
Efficient Multi-View Fusion and Flexible Adaptation to View Missing in Cardiovascular System Signals Qihan Hu Daomiao Wang Hong Wu Jian Liu Cuiwei Yang 92 0 0 13 Jun 2024
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus Matthieu Futeral A. Zebaze Pedro Ortiz Suarez Julien Abadji Rémi Lacroix Cordelia Schmid Rachel Bawden Benoît Sagot 167 3 0 13 Jun 2024