v1v2 (latest)

Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

16 July 2021

Junnan Li

Ramprasaath R. Selvaraju

Akhilesh Deepak Gotmare

ArXiv (abs)PDF HTML Github (1658★)

Papers citing "Align before Fuse: Vision and Language Representation Learning with Momentum Distillation"

50 / 1,231 papers shown

Title
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations? Shouwei Ruan Hanqin Liu Yao Huang Xiaoqi Wang Caixin Kang Hang Su Yinpeng Dong Xingxing Wei VGen 218 0 0 04 Dec 2024
Exploring Large Vision-Language Models for Robust and Efficient Industrial Anomaly Detection Kun Qian Tianyu Sun Wenhong Wang 108 0 0 01 Dec 2024
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding Zilin Du Haoxin Li Jianfei Yu Boyang Li 495 0 0 01 Dec 2024
VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysis Donggoo Kang Dasol Jeong Hyunmin Lee Sangwoo Park Hasil Park Sunkyu Kwon Yeongjoon Kim Joonki Paik MLLM VLM 148 0 0 27 Nov 2024
Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search Shuyu Yang Yaxiong Wang Li Zhu Zhedong Zheng 214 7 0 26 Nov 2024
Leveraging the Power of MLLMs for Gloss-Free Sign Language Translation Jungeun Kim Hyeongwoo Jeon Jongseong Bae Ha Young Kim SLR 122 0 0 25 Nov 2024
ResCLIP: Residual Attention for Training-free Dense Vision-language Inference Yuhang Yang Jinhong Deng Wen Li Lixin Duan VLM 108 1 0 24 Nov 2024
Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment Alvi Md Ishmam Christopher Thomas AAML 180 3 0 23 Nov 2024
Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval Zengbao Sun Ming Zhao Gaorui Liu Andre Kaup 143 4 0 22 Nov 2024
ORID: Organ-Regional Information Driven Framework for Radiology Report Generation Tiancheng Gu Kaicheng Yang Xiang An Ziyong Feng Dongnan Liu Weidong Cai 162 1 0 20 Nov 2024
Joint Vision-Language Social Bias Removal for CLIP Haoyu Zhang Yangyang Guo Mohan S. Kankanhalli VLM 182 1 0 19 Nov 2024
SayComply: Grounding Field Robotic Tasks in Operational Compliance through Retrieval-Based Language Models M. Ginting Dong-Ki Kim Sung-Kyun Kim Bandi Jai Krishna Mykel J. Kochenderfer Shayegan Omidshafiei Ali-akbar Agha-mohammadi LM&Ro 144 0 0 18 Nov 2024
TP-UNet: Temporal Prompt Guided UNet for Medical Image Segmentation Ranmin Wang Limin Zhuang Hongkun Chen Boyan Xu Ruichu Cai 53 0 0 18 Nov 2024
ITACLIP: Boosting Training-Free Semantic Segmentation with Image, Text, and Architectural Enhancements M. Arda Aydın Efe Mert Çırpar Elvin Abdinli Gözde B. Ünal Y. Sahin VLM 291 1 0 18 Nov 2024
TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models Jonathan Fhima Elad Ben Avraham Oren Nuriel Yair Kittenplon Roy Ganz Aviad Aberdam Ron Litman VLM 67 1 0 07 Nov 2024
Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack Xiaojun Jia Sensen Gao Qing Guo Ke Ma Yihao Huang Simeng Qin Yang Liu Ivor Tsang Fellow Xiaochun Cao AAML 87 3 0 04 Nov 2024
SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities Ehsan Faghihi Mohammedreza Zarenejad Ali-Asghar Beheshti Shirazi 72 1 0 04 Nov 2024
Multiple Information Prompt Learning for Cloth-Changing Person Re-Identification Shengxun Wei Zan Gao Yibo Zhao Weili Guan Weili Guan Shengyong Chen 135 2 0 01 Nov 2024
Nearest Neighbor Normalization Improves Multimodal Retrieval Neil Chowdhury Franklin Wang Sumedh Shenoy Douwe Kiela Sarah Schwettmann Tristan Thrush VLM 132 3 0 31 Oct 2024
MoTaDual: Modality-Task Dual Alignment for Enhanced Zero-shot Composed Image Retrieval Haiwen Li Fei Su Zhicheng Zhao 79 0 0 31 Oct 2024
Driving by the Rules: A Benchmark for Integrating Traffic Sign Regulations into Vectorized HD Map Xinyuan Chang Maixuan Xue Xinran Liu Zheng Pan Xing Wei 213 2 0 31 Oct 2024
CLIPErase: Efficient Unlearning of Visual-Textual Associations in CLIP Tianyu Yang Lisen Dai Zheyuan Liu Minhao Cheng Meng Jiang Yapeng Tian VLM MU 96 5 0 30 Oct 2024
Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models Lu Yu Haiyang Zhang Changsheng Xu AAML VLM 102 7 0 29 Oct 2024
Enhancing CTR Prediction in Recommendation Domain with Search Query Representation Yuening Wang M. Chen Yaochen Hu Wei Guo Yingxue Zhang Huifeng Guo Yong Liu Mark Coates 58 1 0 28 Oct 2024
Domain Adaptation with a Single Vision-Language Embedding Mohammad Fahes Tuan-Hung Vu Andrei Bursuc Patrick Pérez Raoul de Charette VLM 71 0 0 28 Oct 2024
ADEM-VL: Adaptive and Embedded Fusion for Efficient Vision-Language Tuning Zhiwei Hao Jianyuan Guo Li Shen Yong Luo Han Hu Yonggang Wen VLM 92 0 0 23 Oct 2024
EntityCLIP: Entity-Centric Image-Text Matching via Multimodal Attentive Contrastive Learning Yaxiong Wang Yijiao Wang Lianwei Wu Lechao Cheng Zhun Zhong Meng Wang VLM 98 0 0 23 Oct 2024
Captions Speak Louder than Images (CASLIE): Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Data Xinyi Ling Bo Peng Hanwen Du Zhihui Zhu Xia Ning 107 0 0 22 Oct 2024
IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing Kang Chen Qingheng Zhang Chengbao Lian Yixin Ji Xuwei Liu Shuguang Han Guoqiang Wu Fei Huang Jufeng Chen 61 2 0 22 Oct 2024
Generalized Multimodal Fusion via Poisson-Nernst-Planck Equation Jiayu Xiong Jing Wang Hengjing Xiang Jun Xue Chen Xu Zhouqiang Jiang 57 0 0 20 Oct 2024
BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping Taolin Zhang Jinqiao Wang Hang Guo Tao Dai Bin Chen Shu-Tao Xia VLM TTA 72 0 0 20 Oct 2024
CMAL: A Novel Cross-Modal Associative Learning Framework for Vision-Language Pre-Training Zhiyuan Ma Jianjun Li Guohui Li Kaiyan Huang VLM 120 9 0 16 Oct 2024
Mind the Gap Between Prototypes and Images in Cross-domain Finetuning Hongduan Tian Feng Liu Zhanke Zhou Tongliang Liu Chengqi Zhang Bo Han VLM 134 1 0 16 Oct 2024
A Survey of Low-shot Vision-Language Model Adaptation via Representer Theorem Kun Ding Ying Wang Gaofeng Meng Shiming Xiang VLM 78 0 0 15 Oct 2024
Multi-modal Vision Pre-training for Medical Image Analysis Shaohao Rui Lingzhi Chen Zhenyu Tang Lilong Wang M. Liu Shanghang Zhang Xiaosong Wang 67 0 0 14 Oct 2024
Leveraging Customer Feedback for Multi-modal Insight Extraction Sandeep Sricharan Mukku Abinesh Kanagarajan Pushpendu Ghosh Chetan Aggarwal 29 0 0 13 Oct 2024
Skipping Computations in Multimodal LLMs Mustafa Shukor Matthieu Cord 63 3 0 12 Oct 2024
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering Ting Yu Kunhao Fu Shuhui Wang Qingming Huang Jun Yu 115 2 0 12 Oct 2024
M3Hop-CoT: Misogynous Meme Identification with Multimodal Multi-hop Chain-of-Thought G. Kumari Kirtan Jain Asif Ekbal 107 4 0 11 Oct 2024
LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts Anh-Quan Cao M. Jaritz Matthieu Guillaumin Raoul de Charette Loris Bazzani VLM CLIP 105 2 0 10 Oct 2024
OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling Linhui Xiao Xiaoshan Yang Fang Peng Yaowei Wang Changsheng Xu ObjD 122 7 0 10 Oct 2024
Structured Spatial Reasoning with Open Vocabulary Object Detectors Negar Nejatishahidin Madhukar Reddy Vongala Jana Kosecka 88 3 0 09 Oct 2024
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey Dianzhi Yu Xinni Zhang Yankai Chen Aiwei Liu Yifei Zhang Philip S. Yu Irwin King VLM CLL 99 13 0 07 Oct 2024
VISTA: A Visual and Textual Attention Dataset for Interpreting Multimodal Models Harshit Tolga Tasdizen CoGe VLM 59 1 0 06 Oct 2024
DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models Sungnyun Kim Haofu Liao Srikar Appalaraju Peng Tang Zhuowen Tu R. Satzoda R. Manmatha Vijay Mahadevan Stefano Soatto 104 0 0 04 Oct 2024
MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents Junpeng Yue Xinru Xu Börje F. Karlsson Zongqing Lu 116 1 0 04 Oct 2024
Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs Wei Wu Chao Wang L. Chen Mingze Yin Yiheng Zhu Kun Fu Jieping Ye Hui Xiong Zheng Wang 143 1 0 04 Oct 2024
Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks Junlin Hou Sicen Liu Yequan Bie Hongmei Wang Andong Tan Luyang Luo Hao Chen XAI 118 5 0 03 Oct 2024
Learning Multimodal Latent Generative Models with Energy-Based Prior Shiyu Yuan Jiali Cui Hanao Li Tian Han 58 1 0 30 Sep 2024
CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches Sifan Wu Amir Khasahmadi Mor Katz P. Jayaraman Yewen Pu K. Willis Bang Liu 3DV 72 9 0 26 Sep 2024