UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning

31 December 2020

Papers citing "UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning"

50 / 86 papers shown

Title
Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering Zeqing Wang Wentao Wan Qiqing Lao Runmeng Chen Minjie Lang Keze Wang Liang Lin Liang Lin LRM 103 3 0 17 Feb 2025
CMAL: A Novel Cross-Modal Associative Learning Framework for Vision-Language Pre-Training Zhiyuan Ma Jianjun Li Guohui Li Kaiyan Huang VLM 56 9 0 16 Oct 2024
ComAlign: Compositional Alignment in Vision-Language Models Ali Abdollah Amirmohammad Izadi Armin Saghafian Reza Vahidimajd Mohammad Mozafari Amirreza Mirzaei Mohammadmahdi Samiei M. Baghshah CoGe VLM 30 0 0 12 Sep 2024
WeCromCL: Weakly Supervised Cross-Modality Contrastive Learning for Transcription-only Supervised Text Spotting Jingjing Wu Zhengyao Fang Pengyuan Lyu Chengquan Zhang Fanglin Chen Guangming Lu Wenjie Pei 50 2 0 28 Jul 2024
Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning Shuvendu Roy Yasaman Parhizkar Franklin Ogidi Vahid Reza Khazaie Michael Colacci Ali Etemad Elham Dolatabadi Arash Afkanpour VLM 52 1 0 11 Jun 2024
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models Hao Fang Jiawei Kong Wenbo Yu Bin Chen Jiawei Li Hao Wu Ke Xu Ke Xu AAML VLM 40 13 0 08 Jun 2024
Modeling Caption Diversity in Contrastive Vision-Language Pretraining Samuel Lavoie Polina Kirichenko Mark Ibrahim Mahmoud Assran Andrew Gordon Wilson Aaron Courville Nicolas Ballas CLIP VLM 67 20 0 30 Apr 2024
VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding Ahmad A Mahmood Ashmal Vayani Muzammal Naseer Salman Khan Fahad Shahbaz Khan LRM 56 7 0 21 Mar 2024
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 47 3 0 04 Mar 2024
UNIMO-G: Unified Image Generation through Multimodal Conditional Diffusion Wei Li Xue Xu Jiachen Liu Xinyan Xiao 25 5 0 24 Jan 2024
Misalign, Contrast then Distill: Rethinking Misalignments in Language-Image Pretraining Bumsoo Kim Yeonsik Jo Jinhyung Kim S. Kim VLM 27 7 0 19 Dec 2023
Expediting Contrastive Language-Image Pretraining via Self-distilled Encoders Bumsoo Kim Jinhyung Kim Yeonsik Jo S. Kim VLM 26 3 0 19 Dec 2023
Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models Jiayun Luo Siddhesh Khandelwal Leonid Sigal Boyang Albert Li MLLM VLM 35 7 0 28 Nov 2023
Vision Language Transformers: A Survey Clayton Fields C. Kennington VLM 28 5 0 06 Jul 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 34 18 0 20 Jun 2023
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models Shuo Chen Jindong Gu Zhen Han Yunpu Ma Philip Torr Volker Tresp VPVLM VLM 34 17 0 03 Jun 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 15 4 0 23 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 48 115 0 18 May 2023
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding Le Xue Ning Yu Shu Zhen Zhang Artemis Panagopoulou Junnan Li ... Jiajun Wu Caiming Xiong Ran Xu Juan Carlos Niebles Silvio Savarese 24 115 0 14 May 2023
Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solution Jianfeng Kuang Wei Hua Dingkang Liang Mingkun Yang Deqiang Jiang Bo Ren Xiang Bai 27 39 0 12 May 2023
Combo of Thinking and Observing for Outside-Knowledge VQA Q. Si Yuchen Mo Zheng Lin Huishan Ji Weiping Wang 46 13 0 10 May 2023
Text with Knowledge Graph Augmented Transformer for Video Captioning Xin Gu G. Chen Yufei Wang Libo Zhang Tiejian Luo Longyin Wen 27 47 0 22 Mar 2023
Multi-modal Large Language Model Enhanced Pseudo 3D Perception Framework for Visual Commonsense Reasoning Jian Zhu Hanli Wang Miaojing Shi LRM 21 4 0 30 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 20 100 0 16 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 19 17 0 12 Jan 2023
SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout Graph Yuxing Long Binyuan Hui Fulong Ye Yanyang Li Zhuoxin Han Caixia Yuan Yongbin Li Xiaojie Wang LLMAG 30 7 0 05 Jan 2023
GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods Da Yin Feng Gao Govind Thattai Michael F. Johnston Kai-Wei Chang VLM 32 15 0 05 Jan 2023
UnICLAM:Contrastive Representation Learning with Adversarial Masking for Unified and Interpretable Medical Vision Question Answering Chenlu Zhan Peng Peng Hongsen Wang Tao Chen Hongwei Wang MedIm 23 3 0 21 Dec 2022
Position-guided Text Prompt for Vision-Language Pre-training Alex Jinpeng Wang Pan Zhou Mike Zheng Shou Shuicheng Yan VLM 24 37 0 19 Dec 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 24 9 0 21 Nov 2022
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works Chao Tao Ji Qi Mingning Guo Qing Zhu Haifeng Li SSL 24 56 0 15 Nov 2022
Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection Yanxin Long Jianhua Han Runhu Huang Xu Hang Yi Zhu Chunjing Xu Xiaodan Liang VLM ObjD 32 18 0 02 Nov 2022
Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering Q. Si Yuanxin Liu Zheng Lin Peng Fu Weiping Wang VLM 42 1 0 26 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 32 62 0 14 Oct 2022
Can Language Representation Models Think in Bets? Zhi–Bin Tang Mayank Kejriwal 15 6 0 14 Oct 2022
Low-resource Neural Machine Translation with Cross-modal Alignment Zhe Yang Qingkai Fang Yang Feng VLM 37 9 0 13 Oct 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Joey Tianyi Zhou VLM 51 28 0 28 Sep 2022
WildQA: In-the-Wild Video Question Answering Santiago Castro Naihao Deng Pingxuan Huang Mihai Burzo Rada Mihalcea 70 7 0 14 Sep 2022
Anticipating the Unseen Discrepancy for Vision and Language Navigation Yujie Lu Huiliang Zhang Ping Nie Weixi Feng Wenda Xu Qing Guo William Yang Wang 35 1 0 10 Sep 2022
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining Xiaoyi Dong Jianmin Bao Yinglin Zheng Ting Zhang Dongdong Chen ... Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP VLM 54 158 0 25 Aug 2022
Interpreting Song Lyrics with an Audio-Informed Pre-trained Language Model Yixiao Zhang Junyan Jiang Gus Xia S. Dixon 33 9 0 24 Aug 2022
Semantic-Enhanced Image Clustering Shao-Qian Cai Li-qing Qiu Xiaojun Chen Qin Zhang Long Chen VLM 31 13 0 21 Aug 2022
CODER: Coupled Diversity-Sensitive Momentum Contrastive Learning for Image-Text Retrieval Haoran Wang Dongliang He Wenhao Wu Boyang Xia Min Yang Fu Li Yunlong Yu Zhong Ji Errui Ding Jingdong Wang 30 22 0 21 Aug 2022
Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics Xiaoyuan Guo Jiali Duan C.-C. Jay Kuo J. Gichoya Imon Banerjee VLM 22 1 0 31 Jul 2022
Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training Haoxuan You Luowei Zhou Bin Xiao Noel Codella Yu Cheng Ruochen Xu Shih-Fu Chang Lu Yuan CLIP VLM 24 48 0 26 Jul 2022
Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding Quan Liu Youpeng Wen Jianhua Han Chunjing Xu Hang Xu Xiaodan Liang VLM 26 67 0 18 Jul 2022
Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detection Jiashuo Yu Jin-Yuan Liu Ying Cheng Rui Feng Yuejie Zhang 19 34 0 12 Jul 2022
VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix Teng Wang Wenhao Jiang Zhichao Lu Feng Zheng Ran Cheng Chengguo Yin Ping Luo VLM 34 43 0 17 Jun 2022
Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval Xiao Dong Xunlin Zhan Yunchao Wei Xiaoyong Wei Yaowei Wang Minlong Lu Xiaochun Cao Xiaodan Liang 24 11 0 17 Jun 2022
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone Zi-Yi Dou Aishwarya Kamath Zhe Gan Pengchuan Zhang Jianfeng Wang ... Ce Liu Yann LeCun Nanyun Peng Jianfeng Gao Lijuan Wang VLM ObjD 30 124 0 15 Jun 2022