v1v2 (latest)

Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts

17 February 2021

Papers citing "Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts"

50 / 871 papers shown

Title
Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality Jialing Yuan Ye Yu Gaurav Mittal Matthew Hall Sandra Sajeev Mei Chen 93 10 0 17 May 2023
IMAD: IMage-Augmented multi-modal Dialogue Viktor Moskvoretskii Anton Frolov Denis Kuznetsov 78 5 0 17 May 2023
CLIP-GCD: Simple Language Guided Generalized Category Discovery Rabah Ouldnoughi Chia-Wen Kuo Z. Kira VLM 82 14 0 17 May 2023
PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering Xiaoman Zhang Chaoyi Wu Ziheng Zhao Weixiong Lin Ya Zhang Yanfeng Wang Weidi Xie LM&MA 159 183 0 17 May 2023
Improved baselines for vision-language pre-training Enrico Fini Pietro Astolfi Adriana Romero Soriano Jakob Verbeek M. Drozdzal SSL CLIP VLM 128 23 0 15 May 2023
On the Hidden Mystery of OCR in Large Multimodal Models Yuliang Liu Zhang Li Mingxin Huang Chunyuan Li Dezhi Peng Mingyu Liu Lianwen Jin Xiang Bai VLM MLLM 142 96 0 13 May 2023
Measuring Progress in Fine-grained Vision-and-Language Understanding Emanuele Bugliarello Laurent Sartran Aishwarya Agrawal Lisa Anne Hendricks Aida Nematzadeh VLM 89 25 0 12 May 2023
An Inverse Scaling Law for CLIP Training Xianhang Li Zeyu Wang Cihang Xie VLM CLIP 115 58 0 11 May 2023
Continual Vision-Language Representation Learning with Off-Diagonal Information Zixuan Ni Longhui Wei Siliang Tang Yueting Zhuang Qi Tian VLM CLL 118 26 0 11 May 2023
VideoChat: Chat-Centric Video Understanding Kunchang Li Yinan He Yi Wang Yizhuo Li Wen Wang Ping Luo Yali Wang Limin Wang Yu Qiao MLLM 118 586 0 10 May 2023
Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception Hassan Akbari Dan Kondratyuk Huayu Chen Rachel Hornung Haoran Wang Hartwig Adam VLM MoE 105 13 0 10 May 2023
Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness Liangliang Cao Bowen Zhang Chen Chen Yinfei Yang Xianzhi Du Wen‐Cheng Zhang Zhiyun Lu Yantao Zheng CLIP VLM 75 15 0 08 May 2023
Otter: A Multi-Modal Model with In-Context Instruction Tuning Yue Liu Yuanhan Zhang Liangyu Chen Jinghao Wang Jingkang Yang Ziwei Liu MLLM 85 522 0 05 May 2023
LMEye: An Interactive Perception Network for Large Language Models Yunxin Li Baotian Hu Xinyu Chen Lin Ma Yong-mei Xu Hao Fei MLLM VLM 91 28 0 05 May 2023
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime Chuhan Zhang Antoine Miech Jiajun Shen Jean-Baptiste Alayrac Pauline Luc VLM VPVLM 90 2 0 03 May 2023
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model Peng Gao Jiaming Han Renrui Zhang Ziyi Lin Shijie Geng ... Pan Lu Conghui He Xiangyu Yue Hongsheng Li Yu Qiao MLLM 118 588 0 28 Apr 2023
DataComp: In search of the next generation of multimodal datasets S. Gadre Gabriel Ilharco Alex Fang J. Hayase Georgios Smyrnis ... A. Dimakis J. Jitsev Y. Carmon Vaishaal Shankar Ludwig Schmidt VLM 120 452 0 27 Apr 2023
Retrieval-based Knowledge Augmented Vision Language Pre-training Jiahua Rao Zifei Shan Long Liu Yao Zhou Yuedong Yang VLM 163 14 0 27 Apr 2023
Hypernymization of named entity-rich captions for grounding-based multi-modal pretraining Giacomo Nebbia Adriana Kovashka 103 0 0 25 Apr 2023
SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models Jonathan Roberts Kai Han Samuel Albanie VLM 94 14 0 23 Apr 2023
OmniLabel: A Challenging Benchmark for Language-Based Object Detection S. Schulter G. VijayKumarB. Yumin Suh Konstantinos M. Dafnis Zhixing Zhang Shiyu Zhao Dimitris N. Metaxas ObjD 70 12 0 22 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 169 2,077 0 20 Apr 2023
Hyperbolic Image-Text Representations Karan Desai Maximilian Nickel Tanmay Rajpurohit Justin Johnson Ramakrishna Vedantam VLM 109 67 0 18 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 582 4,945 0 17 Apr 2023
DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training Yihao Chen Xianbiao Qi Jianan Wang Lei Zhang 82 18 0 17 Apr 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 136 112 0 17 Apr 2023
OPI at SemEval 2023 Task 1: Image-Text Embeddings and Multimodal Information Retrieval for Visual Word Sense Disambiguation Slawomir Dadas 61 5 0 14 Apr 2023
MVP-SEG: Multi-View Prompt Learning for Open-Vocabulary Semantic Segmentation Jie Guo Qimeng Wang Yan Gao Xiaolong Jiang Xu Tang Yao Hu Baochang Zhang VLM 77 11 0 14 Apr 2023
Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text Wanrong Zhu Jack Hessel Anas Awadalla S. Gadre Jesse Dodge Alex Fang Youngjae Yu Ludwig Schmidt William Yang Wang Yejin Choi VLM 112 177 0 14 Apr 2023
Automated Cardiovascular Record Retrieval by Multimodal Learning between Electrocardiogram and Clinical Report Jielin Qiu Jiacheng Zhu Shiqi Liu William Jongwon Han Jingqi Zhang Chaojing Duan Michael A. Rosenberg Emerson Liu Douglas Weber Ding Zhao 41 0 0 13 Apr 2023
MoMo: A shared encoder Model for text, image and multi-Modal representations Rakesh Chada Zhao-Heng Zheng P. Natarajan ViT 62 4 0 11 Apr 2023
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment Lewei Yao Jianhua Han Xiaodan Liang Danqian Xu Wei Zhang Zhenguo Li Hang Xu VLM ObjD CLIP 121 79 0 10 Apr 2023
Probing Conceptual Understanding of Large Visual-Language Models Madeline Chantry Schiappa Raiyaan Abdullah Shehreen Azad Jared Claypoole Michael Cogswell Ajay Divakaran Yogesh S Rawat 81 16 0 07 Apr 2023
What's in a Name? Beyond Class Indices for Image Recognition Kai Han Yandong Li S. Vaze Jie Li Xuhui Jia VLM 85 7 0 05 Apr 2023
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data Vladislav Lialin Stephen Rawls David M. Chan Shalini Ghosh Anna Rumshisky Wael Hamza VLM AI4TS 94 6 0 04 Apr 2023
Associating Spatially-Consistent Grouping with Text-supervised Semantic Segmentation Yabo Zhang Zihao Wang Jun Hao Liew Jingjia Huang Manyu Zhu Jiashi Feng W. Zuo VLM 52 4 0 03 Apr 2023
Vision-Language Models for Vision Tasks: A Survey Jingyi Zhang Jiaxing Huang Sheng Jin Shijian Lu VLM 165 551 0 03 Apr 2023
DIME-FM: DIstilling Multimodal and Efficient Foundation Models Ximeng Sun Pengchuan Zhang Peizhao Zhang Hardik Shah Kate Saenko Xide Xia VLM 109 22 0 31 Mar 2023
Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models Eric Zhang Kai Wang Xingqian Xu Zhangyang Wang Humphrey Shi DiffM 132 193 0 30 Mar 2023
SoftCLIP: Softer Cross-modal Alignment Makes CLIP Stronger Yuting Gao Jinfeng Liu Zi-Han Xu Tong Wu Wen Liu Jie Yang Keren Li Xingen Sun CLIP VLM 64 47 0 30 Mar 2023
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research Xinhao Mei Chutong Meng Haohe Liu Qiuqiang Kong Tom Ko Chengqi Zhao Mark D. Plumbley Yuexian Zou Wenwu Wang 178 220 0 30 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 179 787 0 28 Mar 2023
Variational Distribution Learning for Unsupervised Text-to-Image Generation Minsoo Kang Doyup Lee Jiseob Kim Saehoon Kim Bohyung Han DRL OOD 67 4 0 28 Mar 2023
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Kunchang Li Yali Wang Yizhuo Li Yi Wang Yinan He Limin Wang Yu Qiao VGen 131 169 0 28 Mar 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 300 1,206 0 27 Mar 2023
Prompt-Guided Zero-Shot Anomaly Action Recognition using Pretrained Deep Skeleton Features Fumiaki Sato Ryo Hachiuma Taiki Sekii 71 22 0 27 Mar 2023
Revisiting Multimodal Representation in Contrastive Learning: From Patch and Token Embeddings to Finite Discrete Tokens Yuxiao Chen Jianbo Yuan Yu Tian Shijie Geng Xinyu Li Ding Zhou Dimitris N. Metaxas Hongxia Yang 81 37 0 27 Mar 2023
Accelerating Vision-Language Pretraining with Free Language Modeling Teng Wang Yixiao Ge Feng Zheng Ran Cheng Ying Shan Xiaohu Qie Ping Luo VLM MLLM 113 10 0 24 Mar 2023
Three ways to improve feature alignment for open vocabulary detection Relja Arandjelović A. Andonian A. Mensch Olivier J. Hénaff Jean-Baptiste Alayrac Andrew Zisserman VLM ObjD 120 19 0 23 Mar 2023
Open-Vocabulary Object Detection using Pseudo Caption Labels Han-Cheol Cho Won Young Jhoo Woohyun Kang Byungseok Roh VLM ObjD 60 20 0 23 Mar 2023