v1v2 (latest)

CoCa: Contrastive Captioners are Image-Text Foundation Models

4 May 2022

Mojtaba Seyedhosseini

Papers citing "CoCa: Contrastive Captioners are Image-Text Foundation Models"

50 / 935 papers shown

Title
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion Marco Mistretta Alberto Baldrati Lorenzo Agnolucci Marco Bertini Andrew D. Bagdanov CLIP VLM 170 5 0 06 Feb 2025
Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More Feng Wang Yaodong Yu Guoyizhe Wei Wei Shao Yuyin Zhou Alan Yuille Cihang Xie ViT 147 7 0 06 Feb 2025
Vision-Language Model Selection and Reuse for Downstream Adaptation Hao-Zhe Tan Zhi Zhou Lan-Zhe Guo Yu-Feng Li VLM 152 0 0 30 Jan 2025
Diffusion Generative Modeling for Spatially Resolved Gene Expression Inference from Histology Images Sichen Zhu Yuchen Zhu Molei Tao Peng-Chao Qiu MedIm 95 4 0 28 Jan 2025
Audio-Language Models for Audio-Centric Tasks: A survey Yi Su Jisheng Bai Qisheng Xu Kele Xu Yong Dou AuLLM 164 4 0 28 Jan 2025
Generating customized prompts for Zero-Shot Rare Event Medical Image Classification using LLM Payal Kamboj Ayan Banerjee Bin Xu Sandeep K. S. Gupta VLM MedIm 43 0 0 27 Jan 2025
With Great Backbones Comes Great Adversarial Transferability Erik Arakelyan Karen Hambardzumyan Davit Papikyan Pasquale Minervini Albert Gordo Isabelle Augenstein Aram H. Markosyan AAML 155 0 0 21 Jan 2025
BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature Alejandro Lozano Min Woo Sun James Burgess Liangyu Chen Jeffrey Nirschl ... Xiaohan Wang Yuhui Zhang Alfred Seunghoon Song Robert Tibshirani Serena Yeung-Levy LM&MA VLM MedIm 162 10 0 13 Jan 2025
VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks Ziyan Jiang Rui Meng Xinyi Yang Semih Yavuz Yingbo Zhou Wenhu Chen MLLM VLM 195 29 0 03 Jan 2025
Tuning Vision-Language Models with Candidate Labels by Prompt Alignment Zhifang Zhang Yuwei Niu Xin Liu Beibei Li VPVLM VLM 119 0 0 31 Dec 2024
Improving Generated and Retrieved Knowledge Combination Through Zero-shot Generation Xinkai Du Quanjie Han Chao Lv Yi Liu Yalin Sun Hao Shu Hongbo Shan Maosong Sun RALM 141 2 0 25 Dec 2024
Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations Yi Zhang Chun-Wun Cheng Junyi He Zhihai He Carola-Bibiane Schonlieb Yuyan Chen Angelica I Aviles-Rivero AI4TS 135 0 0 20 Dec 2024
Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimization Yue Zhang Liqiang Jing Vibhav Gogate 205 5 0 19 Dec 2024
Bringing Multimodality to Amazon Visual Search System Xinliang Zhu Michael Huang Han Ding Jinyu Yang Kelvin Chen ... Son Dinh Tran Benjamin Z. Yao Doug Gray Anuj Bindal Arnab Dhua 112 3 0 17 Dec 2024
LLMs are Also Effective Embedding Models: An In-depth Overview Chongyang Tao Tao Shen Shen Gao Junshuo Zhang Zhen Li Zhengwei Tao Shuai Ma 143 11 0 17 Dec 2024
CATSplat: Context-Aware Transformer with Spatial Guidance for Generalizable 3D Gaussian Splatting from A Single-View Image Wonseok Roh Hwanhee Jung Jong Wook Kim Seanie Lee Innfarn Yoo Andreas Lugmayr Seunggeun Chi K. Ramani Sangpil Kim 3DGS 171 2 0 17 Dec 2024
SAMIC: Segment Anything with In-Context Spatial Prompt Engineering S. Nagendra Kashif Rashid Chaopeng Shen Daniel Kifer VLM 143 2 0 16 Dec 2024
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval Haoyu Jiang Zhi-Qi Cheng Gabriel Moreira Jiawen Zhu Jingdong Sun Bukun Ren Jun-Yan He Qi Dai Xian-Sheng Hua VLM 142 0 0 14 Dec 2024
DiffCLIP: Few-shot Language-driven Multimodal Classifier Jiaqing Zhang Mingxiang Cao Xue Yang Kai Jiang Yunsong Li VLM 123 0 0 10 Dec 2024
LLaVA-SpaceSGG: Visual Instruct Tuning for Open-vocabulary Scene Graph Generation with Enhanced Spatial Relations Mingjie Xu Mengyang Wu Yuzhi Zhao Jason Chun Lok Li Weifeng Ou LRM SyDa VLM 129 4 0 09 Dec 2024
Unified Framework for Open-World Compositional Zero-shot Learning Hirunima Jayasekara Khoi Pham Nirat Saini Abhinav Shrivastava 94 0 0 05 Dec 2024
FLAIR: VLM with Fine-grained Language-informed Image Representations Rui Xiao Sanghwan Kim Mariana-Iuliana Georgescu Zeynep Akata Stephan Alaniz VLM CLIP 138 4 0 04 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 346 3 0 02 Dec 2024
CAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives Armin Saghafian Amirmohammad Izadi Negin Hashemi Dijujin M. Baghshah 150 0 0 29 Nov 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 243 12 0 28 Nov 2024
VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysis Donggoo Kang Dasol Jeong Hyunmin Lee Sangwoo Park Hasil Park Sunkyu Kwon Yeongjoon Kim Joonki Paik MLLM VLM 148 0 0 27 Nov 2024
Evaluating Vision-Language Models as Evaluators in Path Planning Mohamed Aghzal Xiang Yue Erion Plaku Ziyu Yao LRM 230 1 0 27 Nov 2024
ResCLIP: Residual Attention for Training-free Dense Vision-language Inference Yuhang Yang Jinhong Deng Wen Li Lixin Duan VLM 108 1 0 24 Nov 2024
Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentation Sule Bai Yong-Jin Liu Yifei Han Haoji Zhang Yansong Tang VLM 325 8 0 24 Nov 2024
Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment Alvi Md Ishmam Christopher Thomas AAML 180 3 0 23 Nov 2024
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining Ming Hu Kun Yuan Yaling Shen Feilong Tang Xiaohao Xu ... Jin Ye N. Padoy Nassir Navab Junjun He Zongyuan Ge VLM CLIP 179 12 0 23 Nov 2024
Towards a Comprehensive Benchmark for Pathological Lymph Node Metastasis in Breast Cancer Sections Xitong Ling Yuanyuan Lei Jiawen Li Junru Cheng Wenting Huang Tian Guan Jian Guan Yonghong He 50 4 0 16 Nov 2024
Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentation Yuheng Shi Minjing Dong Chang Xu VLM 118 3 0 14 Nov 2024
Which Viewpoint Shows it Best? Language for Weakly Supervising View Selection in Multi-view Instructional Videos Sagnik Majumder Tushar Nagarajan Ziad Al-Halah Reina Pradhan Kristen Grauman 80 0 0 13 Nov 2024
Classification Done Right for Vision-Language Pre-Training Zilong Huang Qinghao Ye Bingyi Kang Jiashi Feng Haoqi Fan CLIP VLM 122 4 0 05 Nov 2024
Domain Expansion and Boundary Growth for Open-Set Single-Source Domain Generalization Pengkun Jiao Na Zhao Jingjing Chen Yu-Gang Jiang OOD 142 0 0 05 Nov 2024
INQUIRE: A Natural World Text-to-Image Retrieval Benchmark Edward Vendrow Omiros Pantazis Alexander Shepard Gabriel J. Brostow Kate E. Jones Oisin Mac Aodha Sara Beery Grant Van Horn VLM 105 7 0 04 Nov 2024
Driving by the Rules: A Benchmark for Integrating Traffic Sign Regulations into Vectorized HD Map Xinyuan Chang Maixuan Xue Xinran Liu Zheng Pan Xing Wei 213 2 0 31 Oct 2024
EMMA: End-to-End Multimodal Model for Autonomous Driving Jyh-Jing Hwang Runsheng Xu Hubert Lin Wei-Chih Hung Jingwei Ji ... Benjamin Sapp Yin Zhou James Guo Dragomir Anguelov Mingxing Tan VLM LM&Ro 108 38 0 30 Oct 2024
AlphaChimp: Tracking and Behavior Recognition of Chimpanzees Xiaoxuan Ma Yutang Lin Yuan Xu Stephan P. Kaufhold Jack Terwilliger Andres Meza Yixin Zhu Federico Rossano Yizhou Wang 122 0 0 22 Oct 2024
Beyond Filtering: Adaptive Image-Text Quality Enhancement for MLLM Pretraining Han Huang Yuqi Huo Zijia Zhao Haoyu Lu Shu Wu Bin Wang Qiang Liu Weipeng Chen Liang Wang VLM 67 1 0 21 Oct 2024
TIPS: Text-Image Pretraining with Spatial awareness Kevis-Kokitsi Maninis Kaifeng Chen Soham Ghosh Arjun Karpur Koert Chen ... Jan Dlabal Dan Gnanapragasam Mojtaba Seyedhosseini Howard Zhou Andre Araujo VLM 129 3 0 21 Oct 2024
xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs Michael S Ryoo Honglu Zhou Shrikant B. Kendre Can Qin Le Xue ... Kanchana Ranasinghe Caiming Xiong Ran Xu Caiming Xiong Juan Carlos Niebles VGen 104 15 0 21 Oct 2024
Assistive AI for Augmenting Human Decision-making Natabara Máté Gyöngyössy Bernát Török Csilla Farkas Laura Lucaj Attila Menyhárd Krisztina Menyhárd-Balázs András Simonyi Patrick van der Smagt Zsolt Ződi András Lőrincz 70 0 0 18 Oct 2024
Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models Ce Zhang Simon Stepputtis Katia Sycara Yaqi Xie VLM 102 6 0 16 Oct 2024
DRACO: A Denoising-Reconstruction Autoencoder for Cryo-EM Yingjun Shen Haizhao Dai Qihe Chen Yan Zeng Jiakai Zhang Yuan Pei Jingyi Yu 118 3 0 15 Oct 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Zou Tatsunori Hashimoto VLM 263 7 0 14 Oct 2024
Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models Sathya Kamesh Bhethanabhotla Omar Swelam Julien N. Siems David Salinas Frank Hutter Mamba AI4TS AI4CE 97 8 0 12 Oct 2024
Calibrated Cache Model for Few-Shot Vision-Language Model Adaptation Kun Ding Qiang Yu Haojian Zhang Gaofeng Meng Shiming Xiang VLM 59 0 0 11 Oct 2024
On a Hidden Property in Computational Imaging Yinan Feng Yinpeng Chen Yueh Lee Youzuo Lin 80 0 0 11 Oct 2024