CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment

14 March 2022

Papers citing "CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment"

31 / 31 papers shown

Title
Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarks Chaoyi Wang Junjie Zheng Zihao Chen Shiyu Xia Chaofan Ding Xiaohao Zhang Xi Tao Xiaoming He Xinhan Di AuLLM 120 0 0 30 Apr 2025
Captured by Captions: On Memorization and its Mitigation in CLIP Models Wenhao Wang Adam Dziedzic Grace C. Kim Michael Backes Franziska Boenisch 93 0 0 11 Feb 2025
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion Marco Mistretta Alberto Baldrati Lorenzo Agnolucci Marco Bertini Andrew D. Bagdanov CLIP VLM 104 2 0 06 Feb 2025
R-LLaVA: Improving Med-VQA Understanding through Visual Region of Interest Xupeng Chen Zhixin Lai Kangrui Ruan Shichu Chen Jiaxiang Liu Zuozhu Liu 41 1 0 27 Oct 2024
One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models Hao Fang Jiawei Kong Wenbo Yu Bin Chen Jiawei Li Hao Wu Ke Xu Ke Xu AAML VLM 40 13 0 08 Jun 2024
HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models Gabriel H. Sarch Sahil Somani Raghav Kapoor Michael J. Tarr Katerina Fragkiadaki LM&Ro LLMAG 37 3 0 29 Apr 2024
Cross-Modal Coordination Across a Diverse Set of Input Modalities Jorge Sánchez Rodrigo Laguna VLM 44 0 0 29 Jan 2024
Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Prompts Yunshi Lan Xiang Li Xin Liu Yang Li Wei Qin Weining Qian LRM ReLM 41 24 0 15 Nov 2023
Sentence-level Prompts Benefit Composed Image Retrieval Yang Bai Xinxing Xu Yong-Jin Liu Salman Khan Fahad Khan Wangmeng Zuo Rick Siow Mong Goh Chun-Mei Feng 36 26 0 09 Oct 2023
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding Rui Sun Zhecan Wang Haoxuan You Noel Codella Kai-Wei Chang Shih-Fu Chang CLIP 30 3 0 03 Jul 2023
Modularized Zero-shot VQA with Pre-trained Models Rui Cao Jing Jiang LRM 30 2 0 27 May 2023
CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model Shuai Zhao Xiaohan Wang Linchao Zhu Yezhou Yang CLIP VLM 23 25 0 23 May 2023
ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing Zequn Zeng Hao Zhang Zhengjue Wang Ruiying Lu Dongsheng Wang Bo Chen BDL DiffM 19 33 0 04 Mar 2023
Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learning Ivona Najdenkoska Xiantong Zhen M. Worring VLM 18 18 0 28 Feb 2023
Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation Bingqian Lin Yi Zhu Xiaodan Liang Liang Lin Jian-zhuo Liu CoGe LM&Ro 41 3 0 13 Feb 2023
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval Kuniaki Saito Kihyuk Sohn Xiang Zhang Chun-Liang Li Chen-Yu Lee Kate Saenko Tomas Pfister 30 108 0 06 Feb 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 20 100 0 16 Jan 2023
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction Yue Yang Artemis Panagopoulou Marianna Apidianaki Mark Yatskar Chris Callison-Burch 26 2 0 24 Oct 2022
CPL: Counterfactual Prompt Learning for Vision and Language Models Xuehai He Diji Yang Weixi Feng Tsu-jui Fu Arjun Reddy Akula Varun Jampani P. Narayana Sugato Basu William Yang Wang Qing Guo VPVLM VLM 50 15 0 19 Oct 2022
Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection Shizhen Zhao Xiaojuan Qi 3DPC 48 17 0 11 Oct 2022
Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks Tianwei Chen Noa Garcia Mayu Otani Chenhui Chu Yuta Nakashima Hajime Nagahara VLM 41 0 0 23 Aug 2022
Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models Huy Ha Shuran Song LM&Ro VLM 43 101 0 23 Jul 2022
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action Dhruv Shah B. Osinski Brian Ichter Sergey Levine LM&Ro 158 436 0 10 Jul 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 18 572 0 01 Apr 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 82 213 0 18 Feb 2022
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 202 405 0 13 Jul 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 316 3,708 0 11 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 243 1,919 0 31 Dec 2020
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages Peng Qi Yuhao Zhang Yuhui Zhang Jason Bolton Christopher D. Manning AI4TS 213 1,654 0 16 Mar 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,589 0 21 Jan 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 417 2,588 0 03 Sep 2019