DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training

6 March 2023

Yi Yang

Papers citing "DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training"

21 / 21 papers shown

Title
Instruct-CLIP: Improving Instruction-Guided Image Editing with Automated Data Refinement Using Contrastive Learning Sherry X Chen Misha Sra Pradeep Sen 52 0 0 24 Mar 2025
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion Marco Mistretta Alberto Baldrati Lorenzo Agnolucci Marco Bertini Andrew D. Bagdanov CLIP VLM 104 2 0 06 Feb 2025
Unleashing Text-to-Image Diffusion Prior for Zero-Shot Image Captioning Jianjie Luo Jingwen Chen Yehao Li Yingwei Pan Jianlin Feng Hongyang Chao Ting Yao DiffM VLM 53 0 0 03 Jan 2025
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 84 25 0 04 Oct 2024
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 44 7 0 30 Sep 2024
Segment Any 3D Object with Language Seungjun Lee Yuyang Zhao Gim Hee Lee 41 1 0 02 Apr 2024
Prioritized Semantic Learning for Zero-shot Instance Navigation Xander Sun Louis Lau Hoyard Zhi Ronghe Qiu Junwei Liang 37 8 0 18 Mar 2024
MeaCap: Memory-Augmented Zero-shot Image Captioning Zequn Zeng Yan Xie Hao Zhang Chiyu Chen Zhengjue Wang Boli Chen VLM 39 14 0 06 Mar 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 41 29 0 20 Feb 2024
Zero-shot Translation of Attention Patterns in VQA Models to Natural Language Leonard Salewski A. Sophia Koepke Hendrik P. A. Lensch Zeynep Akata 31 2 0 08 Nov 2023
Weakly-supervised Automated Audio Captioning via text only training Theodoros Kouzelis V. Katsouros CLIP 27 6 0 21 Sep 2023
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning Bang-ju Yang Fenglin Liu X. Wu Yaowei Wang Xu Sun Yuexian Zou VLM CLIP 44 13 0 25 Aug 2023
Transferable Decoding with Visual Entities for Zero-Shot Image Captioning Junjie Fei Teng Wang Jinrui Zhang Zhenyu He Chengjie Wang Feng Zheng VLM 28 34 0 31 Jul 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 47 0 0 10 Jul 2023
Multimodal Knowledge Alignment with Reinforcement Learning Youngjae Yu Jiwan Chung Heeseung Yun Jack Hessel J. Park ... Prithviraj Ammanabrolu Rowan Zellers Ronan Le Bras Gunhee Kim Yejin Choi VLM 115 36 0 25 May 2022
Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation Fenglin Liu Chenyu You Xian Wu Shen Ge Sheng Wang Xu Sun MedIm 81 92 0 08 Nov 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Mohit Bansal Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 196 405 0 13 Jul 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Tsung-Yi Lin Weicheng Kuo Yin Cui VLM ObjD 225 898 0 28 Apr 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 278 1,082 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,700 0 11 Feb 2021
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning Jiasen Lu Caiming Xiong Devi Parikh R. Socher 85 1,442 0 06 Dec 2016