How Much Can CLIP Benefit Vision-and-Language Tasks?

13 July 2021

Papers citing "How Much Can CLIP Benefit Vision-and-Language Tasks?"

50 / 104 papers shown

Title
Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation Junrong Yue Yuhang Zhang Chuan Qin Jing Chen Xiaomin Lie Xinlei Yu Wenxin Zhang Zhendong Zhao 54 0 0 23 Apr 2025
Enhanced Continual Learning of Vision-Language Models with Model Fusion Haoyuan Gao Zicong Zhang Yuqi Wei Linglan Zhao Guilin Li Y. Li Linghe Kong Weiran Huang CLL VLM 167 0 0 12 Mar 2025
Can Hallucination Correction Improve Video-Language Alignment? Lingjun Zhao Mingyang Xie Paola Cascante-Bonilla Hal Daumé III Kwonjoon Lee HILM VLM 62 0 0 20 Feb 2025
Predicate Hierarchies Improve Few-Shot State Classification Emily Jin Joy Hsu Jiajun Wu OffRL 79 0 0 18 Feb 2025
Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models Yue Zhang Ziqiao Ma Jialu Li Yanyuan Qiao Zun Wang J. Chai Qi Wu Joey Tianyi Zhou Parisa Kordjamshidi LRM 63 18 0 31 Dec 2024
Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction Sepideh Maleki Jan-Christian Huetter Kangway V Chuang Gabriele Scalia Tommaso Biancalani Tommaso Biancalani AI4CE 90 2 0 18 Dec 2024
Calibrated Cache Model for Few-Shot Vision-Language Model Adaptation Kun Ding Qiang Yu Haojian Zhang Gaofeng Meng Shiming Xiang VLM 30 0 0 11 Oct 2024
The Wallpaper is Ugly: Indoor Localization using Vision and Language Seth Pate Lawson L. S. Wong 33 0 0 04 Oct 2024
ET tu, CLIP? Addressing Common Object Errors for Unseen Environments Ye Won Byun Cathy Jiao Shahriar Noroozizadeh Jimin Sun Rosa Vitiello VLM 44 1 0 25 Jun 2024
Enhancing Large Vision Language Models with Self-Training on Image Comprehension Yihe Deng Pan Lu Fan Yin Ziniu Hu Sheng Shen James Zou Kai-Wei Chang Wei Wang SyDa VLM LRM 44 36 0 30 May 2024
CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning Yiping Wang Yifang Chen Wendan Yan Alex Fang Wenjing Zhou Kevin G. Jamieson S. Du 36 7 0 29 May 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 44 1 0 06 Feb 2024
CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data Jiangming Shi Shanshan Zheng Xiangbo Yin Yang Lu Yuan Xie Yanyun Qu VLM FedML 44 10 0 14 Dec 2023
What's left can't be right -- The remaining positional incompetence of contrastive vision-language models Nils Hoehing Ellen Rushe Anthony Ventresque VLM 18 2 0 20 Nov 2023
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning Manuele Barraco Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara VLM 55 19 0 23 Aug 2023
Reverse Stable Diffusion: What prompt was used to generate this image? Florinel-Alin Croitoru Vlad Hondru Radu Tudor Ionescu M. Shah VLM DiffM 36 6 0 02 Aug 2023
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding Rui Sun Zhecan Wang Haoxuan You Noel Codella Kai-Wei Chang Shih-Fu Chang CLIP 30 3 0 03 Jul 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 34 18 0 20 Jun 2023
Modularized Zero-shot VQA with Pre-trained Models Rui Cao Jing Jiang LRM 27 2 0 27 May 2023
GeoVLN: Learning Geometry-Enhanced Visual Representation with Slot Attention for Vision-and-Language Navigation Jingyang Huo Qiang Sun Boyan Jiang Haitao Lin Yanwei Fu 36 19 0 26 May 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 31 21 0 25 May 2023
CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model Shuai Zhao Xiaohan Wang Linchao Zhu Yezhou Yang CLIP VLM 23 25 0 23 May 2023
OneCAD: One Classifier for All image Datasets using multimodal learning S. Wadekar Eugenio Culurciello 37 0 0 11 May 2023
Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation Chaoya Jiang Wei Ye Haiyang Xu Miang yan Shikun Zhang Jie Zhang Fei Huang VLM 34 15 0 08 May 2023
IRFL: Image Recognition of Figurative Language Ron Yosef Yonatan Bitton Dafna Shahaf 40 18 0 27 Mar 2023
CoBIT: A Contrastive Bi-directional Image-Text Generation Model Haoxuan You Mandy Guo Zhecan Wang Kai-Wei Chang Jason Baldridge Jiahui Yu DiffM 49 12 0 23 Mar 2023
ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing Zequn Zeng Hao Zhang Zhengjue Wang Ruiying Lu Dongsheng Wang Bo Chen BDL DiffM 19 33 0 04 Mar 2023
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Zhou Yu Xuecheng Ouyang Zhenwei Shao Mei Wang Jun Yu MLLM 94 11 0 03 Mar 2023
Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learning Ivona Najdenkoska Xiantong Zhen M. Worring VLM 13 18 0 28 Feb 2023
Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images Xiaoman Zhang Chaoyi Wu Ya-Qin Zhang Yanfeng Wang Weidi Xie MedIm 44 120 0 27 Feb 2023
Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptions Henrik Voigt J. Hombeck M. Meuschke K. Lawonn Sina Zarrieß VLM 30 1 0 13 Feb 2023
Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation Bingqian Lin Yi Zhu Xiaodan Liang Liang Lin Jian-zhuo Liu CoGe LM&Ro 41 3 0 13 Feb 2023
CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets Zachary Novack Julian McAuley Zachary Chase Lipton Saurabh Garg VLM 29 79 0 06 Feb 2023
Learning Generalized Zero-Shot Learners for Open-Domain Image Geolocalization Lukas Haas Silas Alberti Michal Skreta VLM 32 21 0 01 Feb 2023
ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation KAI-QING Zhou Kai Zheng Connor Pryor Yilin Shen Hongxia Jin Lise Getoor Qing Guo 34 107 0 30 Jan 2023
GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods Da Yin Feng Gao Govind Thattai Michael F. Johnston Kai-Wei Chang VLM 32 15 0 05 Jan 2023
TeTIm-Eval: a novel curated evaluation data set for comparing text-to-image models Federico A. Galatolo M. G. Cimino E. Cogotti 19 4 0 15 Dec 2022
EPCL: Frozen CLIP Transformer is An Efficient Point Cloud Encoder Xiaoshui Huang Zhou Huang Shengjia Li Wentao Qu Tong He Yuenan Hou Yifan Zuo Wanli Ouyang 13 11 0 08 Dec 2022
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Yi Wang Kunchang Li Yizhuo Li Yinan He Bingkun Huang ... Junting Pan Jiashuo Yu Yali Wang Limin Wang Yu Qiao VLM VGen 55 309 0 06 Dec 2022
Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation Jiangyong Huang William Zhu Baoxiong Jia Zan Wang Xiaojian Ma Qing Li Siyuan Huang 37 5 0 28 Nov 2022
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang Limin Wang Yu Qiao ViT 30 107 0 17 Nov 2022
Multi-VQG: Generating Engaging Questions for Multiple Images Min-Hsuan Yeh Vicent Chen Ting-Hao Haung Lun-Wei Ku CoGe 18 7 0 14 Nov 2022
FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralization Junyan Wang Yi Zhang Jitao Sang FaML VLM 34 22 0 26 Oct 2022
MovieCLIP: Visual Scene Recognition in Movies Digbalay Bose Rajat Hebbar Krishna Somandepalli Haoyang Zhang Huayu Chen K. Cole-McLaughlin Haoran Wang Shrikanth Narayanan CLIP 19 21 0 20 Oct 2022
CPL: Counterfactual Prompt Learning for Vision and Language Models Xuehai He Diji Yang Weixi Feng Tsu-jui Fu Arjun Reddy Akula Varun Jampani P. Narayana Sugato Basu William Yang Wang Qing Guo VPVLM VLM 50 15 0 19 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 29 62 0 14 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 28 1 0 12 Oct 2022
Efficient Diffusion Models for Vision: A Survey Anwaar Ulhaq Naveed Akhtar MedIm 32 60 0 07 Oct 2022
A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning Aishwarya Kamath Peter Anderson Su Wang Jing Yu Koh Alexander Ku Austin Waters Yinfei Yang Jason Baldridge Zarana Parekh LM&Ro 22 45 0 06 Oct 2022
Towards Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baseline Lichen Zhao Daigang Cai Jing Zhang Lu Sheng Dong Xu Ruizhi Zheng Yinjie Zhao Lipeng Wang Xibo Fan 6 23 0 24 Sep 2022