How Much Can CLIP Benefit Vision-and-Language Tasks?

13 July 2021

Papers citing "How Much Can CLIP Benefit Vision-and-Language Tasks?"

50 / 89 papers shown

Title
Think Hierarchically, Act Dynamically: Hierarchical Multi-modal Fusion and Reasoning for Vision-and-Language Navigation Junrong Yue Y. Zhang Chuan Qin Bo Li Xiaomin Lie Xinlei Yu Wenxin Zhang Zhendong Zhao 49 0 0 23 Apr 2025
Enhanced Continual Learning of Vision-Language Models with Model Fusion Haoyuan Gao Zicong Zhang Yuqi Wei Linglan Zhao Guilin Li Y. Li Linghe Kong Weiran Huang CLL VLM 151 0 0 12 Mar 2025
Can Hallucination Correction Improve Video-Language Alignment? Lingjun Zhao Mingyang Xie Paola Cascante-Bonilla Hal Daumé III Kwonjoon Lee HILM VLM 57 0 0 20 Feb 2025
Predicate Hierarchies Improve Few-Shot State Classification Emily Jin Joy Hsu Jiajun Wu OffRL 77 0 0 18 Feb 2025
Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models Yue Zhang Ziqiao Ma Jialu Li Yanyuan Qiao Zun Wang J. Chai Qi Wu Mohit Bansal Parisa Kordjamshidi LRM 57 18 0 31 Dec 2024
Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction Sepideh Maleki Jan-Christian Huetter Kangway V Chuang Gabriele Scalia Tommaso Biancalani Tommaso Biancalani AI4CE 85 2 0 18 Dec 2024
Calibrated Cache Model for Few-Shot Vision-Language Model Adaptation Kun Ding Qiang Yu Haojian Zhang Gaofeng Meng Shiming Xiang VLM 25 0 0 11 Oct 2024
The Wallpaper is Ugly: Indoor Localization using Vision and Language Seth Pate Lawson L. S. Wong 33 0 0 04 Oct 2024
ET tu, CLIP? Addressing Common Object Errors for Unseen Environments Ye Won Byun Cathy Jiao Shahriar Noroozizadeh Jimin Sun Rosa Vitiello VLM 32 1 0 25 Jun 2024
Enhancing Large Vision Language Models with Self-Training on Image Comprehension Yihe Deng Pan Lu Fan Yin Ziniu Hu Sheng Shen James Y. Zou Kai-Wei Chang Wei Wang SyDa VLM LRM 36 36 0 30 May 2024
CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning Yiping Wang Yifang Chen Wendan Yan Alex Fang Wenjing Zhou Kevin G. Jamieson S. Du 32 7 0 29 May 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 34 1 0 06 Feb 2024
CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data Jiangming Shi Shanshan Zheng Xiangbo Yin Yang Lu Yuan Xie Yanyun Qu VLM FedML 37 10 0 14 Dec 2023
What's left can't be right -- The remaining positional incompetence of contrastive vision-language models Nils Hoehing Ellen Rushe Anthony Ventresque VLM 15 2 0 20 Nov 2023
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning Manuele Barraco Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara VLM 53 19 0 23 Aug 2023
Reverse Stable Diffusion: What prompt was used to generate this image? Florinel-Alin Croitoru Vlad Hondru Radu Tudor Ionescu M. Shah VLM DiffM 34 6 0 02 Aug 2023
UniFine: A Unified and Fine-grained Approach for Zero-shot Vision-Language Understanding Rui Sun Zhecan Wang Haoxuan You Noel Codella Kai-Wei Chang Shih-Fu Chang CLIP 28 3 0 03 Jul 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 31 18 0 20 Jun 2023
Modularized Zero-shot VQA with Pre-trained Models Rui Cao Jing Jiang LRM 23 2 0 27 May 2023
GeoVLN: Learning Geometry-Enhanced Visual Representation with Slot Attention for Vision-and-Language Navigation Jingyang Huo Qiang Sun Boyan Jiang Haitao Lin Yanwei Fu 32 19 0 26 May 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 27 21 0 25 May 2023
OneCAD: One Classifier for All image Datasets using multimodal learning S. Wadekar Eugenio Culurciello 32 0 0 11 May 2023
Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation Chaoya Jiang Wei Ye Haiyang Xu Miang yan Shikun Zhang Jie Zhang Fei Huang VLM 23 15 0 08 May 2023
IRFL: Image Recognition of Figurative Language Ron Yosef Yonatan Bitton Dafna Shahaf 35 17 0 27 Mar 2023
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Zhou Yu Xuecheng Ouyang Zhenwei Shao Mei Wang Jun Yu MLLM 89 11 0 03 Mar 2023
Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images Xiaoman Zhang Chaoyi Wu Ya-Qin Zhang Yanfeng Wang Weidi Xie MedIm 30 119 0 27 Feb 2023
Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptions Henrik Voigt J. Hombeck M. Meuschke K. Lawonn Sina Zarrieß VLM 25 1 0 13 Feb 2023
Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation Bingqian Lin Yi Zhu Xiaodan Liang Liang Lin Jian-zhuo Liu CoGe LM&Ro 31 3 0 13 Feb 2023
CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets Zachary Novack Julian McAuley Zachary Chase Lipton Saurabh Garg VLM 19 79 0 06 Feb 2023
Learning Generalized Zero-Shot Learners for Open-Domain Image Geolocalization Lukas Haas Silas Alberti Michal Skreta VLM 23 21 0 01 Feb 2023
ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation KAI-QING Zhou Kai Zheng Connor Pryor Yilin Shen Hongxia Jin Lise Getoor X. Wang 18 107 0 30 Jan 2023
GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods Da Yin Feng Gao Govind Thattai Michael F. Johnston Kai-Wei Chang VLM 32 15 0 05 Jan 2023
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Yi Wang Kunchang Li Yizhuo Li Yinan He Bingkun Huang ... Junting Pan Jiashuo Yu Yali Wang Limin Wang Yu Qiao VLM VGen 38 309 0 06 Dec 2022
Perceive, Ground, Reason, and Act: A Benchmark for General-purpose Visual Representation Jiangyong Huang William Zhu Baoxiong Jia Zan Wang Xiaojian Ma Qing Li Siyuan Huang 32 5 0 28 Nov 2022
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang Limin Wang Yu Qiao ViT 25 106 0 17 Nov 2022
Multi-VQG: Generating Engaging Questions for Multiple Images Min-Hsuan Yeh Vicent Chen Ting-Hao Haung Lun-Wei Ku CoGe 13 7 0 14 Nov 2022
CPL: Counterfactual Prompt Learning for Vision and Language Models Xuehai He Diji Yang Weixi Feng Tsu-jui Fu Arjun Reddy Akula Varun Jampani P. Narayana Sugato Basu William Yang Wang X. Wang VPVLM VLM 45 15 0 19 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 26 62 0 14 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 26 1 0 12 Oct 2022
Efficient Diffusion Models for Vision: A Survey Anwaar Ulhaq Naveed Akhtar MedIm 32 60 0 07 Oct 2022
A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning Aishwarya Kamath Peter Anderson Su Wang Jing Yu Koh Alexander Ku Austin Waters Yinfei Yang Jason Baldridge Zarana Parekh LM&Ro 20 45 0 06 Oct 2022
Anticipating the Unseen Discrepancy for Vision and Language Navigation Yujie Lu Huiliang Zhang Ping Nie Weixi Feng Wenda Xu X. Wang William Yang Wang 27 1 0 10 Sep 2022
Masked Vision and Language Modeling for Multi-modal Representation Learning Gukyeong Kwon Zhaowei Cai Avinash Ravichandran Erhan Bas Rahul Bhotika Stefano Soatto 22 67 0 03 Aug 2022
Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models Huy Ha Shuran Song LM&Ro VLM 32 101 0 23 Jul 2022
Personalized Showcases: Generating Multi-Modal Explanations for Recommendations An Yan Zhankui He Jiacheng Li Tianyang Zhang Julian McAuley 22 35 0 30 Jun 2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 34 226 0 16 Jun 2022
Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone Zi-Yi Dou Aishwarya Kamath Zhe Gan Pengchuan Zhang Jianfeng Wang ... Ce Liu Yann LeCun Nanyun Peng Jianfeng Gao Lijuan Wang VLM ObjD 19 124 0 15 Jun 2022
Comprehending and Ordering Semantics for Image Captioning Yehao Li Yingwei Pan Ting Yao Tao Mei 13 87 0 14 Jun 2022
FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation Zi-Yi Dou Nanyun Peng 22 22 0 09 Jun 2022
Revisiting the "Video" in Video-Language Understanding S. Buch Cristobal Eyzaguirre Adrien Gaidon Jiajun Wu L. Fei-Fei Juan Carlos Niebles 27 155 0 03 Jun 2022