Visual Classification via Description from Large Language Models

13 October 2022

Carl Vondrick

Papers citing "Visual Classification via Description from Large Language Models"

50 / 52 papers shown

Title
FedMVP: Federated Multi-modal Visual Prompt Tuning for Vision-Language Models Mainak Singha Subhankar Roy Sarthak Mehrotra Ankit Jha Moloud Abdar Biplab Banerjee Elisa Ricci VLM VPVLM 119 0 0 29 Apr 2025
SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models Justus Westerhoff Erblina Purellku Jakob Hackstein Jonas Loos Leo Pinetzki Lorenz Hufe AAML 28 0 0 07 Apr 2025
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Xia Hu Bo Yuan VLM 59 0 0 24 Mar 2025
TLAC: Two-stage LMM Augmented CLIP for Zero-Shot Classification Ans Munir Faisal Z. Qureshi M. H. Khan Mohsen Ali VLM 70 0 0 15 Mar 2025
ProAPO: Progressively Automatic Prompt Optimization for Visual Classification Xiangyan Qu Gaopeng Gou Jiamin Zhuang Jing Yu Kun Song Qihao Wang Yili Li Gang Xiong VLM 89 0 0 13 Mar 2025
Interactive Medical Image Analysis with Concept-based Similarity Reasoning Ta Duc Huy Sen Kim Tran Phan Nguyen Nguyen Hoang Tran Tran Bao Sam A. Hengel Zhibin Liao Johan W. Verjans Minh Nguyen Nhat To Vu Minh Hieu Phan 50 0 0 10 Mar 2025
InPK: Infusing Prior Knowledge into Prompt for Vision-Language Models Shuchang Zhou Jiwei Wei Shiyuan He Yuyang Zhou Chaoning Zhang Jie Zou Ning Xie Yang Yang VLM VPVLM 81 0 0 27 Feb 2025
DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery Utkarsh Mall Cheng Perng Phoo Mia Chiquier Bharath Hariharan Kavita Bala Carl Vondrick 73 1 0 17 Feb 2025
Zero-shot Action Localization via the Confidence of Large Vision-Language Models Josiah Aklilu Xiaohan Wang Serena Yeung-Levy 57 1 0 18 Oct 2024
Tree of Attributes Prompt Learning for Vision-Language Models Tong Ding Wanhua Li Zhongqi Miao Hanspeter Pfister VLM 52 1 0 15 Oct 2024
CASA: Class-Agnostic Shared Attributes in Vision-Language Models for Efficient Incremental Object Detection Mingyi Guo Yuyang Liu Zongying Lin Peixi Peng Yonghong Tian Yonghong Tian VLM 30 0 0 08 Oct 2024
What Makes a Maze Look Like a Maze? Joy Hsu Jiayuan Mao J. Tenenbaum Noah D. Goodman Jiajun Wu OCL 54 6 0 12 Sep 2024
AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation Yuhan Zhu Yuyang Ji Zhiyu Zhao Gangshan Wu Limin Wang VLM 39 7 0 05 Jul 2024
Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval Hanwen Su G. Song K. Huang Jiyan Wang Ming Yang 48 1 0 01 Jul 2024
Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling Cristian Rodriguez-Opazo Ehsan Abbasnejad Damien Teney Edison Marrese-Taylor Hamed Damirchi A. Hengel VLM 40 1 0 27 May 2024
What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models Abdelrahman Abdelhamed Mahmoud Afifi Alec Go MLLM VLM 33 3 0 24 May 2024
Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model Seonhee Cho Choonghan Kim Jiho Lee Chetan Chilkunda Sujin Choi Joo Heung Yoon 50 0 0 29 Apr 2024
FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization Zhaopeng Gu Bingke Zhu Guibo Zhu Yingying Chen Hao Li Ming Tang Jinqiao Wang 42 15 0 21 Apr 2024
ECOR: Explainable CLIP for Object Recognition Ali Rasekh Sepehr Kazemi Ranjbar Milad Heidari Wolfgang Nejdl VLM 46 4 0 19 Apr 2024
Pre-trained Vision-Language Models Learn Discoverable Visual Concepts Yuan Zang Tian Yun Hao Tan Trung Bui Chen Sun VLM CoGe 50 9 0 19 Apr 2024
Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection Ting Lei Shaofeng Yin Yang Liu VLM 47 9 0 09 Apr 2024
Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning Andrei Semenov Vladimir Ivanov Aleksandr Beznosikov Alexander Gasnikov 37 6 0 04 Apr 2024
Training-Free Semantic Segmentation via LLM-Supervision Wenfang Sun Yingjun Du Gaowen Liu Ramana Rao Kompella Cees G. M. Snoek VLM 44 2 0 31 Mar 2024
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models Elaine Sui Xiaohan Wang Serena Yeung-Levy VLM 30 5 0 19 Mar 2024
DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization Feng Hou Jin Yuan Ying Yang Yang Liu Yang Zhang Cheng Zhong Zhongchao Shi Jianping Fan Yong Rui Zhiqiang He VLM 51 1 0 05 Mar 2024
Leveraging Habitat Information for Fine-grained Bird Identification Tin Nguyen Anh Nguyen Anh Nguyen VLM 44 0 0 22 Dec 2023
Object Recognition as Next Token Prediction Kaiyu Yue Borchun Chen Jonas Geiping Hengduo Li Tom Goldstein Ser-Nam Lim 37 9 0 04 Dec 2023
TeG-DG: Textually Guided Domain Generalization for Face Anti-Spoofing Lianrui Mu Jianhong Bai Xiaoxuan He Jiangnan Ye Xiaoyu Liang Yuchen Yang Jiedong Zhuang Haoji Hu 27 2 0 30 Nov 2023
Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines Hamed Damirchi Cristian Rodriguez-Opazo Ehsan Abbasnejad Damien Teney Javen Qinfeng Shi Stephen Gould A. Hengel VLM 41 0 0 29 Nov 2023
GPT4Vis: What Can GPT-4 Do for Zero-shot Visual Recognition? Wenhao Wu Huanjin Yao Mengxi Zhang Yuxin Song Wanli Ouyang Jingdong Wang VLM 26 29 0 27 Nov 2023
Image Clustering Conditioned on Text Criteria Sehyun Kwon Jaeseung Park Minkyu Kim Jaewoong Cho Ernest K. Ryu Kangwook Lee VLM 34 11 0 27 Oct 2023
Open-Set Image Tagging with Multi-Grained Text Supervision Xinyu Huang Yi-Jie Huang Youcai Zhang Weiwei Tian Rui Feng Yuejie Zhang Yanchun Xie Yaqian Li Lei Zhang VLM 25 28 0 23 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 35 117 0 16 Oct 2023
Prompting Scientific Names for Zero-Shot Species Recognition Shubham Parashar Zhiqiu Lin Yanan Li Shu Kong VLM 23 12 0 15 Oct 2023
Zero-Shot Robustification of Zero-Shot Models Dyah Adila Changho Shin Lin Cai Frederic Sala 38 18 0 08 Sep 2023
AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation Chaofan Ma Yu-Hao Yang Chen Ju Fei Zhang Ya-Qin Zhang Yanfeng Wang VLM 45 17 0 31 Aug 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 35 0 0 29 Aug 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 69 33 0 15 Aug 2023
Few-shot medical image classification with simple shape and texture text descriptors using vision-language models Michal Byra M. F. Rachmadi Henrik Skibbe VLM 28 6 0 08 Aug 2023
Language-based Action Concept Spaces Improve Video Self-Supervised Learning Kanchana Ranasinghe Michael S. Ryoo SSL VLM 34 12 0 20 Jul 2023
In-Context Impersonation Reveals Large Language Models' Strengths and Biases Leonard Salewski Stephan Alaniz Isabel Rio-Torto Eric Schulz Zeynep Akata 41 149 0 24 May 2023
Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Models Lin Li Jun Xiao Guikun Chen Jian Shao Yueting Zhuang Long Chen VLM 27 25 0 21 May 2023
VicTR: Video-conditioned Text Representations for Activity Recognition Kumara Kahatapitiya Anurag Arnab Arsha Nagrani Michael S. Ryoo 33 19 0 05 Apr 2023
ViperGPT: Visual Inference via Python Execution for Reasoning Dídac Surís Sachit Menon Carl Vondrick MLLM LRM ReLM 45 431 0 14 Mar 2023
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners Renrui Zhang Xiangfei Hu Bohao Li Siyuan Huang Hanqiu Deng Hongsheng Li Yu Qiao Peng Gao VLM MLLM 38 170 0 03 Mar 2023
Affective Faces for Goal-Driven Dyadic Communication Scott Geng Revant Teotia Purva Tendulkar Sachit Menon Carl Vondrick VGen 26 18 0 26 Jan 2023
Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification Yue Yang Artemis Panagopoulou Shenghao Zhou Daniel Jin Chris Callison-Burch Mark Yatskar 40 211 0 21 Nov 2022
What does a platypus look like? Generating customized prompts for zero-shot image classification Sarah M Pratt Ian Covert Rosanne Liu Ali Farhadi VLM 128 212 0 07 Sep 2022
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 174 402 0 10 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,700 0 11 Feb 2021