Visual Classification via Description from Large Language Models

13 October 2022

Carl Vondrick

Papers citing "Visual Classification via Description from Large Language Models"

50 / 225 papers shown

Title
What Makes a Maze Look Like a Maze? Joy Hsu Jiayuan Mao J. Tenenbaum Noah D. Goodman Jiajun Wu OCL 54 6 0 12 Sep 2024
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models Gracjan Góral Alicja Ziarko Michal Nauman Maciej Wołczyk LRM 28 1 0 02 Sep 2024
Aligning Medical Images with General Knowledge from Large Language Models X. B. Fang Yi Lin Dong Zhang Kwang-Ting Cheng Hao Chen LM&MA VLM 32 4 0 31 Aug 2024
HPT++: Hierarchically Prompting Vision-Language Models with Multi-Granularity Knowledge Generation and Improved Structure Modeling Yubin Wang Xinyang Jiang De Cheng Wenli Sun Dongsheng Li Cairong Zhao VLM 48 0 0 27 Aug 2024
CLIPCleaner: Cleaning Noisy Labels with CLIP Chen Feng Georgios Tzimiropoulos Ioannis Patras VLM 35 1 0 19 Aug 2024
Efficient Test-Time Prompt Tuning for Vision-Language Models Yuhan Zhu Guozhen Zhang Chen Xu Haocheng Shen Xiaoxin Chen Gangshan Wu Limin Wang VLM 37 2 0 11 Aug 2024
On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Survey Jingcai Guo Zhijie Rao Zhi Chen Song Guo Jingren Zhou Dacheng Tao 33 3 0 09 Aug 2024
Text-Guided Video Masked Autoencoder D. Fan Jue Wang Shuai Liao Zhikang Zhang Vimal Bhat Xinyu Li VGen 33 3 0 01 Aug 2024
SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognition Hao Tan Zichang Tan Jun Li Jun Wan Zhen Lei Stan Z. Li VLM 40 1 0 30 Jul 2024
Category-Extensible Out-of-Distribution Detection via Hierarchical Context Descriptions Kai-Chun Liu Zhihang Fu Chao Chen Sheng Jin Ze Chen Mingyuan Tao Rongxin Jiang Jieping Ye VLM OODD 58 4 0 23 Jul 2024
XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Models Erik Cambria Lorenzo Malandri Fabio Mercorio Navid Nobani Andrea Seveso 50 11 0 21 Jul 2024
Rethinking Visual Content Refinement in Low-Shot CLIP Adaptation Jinda Lu Shuo Wang Yanbin Hao Haifeng Liu Xiang Wang Meng Wang 30 2 0 19 Jul 2024
Robust Calibration of Large Vision-Language Adapters Balamurali Murugesan Julio Silva-Rodríguez Ismail Ben Ayed Jose Dolz OODD VLM 32 6 0 18 Jul 2024
LaMI-DETR: Open-Vocabulary Detection with Language Model Instruction Penghui Du Yu Wang Yifan Sun Luting Wang Yue Liao Gang Zhang Errui Ding Yan Wang Jingdong Wang Si Liu VLM ObjD 43 1 0 16 Jul 2024
Open Vocabulary Multi-Label Video Classification Rohit Gupta Mamshad Nayeem Rizve Jayakrishnan Unnikrishnan Ashish Tawari Son Tran Mubarak Shah Benjamin Z. Yao Trishul Chilimbi VLM 67 1 0 12 Jul 2024
AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation Yuhan Zhu Yuyang Ji Zhiyu Zhao Gangshan Wu Limin Wang VLM 41 7 0 05 Jul 2024
Concept Bottleneck Models Without Predefined Concepts Simon Schrodi Julian Schur Max Argus Thomas Brox 50 9 0 04 Jul 2024
A Survey on Trustworthiness in Foundation Models for Medical Image Analysis Congzhen Shi Ryan Rezai Jiaxi Yang Qi Dou Xiaoxiao Li MedIm 31 4 0 03 Jul 2024
Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval Hanwen Su G. Song K. Huang Jiyan Wang Ming Yang 48 1 0 01 Jul 2024
YouDream: Generating Anatomically Controllable Consistent Text-to-3D Animals Sandeep Mishra Oindrila Saha A. Bovik 37 0 0 24 Jun 2024
Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models Hengyi Wang Shiwei Tan Hao Wang BDL 42 6 0 18 Jun 2024
They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Bias Salma Abdel Magid Jui-Hsien Wang Kushal Kafle Hanspeter Pfister 44 1 0 17 Jun 2024
Conceptual Learning via Embedding Approximations for Reinforcing Interpretability and Transparency Maor Dikter Tsachi Blau Chaim Baskin 41 0 0 13 Jun 2024
ConceptHash: Interpretable Fine-Grained Hashing via Concept Discovery Kam Woh Ng Xiatian Zhu Yi-Zhe Song Tao Xiang 37 2 0 12 Jun 2024
Regularized Training with Generated Datasets for Name-Only Transfer of Vision-Language Models Minho Park S. Park Jooyeol Yun Jaegul Choo VLM 35 0 0 08 Jun 2024
Visual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models Jinhao Li Haopeng Li S. Erfani Lei Feng James Bailey Feng Liu VLM 34 3 0 05 Jun 2024
Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Following Qiaomu Miao Alexandros Graikos Jingwei Zhang Sounak Mondal Minh Hoai Dimitris Samaras 38 0 0 04 Jun 2024
Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection Chentao Cao Zhun Zhong Zhanke Zhou Yang Liu Tongliang Liu Bo Han OODD 26 10 0 02 Jun 2024
Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling Cristian Rodriguez-Opazo Ehsan Abbasnejad Damien Teney Edison Marrese-Taylor Hamed Damirchi Anton Van Den Hengel VLM 40 1 0 27 May 2024
What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models Abdelrahman Abdelhamed Mahmoud Afifi Alec Go MLLM VLM 33 3 0 24 May 2024
SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection Mingxuan Liu Tyler L. Hayes Elisa Ricci G. Csurka Riccardo Volpi ObjD 61 1 0 16 May 2024
Can Better Text Semantics in Prompt Tuning Improve VLM Generalization? Hari Chandana Kuchibhotla Sai Srinivas Kancheti Abbavaram Gowtham Reddy Vineeth N. Balasubramanian VLM 42 0 0 13 May 2024
Improving Concept Alignment in Vision-Language Concept Bottleneck Models Nithish Muthuchamy Selvaraj Xiaobao Guo Bingquan Shen A. Kong Alex C. Kot VLM 46 0 0 03 May 2024
Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model Seonhee Cho Choonghan Kim Jiho Lee Chetan Chilkunda Sujin Choi Joo Heung Yoon 53 0 0 29 Apr 2024
Leveraging Cross-Modal Neighbor Representation for Improved CLIP Classification Chao Yi Lu Ren De-Chuan Zhan Han-Jia Ye CLIP VLM 24 5 0 27 Apr 2024
Multi-Modal Proxy Learning Towards Personalized Visual Multiple Clustering Jiawei Yao Qi Qian Juhua Hu 35 14 0 24 Apr 2024
FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization Zhaopeng Gu Bingke Zhu Guibo Zhu Yingying Chen Hao Li Ming Tang Jinqiao Wang 42 15 0 21 Apr 2024
ECOR: Explainable CLIP for Object Recognition Ali Rasekh Sepehr Kazemi Ranjbar Milad Heidari Wolfgang Nejdl VLM 46 4 0 19 Apr 2024
Pre-trained Vision-Language Models Learn Discoverable Visual Concepts Yuan Zang Tian Yun Hao Tan Trung Bui Chen Sun VLM CoGe 58 9 0 19 Apr 2024
Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model Hao Yan Yuhong Guo VLM FedML 35 2 0 17 Apr 2024
Evolving Interpretable Visual Classifiers with Large Language Models Mia Chiquier Utkarsh Mall Carl Vondrick VLM 30 10 0 15 Apr 2024
The Devil is in the Few Shots: Iterative Visual Knowledge Completion for Few-shot Learning Yaohui Li Qifeng Zhou Haoxing Chen Jianbing Zhang Xinyu Dai Hao Zhou VLM 53 0 0 15 Apr 2024
Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models Simon Schrodi David T. Hoffmann Max Argus Volker Fischer Thomas Brox VLM 58 0 0 11 Apr 2024
Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection Ting Lei Shaofeng Yin Yang Liu VLM 47 9 0 09 Apr 2024
Label Propagation for Zero-shot Classification with Vision-Language Models Vladan Stojnić Yannis Kalantidis Giorgos Tolias VLM 41 8 0 05 Apr 2024
Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning Andrei Semenov Vladimir Ivanov Aleksandr Beznosikov Alexander Gasnikov 42 6 0 04 Apr 2024
Training-Free Semantic Segmentation via LLM-Supervision Wenfang Sun Yingjun Du Gaowen Liu Ramana Rao Kompella Cees G. M. Snoek VLM 44 2 0 31 Mar 2024
Convolutional Prompting meets Language Models for Continual Learning Anurag Roy Riddhiman Moulick Vinay K. Verma Saptarshi Ghosh Abir Das VLM CLL LRM 37 12 0 29 Mar 2024
CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models Saurav Jha Dong Gong Lina Yao CLIP VLM 33 8 0 28 Mar 2024
Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models Yabin Zhang Wen-Qing Zhu Hui Tang Zhiyuan Ma Kaiyang Zhou Lei Zhang VLM 31 22 0 26 Mar 2024