PLOT: Prompt Learning with Optimal Transport for Vision-Language Models

3 October 2022

Xiangchen Song

Papers citing "PLOT: Prompt Learning with Optimal Transport for Vision-Language Models"

50 / 51 papers shown

Title
Learning Clustering-based Prototypes for Compositional Zero-shot Learning Hongyu Qu Jianan Wei Xiangbo Shu Wenguan Wang VLM 112 1 0 10 Feb 2025
Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis Hongyu Sun Qiuhong Ke Yanjie Wang Wang Chen Kang Yang Deying Li Jianfei Cai 3DPC 127 3 0 17 Jan 2025
UNEM: UNrolled Generalized EM for Transductive Few-Shot Learning Long Zhou Fereshteh Shakeri Aymen Sadraoui Mounir Kaaniche J. Pesquet Ismail Ben Ayed VLM 140 0 0 21 Dec 2024
Tree of Attributes Prompt Learning for Vision-Language Models Tong Ding Wanhua Li Zhongqi Miao Hanspeter Pfister VLM 87 1 0 15 Oct 2024
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 339 6,830 0 13 Apr 2022
MotionCLIP: Exposing Human Motion Generation to CLIP Space Guy Tevet Brian Gordon Amir Hertz Amit H. Bermano Daniel Cohen-Or CLIP 110 340 0 15 Mar 2022
Conditional Prompt Learning for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VLM CLIP VPVLM 95 1,339 0 10 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 490 4,324 0 28 Jan 2022
Optimal transport for causal discovery Ruibo Tu Kun Zhang Hedvig Kjellström Cheng Zhang 115 20 0 23 Jan 2022
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models Alex Nichol Prafulla Dhariwal Aditya A. Ramesh Pranav Shyam Pamela Mishkin Bob McGrew Ilya Sutskever Mark Chen 284 3,571 0 20 Dec 2021
VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts Longtian Qiu Renrui Zhang Ziyu Guo Wei Zhang Zilu Guo Ziyao Zeng Guangnan Zhang VLM CLIP 49 45 0 04 Dec 2021
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting Yongming Rao Wenliang Zhao Guangyi Chen Yansong Tang Zheng Zhu Guan Huang Jie Zhou Jiwen Lu VLM CLIP 187 573 0 02 Dec 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 256 399 0 06 Nov 2021
An Empirical Study of Training End-to-End Vision-and-Language Transformers Zi-Yi Dou Yichong Xu Zhe Gan Jianfeng Wang Shuohang Wang ... Pengchuan Zhang Lu Yuan Nanyun Peng Zicheng Liu Michael Zeng VLM 61 376 0 03 Nov 2021
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts Hangbo Bao Wenhui Wang Li Dong Qiang Liu Owais Khan Mohammed Kriti Aggarwal Subhojit Som Furu Wei VLM MLLM MoE 65 552 0 03 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 241 1,035 0 09 Oct 2021
ActionCLIP: A New Paradigm for Video Action Recognition Mengmeng Wang Jiazheng Xing Yong Liu VLM 185 371 0 17 Sep 2021
MURAL: Multimodal, Multitask Retrieval Across Languages Aashi Jain Mandy Guo Krishna Srinivasan Ting-Li Chen Sneha Kudugunta Chao Jia Yinfei Yang Jason Baldridge VLM 145 52 0 10 Sep 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 459 2,382 0 02 Sep 2021
Towards Interpretable Deep Metric Learning with Structural Matching Wenliang Zhao Yongming Rao Ziyi Wang Jiwen Lu Jie Zhou FedML 54 47 0 12 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 185 3,951 0 28 Jul 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 167 1,943 0 16 Jul 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 139 775 0 25 Jun 2021
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding Aishwarya Kamath Mannat Singh Yann LeCun Gabriel Synnaeve Ishan Misra Nicolas Carion ObjD VLM 163 879 0 26 Apr 2021
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery Or Patashnik Zongze Wu Eli Shechtman Daniel Cohen-Or Dani Lischinski CLIP VLM 81 1,205 0 31 Mar 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 155 1,173 0 18 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 808 29,167 0 26 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 417 3,826 0 11 Feb 2021
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Wonjae Kim Bokyung Son Ildoo Kim VLM CLIP 112 1,735 0 05 Feb 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 213 4,238 0 01 Jan 2021
A Recurrent Vision-and-Language BERT for Navigation Yicong Hong Qi Wu Yuankai Qi Cristian Rodriguez-Opazo Stephen Gould LM&Ro 92 300 0 26 Nov 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo D. Song Jacob Steinhardt Justin Gilmer OOD 296 1,727 0 29 Jun 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 356 42,299 0 03 Dec 2019
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 123 1,402 0 28 Nov 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 558 2,660 0 03 Sep 2019
VisualBERT: A Simple and Performant Baseline for Vision and Language Liunian Harold Li Mark Yatskar Da Yin Cho-Jui Hsieh Kai-Wei Chang VLM 130 1,948 0 09 Aug 2019
Natural Adversarial Examples Dan Hendrycks Kevin Zhao Steven Basart Jacob Steinhardt D. Song OODD 193 1,465 0 16 Jul 2019
Learning Robust Global Representations by Penalizing Local Predictive Power Haohan Wang Songwei Ge Eric Xing Zachary Chase Lipton OOD 104 955 0 29 May 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 103 1,709 0 13 Feb 2019
Improving Sequence-to-Sequence Learning via Optimal Transport Liqun Chen Yizhe Zhang Ruiyi Zhang Chenyang Tao Zhe Gan Haichao Zhang Bai Li Dinghan Shen Changyou Chen Lawrence Carin OT 44 94 0 18 Jan 2019
Gromov-Wasserstein Learning for Graph Matching and Node Embedding Hongteng Xu Dixin Luo H. Zha Lawrence Carin 81 259 0 17 Jan 2019
Improving GANs Using Optimal Transport Tim Salimans Han Zhang Alec Radford Dimitris N. Metaxas OT GAN 54 324 0 15 Mar 2018
Computational Optimal Transport Gabriel Peyré Marco Cuturi OT 185 2,143 0 01 Mar 2018
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification P. Helber B. Bischke Andreas Dengel Damian Borth 125 1,806 0 31 Aug 2017
Co-clustering through Optimal Transport Charlotte Laclau I. Redko Basarab Matei Younès Bennani V. Brault OT 18 42 0 17 May 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.9K 193,426 0 10 Dec 2015
Describing Textures in the Wild Mircea Cimpoi Subhransu Maji Iasonas Kokkinos S. Mohamed Andrea Vedaldi 3DV 92 2,661 0 14 Nov 2013
Fine-Grained Visual Classification of Aircraft Subhransu Maji Esa Rahtu Arno Solin Matthew Blaschko Andrea Vedaldi 107 2,252 0 21 Jun 2013
Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances Marco Cuturi OT 182 4,251 0 04 Jun 2013
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild K. Soomro Amir Zamir M. Shah CLIP VGen 113 6,134 0 03 Dec 2012