v1v2 (latest)

CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models

28 March 2024

ArXiv (abs)PDF HTML Github (34★)

Papers citing "CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models"

50 / 56 papers shown

Title
Adaptive Rank, Reduced Forgetting: Knowledge Retention in Continual Learning Vision-Language Models with Dynamic Rank-Selective LoRA Haodong Lu Chongyang Zhao Jason Xue Lina Yao Kristen Moore Dong Gong CLL KELM VLM 169 7 0 01 Dec 2024
Mining Your Own Secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models Saurav Jha Shiqi Yang Masato Ishii Mengjie Zhao Christian Simon Muhammad Jehanzeb Mirza Dong Gong Lina Yao Shusuke Takahashi Yuki Mitsufuji DiffM 124 2 0 01 Oct 2024
Just Say the Name: Online Continual Learning with Category Names Only via Data Generation Minhyuk Seo Diganta Misra Seongwon Cho Minjae Lee Jonghyun Choi CLL 83 9 0 16 Mar 2024
Learning to Prompt with Text Only Supervision for Vision-Language Models Muhammad Uzair Khattak Muhammad Ferjad Naeem Muzammal Naseer Luc Van Gool F. Tombari VLM VPVLM 83 21 0 04 Jan 2024
RanPAC: Random Projections and Pre-trained Models for Continual Learning Mark D Mcdonnell Dong Gong Amin Parvaneh Ehsan Abbasnejad Anton Van Den Hengel VLM CLL 84 108 0 05 Jul 2023
Learning without Forgetting for Vision-Language Models Da-Wei Zhou Yuanhan Zhang Jingyi Ning Jingyi Ning De-Chuan Zhan De-Chuan Zhan Ziwei Liu VLM CLL 115 42 0 30 May 2023
Continual Vision-Language Representation Learning with Off-Diagonal Information Zixuan Ni Longhui Wei Siliang Tang Yueting Zhuang Qi Tian VLM CLL 86 25 0 11 May 2023
Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models Zangwei Zheng Mingyu Ma Kai Wang Ziheng Qin Xiangyu Yue Yang You CLL VLM 136 77 0 12 Mar 2023
Debiased Fine-Tuning for Vision-language Models by Prompt Regularization Beier Zhu Yulei Niu Saeil Lee Minhoe Hur Hanwang Zhang VLM VPVLM 81 24 0 29 Jan 2023
Bayesian posterior approximation with stochastic ensembles Oleksandr Balabanov Bernhard Mehlig Hampus Linander BDL UQCV 82 5 0 15 Dec 2022
Grow and Merge: A Unified Framework for Continuous Categories Discovery Xinwei Zhang Jianwen Jiang Yutong Feng Zhi-Fan Wu Xibin Zhao Hai Wan Mingqian Tang Rong Jin Yue Gao CLL 68 31 0 09 Oct 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan Fahad Shahbaz Khan VPVLM VLM 256 566 0 06 Oct 2022
CLIP model is an Efficient Continual Learner Vishal Thengane Salman Khan Munawar Hayat Fahad Shahbaz Khan BDL VLM CLL 155 51 0 06 Oct 2022
Bayesian Prompt Learning for Image-Language Model Generalization Mohammad Mahdi Derakhshani Enrique Sanchez Adrian Bulat Victor G. Turrisi da Costa Cees G. M. Snoek Georgios Tzimiropoulos Brais Martínez VPVLM VLM 145 36 0 05 Oct 2022
What does a platypus look like? Generating customized prompts for zero-shot image classification Sarah M Pratt Ian Covert Rosanne Liu Ali Farhadi VLM 167 223 0 07 Sep 2022
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit H. Bermano Gal Chechik Daniel Cohen-Or 160 1,889 0 02 Aug 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 193 3,898 0 26 Jul 2022
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification Renrui Zhang Zhang Wei Rongyao Fang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 113 316 0 19 Jul 2022
CLiMB: A Continual Learning Benchmark for Vision-and-Language Tasks Tejas Srinivasan Ting-Yun Chang Leticia Pinto-Alva Georgios Chochlakis Mohammad Rostami Jesse Thomason VLM CLL 83 75 0 18 Jun 2022
Continual evaluation for lifelong learning: Identifying the stability gap Matthias De Lange Gido M. van de Ven Tinne Tuytelaars CLL 157 42 0 26 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 392 3,542 0 29 Apr 2022
DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning Zifeng Wang Zizhao Zhang Sayna Ebrahimi Ruoxi Sun Han Zhang ... Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VLM VPVLM 117 490 0 10 Apr 2022
Towards Exemplar-Free Continual Learning in Vision Transformers: an Account of Attention, Functional and Weight Regularization Francesco Pelosin Saurav Jha A. Torsello Bogdan Raducanu Joost van de Weijer CLL 52 31 0 24 Mar 2022
Conditional Prompt Learning for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VLM CLIP VPVLM 139 1,349 0 10 Mar 2022
Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning Dong Gong Qingsen Yan Yuhang Liu Anton Van Den Hengel Javen Qinfeng Shi CLL BDL 70 39 0 21 Feb 2022
Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution Ananya Kumar Aditi Raghunathan Robbie Jones Tengyu Ma Percy Liang OODD 122 681 0 21 Feb 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 458 15,665 0 20 Dec 2021
Learning to Prompt for Continual Learning Zifeng Wang Zizhao Zhang Chen-Yu Lee Han Zhang Ruoxi Sun Xiaoqi Ren Guolong Su Vincent Perot Jennifer Dy Tomas Pfister CLL VPVLM KELM VLM 101 785 0 16 Dec 2021
VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts Longtian Qiu Renrui Zhang Ziyu Guo Wei Zhang Zilu Guo Ziyao Zeng Guangnan Zhang VLM CLIP 55 45 0 04 Dec 2021
Combined Scaling for Zero-shot Transfer Learning Hieu H. Pham Zihang Dai Golnaz Ghiasi Kenji Kawaguchi Hanxiao Liu ... Yi-Ting Chen Minh-Thang Luong Yonghui Wu Mingxing Tan Quoc V. Le VLM 71 199 0 19 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 299 1,042 0 09 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 137 725 0 04 Sep 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 495 2,399 0 02 Sep 2021
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity Sebastian Lee Sebastian Goldt Andrew M. Saxe CLL 72 74 0 09 Jul 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 477 10,367 0 17 Jun 2021
Revisiting the Calibration of Modern Neural Networks Matthias Minderer Josip Djolonga Rob Romijnders F. Hubis Xiaohua Zhai N. Houlsby Dustin Tran Mario Lucic UQCV 100 367 0 15 Jun 2021
Priors in Bayesian Deep Learning: A Review Vincent Fortuin UQCV BDL 116 132 0 14 May 2021
Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis Saurav Jha Martin Schiemer F. Zambonelli Juan Ye 52 50 0 19 Apr 2021
Class-incremental learning: survey and performance evaluation on image classification Marc Masana Xialei Liu Bartlomiej Twardowski Mikel Menta Andrew D. Bagdanov Joost van de Weijer CLL 79 692 0 28 Oct 2020
Energy-based Out-of-distribution Detection Weitang Liu Xiaoyun Wang John Douglas Owens Yixuan Li OODD 271 1,356 0 08 Oct 2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization Dan Hendrycks Steven Basart Norman Mu Saurav Kadavath Frank Wang ... Samyak Parajuli Mike Guo Basel Alomair Jacob Steinhardt Justin Gilmer OOD 343 1,740 0 29 Jun 2020
Continual Deep Learning by Functional Regularisation of Memorable Past Pingbo Pan S. Swaroop Alexander Immer Runa Eschenhagen Richard Turner Mohammad Emtiyaz Khan KELM CLL 52 142 0 29 Apr 2020
Deep Ensembles: A Loss Landscape Perspective Stanislav Fort Huiyi Hu Balaji Lakshminarayanan OOD UQCV 121 628 0 05 Dec 2019
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark Xiaohua Zhai J. Puigcerver Alexander Kolesnikov P. Ruyssen C. Riquelme ... Michael Tschannen Marcin Michalski Olivier Bousquet Sylvain Gelly N. Houlsby SSL 84 440 0 01 Oct 2019
Large Scale Incremental Learning Yue Wu Yinpeng Chen Lijuan Wang Yuancheng Ye Zicheng Liu Yandong Guo Y. Fu CLL 102 1,257 0 30 May 2019
End-to-End Incremental Learning F. M. Castro M. Marín-Jiménez Nicolás Guil Mata Cordelia Schmid Alahari Karteek CLL 87 1,158 0 25 Jul 2018
Towards Robust Evaluations of Continual Learning Sebastian Farquhar Y. Gal CLL 93 307 0 24 May 2018
Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence Arslan Chaudhry P. Dokania Thalaiyasingam Ajanthan Philip Torr CLL 102 1,141 0 30 Jan 2018
Memory Aware Synapses: Learning what (not) to forget Rahaf Aljundi F. Babiloni Mohamed Elhoseiny Marcus Rohrbach Tinne Tuytelaars KELM CLL 87 1,636 0 27 Nov 2017
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification P. Helber B. Bischke Andreas Dengel Damian Borth 135 1,820 0 31 Aug 2017