Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

21 February 2022

Papers citing "Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution"

50 / 145 papers shown

Title
Test-Time Training for Speech Sri Harsha Dumpala Chandramouli Shama Sastry Sageev Oore 39 1 0 19 Sep 2023
LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters Ching Chang Wei-Yao Wang Wenjie Peng Tien-Fu Chen AI4TS 38 45 0 16 Aug 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 72 33 0 15 Aug 2023
FedBug: A Bottom-Up Gradual Unfreezing Framework for Federated Learning Chia-Hsiang Kao Yu-Chiang Frank Wang FedML 24 1 0 19 Jul 2023
Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study Luis P. Silvestrin Shujian Yu Mark Hoogendoorn OOD 39 1 0 17 Jul 2023
Advances and Challenges in Meta-Learning: A Technical Review Anna Vettoruzzo Mohamed-Rafik Bouguelia Joaquin Vanschoren Thorsteinn Rögnvaldsson K. Santosh OffRL 29 70 0 10 Jul 2023
Becoming self-instruct: introducing early stopping criteria for minimal instruct tuning Waseem Alshikh Manhal Daaboul K. Goddard Brock Imel Kiran Kamble Parikshit Kulkarni M. Russak ALM 11 13 0 05 Jul 2023
Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn OOD 19 5 0 19 Jun 2023
Drag-guided diffusion models for vehicle image generation Nikos Arechiga Frank Permenter Binyang Song Chenyang Yuan DiffM 31 12 0 16 Jun 2023
Differentially Private Image Classification by Learning Priors from Random Processes Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal 23 20 0 08 Jun 2023
LRVS-Fashion: Extending Visual Search with Referring Instructions Simon Lepage Jérémie Mary David Picard 25 1 0 05 Jun 2023
Identifying Visible Tissue in Intraoperative Ultrasound Images during Brain Surgery: A Method and Application Alistair Weld L. Dixon Giulio Anichini Michael Dyck Alex Ranne Sophie Camp Stamatia Giannarou 24 2 0 01 Jun 2023
DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection Rui Shao Tianxing Wu Liqiang Nie Ziwei Liu 28 11 0 01 Jun 2023
FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognition Marawan Elbatel Robert Martí Xiaomeng Li AAML 34 10 0 27 May 2023
Out-of-Distribution Generalization in Text Classification: Past, Present, and Future Linyi Yang Yangqiu Song Xuan Ren Chenyang Lyu Yidong Wang Lingqiao Liu Jindong Wang Jennifer Foster Yue Zhang OOD 37 2 0 23 May 2023
SFP: Spurious Feature-targeted Pruning for Out-of-Distribution Generalization Yingchun Wang Jingcai Guo Yi Liu Song Guo Weizhan Zhang Xiangyong Cao Qinghua Zheng AAML OODD 31 11 0 19 May 2023
Differentially Private Adapters for Parameter Efficient Acoustic Modeling Chun-Wei Ho Chao-Han Huck Yang Sabato Marco Siniscalchi 26 1 0 19 May 2023
Learning to Generalize for Cross-domain QA Yingjie Niu Linyi Yang Ruihai Dong Yue Zhang 21 6 0 14 May 2023
How to Train Your CheXDragon: Training Chest X-Ray Models for Transfer to Novel Tasks and Healthcare Systems Cara Van Uden Jeremy Irvin Mars Huang N. Dean J. Carr A. Ng C. Langlotz OOD 29 1 0 13 May 2023
Consistency Regularization for Domain Generalization with Logit Attribution Matching Han Gao Kaican Li Weiyan Xie Zhi Lin Yongxiang Huang Luning Wang Caleb Chen Cao N. Zhang 13 2 0 13 May 2023
VPGTrans: Transfer Visual Prompt Generator across LLMs Ao Zhang Hao Fei Yuan Yao Wei Ji Li Li Zhiyuan Liu Tat-Seng Chua MLLM VLM 32 85 0 02 May 2023
How Will It Drape Like? Capturing Fabric Mechanics from Depth Images Carlos Rodriguez-Pardo Melania Prieto-Martin Dan Casas Elena Garces 31 12 0 13 Apr 2023
ERM++: An Improved Baseline for Domain Generalization Piotr Teterwak Kuniaki Saito Theodoros Tsiligkaridis Kate Saenko Bryan A. Plummer OOD 41 9 0 04 Apr 2023
Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation Julio Silva-Rodríguez Jose Dolz Ismail Ben Ayed 66 13 0 29 Mar 2023
Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection Kuniaki Saito Donghyun Kim Piotr Teterwak Rogerio Feris Kate Saenko 41 1 0 26 Mar 2023
The Role of Pre-training Data in Transfer Learning R. Entezari Mitchell Wortsman O. Saukh M. Shariatnia Hanie Sedghi Ludwig Schmidt 46 20 0 27 Feb 2023
How to prepare your task head for finetuning Yi Ren Shangmin Guo Wonho Bae Danica J. Sutherland 24 14 0 11 Feb 2023
Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn VLM 32 9 0 10 Feb 2023
CLIPood: Generalizing CLIP to Out-of-Distributions Yang Shu Xingzhuo Guo Jialong Wu Ximei Wang Jianmin Wang Mingsheng Long OODD VLM 52 74 0 02 Feb 2023
Debiasing Vision-Language Models via Biased Prompts Ching-Yao Chuang Varun Jampani Yuanzhen Li Antonio Torralba Stefanie Jegelka VLM 30 96 0 31 Jan 2023
Alignment with human representations supports robust few-shot learning Ilia Sucholutsky Thomas L. Griffiths 24 25 0 27 Jan 2023
Personalised Federated Learning On Heterogeneous Feature Spaces A. Rakotomamonjy Maxime Vono H. M. Ruiz L. Ralaivola FedML 18 8 0 26 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 20 100 0 16 Jan 2023
Who Should Predict? Exact Algorithms For Learning to Defer to Humans Hussein Mozannar Hunter Lang Dennis L. Wei P. Sattigeri Subhro Das David Sontag 30 40 0 15 Jan 2023
Improving Reliability of Fine-tuning with Block-wise Optimisation Basel Barakat Qiang Huang 19 1 0 15 Jan 2023
Does progress on ImageNet transfer to real-world datasets? Alex Fang Simon Kornblith Ludwig Schmidt VLM 23 34 0 11 Jan 2023
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 28 18 0 23 Dec 2022
Foundation models in brief: A historical, socio-technical focus Johannes Schneider VLM 29 9 0 17 Dec 2022
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 34 13 0 14 Dec 2022
A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization Ashwinee Panda Xinyu Tang Saeed Mahloujifar Vikash Sehwag Prateek Mittal 43 11 0 08 Dec 2022
Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning Hongbin Liu Wenjie Qu Jinyuan Jia Neil Zhenqiang Gong SSL 28 6 0 06 Dec 2022
Fair Generative Models via Transfer Learning Christopher T. H. Teo Milad Abdollahzadeh Ngai-man Cheung 21 24 0 02 Dec 2022
Finetune like you pretrain: Improved finetuning of zero-shot vision models Sachin Goyal Ananya Kumar Sankalp Garg Zico Kolter Aditi Raghunathan CLIP VLM 50 136 0 01 Dec 2022
Context-Aware Robust Fine-Tuning Xiaofeng Mao YueFeng Chen Xiaojun Jia Rong Zhang Hui Xue Zhao Li VLM CLIP 35 24 0 29 Nov 2022
Learning to Generate Image Embeddings with User-level Differential Privacy Zheng Xu Maxwell D. Collins Yuxiao Wang Liviu Panait Sewoong Oh S. Augenstein Ting Liu Florian Schroff H. B. McMahan FedML 30 29 0 20 Nov 2022
SolderNet: Towards Trustworthy Visual Inspection of Solder Joints in Electronics Manufacturing Using Explainable Artificial Intelligence Hayden Gunraj P. Guerrier Sheldon Fernandez Alexander Wong 9 8 0 18 Nov 2022
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective Linyi Yang Shuibai Zhang Libo Qin Yafu Li Yidong Wang Hanmeng Liu Jindong Wang Xingxu Xie Yue Zhang ELM 44 79 0 15 Nov 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 43 14 0 30 Oct 2022
Towards Generalized Few-Shot Open-Set Object Detection Binyi Su Hua Zhang Jingzhi Li Zhongjun Zhou 48 9 0 28 Oct 2022