Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

21 February 2022

Papers citing "Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution"

50 / 492 papers shown

Title
CyFormer: Accurate State-of-Health Prediction of Lithium-Ion Batteries via Cyclic Attention Zhiqiang Nie Jiankun Zhao Qicheng Li Yong Qin 21 3 0 17 Apr 2023
Exploring Incompatible Knowledge Transfer in Few-shot Image Generation Yunqing Zhao Chao Du Milad Abdollahzadeh Tianyu Pang Min Lin Shuicheng Yan Ngai-Man Cheung 101 21 0 15 Apr 2023
How Will It Drape Like? Capturing Fabric Mechanics from Depth Images Carlos Rodriguez-Pardo Melania Prieto-Martin Dan Casas Elena Garces 81 12 0 13 Apr 2023
Controllable Textual Inversion for Personalized Text-to-Image Generation Jianan Yang Haobo Wang Yanming Zhang Rui Xiao Sai Wu Gang Chen Jiaqi Zhao DiffM 98 12 0 11 Apr 2023
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Simon Lacoste-Julien MoMe 110 21 0 06 Apr 2023
ERM++: An Improved Baseline for Domain Generalization Piotr Teterwak Kuniaki Saito Theodoros Tsiligkaridis Kate Saenko Bryan A. Plummer OOD 93 10 0 04 Apr 2023
Few-shot Fine-tuning is All You Need for Source-free Domain Adaptation Suho Lee SeungWon Seo Jihyo Kim Yejin Lee Sangheum Hwang 97 5 0 03 Apr 2023
Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? Arjun Majumdar Karmesh Yadav Sergio Arnaud Yecheng Jason Ma Claire Chen ... Dhruv Batra Yixin Lin Oleksandr Maksymets Aravind Rajeswaran Franziska Meier LM&Ro 76 185 0 31 Mar 2023
ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing Xiaodan Li YueFeng Chen Yao Zhu Shuhui Wang Rong Zhang Hui Xue 82 25 0 30 Mar 2023
Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentation Julio Silva-Rodríguez Jose Dolz Ismail Ben Ayed 203 14 0 29 Mar 2023
BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning Changdae Oh Hyeji Hwang Hee-young Lee Yongtaek Lim Geunyoung Jung Jiyoung Jung Hosik Choi Kyungwoo Song VLM VPVLM 139 62 0 26 Mar 2023
Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection Kuniaki Saito Donghyun Kim Piotr Teterwak Rogerio Feris Kate Saenko 83 1 0 26 Mar 2023
MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation Constantin Ulrich Fabian Isensee Tassilo Wald M. Zenk Michael Baumgartner Klaus H. Maier-Hein 81 39 0 25 Mar 2023
CCL: Continual Contrastive Learning for LiDAR Place Recognition Jiafeng Cui Xieyuanli Chen CLL 96 8 0 24 Mar 2023
A Closer Look at Model Adaptation using Feature Distortion and Simplicity Bias Puja Trivedi Danai Koutra Jayaraman J. Thiagarajan AAML 72 18 0 23 Mar 2023
Trainable Projected Gradient Method for Robust Fine-tuning Junjiao Tian Xiaoliang Dai Chih-Yao Ma Zecheng He Yen-Cheng Liu Z. Kira 115 41 0 19 Mar 2023
A Unified Continual Learning Framework with General Parameter-Efficient Tuning Qiankun Gao Chen Zhao Yifan Sun Teng Xi Gang Zhang Guohao Li Shuai Liu CLL 123 98 0 17 Mar 2023
Efficient Computation Sharing for Multi-Task Visual Scene Understanding Sara Shoouri Mingyu Yang Zichen Fan Hun-Seok Kim MoE 81 3 0 16 Mar 2023
Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning Zhuowei Li Long Zhao Zizhao Zhang Han Zhang Diya Liu Ting Liu Dimitris N. Metaxas CLL VLM 103 21 0 16 Mar 2023
A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation Hui Tang Kui Jia OOD 84 14 0 16 Mar 2023
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning Haoyu He Jianfei Cai Jing Zhang Dacheng Tao Bohan Zhuang VPVLM 89 57 0 15 Mar 2023
Is forgetting less a good inductive bias for forward transfer? Jiefeng Chen Timothy Nguyen Dilan Görür Arslan Chaudhry CLL 109 14 0 14 Mar 2023
Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need Da-Wei Zhou Han-Jia Ye De-Chuan Zhan Ziwei Liu CLL 106 111 0 13 Mar 2023
Model-tuning Via Prompts Makes NLP Models Adversarially Robust Mrigank Raman Pratyush Maini J. Zico Kolter Zachary Chase Lipton Danish Pruthi AAML 71 17 0 13 Mar 2023
Masked Images Are Counterfactual Samples for Robust Fine-tuning Yao Xiao Ziyi Tang Pengxu Wei Cong Liu Liang Lin 131 19 0 06 Mar 2023
Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries Charlotte Loh Seung-Jun Han Shivchander Sudalairaj Rumen Dangovski Kai Xu F. Wenzel Marin Soljacic Akash Srivastava UQCV 92 1 0 04 Mar 2023
Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning Rundong Luo Yifei Wang Yisen Wang 76 26 0 02 Mar 2023
UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction Andrianos Michail Stefanos Konstantinou Simon Clematide 58 21 0 02 Mar 2023
The Role of Pre-training Data in Transfer Learning R. Entezari Mitchell Wortsman O. Saukh M. Shariatnia Hanie Sedghi Ludwig Schmidt 98 23 0 27 Feb 2023
Improving Representational Continuity via Continued Pretraining Michael Sun Ananya Kumar Divyam Madaan Percy Liang CLL 24 2 0 26 Feb 2023
How to prepare your task head for finetuning Yi Ren Shangmin Guo Wonho Bae Danica J. Sutherland 68 14 0 11 Feb 2023
Cross-Modal Fine-Tuning: Align then Refine Junhong Shen Liam Li Lucio Dery Corey Staten M. Khodak Graham Neubig Ameet Talwalkar 80 41 0 11 Feb 2023
Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn VLM 88 11 0 10 Feb 2023
Representation Deficiency in Masked Language Modeling Yu Meng Jitin Krishnan Sinong Wang Qifan Wang Yuning Mao Han Fang Marjan Ghazvininejad Jiawei Han Luke Zettlemoyer 140 7 0 04 Feb 2023
Gradient Estimation for Unseen Domain Risk Minimization with Pre-Trained Models Byunggyu Lew Donghyun Son Buru Chang 70 6 0 03 Feb 2023
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data Zhouxing Shi Nicholas Carlini Ananth Balashankar Ludwig Schmidt Cho-Jui Hsieh Alex Beutel Yao Qin OOD 64 10 0 02 Feb 2023
CLIPood: Generalizing CLIP to Out-of-Distributions Yang Shu Xingzhuo Guo Jialong Wu Ximei Wang Jianmin Wang Mingsheng Long OODD VLM 149 79 0 02 Feb 2023
Debiasing Vision-Language Models via Biased Prompts Ching-Yao Chuang Varun Jampani Yuanzhen Li Antonio Torralba Stefanie Jegelka VLM 122 107 0 31 Jan 2023
Understanding Self-Distillation in the Presence of Label Noise Rudrajit Das Sujay Sanghavi 137 17 0 30 Jan 2023
Fine-Tuning Deteriorates General Textual Out-of-Distribution Detection by Distorting Task-Agnostic Features Sishuo Chen Wenkai Yang Xiaohan Bi Xu Sun OODD 60 15 0 30 Jan 2023
Alignment with human representations supports robust few-shot learning Ilia Sucholutsky Thomas Griffiths 62 27 0 27 Jan 2023
Personalised Federated Learning On Heterogeneous Feature Spaces A. Rakotomamonjy Maxime Vono H. M. Ruiz L. Ralaivola FedML 75 8 0 26 Jan 2023
Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment Carsten T. Lüth Till J. Bungert Lukas Klein Paul F. Jaeger 99 10 0 25 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 62 3 0 24 Jan 2023
Estimating Remaining Lifespan from the Face Amir Fekrazad MedIm 13 1 0 19 Jan 2023
NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis Allan Zhou Moo Jin Kim Lirui Wang Peter R. Florence Chelsea Finn 103 50 0 18 Jan 2023
Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models Zhiqiu Lin Samuel Yu Zhiyi Kuang Deepak Pathak Deva Ramana VLM 152 116 0 16 Jan 2023
Who Should Predict? Exact Algorithms For Learning to Defer to Humans Hussein Mozannar Hunter Lang Dennis L. Wei P. Sattigeri Subhro Das David Sontag 100 51 0 15 Jan 2023
Improving Reliability of Fine-tuning with Block-wise Optimisation Basel Barakat Qiang Huang 60 1 0 15 Jan 2023
Does progress on ImageNet transfer to real-world datasets? Alex Fang Simon Kornblith Ludwig Schmidt VLM 87 38 0 11 Jan 2023