Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

21 February 2022

Papers citing "Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution"

50 / 145 papers shown

Title
Improving Out-of-Domain Robustness with Targeted Augmentation in Frequency and Pixel Spaces Ruoqi Wang Haitao Wang Shaojie Guo Qiong Luo OOD 4 0 0 18 May 2025
Heterogeneous Data Game: Characterizing the Model Competition Across Multiple Data Sources Renzhe Xu Kun Wang Bo Li 36 0 0 12 May 2025
Query-based Knowledge Transfer for Heterogeneous Learning Environments Norah Alballa Wenxuan Zhang Ziquan Liu A. Abdelmoniem Mohamed Elhoseiny Marco Canini 38 0 0 12 Apr 2025
Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters Ruiqi Zhu Endong Sun Guanhe Huang Oya Celiktutan CLL OnRL 65 0 0 24 Mar 2025
Statistical Deficiency for Task Inclusion Estimation Loïc Fosse Frédéric Béchet Benoit Favre Géraldine Damnati Gwénolé Lecorvé Maxime Darrin Philippe Formont Pablo Piantanida 139 0 0 07 Mar 2025
Efficient Algorithms for Verifying Kruskal Rank in Sparse Linear Regression and Related Applications Fengqin Zhou 58 3 0 06 Mar 2025
Generalizable Prompt Learning of CLIP: A Brief Overview Fangming Cui Yonggang Zhang Xuan Wang Xule Wang Liang Xiao VPVLM VLM 161 0 0 03 Mar 2025
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation Zhiyuan Liu Yanchen Luo Han Huang Enzhi Zhang Sihang Li Fan Zhang Yaorui Shi Xuben Wang Kenji Kawaguchi Tat-Seng Chua 102 3 0 18 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 99 2 0 06 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 117 3 0 06 Feb 2025
Understanding the Capabilities and Limitations of Weak-to-Strong Generalization Wei Yao Wenkai Yang Zhilin Wang Yankai Lin Yong Liu ELM 107 1 0 03 Feb 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 48 0 0 28 Jan 2025
Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Scheduling Nannan Li Kevin J. Shih Bryan A. Plummer DiffM 54 0 0 08 Jan 2025
Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models Benjamin Laufer Jon M. Kleinberg Hoda Heidari 55 8 0 03 Jan 2025
Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes Aodi Li Liansheng Zhuang Xiao Long Minghong Yao Shafei Wang 186 0 0 18 Dec 2024
Meta Curvature-Aware Minimization for Domain Generalization Zhaoyu Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 197 1 0 16 Dec 2024
LAGUNA: LAnguage Guided UNsupervised Adaptation with structured spaces Anxhelo Diko Antonino Furnari Luigi Cinque G. Farinella 98 0 0 23 Nov 2024
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 39 1 0 07 Nov 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 134 1 0 11 Oct 2024
DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation Changdae Oh Yixuan Li Kyungwoo Song Sangdoo Yun Dongyoon Han OOD MoMe 45 4 0 03 Oct 2024
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 39 0 0 01 Oct 2024
Zero-shot forecasting of chaotic systems Yuanzhao Zhang William Gilpin AI4TS 37 4 0 24 Sep 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 57 3 0 03 Jul 2024
Macroeconomic Forecasting with Large Language Models Andrea Carriero Davide Pettenuzzo Shubhranshu Shekhar 54 4 0 01 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 62 14 0 24 Jun 2024
Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs Rui Yang Ruomeng Ding Yong Lin Huan Zhang Tong Zhang 44 43 0 14 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 77 3 0 05 Jun 2024
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate A. Fuller Daniel G. Kyrollos Yousef Yassin James R. Green 52 2 0 22 May 2024
Understanding Multimodal Deep Neural Networks: A Concept Selection View Chenming Shang Hengyuan Zhang Hao Wen Yujiu Yang 46 5 0 13 Apr 2024
Mixture of Low-rank Experts for Transferable AI-Generated Image Detection Zihan Liu Hanyi Wang Yaoyu Kang Shilin Wang MoE 41 12 0 07 Apr 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 82 3 0 18 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 44 1 0 01 Mar 2024
Embedding Compression for Teacher-to-Student Knowledge Transfer Yiwei Ding Alexander Lerch 26 1 0 09 Feb 2024
LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views Yuji Roh Qingyun Liu Huan Gui Zhe Yuan Yujin Tang ... Liang Liu Shuchao Bi Lichan Hong Ed H. Chi Zhe Zhao 43 1 0 07 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 37 4 0 31 Jan 2024
LanDA: Language-Guided Multi-Source Domain Adaptation Zhenbin Wang Lei Zhang Lituan Wang Minjuan Zhu 35 10 0 25 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 52 248 0 19 Jan 2024
HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization Guanglin Zhou Zhongyi Han Shiming Chen Erdun Gao Liming Zhu Tongliang Liu Lina Yao Anton van den Hengel 32 3 0 18 Jan 2024
Preserving Silent Features for Domain Generalization Chujie Zhao Tianren Zhang Feng Chen 23 0 0 06 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 52 10 0 23 Dec 2023
A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models Julio Silva-Rodríguez Sina Hajimiri Ismail Ben Ayed Jose Dolz VLM 26 27 0 20 Dec 2023
LaViP:Language-Grounded Visual Prompts Nilakshan Kunananthaseelan Jing Zhang Mehrtash Harandi VLM 25 0 0 18 Dec 2023
Open Domain Generalization with a Single Network by Regularization Exploiting Pre-trained Features Inseop Chung Kiyoon Yoo Nojun Kwak VLM 16 0 0 08 Dec 2023
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 56 1 0 30 Nov 2023
Large Pre-trained time series models for cross-domain Time series analysis tasks Harshavardhan Kamarthi B. A. Prakash VLM AI4TS 15 10 0 19 Nov 2023
Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks Jiarong Xu Renhong Huang Xin Jiang Yuxuan Cao Carl Yang Chunping Wang Yang Yang AI4CE 31 14 0 02 Nov 2023
Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 29 3 0 17 Oct 2023
Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies Kieran Wood Samuel Kessler Stephen J. Roberts S. Zohren AI4TS 31 2 0 16 Oct 2023
Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning Lapo Frati Neil Traft Jeff Clune Nick Cheney CLL 27 0 0 12 Oct 2023
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection Jiaying Wu Xinyu Chen Haobin Yang Qi Zhao Yuhui Shi AAML 27 11 0 28 Sep 2023