Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution

21 February 2022

Papers citing "Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution"

50 / 130 papers shown

Title
Heterogeneous Data Game: Characterizing the Model Competition Across Multiple Data Sources Renzhe Xu K. Wang Bo Li 31 0 0 12 May 2025
Query-based Knowledge Transfer for Heterogeneous Learning Environments Norah Alballa Wenxuan Zhang Ziquan Liu A. Abdelmoniem Mohamed Elhoseiny Marco Canini 36 0 0 12 Apr 2025
Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters Ruiqi Zhu Endong Sun Guanhe Huang Oya Celiktutan CLL OnRL 61 0 0 24 Mar 2025
Statistical Deficiency for Task Inclusion Estimation Loïc Fosse Frédéric Béchet Benoit Favre Géraldine Damnati Gwénolé Lecorvé Maxime Darrin Philippe Formont Pablo Piantanida 136 0 0 07 Mar 2025
Efficient Algorithms for Verifying Kruskal Rank in Sparse Linear Regression and Related Applications Fengqin Zhou 52 3 0 06 Mar 2025
Generalizable Prompt Learning of CLIP: A Brief Overview Fangming Cui Yonggang Zhang Xuan Wang Xule Wang Liang Xiao VPVLM VLM 153 0 0 03 Mar 2025
NExT-Mol: 3D Diffusion Meets 1D Language Modeling for 3D Molecule Generation Zhiyuan Liu Yanchen Luo Han Huang Enzhi Zhang Sihang Li Junfeng Fang Yaorui Shi X. Wang Kenji Kawaguchi Tat-Seng Chua 100 3 0 18 Feb 2025
The Best Instruction-Tuning Data are Those That Fit Dylan Zhang Qirun Dai Hao Peng ALM 115 3 0 06 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 93 1 0 06 Feb 2025
Understanding the Capabilities and Limitations of Weak-to-Strong Generalization Wei Yao Wenkai Yang Z. Wang Yankai Lin Yong Liu ELM 103 1 0 03 Feb 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 48 0 0 28 Jan 2025
Enhancing Virtual Try-On with Synthetic Pairs and Error-Aware Noise Scheduling Nannan Li Kevin J. Shih Bryan A. Plummer DiffM 54 0 0 08 Jan 2025
Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models Benjamin Laufer Jon M. Kleinberg Hoda Heidari 55 8 0 03 Jan 2025
Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes Aodi Li Liansheng Zhuang Xiao Long Minghong Yao Shafei Wang 180 0 0 18 Dec 2024
Meta Curvature-Aware Minimization for Domain Generalization Z. Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 191 1 0 16 Dec 2024
LAGUNA: LAnguage Guided UNsupervised Adaptation with structured spaces Anxhelo Diko Antonino Furnari Luigi Cinque G. Farinella 90 0 0 23 Nov 2024
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 33 1 0 07 Nov 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 131 1 0 11 Oct 2024
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 34 0 0 01 Oct 2024
Zero-shot forecasting of chaotic systems Yuanzhao Zhang William Gilpin AI4TS 37 4 0 24 Sep 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 54 3 0 03 Jul 2024
Macroeconomic Forecasting with Large Language Models Andrea Carriero Davide Pettenuzzo Shubhranshu Shekhar 46 4 0 01 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 54 13 0 24 Jun 2024
Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs Rui Yang Ruomeng Ding Yong Lin Huan Zhang Tong Zhang 32 42 0 14 Jun 2024
Feature contamination: Neural networks learn uncorrelated features and fail to generalize Tianren Zhang Chujie Zhao Guanyu Chen Yizhou Jiang Feng Chen OOD MLT OODD 77 3 0 05 Jun 2024
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate A. Fuller Daniel G. Kyrollos Yousef Yassin James R. Green 46 2 0 22 May 2024
Understanding Multimodal Deep Neural Networks: A Concept Selection View Chenming Shang Hengyuan Zhang Hao Wen Yujiu Yang 43 5 0 13 Apr 2024
Mixture of Low-rank Experts for Transferable AI-Generated Image Detection Zihan Liu Hanyi Wang Yaoyu Kang Shilin Wang MoE 36 12 0 07 Apr 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 82 3 0 18 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 39 1 0 01 Mar 2024
Embedding Compression for Teacher-to-Student Knowledge Transfer Yiwei Ding Alexander Lerch 26 1 0 09 Feb 2024
LEVI: Generalizable Fine-tuning via Layer-wise Ensemble of Different Views Yuji Roh Qingyun Liu Huan Gui Zhe Yuan Yujin Tang ... Liang Liu Shuchao Bi Lichan Hong Ed H. Chi Zhe Zhao 43 1 0 07 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 34 4 0 31 Jan 2024
LanDA: Language-Guided Multi-Source Domain Adaptation Zhenbin Wang Lei Zhang Lituan Wang Minjuan Zhu 27 10 0 25 Jan 2024
Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads Tianle Cai Yuhong Li Zhengyang Geng Hongwu Peng Jason D. Lee De-huai Chen Tri Dao 44 247 0 19 Jan 2024
HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization Guanglin Zhou Zhongyi Han Shiming Chen Biwei Huang Liming Zhu Tongliang Liu Lina Yao Kun Zhang 32 3 0 18 Jan 2024
Preserving Silent Features for Domain Generalization Chujie Zhao Tianren Zhang Feng Chen 23 0 0 06 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel S. Pokutta VLM 50 10 0 23 Dec 2023
LaViP:Language-Grounded Visual Prompts Nilakshan Kunananthaseelan Jing Zhang Mehrtash Harandi VLM 14 0 0 18 Dec 2023
Open Domain Generalization with a Single Network by Regularization Exploiting Pre-trained Features Inseop Chung Kiyoon Yoo Nojun Kwak VLM 16 0 0 08 Dec 2023
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 53 1 0 30 Nov 2023
Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks Jiarong Xu Renhong Huang Xin Jiang Yuxuan Cao Carl Yang Chunping Wang Yang Yang AI4CE 31 14 0 02 Nov 2023
Few-Shot Learning Patterns in Financial Time-Series for Trend-Following Strategies Kieran Wood Samuel Kessler Stephen J. Roberts S. Zohren AI4TS 23 2 0 16 Oct 2023
Reset It and Forget It: Relearning Last-Layer Weights Improves Continual and Transfer Learning Lapo Frati Neil Traft Jeff Clune Nick Cheney CLL 19 0 0 12 Oct 2023
Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection Jiaying Wu Xinyu Chen Haobin Yang Qi Zhao Yuhui Shi AAML 21 11 0 28 Sep 2023
Test-Time Training for Speech Sri Harsha Dumpala Chandramouli Shama Sastry Sageev Oore 39 1 0 19 Sep 2023
LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters Ching Chang Wei-Yao Wang Wenjie Peng Tien-Fu Chen AI4TS 38 45 0 16 Aug 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 64 33 0 15 Aug 2023
Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study Luis P. Silvestrin Shujian Yu Mark Hoogendoorn OOD 31 1 0 17 Jul 2023
Advances and Challenges in Meta-Learning: A Technical Review Anna Vettoruzzo Mohamed-Rafik Bouguelia Joaquin Vanschoren Thorsteinn Rögnvaldsson K. Santosh OffRL 19 70 0 10 Jul 2023