Sparse is Enough in Fine-tuning Pre-trained Large Language Models

Sparse is Enough in Fine-tuning Pre-trained Large Language Models

19 December 2023

Bo Du

Papers citing "Sparse is Enough in Fine-tuning Pre-trained Large Language Models"

9 / 9 papers shown

Title
Memory-Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation Fei Wu Jia Hu Geyong Min Shiqiang Wang 17 0 0 16 May 2025
How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients Ming Li Yong Li Ziyue Li Tianyi Zhou LRM 27 1 0 14 Apr 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 65 1 0 06 Mar 2025
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 31 4 0 11 Oct 2024
Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models Luohe Shi Yao Yao Zuchao Li Lefei Zhang Hai Zhao 29 0 0 30 Sep 2024
Sparse Matrix in Large Language Model Fine-tuning Haoze He Juncheng Billy Li Xuan Jiang Heather Miller MoE 27 3 0 24 May 2024
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016
Pac-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning O. Catoni 151 454 0 03 Dec 2007