Selecting Large Language Model to Fine-tune via Rectified Scaling Law

Selecting Large Language Model to Fine-tune via Rectified Scaling Law

4 February 2024

Sujian Li

Xiaojun Wan

Papers citing "Selecting Large Language Model to Fine-tune via Rectified Scaling Law"

14 / 14 papers shown

Title
LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection Xinyue Zeng Haohui Wang Junhong Lin Jun Wu Tyler Cody Dawei Zhou 100 0 0 01 May 2025
Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Framework Yushen Lin Ruichen Zhang Wenqi Huang Kaidi Wang Z. Ding Daniel K. C. So Dusit Niyato 59 0 0 17 Jan 2025
Compute-Constrained Data Selection Junjie Oscar Yin Alexander M. Rush 39 0 0 21 Oct 2024
Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen Binxuan Huang Yifan Gao Zhengyang Wang Jingfeng Yang Heng Ji LRM 45 8 0 11 Oct 2024
Scaling Optimal LR Across Token Horizons Johan Bjorck Alon Benhaim Vishrav Chaudhary Furu Wei Xia Song 54 4 0 30 Sep 2024
CLoG: Benchmarking Continual Learning of Image Generation Models Haotian Zhang Junting Zhou Haowei Lin Hang Ye Jianhua Zhu Zihao Wang Liangcai Gao Yizhou Wang Yitao Liang DiffM VLM 34 1 0 07 Jun 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 139 306 0 05 Jan 2024
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions Minghao Wu Abdul Waheed Chiyu Zhang Muhammad Abdul-Mageed Alham Fikri Aji ALM 132 119 0 27 Apr 2023
Revisiting Neural Scaling Laws in Language and Vision Ibrahim M. Alabdulmohsin Behnam Neyshabur Xiaohua Zhai 156 102 0 13 Sep 2022
Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance Shibal Ibrahim Natalia Ponomareva Rahul Mazumder AAML 106 16 0 13 Oct 2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 203 110 0 22 Sep 2021
A linearized framework and a new benchmark for model selection for fine-tuning Aditya Deshpande Alessandro Achille Avinash Ravichandran Hao Li L. Zancato Charless C. Fowlkes Rahul Bhotika Stefano Soatto Pietro Perona ALM 112 46 0 29 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 240 4,469 0 23 Jan 2020
Transferability and Hardness of Supervised Classification Tasks Anh Tran Cuong V Nguyen Tal Hassner 134 164 0 21 Aug 2019