How Feature Learning Can Improve Neural Scaling Laws

26 September 2024

Papers citing "How Feature Learning Can Improve Neural Scaling Laws"

11 / 11 papers shown

Title
Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures Francesco Cagnetta Alessandro Favero Antonio Sclocchi M. Wyart 26 0 0 11 May 2025
Corner Gradient Descent Dmitry Yarotsky 36 0 0 16 Apr 2025
Dynamically Learning to Integrate in Recurrent Neural Networks Blake Bordelon Jordan Cotler C. Pehlevan Jacob A. Zavatone-Veth 53 2 0 24 Mar 2025
A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules Kairong Luo Haodong Wen Shengding Hu Zhenbo Sun Zhiyuan Liu Maosong Sun Kaifeng Lyu Wenguang Chen CLL 59 1 0 17 Mar 2025
Uncertainty Quantification From Scaling Laws in Deep Neural Networks Ibrahim Elsharkawy Yonatan Kahn Benjamin Hooberman UQCV 45 0 0 07 Mar 2025
Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer Blake Bordelon C. Pehlevan AI4CE 61 1 0 04 Feb 2025
Loss-to-Loss Prediction: Scaling Laws for All Datasets David Brandfonbrener Nikhil Anand Nikhil Vyas Eran Malach Sham Kakade 77 3 0 19 Nov 2024
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws M. E. Ildiz Halil Alperen Gozeten Ege Onur Taga Marco Mondelli Samet Oymak 54 2 0 24 Oct 2024
Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra Roman Worschech B. Rosenow 39 0 0 11 Oct 2024
The Optimization Landscape of SGD Across the Feature Learning Strength Alexander B. Atanasov Alexandru Meterez James B. Simon C. Pehlevan 43 2 0 06 Oct 2024
A Generalization Bound for Nearly-Linear Networks Eugene Golikov 26 0 0 09 Jul 2024