Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization

7 November 2023

Andrej Risteski

Papers citing "Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization"

4 / 4 papers shown

Title
Hidden Breakthroughs in Language Model Training Sara Kangaslahti Elan Rosenfeld Naomi Saphra 7 0 0 18 Jun 2025
Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More Geonhui Yoo Minhak Song Chulhee Yun FAtt 22 0 0 07 Jun 2025
Understanding Why Adam Outperforms SGD: Gradient Heterogeneity in Transformers Akiyoshi Tomihari Issei Sato ODL 153 3 0 31 Jan 2025
Training on the Edge of Stability Is Caused by Layerwise Jacobian Alignment Mark Lowell Catharine A. Kastner 109 0 0 31 May 2024