On the Overlooked Structure of Stochastic Gradients

On the Overlooked Structure of Stochastic Gradients

5 December 2022

Zeke Xie

Papers citing "On the Overlooked Structure of Stochastic Gradients"

5 / 5 papers shown

Title
AlphaGrad: Non-Linear Gradient Normalization Optimizer Soham Sane ODL 56 0 0 22 Apr 2025
Model Balancing Helps Low-data Training and Fine-tuning Zihang Liu Y. Hu Tianyu Pang Yefan Zhou Pu Ren Yaoqing Yang 34 2 0 16 Oct 2024
Understanding Adversarially Robust Generalization via Weight-Curvature Index Yuelin Xu Xiao Zhang AAML 32 0 0 10 Oct 2024
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs Satyen Kale Ayush Sekhari Karthik Sridharan 180 29 0 11 Jul 2021
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective Zeke Xie Zhiqiang Xu Jingzhao Zhang Issei Sato Masashi Sugiyama 17 20 0 23 Nov 2020