Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning

8 February 2022

Papers citing "Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning"

33 / 83 papers shown

Title
Gotta match ém all: Solution diversification in graph matching matched filters Zhirui Li Ben Johnson D. Sussman Carey E. Priebe V. Lyzinski 27 0 0 25 Aug 2023
Bias-Aware Minimisation: Understanding and Mitigating Estimator Bias in Private SGD Moritz Knolle R. Dorfman Alexander Ziller Daniel Rueckert Georgios Kaissis 20 2 0 23 Aug 2023
DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning Qinglun Li Li Shen Guang-Ming Li Quanjun Yin Dacheng Tao FedML 31 7 0 16 Aug 2023
G-Mix: A Generalized Mixup Learning Framework Towards Flat Minima Xingyu Li Bo Tang AAML 17 0 0 07 Aug 2023
Flatness-Aware Minimization for Domain Generalization Xingxuan Zhang Renzhe Xu Han Yu Yancheng Dong Pengfei Tian Peng Cu 32 20 0 20 Jul 2023
Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training Yechao Zhang Shengshan Hu Leo Yu Zhang Junyu Shi Minghui Li Xiaogeng Liu Wei Wan Hai Jin AAML 22 21 0 15 Jul 2023
Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer Peng Mi Li Shen Tianhe Ren Yiyi Zhou Tianshuo Xu Xiaoshuai Sun Tongliang Liu Rongrong Ji Dacheng Tao AAML 37 2 0 30 Jun 2023
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization Khashayar Gatmiry Zhiyuan Li Ching-Yao Chuang Sashank J. Reddi Tengyu Ma Stefanie Jegelka ODL 25 11 0 22 Jun 2023
The Split Matters: Flat Minima Methods for Improving the Performance of GNNs N. Lell A. Scherp 43 1 0 15 Jun 2023
Differentially Private Sharpness-Aware Training Jinseong Park Hoki Kim Yujin Choi Jaewook Lee 27 8 0 09 Jun 2023
Boosting Adversarial Transferability by Achieving Flat Local Maxima Zhijin Ge Hongying Liu Xiaosen Wang Fanhua Shang Yuanyuan Liu AAML 14 40 0 08 Jun 2023
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training Yi Shi Yingqi Liu Yan Sun Zihao Lin Li Shen Xueqian Wang Dacheng Tao FedML 45 10 0 24 May 2023
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape Yan Sun Li Shen Shi-Yong Chen Liang Ding Dacheng Tao FedML 34 33 0 19 May 2023
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency Lingfeng Shen Weiting Tan Boyuan Zheng Daniel Khashabi VLM 39 6 0 18 May 2023
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy Yi Shi Kang Wei Li Shen Yingqi Liu Xueqian Wang Bo Yuan Dacheng Tao FedML 38 2 0 01 May 2023
An Adaptive Policy to Employ Sharpness-Aware Minimization Weisen Jiang Hansi Yang Yu Zhang James T. Kwok AAML 83 31 0 28 Apr 2023
Per-Example Gradient Regularization Improves Learning Signals from Noisy Data Xuran Meng Yuan Cao Difan Zou 25 5 0 31 Mar 2023
Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimization Zhuo Huang Miaoxi Zhu Xiaobo Xia Li Shen Jun Yu Chen Gong Bo Han Bo Du Tongliang Liu 35 33 0 23 Mar 2023
Make Landscape Flatter in Differentially Private Federated Learning Yi Shi Yingqi Liu Kang Wei Li Shen Xueqian Wang Dacheng Tao FedML 25 54 0 20 Mar 2023
Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization Xingxuan Zhang Renzhe Xu Han Yu Hao Zou Peng Cui 16 39 0 03 Mar 2023
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy Yan Sun Li Shen Tiansheng Huang Liang Ding Dacheng Tao FedML 36 51 0 21 Feb 2023
Improving Differentiable Architecture Search via Self-Distillation Xunyu Zhu Jian Li Yong Liu Weiping Wang 24 7 0 11 Feb 2023
Improving the Model Consistency of Decentralized Federated Learning Yi Shi Li Shen Kang Wei Yan Sun Bo Yuan Xueqian Wang Dacheng Tao FedML 36 51 0 08 Feb 2023
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Dinh Q. Phung Trung Le 27 11 0 24 Nov 2022
Efficient Generalization Improvement Guided by Random Weight Perturbation Tao Li Wei Yan Zehao Lei Yingwen Wu Kun Fang Ming Yang X. Huang AAML 40 6 0 21 Nov 2022
How Does Sharpness-Aware Minimization Minimize Sharpness? Kaiyue Wen Tengyu Ma Zhiyuan Li AAML 23 47 0 10 Nov 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 29 69 0 11 Oct 2022
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias Ryo Karakida Tomoumi Takase Tomohiro Hayase Kazuki Osawa 15 14 0 06 Oct 2022
On the Interpretability of Regularisation for Neural Networks Through Model Gradient Similarity Vincent Szolnoky Viktor Andersson Balázs Kulcsár Rebecka Jörnsten 42 5 0 25 May 2022
Randomized Sharpness-Aware Training for Boosting Computational Efficiency in Deep Learning Yang Zhao Hao Zhang Xiuyuan Hu 18 9 0 18 Mar 2022
Estimating the Generalization in Deep Neural Networks via Sparsity Yang Zhao Hao Zhang 45 2 0 02 Apr 2021
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective Zeke Xie Zhiqiang Xu Jingzhao Zhang Issei Sato Masashi Sugiyama 19 22 0 23 Nov 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016