TRAM: Bridging Trust Regions and Sharpness Aware Minimization

TRAM: Bridging Trust Regions and Sharpness Aware Minimization

5 October 2023

Papers citing "TRAM: Bridging Trust Regions and Sharpness Aware Minimization"

16 / 16 papers shown

Title
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 142 373 0 13 Feb 2023
SAM as an Optimal Relaxation of Bayes Thomas Möllenhoff Mohammad Emtiyaz Khan BDL 57 34 0 04 Oct 2022
ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets Damien Teney Yong Lin Seong Joon Oh Ehsan Abbasnejad OOD 485 49 0 01 Sep 2022
Sharpness-Aware Minimization Improves Language Model Generalization Dara Bahri H. Mobahi Yi Tay 154 103 0 16 Oct 2021
Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization John Miller Rohan Taori Aditi Raghunathan Shiori Sagawa Pang Wei Koh Vaishaal Shankar Percy Liang Y. Carmon Ludwig Schmidt OODD OOD 65 275 0 09 Jul 2021
ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks Jungmin Kwon Jeongseop Kim Hyunseong Park I. Choi 86 289 0 23 Feb 2021
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 184 1,345 0 03 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 135 2,730 0 05 Jun 2020
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization Haoming Jiang Pengcheng He Weizhu Chen Xiaodong Liu Jianfeng Gao T. Zhao 78 561 0 08 Nov 2019
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 197 6,546 0 05 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 399 20,114 0 23 Oct 2019
FreeLB: Enhanced Adversarial Training for Natural Language Understanding Chen Zhu Yu Cheng Zhe Gan S. Sun Tom Goldstein Jingjing Liu AAML 261 442 0 25 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,152 0 20 Apr 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 517 4,476 0 18 Apr 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 419 2,936 0 15 Sep 2016
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 448 43,277 0 11 Feb 2015