Memory-Efficient Adaptive Optimization

Memory-Efficient Adaptive Optimization

30 January 2019

Papers citing "Memory-Efficient Adaptive Optimization"

18 / 18 papers shown

Title
Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions Vladimir Feinberg Xinyi Chen Y. Jennifer Sun Rohan Anil Elad Hazan 29 12 0 07 Feb 2023
Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language Yusuke Yasuda T. Toda 33 8 0 16 Dec 2022
Pseudo-Riemannian Embedding Models for Multi-Relational Graph Representations Saee Paliwal Angus Brayne Benedek Fabian Maciej Wiatrak Aaron Sim 23 0 0 02 Dec 2022
VeLO: Training Versatile Learned Optimizers by Scaling Up Luke Metz James Harrison C. Freeman Amil Merchant Lucas Beyer ... Naman Agrawal Ben Poole Igor Mordatch Adam Roberts Jascha Narain Sohl-Dickstein 37 60 0 17 Nov 2022
Scientometric Review of Artificial Intelligence for Operations & Maintenance of Wind Turbines: The Past, Present and Future Joyjit Chatterjee Nina Dethlefs 31 83 0 30 Mar 2022
Robust Training of Neural Networks Using Scale Invariant Architectures Zhiyuan Li Srinadh Bhojanapalli Manzil Zaheer Sashank J. Reddi Surinder Kumar 29 27 0 02 Feb 2022
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 32 14 0 01 Nov 2021
8-bit Optimizers via Block-wise Quantization Tim Dettmers M. Lewis Sam Shleifer Luke Zettlemoyer MQ 34 276 0 06 Oct 2021
Large-Scale Differentially Private BERT Rohan Anil Badih Ghazi Vineet Gupta Ravi Kumar Pasin Manurangsi 36 132 0 03 Aug 2021
PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTS Ye Jia Heiga Zen Jonathan Shen Yu Zhang Yonghui Wu SSL 47 81 0 28 Mar 2021
Quasi-symplectic Langevin Variational Autoencoder Zihao Wang H. Delingette BDL DRL 11 4 0 02 Sep 2020
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization Neha S. Wadia Daniel Duckworth S. Schoenholz Ethan Dyer Jascha Narain Sohl-Dickstein 29 13 0 17 Aug 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 42 162 0 03 Jul 2020
The Limit of the Batch Size Yang You Yuhui Wang Huan Zhang Zhao-jie Zhang J. Demmel Cho-Jui Hsieh 16 15 0 15 Jun 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 18 119 0 09 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 132 19,529 0 23 Oct 2019
CTRL: A Conditional Transformer Language Model for Controllable Generation N. Keskar Bryan McCann Lav Varshney Caiming Xiong R. Socher AI4CE 57 1,237 0 11 Sep 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016