Low-rank Gradient Approximation For Memory-Efficient On-device Training
of Deep Neural Network

Low-rank Gradient Approximation For Memory-Efficient On-device Training of Deep Neural Network

24 January 2020

Mary Gooneratne

ArXiv (abs)PDF HTML

Papers citing "Low-rank Gradient Approximation For Memory-Efficient On-device Training of Deep Neural Network"

12 / 12 papers shown

Title
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 91 4 0 31 Dec 2024
Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entities K. Sim F. Beaufays Arnaud Benard Dhruv Guliani Andreas Kabel ... P. Zadrazil Harry Zhang Leif T. Johnson Giovanni Motta Lillian Zhou 65 83 0 14 Dec 2019
An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models K. Sim P. Zadrazil F. Beaufays 76 58 0 14 Sep 2019
Streaming End-to-end Speech Recognition For Mobile Devices Yanzhang He Tara N. Sainath Rohit Prabhavalkar Ian McGraw R. Álvarez ... K. Sim Tom Bagby Shuo-yiin Chang Kanishka Rao A. Gruenstein 114 627 0 15 Nov 2018
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy Asit K. Mishra Debbie Marr FedML 65 331 0 15 Nov 2017
Knowledge Distillation for Small-footprint Highway Networks Liang Lu Michelle Guo Steve Renals 65 73 0 02 Aug 2016
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems Martín Abadi Ashish Agarwal P. Barham E. Brevdo Zhiwen Chen ... Pete Warden Martin Wattenberg Martin Wicke Yuan Yu Xiaoqiang Zheng 282 11,150 0 14 Mar 2016
Deep Learning with Limited Numerical Precision Suyog Gupta A. Agrawal K. Gopalakrishnan P. Narayanan HAI 207 2,049 0 09 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014
GPU Asynchronous Stochastic Gradient Descent to Speed Up Neural Network Training T. Paine Hailin Jin Jianchao Yang Zhe Lin Thomas Huang 108 98 0 21 Dec 2013
Multi-GPU Training of ConvNets Guillermo A. Castillo Keith Adams Yaniv Taigman Ayonga Hereid 67 101 0 20 Dec 2013
Sequence Transduction with Recurrent Neural Networks Alex Graves 193 1,871 0 14 Nov 2012