v1v2v3 (latest)

Masked Training of Neural Networks with Partial Gradients

16 June 2021

Amirkeivan Mohtashami

Martin Jaggi

Sebastian U. Stich

ArXiv (abs)PDF HTML

Papers citing "Masked Training of Neural Networks with Partial Gradients"

21 / 21 papers shown

Title
Delayed Random Partial Gradient Averaging for Federated Learning Xinyi Hu FedML 104 0 0 31 Dec 2024
Characterising Bias in Compressed Models Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio Emily L. Denton 70 185 0 06 Oct 2020
Dynamic Sparsity Neural Networks for Automatic Speech Recognition Zhaofeng Wu Ding Zhao Qiao Liang Jiahui Yu Anmol Gulati Ruoming Pang 47 41 0 16 May 2020
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models Jiahui Yu Pengchong Jin Hanxiao Liu Gabriel Bender Pieter-Jan Kindermans Mingxing Tan Thomas Huang Xiaodan Song Ruoming Pang Quoc V. Le 84 304 0 24 Mar 2020
Almost Sure Convergence of Dropout Algorithms for Neural Networks Albert Senen-Cerda J. Sanders 46 8 0 06 Feb 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 159 628 0 11 Dec 2019
Distributed Learning of Deep Neural Networks using Independent Subnet Training John Shelton Hyatt Cameron R. Wolfe Michael Lee Yuxin Tang Anastasios Kyrillidis Christopher M. Jermaine OOD 84 38 0 04 Oct 2019
The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication Sebastian U. Stich Sai Praneeth Karimireddy FedML 62 20 0 11 Sep 2019
Once-for-All: Train One Network and Specialize it for Efficient Deployment Han Cai Chuang Gan Tianzhe Wang Zhekai Zhang Song Han OOD 113 1,283 0 26 Aug 2019
Unified Optimal Analysis of the (Stochastic) Gradient Method Sebastian U. Stich 54 113 0 09 Jul 2019
fairseq: A Fast, Extensible Toolkit for Sequence Modeling Myle Ott Sergey Edunov Alexei Baevski Angela Fan Sam Gross Nathan Ng David Grangier Michael Auli VLM FaML 114 3,156 0 01 Apr 2019
Slimmable Neural Networks Jiahui Yu L. Yang N. Xu Jianchao Yang Thomas Huang 88 556 0 21 Dec 2018
A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates Yossi Arjevani Ohad Shamir Nathan Srebro 72 63 0 26 Jun 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 253 3,485 0 09 Mar 2018
On First-Order Meta-Learning Algorithms Alex Nichol Joshua Achiam John Schulman 233 2,237 0 08 Mar 2018
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 197 1,281 0 05 Oct 2017
Sparse Communication for Distributed Gradient Descent Alham Fikri Aji Kenneth Heafield 76 741 0 17 Apr 2017
Coordinating Filters for Faster Deep Neural Networks W. Wen Cong Xu Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 43 138 0 28 Mar 2017
DeepFM: A Factorization-Machine based Neural Network for CTR Prediction Huifeng Guo Ruiming Tang Yunming Ye Zhenguo Li Xiuqiang He 122 2,654 0 13 Mar 2017
Wide & Deep Learning for Recommender Systems Heng-Tze Cheng L. Koc Jeremiah Harmsen T. Shaked Tushar Chandra ... Zakaria Haque Lichan Hong Vihan Jain Xiaobing Liu Hemal Shah HAI VLM 192 3,663 0 24 Jun 2016
Learning both Weights and Connections for Efficient Neural Networks Song Han Jeff Pool J. Tran W. Dally CVBM 313 6,700 0 08 Jun 2015