Training Deep Nets with Sublinear Memory Cost

21 April 2016

Papers citing "Training Deep Nets with Sublinear Memory Cost"

32 / 232 papers shown

Title
Blockwise Self-Attention for Long Document Understanding J. Qiu Hao Ma Omer Levy Scott Yih Sinong Wang Jie Tang 11 251 0 07 Nov 2019
On-Device Machine Learning: An Algorithms and Learning Theory Perspective Sauptik Dhar Junyao Guo Jiayi Liu S. Tripathi Unmesh Kurup Mohak Shah 28 141 0 02 Nov 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 85 6,375 0 26 Sep 2019
DUAL-GLOW: Conditional Flow-Based Generative Model for Modality Transfer Haoliang Sun Ronak R. Mehta H. Zhou Z. Huang Sterling C. Johnson V. Prabhakaran Vikas Singh MedIm 31 46 0 21 Aug 2019
Profiling based Out-of-core Hybrid Method for Large Neural Networks Yuki Ito Haruki Imai Tung D. Le Yasushi Negishi K. Kawachiya R. Matsumiya Toshio Endo 24 9 0 11 Jul 2019
Database Meets Deep Learning: Challenges and Opportunities Wei Wang Meihui Zhang Gang Chen H. V. Jagadish Beng Chin Ooi K. Tan 13 147 0 21 Jun 2019
Evaluating Protein Transfer Learning with TAPE Roshan Rao Nicholas Bhattacharya Neil Thomas Yan Duan Xi Chen John F. Canny Pieter Abbeel Yun S. Song SSL 39 782 0 19 Jun 2019
DeepView: View Synthesis with Learned Gradient Descent John Flynn M. Broxton P. Debevec Matthew DuVall Graham Fyffe Ryan S. Overbeck Noah Snavely Richard Tucker 12 446 0 18 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 16 1,848 0 23 Apr 2019
Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism Nikoli Dryden N. Maruyama Tom Benson Tim Moon M. Snir B. Van Essen 26 49 0 15 Mar 2019
SimpleDet: A Simple and Versatile Distributed Framework for Object Detection and Instance Recognition Yuntao Chen Chenxia Han Yanghao Li Zehao Huang Yi-Xin Jiang Naiyan Wang Zhaoxiang Zhang 94 30 0 14 Mar 2019
ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs A. Gholami Kurt Keutzer George Biros 30 166 0 27 Feb 2019
Efficient Memory Management for GPU-based Deep Learning Systems Junzhe Zhang Sai-Ho Yeung Yao Shu Bingsheng He Wei Wang 18 41 0 19 Feb 2019
Training on the Edge: The why and the how Navjot Kukreja Alena Shilova Olivier Beaumont Jan Huckelheim N. Ferrier P. Hovland Gerard Gorman 16 33 0 13 Feb 2019
AccUDNN: A GPU Memory Efficient Accelerator for Training Ultra-deep Neural Networks Jinrong Guo Wantao Liu Wang Wang Q. Lu Songlin Hu Jizhong Han Ruixuan Li 11 9 0 21 Jan 2019
Learning Energy Based Inpainting for Optical Flow Christoph Vogel Huijuan Cao Thomas Pock 3DPC 30 5 0 09 Nov 2018
Supporting Very Large Models using Automatic Dataflow Graph Partitioning Minjie Wang Chien-chin Huang Jinyang Li 40 154 0 24 Jul 2018
Backdrop: Stochastic Backpropagation Siavash Golkar Kyle Cranmer 41 2 0 04 Jun 2018
Collaborative Learning for Deep Neural Networks Guocong Song Wei Chai FedML 21 192 0 30 May 2018
Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training Bojian Zheng Abhishek Tiwari Nandita Vijaykumar Gennady Pekhimenko 24 44 0 22 May 2018
Dynamic Control Flow in Large-Scale Machine Learning Yuan Yu Martín Abadi P. Barham E. Brevdo M. Burrows ... Michael Isard M. Kudlur R. Monga D. Murray Xiaoqiang Zheng AI4CE 22 106 0 04 May 2018
Profile-guided memory optimization for deep neural networks Taro Sekiyama T. Imamichi Haruki Imai Raymond H. Putra 26 22 0 26 Apr 2018
Value-aware Quantization for Training and Inference of Neural Networks Eunhyeok Park S. Yoo Peter Vajda MQ 14 158 0 20 Apr 2018
Learning Longer-term Dependencies in RNNs with Auxiliary Losses Trieu H. Trinh Andrew M. Dai Thang Luong Quoc V. Le 30 179 0 01 Mar 2018
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Tal Ben-Nun Torsten Hoefler GNN 33 702 0 26 Feb 2018
Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics using CNNs Andres Milioto C. Stachniss SSeg 43 86 0 25 Feb 2018
SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks Linnan Wang Jinmian Ye Yiyang Zhao Wei Wu Ang Li Shuaiwen Leon Song Zenglin Xu Tim Kraska 3DH 46 264 0 13 Jan 2018
Memory-Efficient Implementation of DenseNets Geoff Pleiss Danlu Chen Gao Huang Tongcheng Li L. V. D. van der Maaten Kilian Q. Weinberger 36 159 0 21 Jul 2017
UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory Iasonas Kokkinos SSeg SSL 22 669 0 07 Sep 2016
Memory-Efficient Backpropagation Through Time A. Gruslys Rémi Munos Ivo Danihelka Marc Lanctot Alex Graves 32 228 0 10 Jun 2016
Theano: A Python framework for fast computation of mathematical expressions The Theano Development Team Rami Al-Rfou Guillaume Alain Amjad Almahairi Christof Angermüller ... Kelvin Xu Lijun Xue Li Yao Saizheng Zhang Ying Zhang 20 2,335 0 09 May 2016
Joint Training of Deep Boltzmann Machines Ian Goodfellow Aaron Courville Yoshua Bengio FedML 77 28 0 12 Dec 2012