v1v2v3 (latest)

Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines

17 February 2022

Papers citing "Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines"

43 / 43 papers shown

Title
Asynchronous Stochastic Gradient Descent with Decoupled Backpropagation and Layer-Wise Updates Cabrel Teguemne Fokam Khaleelulla Khan Nazeer Lukas König David Kappel Anand Subramoney 81 0 0 08 Oct 2024
Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines Michael Kuchnik Ana Klimovic Jiří Šimša Virginia Smith George Amvrosiadis 97 31 0 07 Nov 2021
Can Single-Shuffle SGD be Better than Reshuffling SGD and GD? Chulhee Yun S. Sra Ali Jadbabaie 48 10 0 12 Mar 2021
tf.data: A Machine Learning Data Processing Framework D. Murray Jiří Šimša Ana Klimovic Ihor Indyk PINN AI4CE LMTD 90 88 0 28 Jan 2021
The Cube++ Illumination Estimation Dataset E. Ershov A. Savchik I. Semenkov Nikola Banić A. Belokopytov Daria Senshina Karlo Koščević M. Subašić Sven Lončarić 56 24 0 19 Nov 2020
Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics Daniel Kang A. Mathur Teja Veeramacheneni Peter Bailis Matei A. Zaharia 65 43 0 25 Jul 2020
Analyzing and Mitigating Data Stalls in DNN Training Jayashree Mohan Amar Phanishayee Ashish Raniwala Vijay Chidambaram 55 106 0 14 Jul 2020
A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs Jiajia Li M. Lakshminarasimhan Xiaolong Wu Ang Li C. Olschanowsky Kevin J. Barker 18 3 0 02 Jan 2020
Common Voice: A Massively-Multilingual Speech Corpus Rosana Ardila Megan Branson Kelly Davis Michael Henretty M. Kohler Josh Meyer Reuben Morais Lindsay Saunders Francis M. Tyers Gregor Weber VLM 91 1,614 0 13 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 526 42,559 0 03 Dec 2019
Performance Analysis of Deep Learning Workloads on Leading-edge Systems Yihui Ren Shinjae Yoo A. Hoisie ELM 33 22 0 21 May 2019
Deep Learning for Audio Signal Processing Hendrik Purwins Yue Liu Tuomas Virtanen Jan Schlüter Shuo-yiin Chang Tara N. Sainath VLM 101 594 0 30 Apr 2019
Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools R. Mayer Hans-Arno Jacobsen GNN 72 190 0 27 Mar 2019
Image Classification at Supercomputer Scale Chris Ying Sameer Kumar Dehao Chen Tao Wang Youlong Cheng VLM 51 123 0 16 Nov 2018
Characterizing Deep-Learning I/O Workloads in TensorFlow Steven W. D. Chien Stefano Markidis C. Sishtla Luís Santos Pawel Herman Sai B. Narasimhamurthy Erwin Laure 58 50 0 06 Oct 2018
Deep Learning Approaches for Understanding Simple Speech Commands R. Solovyev Maxim Vakhrushev Alexander Radionov Vladimir Aliev Alexey A. Shvets VLM 48 31 0 04 Oct 2018
Beyond Data and Model Parallelism for Deep Neural Networks Zhihao Jia Matei A. Zaharia A. Aiken GNN AI4CE 64 505 0 14 Jul 2018
Random Shuffling Beats SGD after Finite Epochs Jeff Z. HaoChen S. Sra 57 99 0 26 Jun 2018
Understanding the Performance of Ceph Block Storage for Hyper-Converged Cloud with All Flash Storage Moo-Ryong Ra 18 4 0 22 Feb 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,221 0 15 Feb 2018
Deep contextualized word representations Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer NAI 227 11,565 0 15 Feb 2018
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 168 1,804 0 10 Oct 2017
Comparison of Time-Frequency Representations for Environmental Sound Classification using Convolutional Neural Networks M. Huzaifah AI4TS 56 148 0 22 Jun 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 357 27,230 0 20 Mar 2017
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 433 18,361 0 27 May 2016
Distributed TensorFlow with MPI Abhinav Vishnu Charles Siegel J. Daily 51 39 0 07 Mar 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin Dario Amodei Rishita Anubhai Eric Battenberg Carl Case Jared Casper ... Chong-Jun Wang Bo Xiao Dani Yogatama J. Zhan Zhenyao Zhu 137 2,974 0 08 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 886 27,412 0 02 Dec 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,854 0 01 Oct 2015
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 228 7,755 0 31 Aug 2015
Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books Yukun Zhu Ryan Kiros R. Zemel Ruslan Salakhutdinov R. Urtasun Antonio Torralba Sanja Fidler 127 2,554 0 22 Jun 2015
Simultaneous Feature Learning and Hash Coding with Deep Neural Networks Hanjiang Lai Yan Pan Ye Liu Shuicheng Yan 57 823 0 14 Apr 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,260 0 22 Dec 2014
Deep Speech: Scaling up end-to-end speech recognition Awni Y. Hannun Carl Case Jared Casper Bryan Catanzaro G. Diamos ... R. Prenger S. Satheesh Shubho Sengupta Adam Coates A. Ng 186 2,128 0 17 Dec 2014
Show and Tell: A Neural Image Caption Generator Oriol Vinyals Alexander Toshev Samy Bengio D. Erhan 3DV 249 6,035 0 17 Nov 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 485 43,685 0 17 Sep 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,479 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,590 0 01 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 424 43,777 0 01 May 2014
One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling Ciprian Chelba Tomas Mikolov M. Schuster Qi Ge T. Brants P. Koehn T. Robinson 190 1,109 0 11 Dec 2013
Visualizing and Understanding Convolutional Networks Matthew D. Zeiler Rob Fergus FAtt SSL 595 15,902 0 12 Nov 2013
Efficient Estimation of Word Representations in Vector Space Tomas Mikolov Kai Chen G. Corrado J. Dean 3DV 680 31,538 0 16 Jan 2013