v1v2 (latest)

Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training

12 August 2022

Papers citing "Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training"

35 / 35 papers shown

Title
MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems Yao Fu Yao Fu Yeqi Huang Ping Nie Zhan Lu ... Dayou Du Tairan Xu Dayou Du Edoardo Ponti Luo Mai MoE 97 1 0 16 May 2025
Green Prompting Marta Adamska Daria Smirnova Hamid Nasiri Zhengxin Yu Peter Garraghan 527 1 0 09 Mar 2025
THOR: A Generic Energy Estimation Approach for On-Device Training Jiaru Zhang Zesong Wang Hao Wang Tao Song Huai-an Su ... Yang Hua Xiangwei Zhou Ruhui Ma Miao Pan Haibing Guan 104 0 0 27 Jan 2025
MLPerf Power: Benchmarking the Energy Efficiency of Machine Learning Systems from Microwatts to Megawatts for Sustainable AI Arya Tschand Arun Tejusve Raghunath Rajan S. Idgunji Anirban Ghosh J. Holleman ... Rowan Taubitz Sean Zhan Scott Wasson David Kanter Vijay Janapa Reddi 122 3 0 15 Oct 2024
Measuring the Carbon Intensity of AI in Cloud Instances Jesse Dodge Taylor Prewitt Rémi Tachet des Combes Erika Odmark Roy Schwartz Emma Strubell A. Luccioni Noah A. Smith Nicole DeCario Will Buchanan 76 193 0 10 Jun 2022
MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflicts Weixin Liang James Zou OOD 91 84 0 14 Feb 2022
Treehouse: A Case For Carbon-Aware Datacenter Software Thomas Anderson Adam Belay Mosharaf Chowdhury Asaf Cidon Irene Zhang 90 63 0 06 Jan 2022
Sustainable AI: Environmental Implications, Challenges and Opportunities Carole-Jean Wu Ramya Raghavendra Udit Gupta Bilge Acun Newsha Ardalani ... Maximilian Balandat Joe Spisak R. Jain Michael G. Rabbat K. Hazelwood 140 414 0 30 Oct 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 341 684 0 21 Apr 2021
GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training Krishnateja Killamsetty D. Sivasubramanian Ganesh Ramakrishnan A. De Rishabh K. Iyer OOD 151 207 0 27 Feb 2021
Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead Maurizio Capra Beatrice Bussolino Alberto Marchisio Guido Masera Maurizio Martina Mohamed Bennai BDL 115 145 0 21 Dec 2020
Oort: Efficient Federated Learning via Guided Participant Selection Fan Lai Xiangfeng Zhu H. Madhyastha Mosharaf Chowdhury FedML OODD 126 275 0 12 Oct 2020
Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning Aurick Qiao Sang Keun Choe Suhas Jayaram Subramanya Willie Neiswanger Qirong Ho Hao Zhang G. Ganger Eric Xing VLM 74 181 0 27 Aug 2020
Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models Lasse F. Wolff Anthony Benjamin Kanding Raghavendra Selvan HAI 69 315 0 06 Jul 2020
Ansor: Generating High-Performance Tensor Programs for Deep Learning Lianmin Zheng Chengfan Jia Minmin Sun Zhao Wu Cody Hao Yu ... Jun Yang Danyang Zhuo Koushik Sen Joseph E. Gonzalez Ion Stoica 144 403 0 11 Jun 2020
Learning under Concept Drift: A Review Jie Lu Anjin Liu Fan Dong Feng Gu João Gama Guangquan Zhang AI4TS 67 1,292 0 13 Apr 2020
Verified Instruction-Level Energy Consumption Measurement for NVIDIA GPUs Yehia Arafa Ammar Elwazir Abdelrahman Elkanishy Youssef Aly Ayatelrahman Elsayed Abdel-Hameed A. Badawy Gopinath Chennupati S. Eidenbenz N. Santhi 56 40 0 18 Feb 2020
Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning Peter Henderson Jie Hu Joshua Romoff Emma Brunskill Dan Jurafsky Joelle Pineau 97 457 0 31 Jan 2020
ALERT: Accurate Learning for Energy and Timeliness Chengcheng Wan M. Santriaji E. Rogers H. Hoffmann Michael Maire Shan Lu AI4CE 85 42 0 31 Oct 2019
Quantifying the Carbon Emissions of Machine Learning Alexandre Lacoste A. Luccioni Victor Schmidt Thomas Dandres 122 712 0 21 Oct 2019
Blink: Fast and Generic Collectives for Distributed ML Guanhua Wang Shivaram Venkataraman Amar Phanishayee J. Thelin Nikhil R. Devanur Ion Stoica VLM 61 140 0 11 Oct 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 375 6,472 0 26 Sep 2019
The Architectural Implications of Facebook's DNN-based Personalized Recommendation Udit Gupta Carole-Jean Wu Xiaodong Wang Maxim Naumov Brandon Reagen ... Andrey Malevich Dheevatsa Mudigere M. Smelyanskiy Liang Xiong Xuan Zhang GNN 91 292 0 06 Jun 2019
An Empirical Model of Large-Batch Training Sam McCandlish Jared Kaplan Dario Amodei OpenAI Dota Team 76 280 0 14 Dec 2018
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent Noah Golmant N. Vemuri Z. Yao Vladimir Feinberg A. Gholami Kai Rothauge Michael W. Mahoney Joseph E. Gonzalez 86 73 0 30 Nov 2018
A System for Massively Parallel Hyperparameter Tuning Liam Li Kevin Jamieson Afshin Rostamizadeh Ekaterina Gonina Moritz Hardt Benjamin Recht Ameet Talwalkar 79 387 0 13 Oct 2018
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design Ningning Ma Xiangyu Zhang Haitao Zheng Jian Sun 185 5,019 0 30 Jul 2018
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark Cody Coleman Daniel Kang Deepak Narayanan Luigi Nardi Tian Zhao Jian Zhang Peter Bailis K. Olukotun Christopher Ré Matei A. Zaharia 60 117 0 04 Jun 2018
Don't Decay the Learning Rate, Increase the Batch Size Samuel L. Smith Pieter-Jan Kindermans Chris Ying Quoc V. Le ODL 124 996 0 01 Nov 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 130 3,688 0 08 Jun 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 444 2,947 0 15 Sep 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 318 8,177 0 16 Jun 2016
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization Lisha Li Kevin Jamieson Giulia DeSalvo Afshin Rostamizadeh Ameet Talwalkar 246 2,336 0 21 Mar 2016
Deep Speech: Scaling up end-to-end speech recognition Awni Y. Hannun Carl Case Jared Casper Bryan Catanzaro G. Diamos ... R. Prenger S. Satheesh Shubho Sengupta Adam Coates A. Ng 195 2,128 0 17 Dec 2014
ADADELTA: An Adaptive Learning Rate Method Matthew D. Zeiler ODL 165 6,635 0 22 Dec 2012