Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle

5 February 2023

Papers citing "Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle"

50 / 56 papers shown

Title
ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Review Sunder Ali Khowaja P. Khuwaja Kapal Dev Weizheng Wang Lewis Nkenyereye 71 82 0 13 Apr 2023
Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AI S. Budennyy V. Lazarev N. Zakharenko A. Korovin Olga Plosskaya ... Ivan Oseledets I. Barsola Ilya M. Egorov A. Kosterina L. Zhukov 70 103 0 31 Jul 2022
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications Han Cai Ji Lin Chengyue Wu Zhijian Liu Haotian Tang Hanrui Wang Ligeng Zhu Song Han 82 112 0 25 Apr 2022
A Survey on Green Deep Learning Jingjing Xu Wangchunshu Zhou Zhiyi Fu Hao Zhou Lei Li VLM 145 83 0 08 Nov 2021
Sustainable AI: Environmental Implications, Challenges and Opportunities Carole-Jean Wu Ramya Raghavendra Udit Gupta Bilge Acun Newsha Ardalani ... Maximilian Balandat Joe Spisak R. Jain Michael G. Rabbat K. Hazelwood 107 405 0 30 Oct 2021
Full-Cycle Energy Consumption Benchmark for Low-Carbon Computer Vision Yue Liu Xinyang Jiang Donglin Bai Yuge Zhang Ningxin Zheng Xuanyi Dong Lu Liu Yuqing Yang Dongsheng Li 36 10 0 30 Aug 2021
Energy Consumption of Deep Generative Audio Models Constance Douwes P. Esling Jean-Pierre Briot MedIm 37 13 0 06 Jul 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 333 671 0 21 Apr 2021
Lessons on Parameter Sharing across Layers in Transformers Sho Takase Shun Kiyono 53 87 0 13 Apr 2021
EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models Omar Shaikh Jon Saad-Falcon Austin P. Wright Nilaksh Das Scott Freitas O. Asensio Duen Horng Chau 44 19 0 30 Mar 2021
M6: A Chinese Multimodal Pretrainer Junyang Lin Rui Men An Yang Chan Zhou Ming Ding ... Yong Li Wei Lin Jingren Zhou J. Tang Hongxia Yang VLM MoE 78 134 0 01 Mar 2021
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective Wuyang Chen Xinyu Gong Zhangyang Wang OOD 103 238 0 23 Feb 2021
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup Cheng Yang Shengnan Wang Chao Yang Yuechuan Li Ru He Jingqiao Zhang 75 25 0 27 Nov 2020
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 87 121 0 25 Nov 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 181 1,585 0 30 Sep 2020
A Survey of Deep Active Learning Pengzhen Ren Yun Xiao Xiaojun Chang Po-Yao (Bernie) Huang Zhihui Li Brij B. Gupta Xiaojiang Chen Xin Wang 99 1,140 0 30 Aug 2020
Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models Lasse F. Wolff Anthony Benjamin Kanding Raghavendra Selvan HAI 59 313 0 06 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 92 1,165 0 30 Jun 2020
A Survey on Transfer Learning in Natural Language Processing Zaid Alyafeai Maged S. Alshaibani Irfan Ahmad 72 75 0 31 May 2020
Training with Quantization Noise for Extreme Model Compression Angela Fan Pierre Stock Benjamin Graham Edouard Grave Remi Gribonval Hervé Jégou Armand Joulin MQ 90 245 0 15 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 356 1,487 0 18 Mar 2020
Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning Peter Henderson Jie Hu Joshua Romoff Emma Brunskill Dan Jurafsky Joelle Pineau 86 453 0 31 Jan 2020
A Comprehensive Survey on Transfer Learning Fuzhen Zhuang Zhiyuan Qi Keyu Duan Dongbo Xi Yongchun Zhu Hengshu Zhu Hui Xiong Qing He 181 4,449 0 07 Nov 2019
Quantifying the Carbon Emissions of Machine Learning Alexandre Lacoste A. Luccioni Victor Schmidt Thomas Dandres 94 707 0 21 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 229 3,490 0 30 Sep 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 366 6,455 0 26 Sep 2019
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 117 593 0 25 Sep 2019
Understanding and Robustifying Differentiable Architecture Search Arber Zela T. Elsken Tonmoy Saikia Yassine Marrakchi Thomas Brox Frank Hutter OOD AAML 132 372 0 20 Sep 2019
Neural Machine Translation with Byte-Level Subwords Changhan Wang Kyunghyun Cho Jiatao Gu 79 178 0 07 Sep 2019
Adaptively Sparse Transformers Gonçalo M. Correia Vlad Niculae André F. T. Martins 84 256 0 30 Aug 2019
Once-for-All: Train One Network and Specialize it for Efficient Deployment Han Cai Chuang Gan Tianzhe Wang Zhekai Zhang Song Han OOD 102 1,281 0 26 Aug 2019
Green AI Roy Schwartz Jesse Dodge Noah A. Smith Oren Etzioni 107 1,144 0 22 Jul 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 122 1,899 0 23 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 241 3,728 0 09 Jan 2019
Slimmable Neural Networks Jiahui Yu L. Yang N. Xu Jianchao Yang Thomas Huang 82 553 0 21 Dec 2018
pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference Mandar Joshi Eunsol Choi Omer Levy Daniel S. Weld Luke Zettlemoyer CoGe 57 47 0 20 Oct 2018
SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation Xinyi Wang Hieu H. Pham Zihang Dai Graham Neubig 64 197 0 22 Aug 2018
Neural Architecture Optimization Renqian Luo Fei Tian Tao Qin Enhong Chen Tie-Yan Liu 3DV 77 654 0 22 Aug 2018
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking Patrick H. Chen Si Si Yang Li Ciprian Chelba Cho-Jui Hsieh 55 69 0 18 Jun 2018
Group Normalization Yuxin Wu Kaiming He 228 3,660 0 22 Mar 2018
FD-MobileNet: Improved MobileNet with a Fast Downsampling Strategy Zheng Qin Zhaoning Zhang Xiaotao Chen Yuxing Peng MQ 57 104 0 11 Feb 2018
Regularized Evolution for Image Classifier Architecture Search Esteban Real A. Aggarwal Yanping Huang Quoc V. Le 160 3,031 0 05 Feb 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 181 19,284 0 13 Jan 2018
SkipNet: Learning Dynamic Routing in Convolutional Networks Xin Wang Feng Yu Zi-Yi Dou Trevor Darrell Joseph E. Gonzalez 101 636 0 26 Nov 2017
BlockDrop: Dynamic Inference Paths in Residual Networks Zuxuan Wu Tushar Nagarajan Abhishek Kumar Steven J. Rennie L. Davis Kristen Grauman Rogerio Feris 92 466 0 22 Nov 2017
Memory-Efficient Implementation of DenseNets Geoff Pleiss Danlu Chen Gao Huang Tongcheng Li Laurens van der Maaten Kilian Q. Weinberger 68 159 0 21 Jul 2017
A Survey of Neuromorphic Computing and Neural Networks in Hardware Catherine D. Schuman T. Potok Robert M. Patton J. Birdwell Mark E. Dean Garrett S. Rose J. Plank 105 696 0 19 May 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.1K 20,837 0 17 Apr 2017
In-Datacenter Performance Analysis of a Tensor Processing Unit N. Jouppi C. Young Nishant Patil David Patterson Gaurav Agrawal ... Vijay Vasudevan Richard Walter Walter Wang Eric Wilcox Doe Hyun Yoon 235 4,635 0 16 Apr 2017
Xception: Deep Learning with Depthwise Separable Convolutions François Chollet MDE BDL PINN 1.4K 14,559 0 07 Oct 2016