The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink

11 April 2022

Papers citing "The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink"

50 / 114 papers shown

Title
Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference Bartosz Wójcik Alessio Devoto Karol Pustelnik Pasquale Minervini Simone Scardapane 86 6 0 15 Dec 2023
Part-time Power Measurements: nvidia-smi's Lack of Attention Zeyu Yang Karel Adámek Wesley Armour 33 5 0 05 Dec 2023
Power Hungry Processing: Watts Driving the Cost of AI Deployment? Sasha Luccioni Yacine Jernite Emma Strubell 95 196 0 28 Nov 2023
Energy and Carbon Considerations of Fine-Tuning BERT Xiaorong Wang Clara Na Emma Strubell Sorelle A. Friedler Sasha Luccioni 84 13 0 17 Nov 2023
AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems Sasu Tarkoma Roberto Morabito Jaakko Sauvola 111 20 0 10 Nov 2023
MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems Samuel Hsia Alicia Golden Bilge Acun Newsha Ardalani Zach DeVito Gu-Yeon Wei David Brooks Carole-Jean Wu MoE 122 9 0 04 Oct 2023
Benchmarking and In-depth Performance Study of Large Language Models on Habana Gaudi Processors Chengming Zhang Baixi Sun Xiaodong Yu Zhen Xie Weijian Zheng K. Iskra Pete Beckman Dingwen Tao 50 5 0 29 Sep 2023
Carbon Containers: A System-level Facility for Managing Application-level Carbon Emissions John Thiede Noman Bashir David Irwin Prashant J. Shenoy 57 8 0 25 Sep 2023
LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Models Ahmad Faiz S. Kaneda Ruhan Wang Rita Osi Parteek Sharma Fan Chen Lei Jiang 82 65 0 25 Sep 2023
Large language models can accurately predict searcher preferences Paul Thomas S. Spielman Nick Craswell Bhaskar Mitra ALM LRM 104 154 0 19 Sep 2023
Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI Dustin Wright Christian Igel Gabrielle Samuel Raghavendra Selvan 120 16 0 05 Sep 2023
Energy Concerns with HPC Systems and Applications Roblex Nana C. Tadonki Petr Dokladal Youssef Mesri 55 0 0 31 Aug 2023
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition Samuel Horváth Stefanos Laskaridis Shashank Rajput Hongyi Wang BDL 76 4 0 28 Aug 2023
POLCA: Power Oversubscription in LLM Cloud Providers Pratyush Patel Esha Choukse Chaojie Zhang Íñigo Goiri Brijesh Warrier Nithish Mahalingam Ricardo Bianchini 48 14 0 24 Aug 2023
Batching for Green AI -- An Exploratory Study on Inference Tim Yarally Luís Cruz Daniel Feitosa June Sallou A. V. Deursen 48 5 0 21 Jul 2023
TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge Young D. Kwon Rui Li Stylianos I. Venieris Jagmohan Chauhan Nicholas D. Lane Cecilia Mascolo 65 9 0 19 Jul 2023
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation Hao Peng Qingqing Cao Jesse Dodge Matthew E. Peters Jared Fernandez ... Darrell Plessas Iz Beltagy Evan Pete Walsh Noah A. Smith Hannaneh Hajishirzi 81 7 0 19 Jul 2023
LogPrécis: Unleashing Language Models for Automated Malicious Log Analysis Matteo Boffa R. Valentim L. Vassio Danilo Giordano Idilio Drago Marco Mellia Zied Ben-Houidi 54 9 0 17 Jul 2023
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models Jean Kaddour Oscar Key Piotr Nawrot Pasquale Minervini Matt J. Kusner 110 45 0 12 Jul 2023
A Comparative Study of Machine Learning Algorithms for Anomaly Detection in Industrial Environments: Performance and Environmental Impact Álvaro Huertas-García Carlos Martí-González Rubén García Maezo Alejandro Echeverría Rey 64 3 0 01 Jul 2023
The War of the Efficiencies: Understanding the Tension between Carbon and Energy Optimization Walid A. Hanafy Roozbeh Bostandoost Noman Bashir David Irwin Mohammad Hajiesmaili Prashant J. Shenoy 46 16 0 29 Jun 2023
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research Ji-Ung Lee Haritz Puerto Betty van Aken Yuki Arase Jessica Zosa Forde ... Andreas Rucklé Iryna Gurevych Roy Schwartz Emma Strubell Jesse Dodge 120 6 0 29 Jun 2023
Broadening the perspective for sustainable AI: Comprehensive sustainability criteria and indicators for AI systems Friederike Rohde Josephin Wagner A. Meyer Philipp Reinhard Marcus Voss U. Petschow Anne Mollen 59 3 0 22 Jun 2023
Towards Environmentally Equitable AI via Geographical Load Balancing Pengfei Li Jianyi Yang Adam Wierman Shaolei Ren 86 12 0 20 Jun 2023
[Experiments & Analysis] Evaluating the Feasibility of Sampling-Based Techniques for Training Multilayer Perceptrons Sana Ebrahimi Rishi Advani Abolfazl Asudeh 121 0 0 15 Jun 2023
High-performance deep spiking neural networks with 0.3 spikes per neuron A. Stanojević Stanislaw Wo'zniak G. Bellec G. Cherubini A. Pantazi W. Gerstner 116 19 0 14 Jun 2023
FedZero: Leveraging Renewable Excess Energy in Federated Learning Philipp Wiesner R. Khalili Dennis Grinwald Pratik Agrawal L. Thamsen O. Kao 89 17 0 24 May 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 139 6 0 21 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 132 96 0 19 May 2023
Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study Joel Castaño Silverio Martínez-Fernández Xavier Franch Justus Bogner CVBM 98 37 0 18 May 2023
Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation Natalia Díaz Rodríguez Javier Del Ser Mark Coeckelbergh Marcos López de Prado E. Herrera-Viedma Francisco Herrera XAI 92 303 0 02 May 2023
Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service Baolin Li S. Samsi V. Gadepally Devesh Tiwari 69 32 0 19 Apr 2023
ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Review Sunder Ali Khowaja P. Khuwaja Kapal Dev Weizheng Wang Lewis Nkenyereye 130 88 0 13 Apr 2023
Towards Energy Consumption and Carbon Footprint Testing for AI-driven IoT Services Demetris Trihinas L. Thamsen Jossekin Beilharz Moysis Symeonides 54 13 0 13 Apr 2023
Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Models Pengfei Li Jianyi Yang M. A. Islam Shaolei Ren 205 132 0 06 Apr 2023
TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings N. Jouppi George Kurian Sheng Li Peter C. Ma R. Nagarajan ... Brian Towles C. Young Xiaoping Zhou Zongwei Zhou David A. Patterson BDL VLM 169 371 0 04 Apr 2023
Green Federated Learning Ashkan Yousefpour Sheng Guo Ashish Shenoy Sayan Ghosh Pierre Stock Kiwan Maeng Schalk-Willem Kruger Michael G. Rabbat Carole-Jean Wu Ilya Mironov FedML AI4CE 106 10 0 26 Mar 2023
Large AI Models in Health Informatics: Applications, Challenges, and the Future Jianing Qiu Lin Li Jiankai Sun Jiachuan Peng Peilun Shi ... Bo Xiao Wu Yuan Ningli Wang Dong Xu Benny Lo AI4MH LM&MA 114 140 0 21 Mar 2023
The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humans Bill Tomlinson Rebecca W. Black Donald J. Patterson A. Torrance 114 35 0 08 Mar 2023
Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning A. Luccioni Alex Hernandez-Garcia 93 52 0 16 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 89 11 0 13 Feb 2023
Can Persistent Homology provide an efficient alternative for Evaluation of Knowledge Graph Completion Methods? Anson Bastos Kuldeep Singh Abhishek Nadgeri Johannes Hoffart Toyotaro Suzumura Manish Singh 90 3 0 30 Jan 2023
A Green(er) World for A.I Dan Zhao Nathan C. Frey Joseph McDonald Matthew Hubbell David Bestor Michael Jones Andrew Prout V. Gadepally S. Samsi 69 6 0 27 Jan 2023
A Systematic Review of Green AI Roberto Verdecchia June Sallou Luís Cruz 104 166 0 26 Jan 2023
Hopf Physical Reservoir Computer for Reconfigurable Sound Recognition M. R. E. U. Shougat Xiaofu Li Siyao Shao K. McGarvey E. Perkins 53 12 0 20 Dec 2022
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure Mark Zhao Dhruv Choudhary Devashish Tyagi A. Somani Max Kaplan ... Jongsoo Park Aarti Basant Niket Agarwal Carole-Jean Wu Christos Kozyrakis VLM 94 6 0 09 Nov 2022
Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model A. Luccioni S. Viguier Anne-Laure Ligozat 112 293 0 03 Nov 2022
Language Models Understand Us, Poorly Jared Moore LRM 55 4 0 19 Oct 2022
The Future of Consumer Edge-AI Computing Stefanos Laskaridis Stylianos I. Venieris Alexandros Kouris Rui Li Nicholas D. Lane 111 10 0 19 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 107 51 0 13 Oct 2022