Compute Trends Across Three Eras of Machine Learning

11 February 2022

Papers citing "Compute Trends Across Three Eras of Machine Learning"

50 / 117 papers shown

Title
First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models Naomi Saphra Eve Fleisig Kyunghyun Cho Adam Lopez LRM 27 8 0 08 Nov 2023
Market Concentration Implications of Foundation Models Jai Vipra Anton Korinek ELM 37 16 0 02 Nov 2023
Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models? Advait Sarkar 26 16 0 01 Nov 2023
TRANSOM: An Efficient Fault-Tolerant System for Training LLMs Baodong Wu Lei Xia Qingping Li Kangyu Li Xu Chen Yongqiang Guo Tieyao Xiang Yuheng Chen Shigang Li 37 11 0 16 Oct 2023
Multinational AGI Consortium (MAGIC): A Proposal for International Coordination on AI Jason Hausenloy Andrea Miotti Claire Dennis 25 1 0 13 Oct 2023
Energy Estimates Across Layers of Computing: From Devices to Large-Scale Applications in Machine Learning for Natural Language Processing, Scientific Computing, and Cryptocurrency Mining Sadasivan Shankar 27 3 0 11 Oct 2023
Performance and energy balance: a comprehensive study of state-of-the-art sound event detection systems Francesca Ronchini Romain Serizel 27 10 0 05 Oct 2023
From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference S. Samsi Dan Zhao Joseph McDonald Baolin Li Adam Michaleas Michael Jones William Bergeron J. Kepner Devesh Tiwari V. Gadepally 21 120 0 04 Oct 2023
Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives Elizabeth Seger Noemi Dreksler Richard Moulange Emily Dardaman Jonas Schuett ... Emma Bluemke Michael Aird Patrick Levermore Julian Hazell Abhishek Gupta 20 40 0 29 Sep 2023
Activation Compression of Graph Neural Networks using Block-wise Quantization with Improved Variance Minimization Sebastian Eliassen Raghavendra Selvan GNN 27 3 0 21 Sep 2023
Breaking through the learning plateaus of in-context learning in Transformer Jingwen Fu Tao Yang Yuwang Wang Yan Lu Nanning Zheng 30 1 0 12 Sep 2023
Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI Dustin Wright Christian Igel Gabrielle Samuel Raghavendra Selvan 32 15 0 05 Sep 2023
International Governance of Civilian AI: A Jurisdictional Certification Approach Robert F. Trager Benjamin Harack Anka Reuel A. Carnegie Lennart Heim ... R. Lall Owen Larter Seán Ó hÉigeartaigh Simon Staffell José Jaime Villalobos 26 20 0 29 Aug 2023
Computer vision-enriched discrete choice models, with an application to residential location choice S. Cranenburgh Francisco Garrido-Valenzuela 22 2 0 16 Aug 2023
Spatially Varying Nanophotonic Neural Networks Kaixuan Wei Xiao Li Johannes E. Froech Praneeth Chakravarthula James E. M. Whitehead Ethan Tseng A. Majumdar Felix Heide 17 11 0 07 Aug 2023
DaphneSched: A Scheduler for Integrated Data Analysis Pipelines A. Eleliemy F. Ciorba AI4CE 11 0 0 03 Aug 2023
Frontier AI Regulation: Managing Emerging Risks to Public Safety Markus Anderljung Joslyn Barnhart Anton Korinek Jade Leung Cullen O'Keefe ... Jonas Schuett Yonadav Shavit Divya Siddarth Robert F. Trager Kevin J. Wolf SILM 44 118 0 06 Jul 2023
EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models Michael Wornow Rahul Thapa E. Steinberg Jason Alan Fries N. Shah VLM OOD AI4MH 23 36 0 05 Jul 2023
ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design Srivatsan Krishnan Amir Yazdanbaksh Shvetank Prakash Jason J. Jabbour Ikechukwu Uchendu ... Behzad Boroujerdian Daniel Richins Devashree Tripathy Aleksandra Faust Vijay Janapa Reddi 43 11 0 15 Jun 2023
Generative AI for Product Design: Getting the Right Design and the Design Right Matthew K. Hong Shabnam Hakimi Yan-Ying Chen Heishiro Toyoda Charlene C. Wu M. Klenk AI4CE 19 16 0 02 Jun 2023
The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only Guilherme Penedo Quentin Malartic Daniel Hesslow Ruxandra-Aimée Cojocaru Alessandro Cappelli Hamza Alobeidli B. Pannier Ebtesam Almazrouei Julien Launay 27 751 0 01 Jun 2023
MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Models Yu-Hsiang Wang Huan Chen Kai-Wei Chang Winston H. Hsu Hung-yi Lee 24 6 0 30 May 2023
Exploiting Large Neuroimaging Datasets to Create Connectome-Constrained Approaches for more Robust, Efficient, and Adaptable Artificial Intelligence Erik C. Johnson Brian S. Robinson Gautam K. Vallabha Justin Joyce Jordan K. Matelsky ... Matthew J. Roos I-J. Wang Brock Andrew Wester William R. Gray Roncal J. Hoffmann 37 1 0 26 May 2023
Spear Phishing With Large Language Models Julian Hazell 22 46 0 11 May 2023
Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery Debadutta Dash Rahul Thapa Juan M. Banda Akshay Swaminathan Morgan Cheatham ... Garret K. Morris H. Magon M. Lungren Eric Horvitz N. Shah ELM LM&MA AI4MH 68 51 0 26 Apr 2023
STen: Productive and Efficient Sparsity in PyTorch Andrei Ivanov Nikoli Dryden Tal Ben-Nun Saleh Ashkboos Torsten Hoefler 34 4 0 15 Apr 2023
NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems Jason Yik Korneel Van den Berghe Douwe den Blanken Younes Bouhadjar Maxime Fabre ... Fatima Tuz Zohora Charlotte Frenkel Vijay Janapa Reddi Charlotte Frenkel Vijay Janapa Reddi 25 17 0 10 Apr 2023
Eight Things to Know about Large Language Models Sam Bowman ALM 27 113 0 02 Apr 2023
Tetra-AML: Automatic Machine Learning via Tensor Networks A. Naumov Ar. Melnikov V. Abronin F. Oxanichenko K. Izmailov M. Pflitsch A. Melnikov M. Perelshtein 18 11 0 28 Mar 2023
Green Federated Learning Ashkan Yousefpour Sheng Guo Ashish Shenoy Sayan Ghosh Pierre Stock Kiwan Maeng Schalk-Willem Kruger Michael G. Rabbat Carole-Jean Wu Ilya Mironov FedML AI4CE 44 10 0 26 Mar 2023
What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring Yonadav Shavit 31 22 0 20 Mar 2023
Operating critical machine learning models in resource constrained regimes Raghavendra Selvan Julian Schon Erik Dam MedIm 31 8 0 17 Mar 2023
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference Haiyang Huang Newsha Ardalani Anna Y. Sun Liu Ke Hsien-Hsin S. Lee Anjali Sridhar Shruti Bhosale Carole-Jean Wu Benjamin C. Lee MoE 70 23 0 10 Mar 2023
Optical Transformers Maxwell G. Anderson Shifan Ma Tianyu Wang Logan G. Wright Peter L. McMahon 20 20 0 20 Feb 2023
How Generative AI models such as ChatGPT can be (Mis)Used in SPC Practice, Education, and Research? An Exploratory Study F. Megahed Ying-Ju Chen Joshua A. Ferris S. Knoth L. A. Jones‐Farmer 47 117 0 17 Feb 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 26 15 0 16 Feb 2023
Auditing large language models: a three-layered approach Jakob Mokander Jonas Schuett Hannah Rose Kirk Luciano Floridi AILaw MLAU 48 194 0 16 Feb 2023
Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators Malte J. Rasch C. Mackin Manuel Le Gallo An Chen A. Fasoli ... P. Narayanan H. Tsai G. Burr Abu Sebastian Vijay Narayanan 13 83 0 16 Feb 2023
A Green(er) World for A.I Dan Zhao Nathan C. Frey Joseph McDonald Matthew Hubbell David Bestor Michael Jones Andrew Prout V. Gadepally S. Samsi 32 6 0 27 Jan 2023
Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy Michael J. Smith James E. Geach 35 32 0 07 Nov 2022
Class Based Thresholding in Early Exit Semantic Segmentation Networks Alperen Görmez Erdem Koyuncu 23 5 0 27 Oct 2022
Will we run out of data? Limits of LLM scaling based on human-generated data Pablo Villalobos A. Ho J. Sevilla T. Besiroglu Lennart Heim Marius Hobbhahn ALM 33 111 0 26 Oct 2022
OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networks Benoit Steiner Mostafa Elhoushi Jacob Kahn James Hegarty 29 8 0 24 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 31 47 0 13 Oct 2022
EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Search Pedram Bakhtiarifard Christian Igel Raghavendra Selvan 35 6 0 12 Oct 2022
Scaling Laws for a Multi-Agent Reinforcement Learning Model Oren Neumann C. Gros 29 26 0 29 Sep 2022
Enabling Connectivity for Automated Mobility: A Novel MQTT-based Interface Evaluated in a 5G Case Study on Edge-Cloud Lidar Object Detection Lennart Reiher Bastian Lampe Timo Woopen Raphael van Kempen Till Beemelmanns L. Eckstein 10 8 0 08 Sep 2022
The Role Of Biology In Deep Learning Robert Bain 27 0 0 07 Sep 2022
Large Language Models and the Reverse Turing Test T. Sejnowski ELM 26 107 0 28 Jul 2022
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks Tilman Raukur A. Ho Stephen Casper Dylan Hadfield-Menell AAML AI4CE 23 124 0 27 Jul 2022