Machine Learning Model Sizes and the Parameter Gap

5 July 2022

Papers citing "Machine Learning Model Sizes and the Parameter Gap"

33 / 33 papers shown

Title
Neural Interactive Proofs Lewis Hammond Sam Adam-Day AAML 84 2 0 12 Dec 2024
Federated Communication-Efficient Multi-Objective Optimization Baris Askin Pranay Sharma Gauri Joshi Carlee Joe-Wong FedML 66 1 0 21 Oct 2024
FALCON: Pinpointing and Mitigating Stragglers for Large-Scale Hybrid-Parallel Training Tianyuan Wu Wei Wang Yinghao Yu Siran Yang Wenchao Wu Qinkai Duan Guodong Yang Jiamang Wang Lin Qu Liping Zhang 35 6 0 16 Oct 2024
Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting Chunlin Tian Li Li Kahou Tam Yebo Wu Chengzhong Xu FedML 29 1 0 12 Oct 2024
Quantized and Asynchronous Federated Learning Tomàs Ortega Hamid Jafarkhani FedML 31 0 0 30 Sep 2024
FreeRide: Harvesting Bubbles in Pipeline Parallelism Jiashu Zhang Zihan Pan Molly Xu Khuzaima S. Daudjee 90 0 0 11 Sep 2024
Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments Angie Boggust Venkatesh Sivaraman Yannick Assogba Donghao Ren Dominik Moritz Fred Hohman VLM 55 3 0 06 Aug 2024
ConvNLP: Image-based AI Text Detection S. Jambunathan Ashwath Shankarnarayan Parijat Dube DeLMO 44 0 0 09 Jul 2024
A Moonshot for AI Oracles in the Sciences Bryan Kaiser Tailin Wu Maike Sonnewald Colin Thackray Skylar Callis AI4CE 51 0 0 25 Jun 2024
Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach Xiaojing Chen Zhenyuan Li Wei Ni Xin Wang Shunqing Zhang Yanzan Sun Shugong Xu Qingqi Pei 30 2 0 21 Jun 2024
SlipStream: Adapting Pipelines for Distributed Training of Large DNNs Amid Failures Swapnil Gandhi Mark Zhao Athinagoras Skiadopoulos Christos Kozyrakis AI4CE GNN 43 8 0 22 May 2024
Automatic Generation of Model and Data Cards: A Step Towards Responsible AI Jiarui Liu Wenkai Li Zhijing Jin Mona T. Diab SyDa 60 3 0 10 May 2024
I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey Noah Lewis J. L. Bez Suren Byna 57 0 0 16 Apr 2024
Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference Fred Hohman Chaoqun Wang Jinmook Lee Jochen Görtler Dominik Moritz Jeffrey P. Bigham Zhile Ren Cecile Foret Qi Shan Xiaoyi Zhang 32 7 0 03 Apr 2024
Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention Ethan N. Evans Matthew G. Cook Zachary P. Bradshaw Margarite L. LaBorde 48 5 0 21 Mar 2024
Enhancing Instructional Quality: Leveraging Computer-Assisted Textual Analysis to Generate In-Depth Insights from Educational Artifacts Zewei Tian Min Sun Alex Liu Shawon Sarkar Jing Liu 40 5 0 06 Mar 2024
Mobile Fitting Room: On-device Virtual Try-on via Diffusion Models Justin Blalock David Munechika Harsha Karanth Alec Helbling Pratham Mehta Seongmin Lee Duen Horng Chau DiffM 27 0 0 02 Feb 2024
Combining Cloud and Mobile Computing for Machine Learning Ruiqi Xu Tianchi Zhang 34 1 0 20 Jan 2024
The complementary contributions of academia and industry to AI research Lizhen Liang Zhuang Han James Zou Daniel Ernesto Acuna 13 3 0 04 Jan 2024
Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices A. Menon Unnikrishnan Menon Kailash Ahirwar 21 1 0 03 Jan 2024
On the Burstiness of Distributed Machine Learning Traffic Natchanon Luangsomboon Fahimeh Fazel Jorg Liebeherr A. Sobhani Shichao Guan Xingjun Chu 30 1 0 30 Dec 2023
Pathway to a fully data-driven geotechnics: lessons from materials informatics Stephen Wu Yu Otake Yosuke Higo Ikumasa Yoshida AI4CE 21 4 0 01 Dec 2023
The Case for Universal Basic Computing Power Yue Zhu ELM 28 0 0 18 Nov 2023
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences Fred Hohman Mary Beth Kery Donghao Ren Dominik Moritz 24 16 0 06 Oct 2023
Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates Insu Jang Zhenning Yang Zhen Zhang Xin Jin Mosharaf Chowdhury MoE AI4CE OODD 20 44 0 15 Sep 2023
STen: Productive and Efficient Sparsity in PyTorch Andrei Ivanov Nikoli Dryden Tal Ben-Nun Saleh Ashkboos Torsten Hoefler 34 4 0 15 Apr 2023
Auditing large language models: a three-layered approach Jakob Mokander Jonas Schuett Hannah Rose Kirk Luciano Floridi AILaw MLAU 48 194 0 16 Feb 2023
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining Junyang Lin An Yang Jinze Bai Chang Zhou Le Jiang ... Jie Zhang Yong Li Wei Lin Jingren Zhou Hongxia Yang MoE 92 43 0 08 Oct 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 88 152 0 17 Sep 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 244 644 0 21 Apr 2021
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 262 656 0 23 Mar 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 255 4,489 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019