Understanding Training Efficiency of Deep Learning Recommendation Models at Scale

11 November 2020

Papers citing "Understanding Training Efficiency of Deep Learning Recommendation Models at Scale"

34 / 34 papers shown

Title
ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Models Yujeong Choi Jiin Kim Minsoo Rhu 76 2 0 11 Jun 2024
Saturn: An Optimized Data System for Large Model Deep Learning Workloads Kabir Nagrecha Arun Kumar 110 6 0 03 Sep 2023
MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation Samuel Hsia Udit Gupta Bilge Acun Newsha Ardalani Pan Zhong Gu-Yeon Wei David Brooks Carole-Jean Wu 108 17 0 21 Feb 2023
FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models Geet Sethi Pallab Bhattacharya Dhruv Choudhary Carole-Jean Wu Christos Kozyrakis 75 5 0 08 Jan 2023
Systems for Parallel and Distributed Large-Model Deep Learning Training Kabir Nagrecha GNN VLM MoE 74 7 0 06 Jan 2023
A Survey on Federated Recommendation Systems Zehua Sun Yonghui Xu Yang Liu Weiliang He Lanju Kong Fangzhao Wu Yiheng Jiang Li-zhen Cui FedML 113 68 0 27 Dec 2022
Data Leakage via Access Patterns of Sparse Features in Deep Learning-based Recommendation Systems H. Hashemi Wenjie Xiong Liu Ke Kiwan Maeng M. Annavaram G. E. Suh Hsien-Hsin S. Lee 52 6 0 12 Dec 2022
COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training D. Kadiyala Saeed Rashidi Taekyung Heo Abhimanyu Bambhaniya T. Krishna Alexandros Daglis VLM 74 7 0 30 Nov 2022
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure Mark Zhao Dhruv Choudhary Devashish Tyagi A. Somani Max Kaplan ... Jongsoo Park Aarti Basant Niket Agarwal Carole-Jean Wu Christos Kozyrakis VLM 83 6 0 09 Nov 2022
DreamShard: Generalizable Embedding Table Placement for Recommender Systems Daochen Zha Louis Feng Qiaoyu Tan Zirui Liu Kwei-Herng Lai Bhargav Bhushanam Yuandong Tian A. Kejariwal Helen Zhou LMTD OffRL 89 28 0 05 Oct 2022
Understanding Scaling Laws for Recommendation Models Newsha Ardalani Carole-Jean Wu Zeliang Chen Bhargav Bhushanam Adnan Aziz 93 31 0 17 Aug 2022
AutoShard: Automated Embedding Table Sharding for Recommender Systems Daochen Zha Louis Feng Bhargav Bhushanam Dhruv Choudhary Jade Nie Yuandong Tian Jay Chae Yi-An Ma A. Kejariwal Helen Zhou 85 32 0 12 Aug 2022
FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learning Meisam Hejazinia Dzmitry Huba Ilias Leontiadis Kiwan Maeng Mani Malek Luca Melis Ilya Mironov Milad Nasr Kaikai Wang Carole-Jean Wu FedML 75 6 0 07 Jun 2022
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity Kiwan Maeng Haiyu Lu Luca Melis John Nguyen Michael G. Rabbat Carole-Jean Wu FedML 110 32 0 30 May 2022
GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Model Wenbo Su Yuanxing Zhang Yufeng Cai Kaixu Ren Pengjie Wang ... Jing Chen Hongbo Deng Jian Xu Lin Qu Bo Zheng 64 5 0 23 May 2022
Heterogeneous Acceleration Pipeline for Recommendation System Training Muhammad Adnan Yassaman Ebrahimzadeh Maboud Divyat Mahajan Prashant J. Nair 82 19 0 11 Apr 2022
ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications Yifan Yuan Jing-yu Huang Yan Sun Tianchen Wang Jacob Nelson Dan R. K. Ports Yipeng Wang Ren Wang Charlie Tai Nam Sung Kim 57 2 0 16 Mar 2022
GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture Zaid Qureshi Vikram Sharma Mailthody Isaac Gelado S. Min Amna Masood ... Dmitri Vainbrand I-Hsin Chung M. Garland W. Dally Wen-mei W. Hwu GNN 62 22 0 09 Mar 2022
BagPipe: Accelerating Deep Recommendation Model Training Saurabh Agarwal Chengpo Yan Ziyi Zhang Shivaram Venkataraman 93 20 0 24 Feb 2022
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 123 93 0 01 Feb 2022
RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi Bilge Acun Niket Agarwal Christos Kozyrakis Caroline Trippel Carole-Jean Wu 96 70 0 25 Jan 2022
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters Xiangru Lian Binhang Yuan Xuefeng Zhu Yulong Wang Yongjun He ... Lei Yuan Hai-bo Yu Sen Yang Ce Zhang Ji Liu VLM 99 36 0 10 Nov 2021
Modeling Techniques for Machine Learning Fairness: A Survey Mingyang Wan Daochen Zha Ninghao Liu Na Zou SyDa FaML 79 36 0 04 Nov 2021
Sustainable AI: Environmental Implications, Challenges and Opportunities Carole-Jean Wu Ramya Raghavendra Udit Gupta Bilge Acun Newsha Ardalani ... Maximilian Balandat Joe Spisak R. Jain Michael G. Rabbat K. Hazelwood 148 415 0 30 Oct 2021
Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training Mark Zhao Niket Agarwal Aarti Basant B. Gedik Satadru Pan ... Kevin Wilfong Harsha Rastogi Carole-Jean Wu Christos Kozyrakis Parikshit Pol GNN 84 76 0 20 Aug 2021
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning Young Geun Kim Carole-Jean Wu 97 87 0 16 Jul 2021
Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems Xiaocong Du Bhargav Bhushanam Jiecao Yu Dhruv Choudhary Tianxiang Gao Sherman Wong Louis Feng Jongsoo Park Yu Cao A. Kejariwal 71 7 0 04 May 2021
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models Dheevatsa Mudigere Y. Hao Jianyu Huang Zhihao Jia Andrew Tulloch ... Ajit Mathews Lin Qiao M. Smelyanskiy Bill Jia Vijay Rao 111 155 0 12 Apr 2021
ECRM: Efficient Fault Tolerance for Recommendation Model Training via Erasure Coding Kaige Liu J. Kosaian K. V. Rashmi 55 4 0 05 Apr 2021
Accelerating Recommendation System Training by Leveraging Popular Choices Muhammad Adnan Yassaman Ebrahimzadeh Maboud Divyat Mahajan Prashant J. Nair 86 60 0 01 Mar 2021
TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models Chunxing Yin Bilge Acun Xing Liu Carole-Jean Wu 99 106 0 25 Jan 2021
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery Kiwan Maeng Shivam Bharuka Isabel Gao M. C. Jeffrey V. Saraph ... Caroline Trippel Jiyan Yang Michael G. Rabbat Brandon Lucia Carole-Jean Wu OffRL 82 33 0 05 Nov 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 110 85 0 02 Jul 2020
Billion-scale similarity search with GPUs Jeff Johnson Matthijs Douze Hervé Jégou 371 3,749 0 28 Feb 2017