Title
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 212 3 0 20 Nov 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Yunfan LU Kurt Keutzer Jianfei Chen Song Han MQ 75 9 0 25 Oct 2024
CompAct: Compressed Activations for Memory-Efficient LLM Training Yara Shamshoum Nitzan Hodos Yuval Sieradzki Assaf Schuster MQ VLM 52 0 0 20 Oct 2024
Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation Yuchen Yang Yingdong Shi Cheems Wang Xiantong Zhen Yuxuan Shi Jun Xu 40 1 0 24 Jun 2024
Face processing emerges from object-trained convolutional neural networks Zhenhua Zhao Ji Chen Zhicheng Lin Haojiang Ying OOD CVBM 67 0 0 29 May 2024
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation Sunghyeon Woo Baeseong Park Byeongwook Kim Minjung Jo S. Kwon Dongsuk Jeon Dongsoo Lee 65 2 0 27 Feb 2024
An Evaluation of Memory Optimization Methods for Training Neural Networks Xiaoxuan Liu Siddharth Jha Alvin Cheung 29 0 0 26 Mar 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training Joya Chen Kai Xu Yuhui Wang Yifei Cheng Angela Yao 19 7 0 28 Feb 2022
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 416 0 18 Jan 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018