COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training

25 October 2024

Papers citing "COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training"

5 / 5 papers shown

Title
Oscillation-Reduced MXFP4 Training for Vision Transformers Yuxiang Chen Haocheng Xi Jun Zhu Jianfei Chen MQ 54 1 0 28 Feb 2025
Stable-SPAM: How to Train in 4-Bit More Stably than 16-Bit Adam Tianjin Huang Haotian Hu Zhenyu (Allen) Zhang Gaojie Jin X. Li ... Tianlong Chen Lu Liu Qingsong Wen Zhangyang Wang Shiwei Liu MQ 37 0 0 24 Feb 2025
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache Rishabh Tiwari Haocheng Xi Aditya Tomar Coleman Hooper Sehoon Kim Maxwell Horton Mahyar Najibi Michael W. Mahoney K. K. Amir Gholami MQ 56 1 0 05 Feb 2025
CompAct: Compressed Activations for Memory-Efficient LLM Training Yara Shamshoum Nitzan Hodos Yuval Sieradzki Assaf Schuster MQ VLM 42 0 0 20 Oct 2024
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration Jintao Zhang Jia wei Pengle Zhang Jun-Jie Zhu Jun Zhu Jianfei Chen VLM MQ 82 18 0 03 Oct 2024