Optimizing Large Language Model Training Using FP4 Quantization

28 January 2025

Papers citing "Optimizing Large Language Model Training Using FP4 Quantization"

5 / 5 papers shown

Title
Quartet: Native FP4 Training Can Be Optimal for Large Language Models Roberto L. Castro Andrei Panferov Soroush Tabesh Oliver Sieberling Jiale Chen Mahdi Nikdan Saleh Ashkboos Dan Alistarh MQ 9 0 0 20 May 2025
Scaling Law for Quantization-Aware Training Mengzhao Chen Chaoyi Zhang Jing Liu Yutao Zeng Zeyue Xue ... Yunshui Li Jin Ma Jie Huang Xun Zhou Ping Luo MQ 22 0 0 20 May 2025
Gaussian Weight Sampling for Scalable, Efficient and Stable Pseudo-Quantization Training Myeonghwan Ahn Sungjoo Yoo MQ 27 0 0 16 May 2025
Alchemist: Towards the Design of Efficient Online Continual Learning System Yuyang Huang Yuhan Liu Haryadi S. Gunawi Beibin Li Changho Hwang CLL OnRL 112 0 0 03 Mar 2025
Kanana: Compute-efficient Bilingual Language Models Kanana LLM Team Yunju Bak Hojin Lee Minho Ryu Jiyeon Ham ... Daniel Lee Minchul Lee MinHyung Lee Shinbok Lee Gaeun Seo 98 1 0 26 Feb 2025