Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language
Models

v1v2v3 (latest)

Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models

14 December 2023

Alireza Ghaffari

Mahsa Ghazvini Nejad

Vahid Partovi Nia

ArXiv (abs)PDF HTML

Papers citing "Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models"

8 / 8 papers shown

Title
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin Jiaming Tang Haotian Tang Shang Yang Wei-Ming Chen Wei-Chen Wang Guangxuan Xiao Xingyu Dang Chuang Gan Song Han EDL MQ 101 576 0 01 Jun 2023
RPTQ: Reorder-based Post-training Quantization for Large Language Models Zhihang Yuan Lin Niu Jia-Wen Liu Wenyu Liu Xinggang Wang Yuzhang Shang Guangyu Sun Qiang Wu Jiaxiang Wu Bingzhe Wu MQ 98 87 0 03 Apr 2023
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models Xiuying Wei Yunchen Zhang Xiangguo Zhang Ruihao Gong Shanghang Zhang Qi Zhang F. Yu Xianglong Liu MQ 96 152 0 27 Sep 2022
Is Integer Arithmetic Enough for Deep Learning Training? Alireza Ghaffari Marzieh S. Tahaei Mohammadreza Tayaranian M. Asgharian V. Nia MQ 25 16 0 18 Jul 2022
Towards Unified INT8 Training for Convolutional Neural Network Feng Zhu Ruihao Gong F. Yu Xianglong Liu Yanfei Wang Zhelong Li Xiuqi Yang Junjie Yan MQ 88 152 0 29 Dec 2019
Improving Neural Network Quantization without Retraining using Outlier Channel Splitting Ritchie Zhao Yuwei Hu Jordan Dotzel Christopher De Sa Zhiru Zhang OODD MQ 101 311 0 28 Jan 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,169 0 16 Jun 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.