Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models

20 October 2023

Miaoxi Zhu

Qihuang Zhong

Li Shen

Liang Ding

Bo Du

Papers citing "Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models"

33 / 33 papers shown

Title
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE Qihuang Zhong Liang Ding Yibing Zhan Yu Qiao Yonggang Wen ... Yixin Chen Xinbo Gao Steven C. H. Hoi Xiaoou Tang Dacheng Tao VLM ELM 109 35 0 04 Dec 2022
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Z. Yao Reza Yazdani Aminabadi Minjia Zhang Xiaoxia Wu Conglong Li Yuxiong He VLM MQ 139 479 0 04 Jun 2022
It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher Kanghyun Choi Hye Yoon Lee Deokki Hong Joonsang Yu Noseong Park Youngsok Kim Jinho Lee MQ 74 33 0 31 Mar 2022
Compression of Generative Pre-trained Language Models via Quantization Chaofan Tao Lu Hou Wei Zhang Lifeng Shang Xin Jiang Qun Liu Ping Luo Ngai Wong MQ 74 104 0 21 Mar 2022
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity Junchi Yang Antonio Orvieto Aurelien Lucchi Niao He 98 64 0 10 Dec 2021
Sharpness-aware Quantization for Deep Neural Networks Jing Liu Jianfei Cai Bohan Zhuang MQ 106 25 0 24 Nov 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 183 190 0 13 Sep 2021
On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptation Ruidan He Linlin Liu Hai Ye Qingyu Tan Bosheng Ding Liying Cheng Jia-Wei Low Lidong Bing Luo Si 56 202 0 06 Jun 2021
Lessons on Parameter Sharing across Layers in Transformers Sho Takase Shun Kiyono 58 87 0 13 Apr 2021
Zero-shot Adversarial Quantization Yuang Liu Wei Zhang Jun Wang MQ 105 79 0 29 Mar 2021
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction Yuhang Li Ruihao Gong Xu Tan Yang Yang Peng Hu Qi Zhang F. Yu Wei Wang Shi Gu MQ 149 444 0 10 Feb 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 162 352 0 05 Jan 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 217 227 0 31 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 84 406 0 14 Dec 2020
Towards Accurate Quantization and Pruning via Data-free Knowledge Transfer Chen Zhu Zheng Xu Ali Shafahi Manli Shu Amin Ghiasi Tom Goldstein MQ 47 3 0 14 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 199 1,359 0 03 Oct 2020
Data-Free Network Quantization With Adversarial Knowledge Distillation Yoojin Choi Jihwan P. Choi Mostafa El-Khamy Jungwon Lee MQ 74 121 0 08 May 2020
Up or Down? Adaptive Rounding for Post-Training Quantization Markus Nagel Rana Ali Amjad M. V. Baalen Christos Louizos Tijmen Blankevoort MQ 95 588 0 22 Apr 2020
ZeroQ: A Novel Zero Shot Quantization Framework Yaohui Cai Z. Yao Zhen Dong A. Gholami Michael W. Mahoney Kurt Keutzer MQ 101 399 0 01 Jan 2020
Loss Aware Post-training Quantization Yury Nahshan Brian Chmiel Chaim Baskin Evgenii Zheltonozhskii Ron Banner A. Bronstein A. Mendelson MQ 93 166 0 17 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 490 20,342 0 23 Oct 2019
Fully Quantized Transformer for Machine Translation Gabriele Prato Ella Charlaix Mehdi Rezagholizadeh MQ 59 70 0 17 Oct 2019
Green AI Roy Schwartz Jesse Dodge Noah A. Smith Oren Etzioni 121 1,149 0 22 Jul 2019
Data-Free Quantization Through Weight Equalization and Bias Correction Markus Nagel M. V. Baalen Tijmen Blankevoort Max Welling MQ 75 515 0 11 Jun 2019
On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems Tianyi Lin Chi Jin Michael I. Jordan 129 508 0 02 Jun 2019
Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding Xiaodong Liu Pengcheng He Weizhu Chen Jianfeng Gao FedML 62 182 0 20 Apr 2019
Improving Neural Network Quantization without Retraining using Outlier Channel Splitting Ritchie Zhao Yuwei Hu Jordan Dotzel Christopher De Sa Zhiru Zhang OODD MQ 113 311 0 28 Jan 2019
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 244 1,413 0 31 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,201 0 20 Apr 2018
SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation Daniel Cer Mona T. Diab Eneko Agirre I. Lopez-Gazpio Lucia Specia 445 1,891 0 31 Jul 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,497 0 18 Apr 2017
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 343 2,900 0 26 Sep 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 316 8,174 0 16 Jun 2016