Understanding and Overcoming the Challenges of Efficient Transformer Quantization

27 September 2021

Papers citing "Understanding and Overcoming the Challenges of Efficient Transformer Quantization"

12 / 62 papers shown

Title
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,154 0 20 Apr 2018
Training and Inference with Integers in Deep Neural Networks Shuang Wu Guoqi Li F. Chen Luping Shi MQ 60 391 0 13 Feb 2018
Alternating Multi-bit Quantization for Recurrent Neural Networks Chen Xu Jianqiang Yao Zhouchen Lin Wenwu Ou Yuanbin Cao Zhirong Wang H. Zha MQ 69 116 0 01 Feb 2018
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob S. Kligys Bo Chen Menglong Zhu Matthew Tang Andrew G. Howard Hartwig Adam Dmitry Kalenichenko MQ 150 3,123 0 15 Dec 2017
Adaptive Quantization for Deep Neural Network Yiren Zhou Seyed-Mohsen Moosavi-Dezfooli Ngai-Man Cheung P. Frossard MQ 62 183 0 04 Dec 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 687 131,526 0 12 Jun 2017
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations Itay Hubara Matthieu Courbariaux Daniel Soudry Ran El-Yaniv Yoshua Bengio MQ 149 1,864 0 22 Sep 2016
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients Shuchang Zhou Yuxin Wu Zekun Ni Xinyu Zhou He Wen Yuheng Zou MQ 119 2,086 0 20 Jun 2016
Fixed Point Quantization of Deep Convolutional Networks D. Lin S. Talathi V. Annapureddy MQ 92 815 0 19 Nov 2015
Deep Learning with Limited Numerical Precision Suyog Gupta A. Agrawal K. Gopalakrishnan P. Narayanan HAI 199 2,047 0 09 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.8K 150,039 0 22 Dec 2014
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation Yoshua Bengio Nicholas Léonard Aaron Courville 379 3,129 0 15 Aug 2013