Towards Unified INT8 Training for Convolutional Neural Network

29 December 2019

Xianglong Liu

Papers citing "Towards Unified INT8 Training for Convolutional Neural Network"

46 / 46 papers shown

Title
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang MQ ALM 169 0 0 18 Feb 2025
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Yaojie Lu Kurt Keutzer Jianfei Chen Song Han MQ 129 10 0 25 Oct 2024
Hierarchical Feature Embedding for Attribute Recognition Jie Yang Jiarou Fan Yiru Wang Yige Wang Weihao Gan Lin Liu Wei Wu CVBM 169 43 0 23 May 2020
Quantization Networks Jiwei Yang Xu Shen Jun Xing Xinmei Tian Houqiang Li Bing Deng Jianqiang Huang Xiansheng Hua MQ 79 350 0 21 Nov 2019
Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers Yukuan Yang Shuang Wu Lei Deng Tianyi Yan Yuan Xie Guoqi Li MQ 145 112 0 05 Sep 2019
Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks Ruihao Gong Xianglong Liu Shenghu Jiang Tian-Hao Li Peng Hu Jiazhen Lin F. Yu Junjie Yan MQ 70 459 0 14 Aug 2019
On the Convergence of Adam and Beyond Sashank J. Reddi Satyen Kale Surinder Kumar 106 2,505 0 19 Apr 2019
Regularizing Activation Distribution for Training Binarized Deep Networks Ruizhou Ding Ting-Wu Chin Z. Liu Diana Marculescu MQ 65 149 0 04 Apr 2019
Adaptive Gradient Methods with Dynamic Bound of Learning Rate Liangchen Luo Yuanhao Xiong Yan Liu Xu Sun ODL 80 602 0 26 Feb 2019
Learned Step Size Quantization S. K. Esser J. McKinstry Deepika Bablani R. Appuswamy D. Modha MQ 75 809 0 21 Feb 2019
Dynamic Curriculum Learning for Imbalanced Data Classification Yiru Wang Weihao Gan Jie Yang Wei Wu Junjie Yan 78 222 0 21 Jan 2019
Per-Tensor Fixed-Point Quantization of the Back-Propagation Algorithm Charbel Sakr Naresh R Shanbhag MQ 69 43 0 31 Dec 2018
Training Deep Neural Networks with 8-bit Floating Point Numbers Naigang Wang Jungwook Choi D. Brand Chia-Yu Chen K. Gopalakrishnan MQ 65 503 0 19 Dec 2018
HAQ: Hardware-Aware Automated Quantization with Mixed Precision Kuan-Chieh Wang Zhijian Liu Chengyue Wu Ji Lin Song Han MQ 129 884 0 21 Nov 2018
Post-training 4-bit quantization of convolution networks for rapid-deployment Ron Banner Yury Nahshan Elad Hoffer Daniel Soudry MQ 62 94 0 02 Oct 2018
Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation Zhezhi He Deliang Fan MQ 53 67 0 02 Oct 2018
Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference J. McKinstry S. K. Esser R. Appuswamy Deepika Bablani John V. Arthur Izzet B. Yildiz D. Modha MQ 54 94 0 11 Sep 2018
Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss S. Jung Changyong Son Seohyung Lee JinWoo Son Youngjun Kwak Jae-Joon Han Sung Ju Hwang Changkyu Choi MQ 48 375 0 17 Aug 2018
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks Penghang Yin Shuai Zhang J. Lyu Stanley Osher Y. Qi Jack Xin MQ 79 62 0 15 Aug 2018
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks Dongqing Zhang Jiaolong Yang Dongqiangzi Ye G. Hua MQ 63 703 0 26 Jul 2018
Quantizing deep convolutional networks for efficient inference: A whitepaper Raghuraman Krishnamoorthi MQ 141 1,021 0 21 Jun 2018
Scalable Methods for 8-bit Training of Neural Networks Ron Banner Itay Hubara Elad Hoffer Daniel Soudry MQ 84 339 0 25 May 2018
PACT: Parameterized Clipping Activation for Quantized Neural Networks Jungwook Choi Zhuo Wang Swagath Venkataramani P. Chuang Vijayalakshmi Srinivasan K. Gopalakrishnan MQ 68 955 0 16 May 2018
Loss-aware Weight Quantization of Deep Networks Lu Hou James T. Kwok MQ 85 127 0 23 Feb 2018
Training and Inference with Integers in Deep Neural Networks Shuang Wu Guoqi Li F. Chen Luping Shi MQ 65 391 0 13 Feb 2018
Mixed Precision Training of Convolutional Neural Networks using Integer Operations Dipankar Das Naveen Mellempudi Dheevatsa Mudigere Dhiraj D. Kalamkar Sasikanth Avancha ... J. Corbal N. Shustrov R. Dubtsov Evarist Fomenko V. Pirogov MQ 63 154 0 03 Feb 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 204 19,333 0 13 Jan 2018
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob S. Kligys Bo Chen Menglong Zhu Matthew Tang Andrew G. Howard Hartwig Adam Dmitry Kalenichenko MQ 164 3,141 0 15 Dec 2017
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy Asit K. Mishra Debbie Marr FedML 65 331 0 15 Nov 2017
Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks Urs Koster T. Webb Xin Eric Wang Marcel Nassar Arjun K. Bansal ... Luke Hornof A. Khosrowshahi Carey Kloss Ruby J. Pai N. Rao MQ 47 262 0 06 Nov 2017
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 174 1,805 0 10 Oct 2017
Training Quantized Nets: A Deeper Understanding Hao Li Soham De Zheng Xu Christoph Studer H. Samet Tom Goldstein MQ 55 211 0 07 Jun 2017
In-Datacenter Performance Analysis of a Tensor Processing Unit N. Jouppi C. Young Nishant Patil David Patterson Gaurav Agrawal ... Vijay Vasudevan Richard Walter Walter Wang Eric Wilcox Doe Hyun Yoon 237 4,644 0 16 Apr 2017
Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights Aojun Zhou Anbang Yao Yiwen Guo Lin Xu Yurong Chen MQ 395 1,055 0 10 Feb 2017
Deep Learning with Low Precision by Half-wave Gaussian Quantization Zhaowei Cai Xiaodong He Jian Sun Nuno Vasconcelos MQ 138 505 0 03 Feb 2017
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients Shuchang Zhou Yuxin Wu Zekun Ni Xinyu Zhou He Wen Yuheng Zou MQ 129 2,090 0 20 Jun 2016
R-FCN: Object Detection via Region-based Fully Convolutional Networks Jifeng Dai Yi Li Kaiming He Jian Sun ObjD 177 5,642 0 20 May 2016
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi MQ 175 4,369 0 16 Mar 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,426 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 886 27,416 0 02 Dec 2015
BinaryConnect: Training Deep Neural Networks with binary weights during propagations Matthieu Courbariaux Yoshua Bengio J. David MQ 212 2,992 0 02 Nov 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 528 62,377 0 04 Jun 2015
Deep Learning with Limited Numerical Precision Suyog Gupta A. Agrawal K. Gopalakrishnan P. Narayanan HAI 207 2,049 0 09 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014
Caffe: Convolutional Architecture for Fast Feature Embedding Yangqing Jia Evan Shelhamer Jeff Donahue Sergey Karayev Jonathan Long Ross B. Girshick S. Guadarrama Trevor Darrell VLM BDL 3DV 280 14,712 0 20 Jun 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 426 43,814 0 01 May 2014