Towards Unified INT8 Training for Convolutional Neural Network

29 December 2019

Xianglong Liu

Papers citing "Towards Unified INT8 Training for Convolutional Neural Network"

50 / 73 papers shown

Title
PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs Lukas Meiner Jens Mehnert A. P. Condurache MQ 42 0 0 06 May 2025
Accurate INT8 Training Through Dynamic Block-Level Fallback Pengle Zhang Jia wei Jintao Zhang Jun-Jie Zhu Jianfei Chen MQ 82 3 0 13 Mar 2025
Oscillation-Reduced MXFP4 Training for Vision Transformers Yuxiang Chen Haocheng Xi Jun Zhu Jianfei Chen MQ 56 2 0 28 Feb 2025
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 90 0 0 18 Feb 2025
Rethinking Post-Training Quantization: Introducing a Statistical Pre-Calibration Approach Alireza Ghaffari Sharareh Younesian Boxing Chen Vahid Partovi Nia M. Asgharian MQ 61 0 0 17 Jan 2025
SuperGCN: General and Scalable Framework for GCN Training on CPU-powered Supercomputers Chen Zhuang Peng Chen Xin Liu Rio Yokota Nikoli Dryden Toshio Endo Satoshi Matsuoka M. Wahib GNN 67 0 0 25 Nov 2024
Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach Yequan Zhao Hai Li Ian Young Zheng-Wei Zhang MQ 39 2 0 07 Nov 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Yaojie Lu Kurt Keutzer Jianfei Chen Song Han MQ 75 9 0 25 Oct 2024
1-Bit FQT: Pushing the Limit of Fully Quantized Training to 1-bit Chang Gao Jianfei Chen Kang Zhao Jiaqi Wang Liping Jing MQ 41 2 0 26 Aug 2024
Toward INT4 Fixed-Point Training via Exploring Quantization Error for Gradients Dohyung Kim Junghyup Lee Jeimin Jeon Jaehyeon Moon Bumsub Ham MQ 32 0 0 17 Jul 2024
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients Zhenyu (Allen) Zhang Ajay Jaiswal L. Yin Shiwei Liu Jiawei Zhao Yuandong Tian Zhangyang Wang VLM 33 16 0 11 Jul 2024
OvSW: Overcoming Silent Weights for Accurate Binary Neural Networks Jingyang Xiang Zuohui Chen Siqi Li Qing Wu Yong-Jin Liu 26 1 0 07 Jul 2024
AdpQ: A Zero-shot Calibration Free Adaptive Post Training Quantization Method for LLMs Alireza Ghaffari Sharareh Younesian Vahid Partovi Nia Boxing Chen M. Asgharian MQ 47 0 0 22 May 2024
Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attacks Boheng Li Yishuo Cai Haowei Li Feng Xue Zhifeng Li Yiming Li MQ AAML 35 20 0 21 May 2024
Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes Ruihao Gong Yang Yong Zining Wang Jinyang Guo Xiuying Wei Yuqing Ma Xianglong Liu 35 5 0 09 May 2024
Lightweight Change Detection in Heterogeneous Remote Sensing Images with Online All-Integer Pruning Training Chengyang Zhang Weiming Li Gang Li Huina Song Zhaohui Song Xueqian Wang Antonio Plaza 38 0 0 03 May 2024
Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization Haocheng Xi Yuxiang Chen Kang Zhao Kaijun Zheng Jianfei Chen Jun Zhu MQ 39 20 0 19 Mar 2024
IM-Unpack: Training and Inference with Arbitrarily Low Precision Integers Zhanpeng Zeng Karthikeyan Sankaralingam Vikas Singh 58 1 0 12 Mar 2024
EdgeOL: Efficient in-situ Online Learning on Edge Devices Sheng Li Geng Yuan Yawen Wu Yuezhen Dai Chao Wu Alex K. Jones Jingtong Hu Yanzhi Wang Xulong Tang 41 0 0 30 Jan 2024
Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity Yiyue Chen H. Vikalo C. Wang FedML 41 5 0 20 Dec 2023
Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models Alireza Ghaffari Justin Yu Mahsa Ghazvini Nejad M. Asgharian Boxing Chen Vahid Partovi Nia 13 2 0 14 Dec 2023
ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout Huiyao Shu Ang Wang Ziji Shi Hanyu Zhao Yong Li Lu Lu OffRL 31 2 0 30 Oct 2023
Generative Model for Models: Rapid DNN Customization for Diverse Tasks and Resource Constraints Wenxing Xu Yuanchun Li Jiacheng Liu Yiyou Sun Zhengyang Cao Yixuan Li Hao Wen Yunxin Liu 27 0 0 29 Aug 2023
Tango: rethinking quantization for graph neural network training on GPUs Shiyang Chen Da Zheng Caiwen Ding Chengying Huan Yuede Ji Hang Liu GNN MQ 31 5 0 02 Aug 2023
EdgeConvEns: Convolutional Ensemble Learning for Edge Intelligence Ilkay Sikdokur Inci M. Baytas A. Yurdakul FedML 20 0 0 25 Jul 2023
Training Transformers with 4-bit Integers Haocheng Xi Changhao Li Jianfei Chen Jun Zhu MQ 25 47 0 21 Jun 2023
Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN Training Borui Wan Juntao Zhao Chuan Wu GNN 14 15 0 02 Jun 2023
NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient Inference Ruiqi Sun Siwei Ye Jie Zhao Xin He Yiran Li An Zou 35 0 0 23 May 2023
Stable and low-precision training for large-scale vision-language models Mitchell Wortsman Tim Dettmers Luke Zettlemoyer Ari S. Morcos Ali Farhadi Ludwig Schmidt MQ MLLM VLM 24 38 0 25 Apr 2023
Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspective Yuexiao Ma Huixia Li Xiawu Zheng Xuefeng Xiao Rui Wang Shilei Wen Xin Pan Rongrong Ji Rongrong Ji MQ 15 12 0 21 Mar 2023
AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environments Hao Wen Yuanchun Li Zunshuai Zhang Shiqi Jiang Xiaozhou Ye Ouyang Ye Yaqin Zhang Yunxin Liu 90 29 0 13 Mar 2023
MetaGrad: Adaptive Gradient Quantization with Hypernetworks Kaixin Xu Alina Hui Xiu Lee Ziyuan Zhao Zhe Wang Min-man Wu Weisi Lin MQ 20 1 0 04 Mar 2023
Ultra-low Precision Multiplication-free Training for Deep Neural Networks Chang-Shu Liu Rui Zhang Xishan Zhang Yifan Hao Zidong Du Xingui Hu Ling Li Qi Guo MQ 34 1 0 28 Feb 2023
Towards Optimal Compression: Joint Pruning and Quantization Ben Zandonati Glenn Bucagu Adrian Alan Pol M. Pierini Olya Sirkin Tal Kopetz MQ 22 2 0 15 Feb 2023
Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learning Yingchun Wang Jingcai Guo Song Guo Weizhan Zhang MQ 29 20 0 09 Feb 2023
Learning Discretized Neural Networks under Ricci Flow Jun Chen Han Chen Mengmeng Wang Guang Dai Ivor W. Tsang Yong-Jin Liu 25 2 0 07 Feb 2023
Quantized Distributed Training of Large Models with Convergence Guarantees I. Markov Adrian Vladu Qi Guo Dan Alistarh MQ 31 11 0 05 Feb 2023
An Optical XNOR-Bitcount Based Accelerator for Efficient Inference of Binary Neural Networks Sairam Sri Vatsavai Venkata Sai Praneeth Karempudi Ishan G. Thakkar MQ 28 4 0 03 Feb 2023
MobileTL: On-device Transfer Learning with Inverted Residual Blocks HungYueh Chiang N. Frumkin Feng Liang Diana Marculescu MQ 29 12 0 05 Dec 2022
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Yunshan Zhong Gongrui Nan Yu-xin Zhang Rongrong Ji Rongrong Ji MQ 18 3 0 12 Nov 2022
OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networks Benoit Steiner Mostafa Elhoushi Jacob Kahn James Hegarty 29 8 0 24 Oct 2022
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models Xiuying Wei Yunchen Zhang Xiangguo Zhang Ruihao Gong Shanghang Zhang Qi Zhang F. Yu Xianglong Liu MQ 27 145 0 27 Sep 2022
Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tile Renzo Andri Beatrice Bussolino A. Cipolletta Lukas Cavigelli Zhe Wang MQ 26 13 0 26 Sep 2022
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation Mohammadreza Tayaranian Alireza Ghaffari Marzieh S. Tahaei Mehdi Rezagholizadeh M. Asgharian V. Nia MQ 31 6 0 20 Sep 2022
Is Integer Arithmetic Enough for Deep Learning Training? Alireza Ghaffari Marzieh S. Tahaei Mohammadreza Tayaranian M. Asgharian V. Nia MQ 11 16 0 18 Jul 2022
BiT: Robustly Binarized Multi-distilled Transformer Zechun Liu Barlas Oğuz Aasish Pappu Lin Xiao Scott Yih Meng Li Raghuraman Krishnamoorthi Yashar Mehdad MQ 53 52 0 25 May 2022
Engineering the Neural Automatic Passenger Counter Nico Jahn Michael Siebert 13 2 0 02 Mar 2022
Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization Jaehong Yoon Geondo Park Wonyong Jeong Sung Ju Hwang FedML 24 19 0 23 Feb 2022
F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization Qing Jin Jian Ren Richard Zhuang Sumant Hanumante Zhengang Li Zhiyu Chen Yanzhi Wang Kai-Min Yang Sergey Tulyakov MQ 24 48 0 10 Feb 2022
FAT: An In-Memory Accelerator with Fast Addition for Ternary Weight Neural Networks Shien Zhu Luan H. K. Duong Hui Chen Di Liu Weichen Liu MQ 24 5 0 19 Jan 2022