v1v2v3v4 (latest)

Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets

13 March 2019

Stanley Osher

Papers citing "Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets"

38 / 38 papers shown

Title
A Principled Bayesian Framework for Training Binary and Spiking Neural Networks James A. Walker M. Khajehnejad Adeel Razi BDL 142 0 0 23 May 2025
Choose Your Model Size: Any Compression by a Single Gradient Descent Martin Genzel Patrick Putzky Pengfei Zhao Siyang Song Mattes Mollenhauer Robert Seidel Stefan Dietzel Thomas Wollmann 105 0 0 03 Feb 2025
Optimizing Large Language Model Training Using FP4 Quantization Ruizhe Wang Yeyun Gong Xiao Liu Guoshuai Zhao Ziyue Yang Baining Guo Zhengjun Zha Peng Cheng MQ 184 12 0 28 Jan 2025
In-context KV-Cache Eviction for LLMs via Attention-Gate Zihao Zeng Bokai Lin Tianqi Hou Hao Zhang Zhijie Deng 109 2 0 15 Oct 2024
End-to-end Adaptive Dynamic Subsampling and Reconstruction for Cardiac MRI George Yiasemis Jan-Jakob Sonke Jonas Teuwen 93 2 0 15 Mar 2024
Neural Lattice Reduction: A Self-Supervised Geometric Deep Learning Approach Giovanni Luca Marchetti Gabriele Cesa Kumar Pratik Arash Behboodi 143 2 0 14 Nov 2023
Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training Xiaoying Zhi Varun Babbar P. Sun Fran Silavong Ruibo Shi Sean J. Moran Sean Moran 140 1 0 17 Feb 2023
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks Penghang Yin Shuai Zhang J. Lyu Stanley Osher Y. Qi Jack Xin MQ 91 62 0 15 Aug 2018
PACT: Parameterized Clipping Activation for Quantized Neural Networks Jungwook Choi Zhuo Wang Swagath Venkataramani P. Chuang Vijayalakshmi Srinivasan K. Gopalakrishnan MQ 80 955 0 16 May 2018
An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks Qianxiao Li Shuji Hao 92 76 0 04 Mar 2018
Loss-aware Weight Quantization of Deep Networks Lu Hou James T. Kwok MQ 104 127 0 23 Feb 2018
Learning One Convolutional Layer with Overlapping Patches Surbhi Goel Adam R. Klivans Raghu Meka MLT 80 81 0 07 Feb 2018
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples Anish Athalye Nicholas Carlini D. Wagner AAML 249 3,195 0 01 Feb 2018
BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights Penghang Yin Shuai Zhang J. Lyu Stanley Osher Y. Qi Jack Xin MQ 86 79 0 19 Jan 2018
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima S. Du Jason D. Lee Yuandong Tian Barnabás Póczós Aarti Singh MLT 157 236 0 03 Dec 2017
Deep Learning as a Mixed Convex-Combinatorial Optimization Problem A. Friesen Pedro M. Domingos 46 20 0 31 Oct 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 183 337 0 10 Jun 2017
Training Quantized Nets: A Deeper Understanding Hao Li Soham De Zheng Xu Christoph Studer H. Samet Tom Goldstein MQ 64 211 0 07 Jun 2017
Convergence Analysis of Two-layer Neural Networks with ReLU Activation Yuanzhi Li Yang Yuan MLT 166 654 0 28 May 2017
Learning ReLUs via Gradient Descent Mahdi Soltanolkotabi MLT 96 183 0 10 May 2017
An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis Yuandong Tian MLT 218 217 0 02 Mar 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 176 313 0 26 Feb 2017
Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights Aojun Zhou Anbang Yao Yiwen Guo Lin Xu Yurong Chen MQ 399 1,055 0 10 Feb 2017
Deep Learning with Low Precision by Half-wave Gaussian Quantization Zhaowei Cai Xiaodong He Jian Sun Nuno Vasconcelos MQ 142 505 0 03 Feb 2017
Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection Penghang Yin Shuai Zhang Y. Qi Jack Xin MQ 169 42 0 19 Dec 2016
Trained Ternary Quantization Chenzhuo Zhu Song Han Huizi Mao W. Dally MQ 153 1,035 0 04 Dec 2016
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations Itay Hubara Matthieu Courbariaux Daniel Soudry Ran El-Yaniv Yoshua Bengio MQ 168 1,871 0 22 Sep 2016
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients Shuchang Zhou Yuxin Wu Zekun Ni Xinyu Zhou He Wen Yuheng Zou MQ 135 2,090 0 20 Jun 2016
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi MQ 181 4,369 0 16 Mar 2016
Binarized Neural Networks Itay Hubara Daniel Soudry Ran El-Yaniv MQ 214 1,348 0 08 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
BinaryConnect: Training Deep Neural Networks with binary weights during propagations Matthieu Courbariaux Yoshua Bengio J. David MQ 225 2,994 0 02 Nov 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 545 62,477 0 04 Jun 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 471 43,357 0 11 Feb 2015
Difference Target Propagation Dong-Hyun Lee Saizheng Zhang Asja Fischer Yoshua Bengio AAML 127 353 0 23 Dec 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,575 0 04 Sep 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 297 14,978 1 21 Dec 2013
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation Yoshua Bengio Nicholas Léonard Aaron Courville 406 3,158 0 15 Aug 2013