SWALP : Stochastic Weight Averaging in Low-Precision Training

26 April 2019

Papers citing "SWALP : Stochastic Weight Averaging in Low-Precision Training"

50 / 52 papers shown

Title
Silenzio: Secure Non-Interactive Outsourced MLP Training Jonas Sander T. Eisenbarth 33 0 0 24 Apr 2025
Unmasking the Unknown: Facial Deepfake Detection in the Open-Set Paradigm Nadarasar Bahavan Sanjay Saha Ken Chen Sachith Seneviratne Sanka Rasnayaka Saman K. Halgamuge 62 0 0 11 Mar 2025
When, Where and Why to Average Weights? Niccolò Ajroldi Antonio Orvieto Jonas Geiping MoMe 96 0 0 10 Feb 2025
Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits Daniel Morales-Brotons Thijs Vogels Hadrien Hendrikx 126 17 0 27 Nov 2024
Task Arithmetic Through The Lens Of One-Shot Federated Learning Zhixu Tao I. Mason Sanjeev R. Kulkarni Xavier Boix MoMe FedML 84 3 0 27 Nov 2024
On Exact Bit-level Reversible Transformers Without Changing Architectures Guoqiang Zhang J. P. Lewis W. Kleijn MQ AI4CE 32 0 0 12 Jul 2024
Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients Zhenyu (Allen) Zhang Ajay Jaiswal L. Yin Shiwei Liu Jiawei Zhao Yuandong Tian Zhangyang Wang VLM 33 16 0 11 Jul 2024
VIPriors 4: Visual Inductive Priors for Data-Efficient Deep Learning Challenges Robert-Jan Bruintjes A. Lengyel Marcos Baptista-Rios O. Kayhan Davide Zambrano Nergis Tomen J. C. V. Gemert VLM 44 0 0 26 Jun 2024
Weighted Ensemble Models Are Strong Continual Learners Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 32 16 0 14 Dec 2023
Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization Yaohua Liu Jiaxin Gao Xianghao Jiao Zhu Liu Xin-Yue Fan Risheng Liu AAML 43 0 0 19 Oct 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 33 52 0 27 Sep 2023
Tango: rethinking quantization for graph neural network training on GPUs Shiyang Chen Da Zheng Caiwen Ding Chengying Huan Yuede Ji Hang Liu GNN MQ 31 5 0 02 Aug 2023
Number Systems for Deep Neural Network Architectures: A Survey Ghada Alsuhli Vasileios Sakellariou H. Saleh Mahmoud Al-Qutayri Baker Mohammad T. Stouraitis 16 3 0 11 Jul 2023
The Split Matters: Flat Minima Methods for Improving the Performance of GNNs N. Lell A. Scherp 43 1 0 15 Jun 2023
Early Weight Averaging meets High Learning Rates for LLM Pre-training Sunny Sanyal A. Neerkaje Jean Kaddour Abhishek Kumar Sujay Sanghavi MoMe 33 17 0 05 Jun 2023
Hierarchical Weight Averaging for Deep Neural Networks Xiaozhe Gu Zixun Zhang Yuncheng Jiang Tao Luo Ruimao Zhang Shuguang Cui Zhuguo Li 27 5 0 23 Apr 2023
A Survey of Historical Learning: Learning Models with Learning History Xiang Li Ge Wu Lingfeng Yang Wenzhe Wang Renjie Song Jian Yang MU AI4TS 31 2 0 23 Mar 2023
Dynamic Stashing Quantization for Efficient Transformer Training Guofu Yang Daniel Lo Robert D. Mullins Yiren Zhao MQ 31 8 0 09 Mar 2023
Training with Mixed-Precision Floating-Point Assignments Wonyeol Lee Rahul Sharma A. Aiken MQ 24 2 0 31 Jan 2023
Variants of SGD for Lipschitz Continuous Loss Functions in Low-Precision Environments Michael R. Metel 30 1 0 09 Nov 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 31 47 0 13 Oct 2022
Low-Precision Arithmetic for Fast Gaussian Processes Wesley J. Maddox Andres Potapczynski A. Wilson 27 12 0 14 Jul 2022
ST-CoNAL: Consistency-Based Acquisition Criterion Using Temporal Self-Ensemble for Active Learning J. Baik In Young Yoon J. Choi 14 0 0 05 Jul 2022
DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware H. Hashemi Yongqin Wang M. Annavaram FedML 26 58 0 30 Jun 2022
Low-Precision Stochastic Gradient Langevin Dynamics Ruqi Zhang A. Wilson Chris De Sa BDL 21 14 0 20 Jun 2022
Trainable Weight Averaging: Accelerating Training and Improving Generalization Tao Li Zhehao Huang Yingwen Wu Zhengbao He Qinghua Tao X. Huang Chih-Jen Lin MoMe 52 3 0 26 May 2022
On Distributed Adaptive Optimization with Gradient Compression Xiaoyun Li Belhal Karimi Ping Li 15 25 0 11 May 2022
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation An Xu Wenqi Li Pengfei Guo Dong Yang H. Roth Ali Hatamizadeh Can Zhao Daguang Xu Heng-Chiao Huang Ziyue Xu FedML 36 51 0 18 Mar 2022
QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization Xiuying Wei Ruihao Gong Yuhang Li Xianglong Liu F. Yu MQ VLM 19 166 0 11 Mar 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 34 8 0 09 Jan 2022
SAE: Sequential Anchored Ensembles Arnaud Delaunoy Gilles Louppe UQCV BDL 11 0 0 30 Dec 2021
Whole Brain Segmentation with Full Volume Neural Network Yeshu Li Jianwei Cui Yilun Sheng Xiao Liang Jingdong Wang E. Chang Yan Xu 32 11 0 29 Oct 2021
Perturbated Gradients Updating within Unit Space for Deep Learning Ching-Hsun Tseng Liu Cheng Shin-Jye Lee Xiaojun Zeng 40 5 0 01 Oct 2021
Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression Masaki Morimoto Kai Fukami R. Maulik Ricardo Vinuesa K. Fukagata UQCV 38 30 0 16 Sep 2021
Noisy Truncated SGD: Optimization and Generalization Yingxue Zhou Xinyan Li A. Banerjee 19 3 0 26 Feb 2021
Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precision Johan Bjorck Xiangyu Chen Christopher De Sa Carla P. Gomes Kilian Q. Weinberger 23 6 0 26 Feb 2021
SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training Xiaohan Chen Yang Katie Zhao Yue Wang Pengfei Xu Haoran You Chaojian Li Y. Fu Yingyan Lin Zhangyang Wang 38 1 0 04 Jan 2021
Gradient Descent Averaging and Primal-dual Averaging for Strongly Convex Optimization Wei Tao Wei Li Zhisong Pan Qing Tao MoMe 11 4 0 29 Dec 2020
An FPGA Accelerated Method for Training Feed-forward Neural Networks Using Alternating Direction Method of Multipliers and LSMR Seyedeh Niusha Alavi Foumani Ce Guo Wayne Luk 19 3 0 06 Sep 2020
SQWA: Stochastic Quantized Weight Averaging for Improving the Generalization Capability of Low-Precision Deep Neural Networks Sungho Shin Yoonho Boo Wonyong Sung MQ 22 3 0 02 Feb 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 38 55 0 07 Jan 2020
Sparse Weight Activation Training Md Aamir Raihan Tor M. Aamodt 34 73 0 07 Jan 2020
Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Data Qian Lou Bo Feng Geoffrey C. Fox Lei Jiang FedML 14 81 0 16 Nov 2019
On-Device Machine Learning: An Algorithms and Learning Theory Perspective Sauptik Dhar Junyao Guo Jiayi Liu S. Tripathi Unmesh Kurup Mohak Shah 22 141 0 02 Nov 2019
E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savings Yue Wang Ziyu Jiang Xiaohan Chen Pengfei Xu Yang Katie Zhao Yingyan Lin Zhangyang Wang MQ 21 83 0 29 Oct 2019
QPyTorch: A Low-Precision Arithmetic Simulation Framework Tianyi Zhang Zhiqiu Lin Guandao Yang Christopher De Sa MQ 26 64 0 09 Oct 2019
CAT: Compression-Aware Training for bandwidth reduction Chaim Baskin Brian Chmiel Evgenii Zheltonozhskii Ron Banner A. Bronstein A. Mendelson MQ 20 10 0 25 Sep 2019
Feature Map Transform Coding for Energy-Efficient CNN Inference Brian Chmiel Chaim Baskin Ron Banner Evgenii Zheltonozhskii Yevgeny Yermolin Alex Karbachevsky A. Bronstein A. Mendelson 23 24 0 26 May 2019
A Simple Baseline for Bayesian Uncertainty in Deep Learning Wesley J. Maddox T. Garipov Pavel Izmailov Dmitry Vetrov A. Wilson BDL UQCV 28 795 0 07 Feb 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 20 147 0 02 Feb 2019