Model compression via distillation and quantization

15 February 2018

Dan Alistarh

Papers citing "Model compression via distillation and quantization"

50 / 171 papers shown

Title
FedSynth: Gradient Compression via Synthetic Data in Federated Learning Shengyuan Hu Jack Goetz Kshitiz Malik Hongyuan Zhan Zhe Liu Yue Liu DD FedML 45 38 0 04 Apr 2022
Update Compression for Deep Neural Networks on the Edge Bo Chen A. Bakhshi Gustavo E. A. P. A. Batista Brian Ng Tat-Jun Chin 31 17 0 09 Mar 2022
Deadwooding: Robust Global Pruning for Deep Neural Networks Sawinder Kaur Ferdinando Fioretto Asif Salekin 32 4 0 10 Feb 2022
Robust Binary Models by Pruning Randomly-initialized Networks Chen Liu Ziqi Zhao Sabine Süsstrunk Mathieu Salzmann TPM AAML MQ 32 4 0 03 Feb 2022
Iterative Activation-based Structured Pruning Kaiqi Zhao Animesh Jain Ming Zhao 42 0 0 22 Jan 2022
Enabling Deep Learning on Edge Devices through Filter Pruning and Knowledge Transfer Kaiqi Zhao Yitao Chen Ming Zhao 27 3 0 22 Jan 2022
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems Yoshitomo Matsubara Luca Soldaini Eric Lind Alessandro Moschitti 37 6 0 15 Jan 2022
Problem-dependent attention and effort in neural networks with applications to image resolution and model selection Chris Rohlfs 34 4 0 05 Jan 2022
Data-Free Knowledge Transfer: A Survey Yuang Liu Wei Zhang Jun Wang Jianyong Wang 40 48 0 31 Dec 2021
Learning Robust and Lightweight Model through Separable Structured Transformations Xian Wei Yanhui Huang Yang Xu Mingsong Chen Hai Lan Yuanxiang Li Zhongfeng Wang Xuan Tang OOD 24 0 0 27 Dec 2021
Multi-Modality Distillation via Learning the teacher's modality-level Gram Matrix Peng Liu 24 0 0 21 Dec 2021
Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection Yifan Zhuang Ziyuan Pu Jia Hu Yinhai Wang 35 24 0 09 Dec 2021
Automatic Neural Network Pruning that Efficiently Preserves the Model Accuracy Thibault Castells Seul-Ki Yeom 3DV 23 3 0 18 Nov 2021
Arch-Net: Model Distillation for Architecture Agnostic Model Deployment Weixin Xu Zipeng Feng Shuangkang Fang Song Yuan Yi Yang Shuchang Zhou MQ 30 1 0 01 Nov 2021
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation Sumanth Chennupati Mohammad Mahdi Kamani Zhongwei Cheng Lin Chen 35 4 0 19 Oct 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 36 48 0 11 Oct 2021
Visualizing the embedding space to explain the effect of knowledge distillation Hyun Seung Lee C. Wallraven 31 1 0 09 Oct 2021
Generative Optimization Networks for Memory Efficient Data Generation Shreshth Tuli Shikhar Tuli G. Casale N. Jennings 33 12 0 06 Oct 2021
Auto-Split: A General Framework of Collaborative Edge-Cloud AI Amin Banitalebi-Dehkordi Naveen Vedula J. Pei Fei Xia Lanjun Wang Yong Zhang 22 89 0 30 Aug 2021
DKM: Differentiable K-Means Clustering Layer for Neural Network Compression Minsik Cho Keivan Alizadeh Vahid Saurabh N. Adya Mohammad Rastegari 44 34 0 28 Aug 2021
Privacy-Preserving Machine Learning: Methods, Challenges and Directions Runhua Xu Nathalie Baracaldo J. Joshi 32 99 0 10 Aug 2021
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning Kaan Ozkara Navjot Singh Deepesh Data Suhas Diggavi FedML MQ 29 56 0 29 Jul 2021
Follow Your Path: a Progressive Method for Knowledge Distillation Wenxian Shi Yuxuan Song Hao Zhou Bohan Li Lei Li 17 15 0 20 Jul 2021
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference Chaojian Li Wuyang Chen Yuchen Gu Tianlong Chen Yonggan Fu Zhangyang Wang Yingyan Lin 35 0 0 16 Jul 2021
Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion Mingbao Lin Bohong Chen Rongrong Ji Rongrong Ji VLM 35 23 0 14 Jul 2021
Improving the Efficiency of Transformers for Resource-Constrained Devices Hamid Tabani Ajay Balasubramaniam Shabbir Marzban Elahe Arani Bahram Zonooz 46 20 0 30 Jun 2021
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better Gaurav Menghani VLM MedIm 23 367 0 16 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 60 818 0 14 Jun 2021
BoolNet: Minimizing The Energy Consumption of Binary Neural Networks Nianhui Guo Joseph Bethge Haojin Yang Kai Zhong Xuefei Ning Christoph Meinel Yu Wang MQ 29 11 0 13 Jun 2021
Rethinking Transfer Learning for Medical Image Classification Le Peng Hengyue Liang Gaoxiang Luo Taihui Li Ju Sun VLM LM&MA 16 5 0 09 Jun 2021
RBNN: Memory-Efficient Reconfigurable Deep Binary Neural Network with IP Protection for Internet of Things Huming Qiu Hua Ma Zhi-Li Zhang Yifeng Zheng Anmin Fu Pan Zhou Yansong Gao Derek Abbott S. Al-Sarawi MQ 24 9 0 09 May 2021
Self-paced Resistance Learning against Overfitting on Noisy Labels Xiaoshuang Shi Zhenhua Guo Fuyong Xing Yun Liang Xiaofeng Zhu NoLa 23 20 0 07 May 2021
Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems Xiaocong Du Bhargav Bhushanam Jiecao Yu Dhruv Choudhary Tianxiang Gao Sherman Wong Louis Feng Jongsoo Park Yu Cao A. Kejariwal 34 5 0 04 May 2021
Differentiable Model Compression via Pseudo Quantization Noise Alexandre Défossez Yossi Adi Gabriel Synnaeve DiffM MQ 26 48 0 20 Apr 2021
Compact CNN Structure Learning by Knowledge Distillation Waqar Ahmed Andrea Zunino Pietro Morerio Vittorio Murino 38 5 0 19 Apr 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 46 36 0 16 Apr 2021
Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer Phuoc Pham J. Abraham Jaeyong Chung MQ 53 11 0 01 Apr 2021
Student Network Learning via Evolutionary Knowledge Distillation Kangkai Zhang Chunhui Zhang Shikun Li Dan Zeng Shiming Ge 24 83 0 23 Mar 2021
CDFI: Compression-Driven Network Design for Frame Interpolation Tianyu Ding Luming Liang Zhihui Zhu Ilya Zharkov 29 93 0 18 Mar 2021
Learnable Companding Quantization for Accurate Low-bit Neural Networks Kohei Yamamoto MQ 36 64 0 12 Mar 2021
Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices Md Mohaimenuzzaman Christoph Bergmeir I. West B. Meyer 22 41 0 05 Mar 2021
Diversifying Sample Generation for Accurate Data-Free Quantization Xiangguo Zhang Haotong Qin Yifu Ding Ruihao Gong Qing Yan Renshuai Tao Yuhang Li F. Yu Xianglong Liu MQ 58 94 0 01 Mar 2021
BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization Huanrui Yang Lin Duan Yiran Chen Hai Helen Li MQ 21 64 0 20 Feb 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 95 35 0 16 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 247 190 0 12 Jan 2021
I-BERT: Integer-only BERT Quantization Sehoon Kim A. Gholami Z. Yao Michael W. Mahoney Kurt Keutzer MQ 107 345 0 05 Jan 2021
Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search Y. Fu Zhongzhi Yu Yongan Zhang Yingyan Lin 27 4 0 24 Dec 2020
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 45 288 0 06 Dec 2020
Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression Cody Blakeney Xiaomin Li Yan Yan Ziliang Zong 53 40 0 05 Dec 2020
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 54 116 0 25 Nov 2020