Compression of Deep Learning Models for Text: A Survey

Compression of Deep Learning Models for Text: A Survey

12 August 2020

Papers citing "Compression of Deep Learning Models for Text: A Survey"

16 / 16 papers shown

Title
Knowledge Augmentation in Federation: Rethinking What Collaborative Learning Can Bring Back to Decentralized Data Wentai Wu Ligang He Saiqin Long Ahmed M. Abdelmoniem Yingliang Wu Rui Mao 60 0 0 05 Mar 2025
MoDeGPT: Modular Decomposition for Large Language Model Compression Chi-Heng Lin Shangqian Gao James Seale Smith Abhishek Patel Shikhar Tuli Yilin Shen Hongxia Jin Yen-Chang Hsu 71 7 0 19 Aug 2024
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging Deyuan Liu Zhanyue Qin Hairu Wang Zhao Yang Zecheng Wang ... Zhao Lv Zhiying Tu Dianhui Chu Bo Li Dianbo Sui 22 2 0 24 Jun 2024
Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation Yun-Wei Chu Dong-Jun Han Christopher G. Brinton 28 4 0 15 Jan 2024
LACoS-BLOOM: Low-rank Adaptation with Contrastive objective on 8 bits Siamese-BLOOM Wenhui Hua Brian Williams Davood Shamsi 28 3 0 10 May 2023
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training Nitay Calderon Subhabrata Mukherjee Roi Reichart Amir Kantor 33 17 0 03 May 2023
Efficiently Scaling Transformer Inference Reiner Pope Sholto Douglas Aakanksha Chowdhery Jacob Devlin James Bradbury Anselm Levskaya Jonathan Heek Kefan Xiao Shivani Agrawal J. Dean 34 295 0 09 Nov 2022
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models Bowen Shen Zheng Lin Yuanxin Liu Zhengxiao Liu Lei Wang Weiping Wang VLM 47 4 0 27 Oct 2022
Stochastic gradient descent with gradient estimator for categorical features Paul Peseux Maxime Bérar Thierry Paquet Victor Nicollet 30 0 0 08 Sep 2022
Can Model Compression Improve NLP Fairness Guangxuan Xu Qingyuan Hu 28 26 0 21 Jan 2022
A Short Study on Compressing Decoder-Based Language Models Tianda Li Yassir El Mesbahi I. Kobyzev Ahmad Rashid A. Mahmud Nithin Anchuri Habib Hajimolahoseini Yang Liu Mehdi Rezagholizadeh 93 25 0 16 Oct 2021
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 221 197 0 07 Feb 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 233 576 0 12 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018