Exploring Extreme Parameter Compression for Pre-trained Language Models

Exploring Extreme Parameter Compression for Pre-trained Language Models

20 May 2022

Lifeng Shang

Xin Jiang

Qun Liu

ArXiv (abs)PDF HTML

Papers citing "Exploring Extreme Parameter Compression for Pre-trained Language Models"

6 / 6 papers shown

Title
Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI Dustin Wright Christian Igel Gabrielle Samuel Raghavendra Selvan 120 16 0 05 Sep 2023
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models Mohammadreza Banaei Klaudia Bałazy Artur Kasymov R. Lebret Jacek Tabor Karl Aberer OffRL 45 0 0 08 Feb 2023
Tensor Networks Meet Neural Networks: A Survey and Future Perspectives Maolin Wang Yu Pan Zenglin Xu Xiangli Yang Guangxi Li A. Cichocki Andrzej Cichocki 202 22 0 22 Jan 2023
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 80 137 0 06 Dec 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 108 15 0 10 Oct 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 72 65 0 29 Mar 2022