Exploring Extreme Parameter Compression for Pre-trained Language Models

Exploring Extreme Parameter Compression for Pre-trained Language Models

20 May 2022

Lifeng Shang

Xin Jiang

Qun Liu

ArXiv (abs)PDF HTML

Papers citing "Exploring Extreme Parameter Compression for Pre-trained Language Models"

6 / 6 papers shown

Title
Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI Dustin Wright Christian Igel Gabrielle Samuel Raghavendra Selvan 120 16 0 05 Sep 2023
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models Mohammadreza Banaei Klaudia Bałazy Artur Kasymov R. Lebret Jacek Tabor Karl Aberer OffRL 45 0 0 08 Feb 2023
Tensor Networks Meet Neural Networks: A Survey and Future Perspectives Maolin Wang Yu Pan Zenglin Xu Xiangli Yang Guangxi Li A. Cichocki Andrzej Cichocki 202 22 0 22 Jan 2023
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 80 137 0 06 Dec 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 108 15 0 10 Oct 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 72 65 0 29 Mar 2022

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.