MiniALBERT: Model Distillation via Parameter-Efficient Recursive
Transformers

MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers

12 October 2022

Mohammadmahdi Nouriborji

Samaneh Kouchaki

David A. Clifton

Papers citing "MiniALBERT: Model Distillation via Parameter-Efficient Recursive Transformers"

10 / 10 papers shown

Title
DistilXLSR: A Light Weight Cross-Lingual Speech Representation Model Haoyu Wang Siyuan Wang Weiqiang Zhang Jinfeng Bai 32 2 0 02 Jun 2023
Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture Peiyu Liu Ze-Feng Gao Yushuo Chen Wayne Xin Zhao Ji-Rong Wen MoE 30 0 0 27 Mar 2023
Lightweight Transformers for Clinical Natural Language Processing Omid Rohanian Mohammadmahdi Nouriborji Hannah Jauncey Samaneh Kouchaki Isaric Clinical Characterisation Group Lei A. Clifton L. Merson David A. Clifton MedIm LM&MA 16 12 0 09 Feb 2023
Using Bottleneck Adapters to Identify Cancer in Clinical Notes under Low-Resource Constraints Omid Rohanian Hannah Jauncey Mohammadmahdi Nouriborji Vinod Kumar Chauhan Bronner P. Gonccalves Christiana Kartsonaki Isaric Clinical Characterisation Group L. Merson David A. Clifton 11 7 0 17 Oct 2022
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics Daniil Larionov Jens Grunwald Christoph Leiter Steffen Eger 20 5 0 20 Sep 2022
On The Computational Complexity of Self-Attention Feyza Duman Keles Pruthuvi Maheshakya Wijewardena C. Hegde 68 108 0 11 Sep 2022
On the Effectiveness of Compact Biomedical Transformers Omid Rohanian Mohammadmahdi Nouriborji Samaneh Kouchaki David A. Clifton MedIm 18 31 0 07 Sep 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 233 576 0 12 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018