CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade

29 December 2020

Lei Li

Yankai Lin

Deli Chen

Shuhuai Ren

Peng Li

Jie Zhou

Xu Sun

ArXiv PDF HTML

Papers citing "CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade"

30 / 30 papers shown

Title
A Unified Approach to Routing and Cascading for LLMs Jasper Dekoninck Maximilian Baader Martin Vechev 91 2 0 14 Oct 2024
Accelerating BERT Inference for Sequence Labeling via Early-Exit Xiaonan Li Yunfan Shao Tianxiang Sun Hang Yan Xipeng Qiu Xuanjing Huang 44 40 0 28 May 2021
Early Exiting with Ensemble Internal Classifiers Tianxiang Sun Yunhua Zhou Xiangyang Liu Xinyu Zhang Hao Jiang Bo Zhao Xuanjing Huang Xipeng Qiu 47 30 0 28 May 2021
Learning to Cascade: Confidence Calibration for Improving the Accuracy and Computational Cost of Cascade Inference Systems Shohei Enomoto Takeharu Eda UQCV 56 17 0 15 Apr 2021
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 68 209 0 27 Sep 2020
BERT Loses Patience: Fast and Robust Inference with Early Exit Wangchunshu Zhou Canwen Xu Tao Ge Julian McAuley Ke Xu Furu Wei 31 337 0 07 Jun 2020
The Cascade Transformer: an Application for Efficient Answer Sentence Selection Luca Soldaini Alessandro Moschitti 48 44 0 05 May 2020
What Happens To BERT Embeddings During Fine-tuning? Amil Merchant Elahe Rahimtoroghi Ellie Pavlick Ian Tenney 40 186 0 29 Apr 2020
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference Ji Xin Raphael Tang Jaejun Lee Yaoliang Yu Jimmy J. Lin 42 370 0 27 Apr 2020
The Right Tool for the Job: Matching Model and Instance Complexities Roy Schwartz Gabriel Stanovsky Swabha Swayamdipta Jesse Dodge Noah A. Smith 71 168 0 16 Apr 2020
FastBERT: a Self-distilling BERT with Adaptive Inference Time Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Haotang Deng Qi Ju 73 356 0 05 Apr 2020
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 252 199 0 07 Feb 2020
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 133 10,720 0 29 Oct 2019
Q8BERT: Quantized 8Bit BERT Ofir Zafrir Guy Boudoukh Peter Izsak Moshe Wasserblat MQ 57 502 0 14 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 126 7,437 0 02 Oct 2019
Reducing Transformer Depth on Demand with Structured Dropout Angela Fan Edouard Grave Armand Joulin 88 588 0 25 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 59 1,847 0 23 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 101 833 0 25 Aug 2019
Well-Read Students Learn Better: On the Importance of Pre-training Compact Models Iulia Turc Ming-Wei Chang Kenton Lee Kristina Toutanova 49 224 0 23 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 406 24,160 0 26 Jul 2019
Are Sixteen Heads Really Better than One? Paul Michel Omer Levy Graham Neubig MoE 73 1,051 0 25 May 2019
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned Elena Voita David Talbot F. Moiseev Rico Sennrich Ivan Titov 76 1,120 0 23 May 2019
BERT Rediscovers the Classical NLP Pipeline Ian Tenney Dipanjan Das Ellie Pavlick MILM SSeg 100 1,458 0 15 May 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 951 93,936 0 11 Oct 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 652 7,080 0 20 Apr 2018
On Calibration of Modern Neural Networks Chuan Guo Geoff Pleiss Yu Sun Kilian Q. Weinberger UQCV 197 5,774 0 14 Jun 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 397 4,444 0 18 Apr 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 151 8,067 0 16 Jun 2016
Adaptive Computation Time for Recurrent Neural Networks Alex Graves 46 544 0 29 Mar 2016
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 236 19,523 0 09 Mar 2015