BERT Loses Patience: Fast and Robust Inference with Early Exit

7 June 2020

Papers citing "BERT Loses Patience: Fast and Robust Inference with Early Exit"

30 / 80 papers shown

Title
OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses Robik Shrestha Kushal Kafle Christopher Kanan CML 35 13 0 05 Apr 2022
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection Xin Huang A. Khetan Rene Bidart Zohar Karnin 19 14 0 27 Mar 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 41 37 0 12 Mar 2022
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation Tianxiang Sun Xiangyang Liu Wei-wei Zhu Zhichao Geng Lingling Wu Yilong He Yuan Ni Guotong Xie Xuanjing Huang Xipeng Qiu 39 40 0 03 Mar 2022
Revisiting Over-smoothing in BERT from the Perspective of Graph Han Shi Jiahui Gao Hang Xu Xiaodan Liang Zhenguo Li Lingpeng Kong Stephen M. S. Lee James T. Kwok 27 71 0 17 Feb 2022
A Survey on Model Compression and Acceleration for Pretrained Language Models Canwen Xu Julian McAuley 28 58 0 15 Feb 2022
Towards Disentangling Information Paths with Coded ResNeXt Apostolos Avranas Marios Kountouris FAtt 30 1 0 10 Feb 2022
Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning Utku Evci Vincent Dumoulin Hugo Larochelle Michael C. Mozer 30 83 0 10 Jan 2022
AdaViT: Adaptive Tokens for Efficient Vision Transformer Hongxu Yin Arash Vahdat J. Álvarez Arun Mallya Jan Kautz Pavlo Molchanov ViT 35 320 0 14 Dec 2021
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline Xiangyang Liu Tianxiang Sun Junliang He Jiawen Wu Lingling Wu Xinyu Zhang Hao Jiang Bo Zhao Xuanjing Huang Xipeng Qiu ELM 28 46 0 13 Oct 2021
Towards Efficient Post-training Quantization of Pre-trained Language Models Haoli Bai Lu Hou Lifeng Shang Xin Jiang Irwin King M. Lyu MQ 82 47 0 30 Sep 2021
DACT-BERT: Differentiable Adaptive Computation Time for an Efficient BERT Inference Cristobal Eyzaguirre Felipe del-Rio Vladimir Araujo Alvaro Soto 16 7 0 24 Sep 2021
PluGeN: Multi-Label Conditional Generation From Pre-Trained Models Maciej Wołczyk Magdalena Proszewska Lukasz Maziarka Maciej Ziȩba Patryk Wielopolski Rafał Kurczab Marek Śmieja DRL 29 5 0 18 Sep 2021
Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers Jason Phang Haokun Liu Samuel R. Bowman 30 25 0 17 Sep 2021
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression Canwen Xu Wangchunshu Zhou Tao Ge Kelvin J. Xu Julian McAuley Furu Wei 21 41 0 07 Sep 2021
End-to-End Self-Debiasing Framework for Robust NLU Training Abbas Ghaddar Philippe Langlais Mehdi Rezagholizadeh Ahmad Rashid UQCV 34 36 0 05 Sep 2021
Exceeding the Limits of Visual-Linguistic Multi-Task Learning Cameron R. Wolfe Keld T. Lundgaard VLM 45 2 0 27 Jul 2021
Elbert: Fast Albert with Confidence-Window Based Early Exit Keli Xie Siyuan Lu Meiqi Wang Zhongfeng Wang 19 20 0 01 Jul 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 53 1,089 0 08 Jun 2021
Accelerating BERT Inference for Sequence Labeling via Early-Exit Xiaonan Li Yunfan Shao Tianxiang Sun Hang Yan Xipeng Qiu Xuanjing Huang 24 40 0 28 May 2021
The Power of Log-Sum-Exp: Sequential Density Ratio Matrix Estimation for Speed-Accuracy Optimization Taiki Miyagawa Akinori F. Ebihara 25 3 0 28 May 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 33 199 0 08 Mar 2021
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting Wangchunshu Zhou Tao Ge Canwen Xu Ke Xu Furu Wei LRM 16 15 0 02 Jan 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 145 221 0 31 Dec 2020
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade Lei Li Yankai Lin Deli Chen Shuhuai Ren Peng Li Jie Zhou Xu Sun 29 51 0 29 Dec 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 246 1,454 0 18 Mar 2020
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 231 198 0 07 Feb 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,833 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 304 6,996 0 20 Apr 2018
Adversarial examples in the physical world Alexey Kurakin Ian Goodfellow Samy Bengio SILM AAML 326 5,847 0 08 Jul 2016