NarrowBERT: Accelerating Masked Language Model Pretraining and Inference

11 January 2023

Papers citing "NarrowBERT: Accelerating Masked Language Model Pretraining and Inference"

6 / 6 papers shown

Title
Efficient Sparse Attention needs Adaptive Token Release Chaoran Zhang Lixin Zou Dan Luo Min Tang Xiangyang Luo Zihao Li Chenliang Li 41 2 0 02 Jul 2024
MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining Jacob P. Portes Alex Trott Sam Havens Daniel King Abhinav Venigalla Moin Nadeem Nikhil Sardana D. Khudia Jonathan Frankle 20 16 0 29 Dec 2023
One-Versus-Others Attention: Scalable Multimodal Integration for Clinical Data Michal Golovanevsky Eva Schiller Akira Nair Ritambhara Singh Carsten Eickhoff 16 2 0 11 Jul 2023
ABC: Attention with Bounded-memory Control Hao Peng Jungo Kasai Nikolaos Pappas Dani Yogatama Zhaofeng Wu Lingpeng Kong Roy Schwartz Noah A. Smith 76 22 0 06 Oct 2021
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 249 205 0 25 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018