NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents

27 February 2024

Papers citing "NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents"

4 / 4 papers shown

Title
Autoregressive Image Generation using Residual Quantization Doyup Lee Chiheon Kim Saehoon Kim Minsu Cho Wook-Shin Han VGen 175 330 0 03 Mar 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 698 0 27 Aug 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 279 1,996 0 31 Dec 2020
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers Shusheng Xu Xingxing Zhang Yi Wu Furu Wei Ming Zhou 65 44 0 16 Oct 2020