Extending Input Contexts of Language Models through Training on Segmented Sequences

23 October 2023

Julian McAuley

George Karypis

Papers citing "Extending Input Contexts of Language Models through Training on Segmented Sequences"

3 / 3 papers shown

Title
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 78 122 0 03 Jul 2022
SHAPE: Shifted Absolute Position Embedding for Transformers Shun Kiyono Sosuke Kobayashi Jun Suzuki Kentaro Inui 236 45 0 13 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 695 0 27 Aug 2021