Adapting Pretrained Text-to-Text Models for Long Text Sequences

Adapting Pretrained Text-to-Text Models for Long Text Sequences

21 September 2022

Shubham Toshniwal

Yashar Mehdad

Papers citing "Adapting Pretrained Text-to-Text Models for Long Text Sequences"

11 / 11 papers shown

Title
Can Large Language Model Summarizers Adapt to Diverse Scientific Communication Goals? Marcio Fonseca Shay B. Cohen 39 10 0 18 Jan 2024
Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention Ziwei He Jian Yuan Le Zhou Jingwen Leng Bo Jiang 32 1 0 13 Nov 2023
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding Uri Shaham Maor Ivgi Avia Efrat Jonathan Berant Omer Levy VLM 41 126 0 23 May 2023
A Method to Automate the Discharge Summary Hospital Course for Neurology Patients Vince C. Hartman Sanika S. Bapat M. Weiner B. Navi E. Sholle T. Campion 32 18 0 10 May 2023
MUG: A General Meeting Understanding and Generation Benchmark Qinglin Zhang Chong Deng Jiaqing Liu Hai Yu Qian Chen Wen Wang Zhijie Yan Jinglin Liu Yi Ren Zhou Zhao 49 7 0 24 Mar 2023
Visconde: Multi-document QA with GPT-3 and Neural Reranking Jayr Alencar Pereira R. Fidalgo R. Lotufo Rodrigo Nogueira BDL RALM 23 31 0 19 Dec 2022
GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state Junyi Bian Xiaodi Huang Hong Zhou Shanfeng Zhu 27 11 0 18 Nov 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 261 1,996 0 31 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 285 2,015 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 246 580 0 12 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018