Sparsifying Transformer Models with Trainable Representation Pooling

10 September 2020

Papers citing "Sparsifying Transformer Models with Trainable Representation Pooling"

10 / 10 papers shown

Title
Self-Satisfied: An end-to-end framework for SAT generation and prediction Christopher R. Serrano Jonathan Gallagher Kenji Yamada Alexei Kopylov Michael A. Warren 29 0 0 18 Oct 2024
Arctic-TILT. Business Document Understanding at Sub-Billion Scale Łukasz Borchmann Michał Pietruszka Wojciech Ja'skowski Dawid Jurkiewicz Piotr Halama ... Gabriela Nowakowska Artur Zawłocki Łukasz Duhr Paweł Dyda Michał Turski VLM 39 1 0 08 Aug 2024
TRAMS: Training-free Memory Selection for Long-range Language Modeling Haofei Yu Cunxiang Wang Yue Zhang Wei Bi RALM 41 6 0 24 Oct 2023
Named Entity Inclusion in Abstractive Text Summarization S. Berezin Tatiana Batura 39 7 0 05 Jul 2023
How Far are We from Robust Long Abstractive Summarization? Huan Yee Koh Jiaxin Ju He Zhang Ming Liu Shirui Pan HILM 31 39 0 30 Oct 2022
Leveraging Locality in Abstractive Text Summarization Yixin Liu Ansong Ni Linyong Nan Budhaditya Deb Chenguang Zhu Ahmed Hassan Awadallah Dragomir R. Radev 33 18 0 25 May 2022
Linear Complexity Randomized Self-attention Mechanism Lin Zheng Chong-Jun Wang Lingpeng Kong 22 31 0 10 Apr 2022
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 285 2,017 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 252 580 0 12 Mar 2020
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models Sandeep Subramanian Raymond Li Jonathan Pilault C. Pal 246 215 0 07 Sep 2019