ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through
Regularized Self-Attention

ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention

23 March 2022

Yang Liu

Papers citing "ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention"

6 / 6 papers shown

Title
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models Thibaut Thonet Jos Rozen Laurent Besacier RALM 137 2 0 20 Jan 2025
HDT: Hierarchical Document Transformer Haoyu He Markus Flicke Jan Buchmann Iryna Gurevych Andreas Geiger 43 0 0 11 Jul 2024
Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks Antoine Louis Gijs van Dijck Gerasimos Spanakis AILaw 21 9 0 30 Jan 2023
An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification Ilias Chalkidis Xiang Dai Manos Fergadiotis Prodromos Malakasiotis Desmond Elliott 34 33 0 11 Oct 2022
Recurrent Memory Transformer Aydar Bulatov Yuri Kuratov Mikhail Burtsev CLL 13 102 0 14 Jul 2022
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 282 2,015 0 28 Jul 2020