Adaptive Attention Span in Transformers

19 May 2019

Papers citing "Adaptive Attention Span in Transformers"

8 / 8 papers shown

Title
DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models Saeed Ranjbar Alvar Gursimran Singh Mohammad Akbari Yong Zhang VLM 118 0 0 04 Mar 2025
Enhancing RWKV-based Language Models for Long-Sequence Text Generation Xinghan Pan 75 0 0 21 Feb 2025
Whole Genome Transformer for Gene Interaction Effects in Microbiome Habitat Specificity Zhufeng Li S. S. Cranganore Nicholas D. Youngblut Niki Kilbertus 70 2 0 09 May 2024
Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction Jun Chen Ming Hu Boyang Albert Li Mohamed Elhoseiny 82 36 0 01 Jun 2022
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 133 3,707 0 09 Jan 2019
Variable Computation in Recurrent Neural Networks Yacine Jernite Edouard Grave Armand Joulin Tomas Mikolov 55 59 0 18 Nov 2016
Adaptive Computation Time for Recurrent Neural Networks Alex Graves 43 544 0 29 Mar 2016
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 307 7,951 0 17 Aug 2015