On the Anatomy of Attention

On the Anatomy of Attention

2 July 2024

Tuomas Laakkonen

Vincent Wang-Ma'scianica

Papers citing "On the Anatomy of Attention"

11 / 11 papers shown

Title
A Pattern Language for Machine Learning Tasks Benjamin Rodatz Ian Fan Tuomas Laakkonen Neil John Ortega Thomas Hoffman Vincent Wang-Ma'scianica 66 3 0 02 Jul 2024
Distilling Text into Circuits Vincent Wang-Ma'scianica Jonathon Liu B. Coecke 49 11 0 25 Jan 2023
De Finetti's Theorem in Categorical Probability Tobias Fritz Tomáš Gonda Paolo Perrone 24 32 0 06 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 374 2,638 0 04 May 2021
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 113 1,548 0 30 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 455 2,051 0 28 Jul 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 141 1,678 0 08 Jun 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 63 3,996 0 10 Apr 2020
Star-Transformer Qipeng Guo Xipeng Qiu Pengfei Liu Yunfan Shao Xiangyang Xue Zheng Zhang 48 262 0 25 Feb 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 107 3,707 0 09 Jan 2019
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 331 27,205 0 01 Sep 2014