You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli
Sampling

You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling

18 November 2021

Shailesh Acharya

Papers citing "You Only Sample (Almost) Once: Linear Cost Self-Attention Via Bernoulli Sampling"

9 / 9 papers shown

Title
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 38 4 0 07 Dec 2023
Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens Zhanpeng Zeng Cole Hawkins Min-Fong Hong Aston Zhang Nikolaos Pappas Vikas Singh Shuai Zheng 21 6 0 07 May 2023
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 43 9 0 14 Oct 2022
Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction Yue Cao Xiaojiang Zhou Jiaqi Feng Peihao Huang Yao Xiao Dayao Chen Sheng Chen 82 40 0 20 May 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 44 150 0 27 Apr 2022
Deep Unlearning via Randomized Conditionally Independent Hessians Ronak R. Mehta Sourav Pal Vikas Singh Sathya Ravi MU 27 81 0 15 Apr 2022
Flowformer: Linearizing Transformers with Conservation Flows Haixu Wu Jialong Wu Jiehui Xu Jianmin Wang Mingsheng Long 14 90 0 13 Feb 2022
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 288 2,017 0 28 Jul 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018