Reformer: The Efficient Transformer

13 January 2020

Papers citing "Reformer: The Efficient Transformer"

50 / 505 papers shown

Title
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining Hongwu Peng Shaoyi Huang Shiyang Chen Bingbing Li Tong Geng ... Weiwen Jiang Wujie Wen J. Bi Hang Liu Caiwen Ding 47 54 0 07 Aug 2022
Vision-Centric BEV Perception: A Survey Yuexin Ma Tai Wang Xuyang Bai Huitong Yang Yuenan Hou Yaming Wang Yu Qiao Ruigang Yang Tianyi Zhou Xinge Zhu 66 130 0 04 Aug 2022
SpanDrop: Simple and Effective Counterfactual Learning for Long Sequences Peng Qi Guangtao Wang Jing Huang 24 0 0 03 Aug 2022
Efficient Long-Text Understanding with Short-Text Models Maor Ivgi Uri Shaham Jonathan Berant VLM 38 76 0 01 Aug 2022
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization T. Nguyen Richard G. Baraniuk Robert M. Kirby Stanley J. Osher Bao Wang 42 9 0 01 Aug 2022
Neural Architecture Search on Efficient Transformers and Beyond Zexiang Liu Dong Li Kaiyue Lu Zhen Qin Weixuan Sun Jiacheng Xu Yiran Zhong 35 19 0 28 Jul 2022
Conditional DETR V2: Efficient Detection Transformer with Box Queries Xiaokang Chen Fangyun Wei Gang Zeng Jingdong Wang ViT 30 33 0 18 Jul 2022
QSAN: A Near-term Achievable Quantum Self-Attention Network Jinjing Shi Ren-Xin Zhao Wenxuan Wang Shenmin Zhang Xuelong Li 28 20 0 14 Jul 2022
DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation Songhua Liu Jingwen Ye Sucheng Ren Xinchao Wang 27 44 0 13 Jul 2022
Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation Yuhao Yang Chao Huang Lianghao Xia Keli Zhang Yanwei Yu Chenliang Li HAI 18 121 0 12 Jul 2022
HelixFold: An Efficient Implementation of AlphaFold2 using PaddlePaddle Guoxia Wang Xiaomin Fang Zhihua Wu Yiqun Liu Yang Xue Yingfei Xiang Dianhai Yu Fan Wang Yanjun Ma 36 31 0 12 Jul 2022
Horizontal and Vertical Attention in Transformers Litao Yu Jing Zhang ViT 25 1 0 10 Jul 2022
Deformable Graph Transformer Jinyoung Park Seongjun Yun Hyeon-ju Park Jaewoo Kang Jisu Jeong KyungHyun Kim Jung-Woo Ha Hyunwoo J. Kim 93 7 0 29 Jun 2022
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network Vitaliy Chiley Vithursan Thangarasa Abhay Gupta Anshul Samar Joel Hestness D. DeCoste 52 8 0 28 Jun 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 39 232 0 27 Jun 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 39 32 0 19 Jun 2022
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation Qihang Yu Huiyu Wang Dahun Kim Siyuan Qiao Maxwell D. Collins Yukun Zhu Hartwig Adam Alan Yuille Liang-Chieh Chen ViT MedIm 32 90 0 17 Jun 2022
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning Yi-Lin Sung Jaemin Cho Joey Tianyi Zhou VLM 21 237 0 13 Jun 2022
DRAformer: Differentially Reconstructed Attention Transformer for Time-Series Forecasting Benhan Li Shengdong Du Tianrui Li Jie Hu Zhenglong Jia AI4TS 19 2 0 11 Jun 2022
Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting Amin Shabani A. Abdi Li Meng Tristan Sylvain AI4TS 27 61 0 08 Jun 2022
Separable Self-attention for Mobile Vision Transformers Sachin Mehta Mohammad Rastegari ViT MQ 34 253 0 06 Jun 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 26 348 0 02 Jun 2022
Dynamic Linear Transformer for 3D Biomedical Image Segmentation Zheyu Zhang Ulas Bagci ViT MedIm 28 12 0 01 Jun 2022
Chefs' Random Tables: Non-Trigonometric Random Features Valerii Likhosherstov K. Choromanski Kumar Avinava Dubey Frederick Liu Tamás Sarlós Adrian Weller 38 17 0 30 May 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 121 17 0 30 May 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 116 2,055 0 27 May 2022
Are Transformers Effective for Time Series Forecasting? Ailing Zeng Mu-Hwa Chen L. Zhang Qiang Xu AI4TS 90 1,620 0 26 May 2022
VTP: Volumetric Transformer for Multi-view Multi-person 3D Pose Estimation Yuxing Chen Renshu Gu Ouhan Huang Gangyong Jia 3DH 49 11 0 25 May 2022
Leveraging Locality in Abstractive Text Summarization Yixin Liu Ansong Ni Linyong Nan Budhaditya Deb Chenguang Zhu Ahmed Hassan Awadallah Dragomir R. Radev 35 18 0 25 May 2022
Recipe for a General, Powerful, Scalable Graph Transformer Ladislav Rampášek Mikhail Galkin Vijay Prakash Dwivedi A. Luu Guy Wolf Dominique Beaini 78 522 0 25 May 2022
FreDo: Frequency Domain-based Long-Term Time Series Forecasting Fan-Keng Sun Duane S. Boning AI4TS 53 11 0 24 May 2022
ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions Difan Liu Sandesh Shetty Tobias Hinz Matthew Fisher Richard Y. Zhang Taesung Park E. Kalogerakis ViT 32 30 0 24 May 2022
Dynamic Query Selection for Fast Visual Perceiver Corentin Dancette Matthieu Cord 36 1 0 22 May 2022
Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction Yue Cao Xiaojiang Zhou Jiaqi Feng Peihao Huang Yao Xiao Dayao Chen Sheng Chen 82 40 0 20 May 2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers Arda Sahiner Tolga Ergen Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 43 33 0 17 May 2022
Transkimmer: Transformer Learns to Layer-wise Skim Yue Guan Zhengyi Li Jingwen Leng Zhouhan Lin Minyi Guo 80 38 0 15 May 2022
LayoutBERT: Masked Language Layout Model for Object Insertion Kerem Turgutlu Sanatan Sharma J. Kumar VLM DiffM 35 2 0 30 Apr 2022
Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Version Razvan-Gabriel Cirstea Chenjuan Guo B. Yang Tung Kieu Xuanyi Dong Shirui Pan AI4TS 37 108 0 28 Apr 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 49 150 0 27 Apr 2022
ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching Yanxing Shi Junxiong Cai Yoli Shavit Tai-Jiang Mu Wensen Feng Kai Zhang GNN 27 77 0 25 Apr 2022
Paramixer: Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product Self-Attention Tong Yu Ruslan Khalitov Lei Cheng Zhirong Yang MoE 27 10 0 22 Apr 2022
Efficient Linear Attention for Fast and Accurate Keypoint Matching Suwichaya Suwanwimolkul S. Komorita 3DPC 3DV 22 11 0 16 Apr 2022
Revisiting Transformer-based Models for Long Document Classification Xiang Dai Ilias Chalkidis S. Darkner Desmond Elliott VLM 25 68 0 14 Apr 2022
Malceiver: Perceiver with Hierarchical and Multi-modal Features for Android Malware Detection Niall McLaughlin 32 2 0 12 Apr 2022
A Call for Clarity in Beam Search: How It Works and When It Stops Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Dragomir R. Radev Yejin Choi Noah A. Smith 28 6 0 11 Apr 2022
Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models Patrick Huber Giuseppe Carenini 20 11 0 08 Apr 2022
Few-Shot Forecasting of Time-Series with Heterogeneous Channels L. Brinkmeyer Rafael Rêgo Drumond Johannes Burchert Lars Schmidt-Thieme AI4TS 28 7 0 07 Apr 2022
Accelerating Attention through Gradient-Based Learned Runtime Pruning Zheng Li Soroush Ghodrati Amir Yazdanbakhsh H. Esmaeilzadeh Mingu Kang 27 17 0 07 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 148 6,035 0 05 Apr 2022
Long Movie Clip Classification with State-Space Video Models Md. Mohaiminul Islam Gedas Bertasius VLM 51 102 0 04 Apr 2022