Reformer: The Efficient Transformer

13 January 2020

Papers citing "Reformer: The Efficient Transformer"

50 / 505 papers shown

Title
Towards Accurate Post-Training Quantization for Vision Transformer Yifu Ding Haotong Qin Qing-Yu Yan Z. Chai Junjie Liu Xiaolin K. Wei Xianglong Liu MQ 54 69 0 25 Mar 2023
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan ViT 37 155 0 24 Mar 2023
Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion Bohan Li Yasheng Sun Zhujin Liang Dalong Du Zhengbiao Zhu Xiaoefeng Wang Yunpeng Zhang Han Xiao Wenjun Zeng 3DV 33 10 0 24 Mar 2023
XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages Dhaval Taunk Shivprasad Sagare Anupam Patil Shivansh Subramanian Manish Gupta Vasudeva Varma 25 3 0 22 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 46 47 0 21 Mar 2023
Discovering Predictable Latent Factors for Time Series Forecasting Jingyi Hou Zhen Dong Jiayu Zhou Zhijie Liu AI4TS BDL 35 1 0 18 Mar 2023
HDformer: A Higher Dimensional Transformer for Diabetes Detection Utilizing Long Range Vascular Signals Ella Lan MedIm 20 1 0 17 Mar 2023
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Bo He Jun Wang Jielin Qiu Trung Bui Abhinav Shrivastava Zhaowen Wang 22 66 0 13 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 38 509 0 07 Mar 2023
Gradient-Free Structured Pruning with Unlabeled Data Azade Nova H. Dai Dale Schuurmans SyDa 40 20 0 07 Mar 2023
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration Yawei Li Yuchen Fan Xiaoyu Xiang D. Demandolx Rakesh Ranjan Radu Timofte Luc Van Gool 35 173 0 01 Mar 2023
Single-Cell Multimodal Prediction via Transformers Wenzhuo Tang Haifang Wen Renming Liu Jiayuan Ding Wei Jin Yuying Xie Hui Liu Jiliang Tang AI4CE 29 11 0 01 Mar 2023
A Survey on Long Text Modeling with Transformers Zican Dong Tianyi Tang Lunyi Li Wayne Xin Zhao VLM 26 54 0 28 Feb 2023
Elementwise Language Representation Du-Yeong Kim Jeeeun Kim 38 0 0 27 Feb 2023
Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations Xun Zhu Yutong Xiong Ming Wu Gaozhen Nie Bin Zhang Ziheng Yang AI4TS 30 17 0 21 Feb 2023
Efficiency 360: Efficient Vision Transformers Badri N. Patro Vijay Srinivas Agneeswaran 33 6 0 16 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 38 10 0 13 Feb 2023
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 34 18 0 09 Feb 2023
Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis Akshita Jha Adithya Samavedhi Vineeth Rakesh J. Chandrashekar Chandan K. Reddy 19 0 0 07 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 39 8 0 03 Feb 2023
Mnemosyne: Learning to Train Transformers with Transformers Deepali Jain K. Choromanski Kumar Avinava Dubey Sumeet Singh Vikas Sindhwani Tingnan Zhang Jie Tan OffRL 44 9 0 02 Feb 2023
FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecasting Jianqing Zhu Juncai He Lian Zhang Jinchao Xu 39 3 0 02 Feb 2023
Transformers Meet Directed Graphs Simon Geisler Yujia Li D. Mankowitz A. Cemgil Stephan Günnemann Cosmin Paduraru 33 36 0 31 Jan 2023
Exploring Attention Map Reuse for Efficient Transformer Neural Networks Kyuhong Shim Jungwook Choi Wonyong Sung ViT 26 3 0 29 Jan 2023
A Comparative Study of Pretrained Language Models for Long Clinical Text Yikuan Li R. M. Wehbe F. Ahmad Hanyin Wang Yuan Luo LM&MA ELM VLM MedIm 29 79 0 27 Jan 2023
Open Problems in Applied Deep Learning M. Raissi AI4CE 53 2 0 26 Jan 2023
Understanding and Improving Deep Graph Neural Networks: A Probabilistic Graphical Model Perspective Jiayuan Chen Xiang Zhang Yinfei Xu Tianli Zhao Renjie Xie Wei Xu GNN BDL 26 0 0 25 Jan 2023
Effective End-to-End Vision Language Pretraining with Semantic Visual Loss Xiaofeng Yang Fayao Liu Guosheng Lin VLM 26 7 0 18 Jan 2023
Dynamic Grained Encoder for Vision Transformers Lin Song Songyang Zhang Songtao Liu Zeming Li Xuming He Hongbin Sun Jian Sun Nanning Zheng ViT 26 34 0 10 Jan 2023
Automating Nearest Neighbor Search Configuration with Constrained Optimization Philip Sun Ruiqi Guo Surinder Kumar 31 7 0 04 Jan 2023
Infomaxformer: Maximum Entropy Transformer for Long Time-Series Forecasting Problem Peiwang Tang Xianchao Zhang AI4TS 45 3 0 04 Jan 2023
Black-box language model explanation by context length probing Ondřej Cífka Antoine Liutkus MILM LRM 24 6 0 30 Dec 2022
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 78 372 0 28 Dec 2022
ORCA: A Challenging Benchmark for Arabic Language Understanding AbdelRahim Elmadany El Moatez Billah Nagoudi Muhammad Abdul-Mageed ELM 17 40 0 21 Dec 2022
JEMMA: An Extensible Java Dataset for ML4Code Applications Anjan Karmakar Miltiadis Allamanis Romain Robbes VLM 29 3 0 18 Dec 2022
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li Jiahao Yu Ce Zhang Gao Huang Yu Tong ViT 27 6 0 16 Dec 2022
First De-Trend then Attend: Rethinking Attention for Time-Series Forecasting Xiyuan Zhang Xiaoyong Jin Karthick Gopalswamy Gaurav Gupta Youngsuk Park Xingjian Shi Hongya Wang Danielle C. Maddix Yuyang Wang AI4TS 30 19 0 15 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 130 36 0 15 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 40 161 0 15 Dec 2022
Full Contextual Attention for Multi-resolution Transformers in Semantic Segmentation Loic Themyr Clément Rambour Nicolas Thome Toby Collins Alexandre Hostettler ViT 27 10 0 15 Dec 2022
Towards Better Long-range Time Series Forecasting using Generative Forecasting Shiyu Liu Rohan Ghosh Mehul Motani AI4TS 99 2 0 09 Dec 2022
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming Yang Fahad Shahbaz Khan MedIm 40 131 0 08 Dec 2022
Transformers for End-to-End InfoSec Tasks: A Feasibility Study Ethan M. Rudd Mohammad Saidur Rahman Philip Tully 30 5 0 05 Dec 2022
FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting Maowei Jiang Pengyu Zeng Kai-Ming Wang Huan Liu Wenbo Chen Haoran Liu AI4TS 35 50 0 02 Dec 2022
Deep neural network techniques for monaural speech enhancement: state of the art analysis P. Ochieng 37 21 0 01 Dec 2022
sEHR-CE: Language modelling of structured EHR data for efficient and generalizable patient cohort expansion Anna Munoz-Farre Harry Rose S. A. Cakiroglu 11 4 0 30 Nov 2022
FsaNet: Frequency Self-attention for Semantic Segmentation Fengyu Zhang Ashkan Panahi Guangjun Gao AI4TS 32 28 0 28 Nov 2022
Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification Yuteng Ye Hang Zhou Jiale Cai Chenxing Gao Youjia Zhang Junle Wang Qiang Hu Junqing Yu Wei Yang 31 6 0 27 Nov 2022
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao Song Ruizhe Zhang Danyang Zhuo 87 31 0 25 Nov 2022
MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention Wenyuan Zeng Meng Li Wenjie Xiong Tong Tong Wen-jie Lu Jin Tan Runsheng Wang Ru Huang 29 21 0 25 Nov 2022