Sparse Binary Transformers for Multivariate Time Series Modeling

9 August 2023

Papers citing "Sparse Binary Transformers for Multivariate Time Series Modeling"

32 / 32 papers shown

Title
Explainable and Interpretable Forecasts on Non-Smooth Multivariate Time Series for Responsible Gameplay Hussain Jagirdar Rukma Talwadker Aditya Pareek Pulkit Agrawal Tridib Mukherjee AI4TS 188 2 0 03 Apr 2025
Energy-Efficient Transformer Inference: Optimization Strategies for Time Series Classification Arshia Kermani Ehsan Zeraatkar Habib Irani 132 3 0 23 Feb 2025
Randomly Initialized Subnetworks with Iterative Weight Recycling Matt Gorbett L. D. Whitley 68 4 0 28 Mar 2023
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting Tian Zhou Ziqing Ma Qingsong Wen Xue Wang Liang Sun Rong Jin AI4TS 256 1,458 0 30 Jan 2022
Sparse is Enough in Scaling Transformers Sebastian Jaszczur Aakanksha Chowdhery Afroz Mohiuddin Lukasz Kaiser Wojciech Gajewski Henryk Michalewski Jonni Kanerva MoE 69 102 0 24 Nov 2021
Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy Jiehui Xu Haixu Wu Jianmin Wang Mingsheng Long AI4TS 95 504 0 06 Oct 2021
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting Haixu Wu Jiehui Xu Jianmin Wang Mingsheng Long AI4TS 113 2,320 0 24 Jun 2021
EfficientNetV2: Smaller Models and Faster Training Mingxing Tan Quoc V. Le EgoV 134 2,730 0 01 Apr 2021
Gated Transformer Networks for Multivariate Time Series Classification Minghao Liu Shengqi Ren Siyuan Ma Jiahui Jiao Yizhou Chen Zhiguang Wang Wei Song AI4TS 62 137 0 26 Mar 2021
Pre-Trained Image Processing Transformer Hanting Chen Yunhe Wang Tianyu Guo Chang Xu Yiping Deng Zhenhua Liu Siwei Ma Chunjing Xu Chao Xu Wen Gao VLM ViT 147 1,681 0 01 Dec 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 186 1,604 0 30 Sep 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 170 1,128 0 14 Sep 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 138 1,194 0 30 Jun 2020
Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient Ankit Pensia Shashank Rajput Alliot Nagle Harit Vishwakarma Dimitris Papailiopoulos 60 104 0 14 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 219 1,719 0 08 Jun 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 187 4,105 0 10 Apr 2020
Binary Neural Networks: A Survey Haotong Qin Ruihao Gong Xianglong Liu Xiao Bai Jingkuan Song N. Sebe MQ 134 471 0 31 Mar 2020
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 283 1,054 0 06 Mar 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 112 276 0 03 Feb 2020
What's Hidden in a Randomly Weighted Neural Network? Vivek Ramanujan Mitchell Wortsman Aniruddha Kembhavi Ali Farhadi Mohammad Rastegari 66 362 0 29 Nov 2019
Self-attention for raw optical Satellite Time Series Classification M. Rußwurm Marco Körner AI4TS 70 267 0 23 Oct 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 113 1,872 0 23 Sep 2019
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting Shiyang Li Xiaoyong Jin Yao Xuan Xiyou Zhou Wenhu Chen Yu Wang Xifeng Yan AI4TS 125 1,429 0 29 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 136 1,919 0 23 Apr 2019
Unsupervised Scalable Representation Learning for Multivariate Time Series Jean-Yves Franceschi Aymeric Dieuleveut Martin Jaggi SSL AI4TS 59 456 0 30 Jan 2019
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 277 3,489 0 09 Mar 2018
Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding K. Hundman V. Constantinou Christopher Laporte Ian Colwell T. Söderström AI4TS 127 1,260 0 13 Feb 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 226 19,353 0 13 Jan 2018
LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection Pankaj Malhotra Anusha Ramakrishnan G. Anand Lovekesh Vig Puneet Agarwal Gautam M. Shroff AI4TS 91 959 0 01 Jul 2016
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi MQ 179 4,369 0 16 Mar 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 355 18,661 0 06 Feb 2015
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation Yoshua Bengio Nicholas Léonard Aaron Courville 398 3,158 0 15 Aug 2013