Efficiently Modeling Long Sequences with Structured State Spaces

31 October 2021

Papers citing "Efficiently Modeling Long Sequences with Structured State Spaces"

45 / 1,145 papers shown

Title
Temporally Consistent Transformers for Video Generation Wilson Yan Danijar Hafner Stephen James Pieter Abbeel DiffM 27 28 0 05 Oct 2022
TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis Haixu Wu Teng Hu Yong Liu Hang Zhou Jianmin Wang Mingsheng Long AI4TS AIFin 61 715 0 05 Oct 2022
WavSpA: Wavelet Space Attention for Boosting Transformers' Long Sequence Learning Ability Yufan Zhuang Zihan Wang Fangbo Tao Jingbo Shang ViT AI4TS 37 3 0 05 Oct 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 107 98 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 49 22 0 22 Sep 2022
Mega: Moving Average Equipped Gated Attention Xuezhe Ma Chunting Zhou Xiang Kong Junxian He Liangke Gui Graham Neubig Jonathan May Luke Zettlemoyer 38 183 0 21 Sep 2022
Adapting Pretrained Text-to-Text Models for Long Text Sequences Wenhan Xiong Anchit Gupta Shubham Toshniwal Yashar Mehdad Wen-tau Yih RALM VLM 62 30 0 21 Sep 2022
Extend and Explain: Interpreting Very Long Language Models Joel Stremmel B. Hill Jeffrey S. Hertzberg Jaime Murillo Llewelyn Allotey Eran Halperin 17 4 0 02 Sep 2022
Diffusion Models: A Comprehensive Survey of Methods and Applications Ling Yang Zhilong Zhang Yingxia Shao Shenda Hong Runsheng Xu Yue Zhao Wentao Zhang Bin Cui Ming-Hsuan Yang DiffM MedIm 226 1,320 0 02 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 40 109 0 31 Aug 2022
Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models Juan Miguel Lopez Alcaraz Nils Strodthoff DiffM 28 168 0 19 Aug 2022
Treeformer: Dense Gradient Trees for Efficient Attention Computation Lovish Madaan Srinadh Bhojanapalli Himanshu Jain Prateek Jain 35 6 0 18 Aug 2022
fMRI-S4: learning short- and long-range dynamic fMRI dependencies using 1D Convolutions and State Space Models A. E. Gazzar R. Thomas G. Wingen 29 3 0 08 Aug 2022
Efficient Long-Text Understanding with Short-Text Models Maor Ivgi Uri Shaham Jonathan Berant VLM 40 76 0 01 Aug 2022
GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning Kyunghoon Hur Jungwoo Oh Junu Kim Jiyoun Kim Min Jae Lee Eunbyeol Choi Seong-Eun Moon Young-Hak Kim Louis Atallah Edward Choi AI4TS 22 22 0 20 Jul 2022
Markovian Gaussian Process Variational Autoencoders Harrison Zhu Carles Balsells Rodas Yingzhen Li BDL AI4TS 51 15 0 12 Jul 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 39 232 0 27 Jun 2022
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections Albert Gu Isys Johnson Aman Timalsina Atri Rudra Christopher Ré Mamba 106 90 0 24 Jun 2022
On the Parameterization and Initialization of Diagonal State Space Models Albert Gu Ankit Gupta Karan Goel Christopher Ré 25 300 0 23 Jun 2022
Adversarial Audio Synthesis with Complex-valued Polynomial Networks Yongtao Wu Grigorios G. Chrysos V. Cevher DiffM 27 4 0 14 Jun 2022
ChordMixer: A Scalable Neural Attention Model for Sequences with Different Lengths Ruslan Khalitov Tong Yu Lei Cheng Zhirong Yang 33 12 0 12 Jun 2022
Towards a General Purpose CNN for Long Range Dependencies in $N$ D David W. Romero David M. Knigge Albert Gu Erik J. Bekkers E. Gavves Jakub M. Tomczak Mark Hoogendoorn 26 19 0 07 Jun 2022
Improving the Diagnosis of Psychiatric Disorders with Self-Supervised Graph State Space Models A. E. Gazzar R. Thomas G. Wingen AI4MH 18 6 0 07 Jun 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 19 5 0 02 Jun 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 116 2,061 0 27 May 2022
Recognition Models to Learn Dynamics from Partial Observations with Neural ODEs Mona Buisson-Fenet V. Morgenthaler Sebastian Trimpe F. D. Meglio 51 6 0 25 May 2022
Recipe for a General, Powerful, Scalable Graph Transformer Ladislav Rampášek Mikhail Galkin Vijay Prakash Dwivedi Anh Tuan Luu Guy Wolf Dominique Beaini 83 527 0 25 May 2022
Realization Theory Of Recurrent Neural ODEs Using Polynomial System Embeddings Martin Gonzalez Thibault Defourneau H. Hajri Mihaly Petreczky 31 2 0 24 May 2022
FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting Tian Zhou Ziqing Ma Xue Wang Qingsong Wen Liang Sun Tao Yao Wotao Yin Rong Jin AI4TS 121 171 0 18 May 2022
Long Movie Clip Classification with State-Space Video Models Md. Mohaiminul Islam Gedas Bertasius VLM 56 102 0 04 Apr 2022
Path Development Network with Finite-dimensional Lie Group Representation Han Lou Siran Li Hao Ni 23 7 0 02 Apr 2022
Diagonal State Spaces are as Effective as Structured State Spaces Ankit Gupta Albert Gu Jonathan Berant 59 293 0 27 Mar 2022
Continuous-Time Audiovisual Fusion with Recurrence vs. Attention for In-The-Wild Affect Recognition Vincent Karas M. Tellamekala Adria Mallol-Ragolta M. Valstar Björn W. Schuller 30 13 0 24 Mar 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 54 32 0 10 Mar 2022
It's Raw! Audio Generation with State-Space Models Karan Goel Albert Gu Chris Donahue Christopher Ré 28 187 0 20 Feb 2022
General-purpose, long-context autoregressive modeling with Perceiver AR Curtis Hawthorne Andrew Jaegle Cătălina Cangea Sebastian Borgeaud C. Nash ... Hannah R. Sheahan Neil Zeghidour Jean-Baptiste Alayrac João Carreira Jesse Engel 43 65 0 15 Feb 2022
Transferable Time-Series Forecasting under Causal Conditional Shift Zijian Li Ruichu Cai T. Fu Zhifeng Hao Kun Zhang TTA OOD AI4TS 28 23 0 05 Nov 2021
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes David W. Romero Robert-Jan Bruintjes Jakub M. Tomczak Erik J. Bekkers Mark Hoogendoorn Jan van Gemert 80 82 0 15 Oct 2021
Networked Time Series Prediction with Incomplete Data via Generative Adversarial Network Yichen Zhu Bo Jiang Haiming Jin Mengtian Zhang Feng Gao Jianqiang Huang Tao Lin Xinbing Wang GNN AI4TS 42 5 0 05 Oct 2021
Closed-form Continuous-time Neural Models Ramin Hasani Mathias Lechner Alexander Amini Lucas Liebenwein Aaron Ray Max Tschaikowski G. Teschl Daniela Rus PINN AI4TS 36 85 0 25 Jun 2021
RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks L. Kozachkov Michaela Ennis Jean-Jacques E. Slotine 24 18 0 16 Jun 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 304 2,611 0 04 May 2021
Creativity and Machine Learning: A Survey Giorgio Franceschelli Mirco Musolesi VLM AI4CE 34 40 0 06 Apr 2021
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 171 3,921 0 14 Dec 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 116 1,104 0 14 Sep 2020