Efficiently Modeling Long Sequences with Structured State Spaces

31 October 2021

Papers citing "Efficiently Modeling Long Sequences with Structured State Spaces"

50 / 1,149 papers shown

Title
Infrastructure-based End-to-End Learning and Prevention of Driver Failure Noam Buckman Shiva Sreeram Mathias Lechner Yutong Ban Ramin Hasani S. Karaman Daniela Rus 32 2 0 21 Mar 2023
What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement Yotam Alexander Nimrod De La Vega Noam Razin Nadav Cohen 32 4 0 20 Mar 2023
Effectively Modeling Time Series with Simple Discrete State Spaces Michael Zhang Khaled Kamal Saab Michael Poli Tri Dao Karan Goel Christopher Ré AI4TS 30 45 0 16 Mar 2023
Transcription free filler word detection with Neural semi-CRFs Ge Zhu Yujia Yan Juan-Pablo Caceres Z. Duan 32 3 0 11 Mar 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 88 271 0 11 Mar 2023
TSMixer: An All-MLP Architecture for Time Series Forecasting Si-An Chen Chun-Liang Li Nate Yoder Sercan O. Arik Tomas Pfister AI4TS 36 159 0 10 Mar 2023
Structured State Space Models for In-Context Reinforcement Learning Chris Xiaoxuan Lu Yannick Schroecker Albert Gu Emilio Parisotto Jakob N. Foerster Satinder Singh Feryal M. P. Behbahani AI4TS 102 84 0 07 Mar 2023
POPGym: Benchmarking Partially Observable Reinforcement Learning Steven D. Morad Ryan Kortvelesy Matteo Bettini Stephan Liwicki Amanda Prorok OffRL 32 38 0 03 Mar 2023
Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections E. Brouwer Rahul G. Krishnan AI4TS 27 0 0 03 Mar 2023
Diagonal State Space Augmented Transformers for Speech Recognition G. Saon Ankit Gupta Xiaodong Cui AI4TS 40 26 0 27 Feb 2023
One Fits All:Power General Time Series Analysis by Pretrained LM Tian Zhou Peisong Niu Xue Wang Liang Sun Rong Jin AI4TS 30 387 0 23 Feb 2023
Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli Stefano Massaroli Eric Q. Nguyen Daniel Y. Fu Tri Dao S. Baccus Yoshua Bengio Stefano Ermon Christopher Ré VLM 28 286 0 21 Feb 2023
A Neural PDE Solver with Temporal Stencil Modeling Zhiqing Sun Yiming Yang Shinjae Yoo DiffM AI4CE 32 14 0 16 Feb 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 25 52 0 13 Feb 2023
Continuous-time convolutions model of event sequences Vladislav Zhuzhel Vsevolod Grabar Galina Boeva Artem Zabolotnyi Alexander Stepikin ... Mikhail Orlov Ivan Kireev Evgeny Burnaev Rodrigo Rivera-Castro Alexey Zaytsev AI4TS 21 0 0 13 Feb 2023
A Unified View of Long-Sequence Models towards Modeling Million-Scale Dependencies Hongyu Hè Marko Kabić 35 2 0 13 Feb 2023
DNArch: Learning Convolutional Neural Architectures by Backpropagation David W. Romero Neil Zeghidour AI4CE 29 4 0 10 Feb 2023
In-Context Learning with Many Demonstration Examples Mukai Li Shansan Gong Jiangtao Feng Yiheng Xu Jinchao Zhang Zhiyong Wu Lingpeng Kong 42 34 0 09 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 42 8 0 03 Feb 2023
Mnemosyne: Learning to Train Transformers with Transformers Deepali Jain K. Choromanski Kumar Avinava Dubey Sumeet Singh Vikas Sindhwani Tingnan Zhang Jie Tan OffRL 49 9 0 02 Feb 2023
Learning PDE Solution Operator for Continuous Modeling of Time-Series Yesom Park Jaemoo Choi Changyeon Yoon Changhoon Song Myung-joo Kang AI4TS AI4CE 27 3 0 02 Feb 2023
Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks Antoine Louis Gijs van Dijck Gerasimos Spanakis AILaw 28 9 0 30 Jan 2023
Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series Abdul Fatir Ansari Alvin Heng Andre Lim Harold Soh BDL AI4TS 36 15 0 26 Jan 2023
Modelling Long Range Dependencies in $N$ D: From Task-Specific to a General Purpose CNN David M. Knigge David W. Romero Albert Gu E. Gavves Erik J. Bekkers Jakub M. Tomczak Mark Hoogendoorn Jan-Jakob Sonke 3DV 40 21 0 25 Jan 2023
Diffusion-based Conditional ECG Generation with Structured State Space Models Juan Miguel Lopez Alcaraz Nils Strodthoff DiffM 43 48 0 19 Jan 2023
Rock Guitar Tablature Generation via Natural Language Processing Josue Casco-Rodriguez 45 1 0 12 Jan 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 33 53 0 05 Jan 2023
Is word segmentation necessary for Vietnamese sentiment classification? Duc-Vu Nguyen Ngan Luu-Thuy Nguyen 32 0 0 01 Jan 2023
Efficient Movie Scene Detection using State-Space Transformers Md. Mohaiminul Islam Mahmudul Hasan Kishan Athrey Tony Braskich Gedas Bertasius ViT 44 44 0 29 Dec 2022
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 78 372 0 28 Dec 2022
Deep Latent State Space Models for Time-Series Generation Linqi Zhou Michael Poli Winnie Xu Stefano Massaroli Stefano Ermon BDL AI4TS 14 34 0 24 Dec 2022
Pretraining Without Attention Junxiong Wang J. Yan Albert Gu Alexander M. Rush 27 48 0 20 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 130 36 0 15 Dec 2022
Simplifying and Understanding State Space Models with Diagonal Linear RNNs Ankit Gupta Harsh Mehta Jonathan Berant 29 21 0 01 Dec 2022
Gated Recurrent Neural Networks with Weighted Time-Delay Feedback N. Benjamin Erichson Soon Hoe Lim Michael W. Mahoney 36 6 0 01 Dec 2022
Predicting Properties of Quantum Systems with Conditional Generative Models Haoxiang Wang Maurice Weber J. Izaac Cedric Yen-Yu Lin AI4CE 39 9 0 30 Nov 2022
DBA: Efficient Transformer with Dynamic Bilinear Low-Rank Attention Bosheng Qin Juncheng Li Siliang Tang Yueting Zhuang 25 2 0 24 Nov 2022
Modeling Multivariate Biosignals With Graph Neural Networks and Structured State Space Models Siyi Tang Jared A. Dunnmon Liangqiong Qu Khaled Kamal Saab T. Baykaner Christopher Lee-Messer D. Rubin 35 21 0 21 Nov 2022
UniHPF : Universal Healthcare Predictive Framework with Zero Domain Knowledge Kyunghoon Hur Jungwoo Oh Junu Kim Jiyoun Kim Min Jae Lee Eunbyeol Cho Seong-Eun Moon Young-Hak Kim Edward Choi 40 5 0 15 Nov 2022
Advancing the State-of-the-Art for ECG Analysis through Structured State Space Models Temesgen Mehari Nils Strodthoff 29 11 0 14 Nov 2022
QuadConv: Quadrature-Based Convolutions with Applications to Non-Uniform PDE Data Compression Kevin Doherty Cooper Simpson Stephen Becker Alireza Doostan 44 7 0 09 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 205 2,327 0 09 Nov 2022
Circling Back to Recurrent Models of Language Gábor Melis 42 0 0 03 Nov 2022
Structured State Space Decoder for Speech Recognition and Synthesis Koichi Miyazaki Masato Murata Tomoki Koriyama 39 12 0 31 Oct 2022
Learning Modular Simulations for Homogeneous Systems Jayesh K. Gupta Sai H. Vemprala Ashish Kapoor 25 6 0 28 Oct 2022
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets Edo Cohen-Karlik Itamar Menuhin-Gruman Raja Giryes Nadav Cohen Amir Globerson 34 4 0 25 Oct 2022
Deep Equilibrium Approaches to Diffusion Models Ashwini Pokle Zhengyang Geng Zico Kolter DiffM 35 39 0 23 Oct 2022
BEANS: The Benchmark of Animal Sounds Masato Hagiwara Benjamin Hoffman Jen-Yu Liu M. Cusimano Felix Effenberger Katie Zacarian 53 26 0 21 Oct 2022
What Makes Convolutional Models Great on Long Sequence Modeling? Yuhong Li Tianle Cai Yi Zhang De-huai Chen Debadeepta Dey VLM 39 96 0 17 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 49 9 0 14 Oct 2022