HiPPO: Recurrent Memory with Optimal Polynomial Projections

17 August 2020

Papers citing "HiPPO: Recurrent Memory with Optimal Polynomial Projections"

50 / 326 papers shown

Title
Facing Off World Model Backbones: RNNs, Transformers, and S4 Fei Deng Junyeong Park Sungjin Ahn 32 24 0 05 Jul 2023
Structured State Space Models for Multiple Instance Learning in Digital Pathology Leo Fillioux Joseph Boyd Maria Vakalopoulou P. Cournède Stergios Christodoulidis 13 21 0 27 Jun 2023
Efficient Dynamics Modeling in Interactive Environments with Koopman Theory Arnab Kumar Mondal Siba Smarak Panigrahi Sai Rajeswar K. Siddiqi Siamak Ravanbakhsh 34 3 0 20 Jun 2023
Sparse Modular Activation for Efficient Sequence Modeling Liliang Ren Yang Liu Shuohang Wang Yichong Xu Chenguang Zhu Chengxiang Zhai 53 13 0 19 Jun 2023
Block-State Transformers Mahan Fathi Jonathan Pilault Orhan Firat C. Pal Pierre-Luc Bacon Ross Goroshin 42 17 0 15 Jun 2023
GCformer: An Efficient Framework for Accurate and Scalable Long-Term Multivariate Time Series Forecasting Yanjun Zhao Ziqing Ma Tian Zhou Liang Sun M. Ye Yi Qian AI4TS 32 19 0 14 Jun 2023
Decision S4: Efficient Sequence-Based RL via State Spaces Layers Shmuel Bar-David Itamar Zimerman Eliya Nachmani Lior Wolf OffRL 23 27 0 08 Jun 2023
Recasting Self-Attention with Holographic Reduced Representations Mohammad Mahmudul Alam Edward Raff Stella Biderman Tim Oates James Holt 8 8 0 31 May 2023
Blockwise Parallel Transformer for Large Context Models Hao Liu Pieter Abbeel 49 11 0 30 May 2023
Inverse Approximation Theory for Nonlinear Recurrent Neural Networks Shida Wang Zhong Li Qianxiao Li 22 7 0 30 May 2023
A Neural State-Space Model Approach to Efficient Speech Separation Chen Chen Chao-Han Huck Yang Kai Li Yuchen Hu Pin-Jui Ku Chng Eng Siong 34 11 0 26 May 2023
Online learning of long-range dependencies Nicolas Zucchet Robert Meier Simon Schug Asier Mujika João Sacramento CLL 52 18 0 25 May 2023
Neural Machine Translation for Code Generation K. Dharma Clayton T. Morrison 32 4 0 22 May 2023
Multi-Head State Space Model for Speech Recognition Yassir Fathullah Chunyang Wu Yuan Shangguan Junteng Jia Wenhan Xiong ... Chunxi Liu Yangyang Shi Ozlem Kalinli M. Seltzer Mark Gales 34 13 0 21 May 2023
Neural Oscillators are Universal S. Lanthaler T. Konstantin Rusch Siddhartha Mishra 35 10 0 15 May 2023
Sequence Modeling with Multiresolution Convolutional Memory Jiaxin Shi Ke Alexander Wang E. Fox 42 13 0 02 May 2023
State Spaces Aren't Enough: Machine Translation Needs Attention Ali Vardasbi Telmo Pires Robin M. Schmidt Stephan Peitz 27 9 0 25 Apr 2023
SMPConv: Self-moving Point Representations for Continuous Convolution Sanghyeon Kim Eunbyung Park 3DPC 42 13 0 05 Apr 2023
Selective Structured State-Spaces for Long-Form Video Understanding Jue Wang Wenjie Zhu Pichao Wang Xiang Yu Linda Liu Mohamed Omar Raffay Hamid 41 94 0 25 Mar 2023
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 88 268 0 11 Mar 2023
Structured State Space Models for In-Context Reinforcement Learning Chris Xiaoxuan Lu Yannick Schroecker Albert Gu Emilio Parisotto Jakob N. Foerster Satinder Singh Feryal M. P. Behbahani AI4TS 97 83 0 07 Mar 2023
Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections E. Brouwer Rahul G. Krishnan AI4TS 22 0 0 03 Mar 2023
Diagonal State Space Augmented Transformers for Speech Recognition G. Saon Ankit Gupta Xiaodong Cui AI4TS 35 26 0 27 Feb 2023
Hyena Hierarchy: Towards Larger Convolutional Language Models Michael Poli Stefano Massaroli Eric Q. Nguyen Daniel Y. Fu Tri Dao S. Baccus Yoshua Bengio Stefano Ermon Christopher Ré VLM 28 285 0 21 Feb 2023
A Neural PDE Solver with Temporal Stencil Modeling Zhiqing Sun Yiming Yang Shinjae Yoo DiffM AI4CE 24 14 0 16 Feb 2023
Simple Hardware-Efficient Long Convolutions for Sequence Modeling Daniel Y. Fu Elliot L. Epstein Eric N. D. Nguyen A. Thomas Michael Zhang Tri Dao Atri Rudra Christopher Ré 16 52 0 13 Feb 2023
A Unified View of Long-Sequence Models towards Modeling Million-Scale Dependencies Hongyu Hè Marko Kabić 25 2 0 13 Feb 2023
Scaling Up Computer Vision Neural Networks Using Fast Fourier Transform Siddharth Agrawal 15 0 0 02 Feb 2023
Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series Abdul Fatir Ansari Alvin Heng Andre Lim Harold Soh BDL AI4TS 36 15 0 26 Jan 2023
Diffusion-based Conditional ECG Generation with Structured State Space Models Juan Miguel Lopez Alcaraz Nils Strodthoff DiffM 38 48 0 19 Jan 2023
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 73 371 0 28 Dec 2022
Deep Latent State Space Models for Time-Series Generation Linqi Zhou Michael Poli Winnie Xu Stefano Massaroli Stefano Ermon BDL AI4TS 11 34 0 24 Dec 2022
Pretraining Without Attention Junxiong Wang J. Yan Albert Gu Alexander M. Rush 27 48 0 20 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 125 36 0 15 Dec 2022
Simplifying and Understanding State Space Models with Diagonal Linear RNNs Ankit Gupta Harsh Mehta Jonathan Berant 29 21 0 01 Dec 2022
Modeling Multivariate Biosignals With Graph Neural Networks and Structured State Space Models Siyi Tang Jared A. Dunnmon Liangqiong Qu Khaled Kamal Saab T. Baykaner Christopher Lee-Messer D. Rubin 30 21 0 21 Nov 2022
Advancing the State-of-the-Art for ECG Analysis through Structured State Space Models Temesgen Mehari Nils Strodthoff 26 11 0 14 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 118 2,315 0 09 Nov 2022
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets Edo Cohen-Karlik Itamar Menuhin-Gruman Raja Giryes Nadav Cohen Amir Globerson 27 4 0 25 Oct 2022
What Makes Convolutional Models Great on Long Sequence Modeling? Yuhong Li Tianle Cai Yi Zhang De-huai Chen Debadeepta Dey VLM 39 96 0 17 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 43 9 0 14 Oct 2022
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces Eric N. D. Nguyen Karan Goel Albert Gu Gordon W. Downs Preey Shah Tri Dao S. Baccus Christopher Ré VLM 22 39 0 12 Oct 2022
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks Kentaro Ohno Sekitoshi Kanai Yasutoshi Ida 23 0 0 04 Oct 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 107 96 0 26 Sep 2022
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases James Harrison Luke Metz Jascha Narain Sohl-Dickstein 47 22 0 22 Sep 2022
Mega: Moving Average Equipped Gated Attention Xuezhe Ma Chunting Zhou Xiang Kong Junxian He Liangke Gui Graham Neubig Jonathan May Luke Zettlemoyer 33 183 0 21 Sep 2022
Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models Juan Miguel Lopez Alcaraz Nils Strodthoff DiffM 28 168 0 19 Aug 2022
fMRI-S4: learning short- and long-range dynamic fMRI dependencies using 1D Convolutions and State Space Models A. E. Gazzar R. Thomas G. Wingen 22 3 0 08 Aug 2022
Offline Reinforcement Learning at Multiple Frequencies Kaylee Burns Tianhe Yu Chelsea Finn Karol Hausman OffRL 33 6 0 26 Jul 2022
Long Range Language Modeling via Gated State Spaces Harsh Mehta Ankit Gupta Ashok Cutkosky Behnam Neyshabur Mamba 37 232 0 27 Jun 2022