HiPPO: Recurrent Memory with Optimal Polynomial Projections

17 August 2020

Papers citing "HiPPO: Recurrent Memory with Optimal Polynomial Projections"

50 / 326 papers shown

Title
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 46 25 0 14 Mar 2024
SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces Yuta Oshima Shohei Taniguchi Masahiro Suzuki Yutaka Matsuo 45 7 0 12 Mar 2024
Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy Jiuming Liu Ruiji Yu Yian Wang Yu Zheng Tianchen Deng Weicai Ye Hesheng Wang 39 43 0 11 Mar 2024
Mastering Memory Tasks with World Models Mohammad Reza Samsami Artem Zholus Janarthanan Rajendran Sarath Chandar CLL OffRL 34 22 0 07 Mar 2024
Accelerating Greedy Coordinate Gradient via Probe Sampling Yiran Zhao Wenyue Zheng Tianle Cai Xuan Long Do Kenji Kawaguchi Anirudh Goyal Michael Shieh 51 2 0 02 Mar 2024
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 61 117 0 29 Feb 2024
Theoretical Foundations of Deep Selective State-Space Models Nicola Muca Cirone Antonio Orvieto Benjamin Walker C. Salvi Terry Lyons Mamba 61 25 0 29 Feb 2024
DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models Wei He Kai Han Yehui Tang Chengcheng Wang Yujie Yang Tianyu Guo Yunhe Wang Mamba 61 25 0 26 Feb 2024
Model Compression Method for S4 with Diagonal State Space Layers using Balanced Truncation Haruka Ezoe Kazuhiro Sato 33 0 0 25 Feb 2024
MambaIR: A Simple Baseline for Image Restoration with State-Space Model Hang Guo Jinmin Li Tao Dai Zhihao Ouyang Xudong Ren Shu-Tao Xia Mamba 58 203 0 23 Feb 2024
State Space Models for Event Cameras Nikola Zubić Mathias Gehrig Davide Scaramuzza 60 39 0 23 Feb 2024
Nonstationary Time Series Forecasting via Unknown Distribution Adaptation Zijian Li Ruichu Cai Zhenhui Yang Haiqin Huang Guan-Hong Chen Yifan Shen Zhengming Chen Xiangchen Song Kun Zhang OOD AI4TS 37 2 0 20 Feb 2024
Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective Jiaxi Hu Yuehong Hu Wei Chen Ming Jin Shirui Pan Qingsong Wen Keli Zhang AI4TS 37 9 0 18 Feb 2024
PointMamba: A Simple State Space Model for Point Cloud Analysis Dingkang Liang Xin Zhou Wei Xu Xingkui Zhu Zhikang Zou Xiaoqing Ye Xinyu Wang Xiang Bai 89 93 0 16 Feb 2024
Graph Mamba: Towards Learning on Graphs with State Space Models Ali Behrouz Farnoosh Hashemi AI4CE 112 59 0 13 Feb 2024
On the Resurgence of Recurrent Models for Long Sequences -- Survey and Research Opportunities in the Transformer Era Matteo Tiezzi Michele Casoni Alessandro Betti Tommaso Guidi Marco Gori S. Melacci 27 9 0 12 Feb 2024
Scalable Diffusion Models with State Space Backbone Zhengcong Fei Mingyuan Fan Changqian Yu Junshi Huang 72 34 0 08 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 41 28 0 05 Feb 2024
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models Xindi Wang Mahsa Salmani Parsa Omidi Xiangyu Ren Mehdi Rezagholizadeh A. Eshaghi LRM 34 35 0 03 Feb 2024
Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary Takashi Morita 24 3 0 31 Jan 2024
$HyperZ$\cdot$Z$\cdot$W Operator Connects Slow-Fast Networks for Full Context Interaction$ HyperZ $\cdot$ Z $\cdot$ W Operator Connects Slow-Fast Networks for Full Context Interaction Harvie Zhang 39 0 0 31 Jan 2024
LOCOST: State-Space Models for Long Document Abstractive Summarization Florian Le Bronnec Song Duong Mathieu Ravaut Alexandre Allauzen Nancy F. Chen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari 48 8 0 31 Jan 2024
LMUFormer: Low Complexity Yet Powerful Spiking Model With Legendre Memory Units Zeyu Liu Gourav Datta Anni Li P. Beerel 35 9 0 20 Jan 2024
U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation Jun Ma Feifei Li Bo Wang Mamba 82 338 0 09 Jan 2024
The Rise of Diffusion Models in Time-Series Forecasting Caspar Meijer Lydia Y. Chen DiffM AI4TS 32 9 0 05 Jan 2024
Learning Long Sequences in Spiking Neural Networks Matei Ioan Stan Oliver Rhodes 37 11 0 14 Dec 2023
Spectral State Space Models Naman Agarwal Daniel Suo Xinyi Chen Elad Hazan 30 12 0 11 Dec 2023
Structured state-space models are deep Wiener models Fabio Bonassi Carl R. Andersson Per Mattsson Thomas B. Schon 36 3 0 11 Dec 2023
TCNCA: Temporal Convolution Network with Chunked Attention for Scalable Sequence Processing Aleksandar Terzić Michael Hersche G. Karunaratne Zixiao Huang Abu Sebastian Abbas Rahimi AI4TS 22 1 0 09 Dec 2023
Recurrent Distance Filtering for Graph Representation Learning Yuhui Ding Antonio Orvieto Bobby He Thomas Hofmann GNN 36 6 0 03 Dec 2023
Diffusion Models Without Attention Jing Nathan Yan Jiatao Gu Alexander M. Rush 35 61 0 30 Nov 2023
On the Long Range Abilities of Transformers Itamar Zimerman Lior Wolf 35 7 0 28 Nov 2023
Stability-Informed Initialization of Neural Ordinary Differential Equations Theodor Westny Arman Mohammadi Daniel Jung Erik Frisk 28 0 0 27 Nov 2023
StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization Shida Wang Qianxiao Li 22 13 0 24 Nov 2023
Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions Stefano Massaroli Michael Poli Daniel Y. Fu Hermann Kumbong Rom N. Parnichkun ... Atri Rudra Ce Zhang Christopher Ré Stefano Ermon Yoshua Bengio 39 19 0 28 Oct 2023
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer Qingru Zhang Dhananjay Ram Cole Hawkins Sheng Zha Tuo Zhao 35 15 0 19 Oct 2023
Reinforcement Learning with Fast and Forgetful Memory Steven D. Morad Ryan Kortvelesy Stephan Liwicki Amanda Prorok OffRL 24 4 0 06 Oct 2023
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors Ido Amos Jonathan Berant Ankit Gupta 33 25 0 04 Oct 2023
Robustifying State-space Models for Long Sequences via Approximate Diagonalization Annan Yu Arnur Nigmetov Dmitriy Morozov Michael W. Mahoney N. Benjamin Erichson 32 12 0 02 Oct 2023
Scaling Representation Learning from Ubiquitous ECG with State-Space Models Kleanthis Avramidis Dominika Kunc Bartosz Perz Kranti Adsul Tiantian Feng Przemyslaw Kazienko Stanisław Saganowski Shrikanth Narayanan AI4TS 36 3 0 26 Sep 2023
State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory Shida Wang Beichen Xue 21 24 0 23 Sep 2023
Spiking Structured State Space Model for Monaural Speech Enhancement Yu Du Xu Liu Yansong Chua 27 16 0 07 Sep 2023
Reasonable Anomaly Detection in Long Sequences Yalong Jiang Changkang Li AI4TS 40 0 0 06 Sep 2023
Traveling Waves Encode the Recent Past and Enhance Sequence Learning Thomas Anderson Keller L. Muller T. Sejnowski Max Welling AI4TS 28 13 0 03 Sep 2023
Towards quantitative precision for ECG analysis: Leveraging state space models, self-supervision and patient metadata Temesgen Mehari Nils Strodthoff 26 17 0 29 Aug 2023
Contaminated Multivariate Time-Series Anomaly Detection with Spatio-Temporal Graph Conditional Diffusion Models Thi Kieu Khanh Ho Narges Armanfard AI4TS 50 1 0 24 Aug 2023
Generative AI for End-to-End Limit Order Book Modelling: A Token-Level Autoregressive Generative Model of Message Flow Using a Deep State Space Network Peer Nagy Sascha Frey Silvia Sapora Kang Li Anisoara Calinescu Stefan Zohren Jakob Foerster 19 16 0 23 Aug 2023
PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers Phillip Lippe Bastiaan S. Veeling P. Perdikaris Richard Turner Johannes Brandstetter DiffM AI4CE 33 78 0 10 Aug 2023
TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer Zhen Qin Dong Li Weigao Sun Weixuan Sun Xuyang Shen ... Yunshen Wei Baohong Lv Xiao Luo Yu Qiao Yiran Zhong 43 15 0 27 Jul 2023
Improve Long-term Memory Learning Through Rescaling the Error Temporally Shida Wang Zhanglu Yan 25 1 0 21 Jul 2023