Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies

2 October 2020

Papers citing "Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies"

50 / 55 papers shown

Title
Learning to Dissipate Energy in Oscillatory State-Space Models Jared Boyer T. Konstantin Rusch Daniela Rus 6 0 0 17 May 2025
Sentiment Analysis of Movie Reviews Using BERT Gibson Nkhata Usman Anjum J. Zhan 34 1 0 26 Feb 2025
On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning Alvaro Arroyo Alessio Gravina Benjamin Gutteridge Federico Barbero Claudio Gallicchio Xiaowen Dong Michael M. Bronstein P. Vandergheynst 42 3 0 15 Feb 2025
Traveling Waves Integrate Spatial Information Through Time Mozes Jacobs Roberto C. Budzinski L. Muller Demba E. Ba T. Anderson Keller 85 0 0 09 Feb 2025
Deep Loss Convexification for Learning Iterative Models Ziming Zhang Yuping Shao Yiqing Zhang Fangzhou Lin Haichong K. Zhang Elke Rundensteiner 3DPC 41 0 0 16 Nov 2024
Artificial Kuramoto Oscillatory Neurons Takeru Miyato Sindy Lowe Andreas Geiger Max Welling AI4CE 77 6 0 17 Oct 2024
Oscillatory State-Space Models T. Konstantin Rusch Daniela Rus AI4TS 162 6 0 04 Oct 2024
Unconditional stability of a recurrent neural circuit implementing divisive normalization Shivang Rawat David J. Heeger Stefano Martiniani 26 0 0 27 Sep 2024
Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation Maximilian Baronig Romain Ferrand Silvester Sabathiel Robert Legenstein 48 3 0 14 Aug 2024
Recurrent neural networks: vanishing and exploding gradients are not the end of the story Nicolas Zucchet Antonio Orvieto ODL AAML 45 9 0 31 May 2024
Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching Fernando Moreno-Pino Alvaro Arroyo H. Waldon Xiaowen Dong Álvaro Cartea AI4TS 40 1 0 31 May 2024
Subhomogeneous Deep Equilibrium Models Pietro Sittoni Francesco Tudisco 29 0 0 01 Mar 2024
Quantized Approximately Orthogonal Recurrent Neural Networks Armand Foucault Franck Mamalet Franccois Malgouyres MQ 29 1 0 05 Feb 2024
Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians Rainer Engelken 29 5 0 28 Dec 2023
Delving Deeper Into Astromorphic Transformers Md. Zesun Ahmed Mia Malyaban Bal Abhronil Sengupta 36 1 0 18 Dec 2023
StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization Shida Wang Qianxiao Li 22 13 0 24 Nov 2023
Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gate Pengfei Sun Jibin Wu Malu Zhang P. Devos Dick Botteldooren 27 5 0 23 Oct 2023
Locality-Aware Graph-Rewiring in GNNs Federico Barbero A. Velingker Amin Saberi Michael M. Bronstein Francesco Di Giovanni 40 27 0 02 Oct 2023
Traveling Waves Encode the Recent Past and Enhance Sequence Learning Thomas Anderson Keller L. Muller T. Sejnowski Max Welling AI4TS 28 13 0 03 Sep 2023
Neural oscillators for magnetic hysteresis modeling Abhishek Chandra Taniya Kapoor B. Daniels M. Curti K. Tiels D. Tartakovsky E. Lomonova 21 3 0 23 Aug 2023
Neural oscillators for generalization of physics-informed machine learning Taniya Kapoor Abhishek Chandra D. Tartakovsky Hongrui Wang Alfredo Núñez R. Dollevoet AI4CE 32 11 0 17 Aug 2023
Fading memory as inductive bias in residual recurrent networks I. Dubinin Felix Effenberger 43 4 0 27 Jul 2023
Improve Long-term Memory Learning Through Rescaling the Error Temporally Shida Wang Zhanglu Yan 25 1 0 21 Jul 2023
Trainability, Expressivity and Interpretability in Gated Neural ODEs T. Kim T. Can K. Krishnamurthy AI4CE 35 4 0 12 Jul 2023
Generating Oscillation Activity with Echo State Network to Mimic the Behavior of a Simple Central Pattern Generator Tham Yik Foong Danilo Vasconcellos Vargas 11 1 0 19 Jun 2023
Generalized Teacher Forcing for Learning Chaotic Dynamics Florian Hess Zahra Monfared Manuela Brenner Daniel Durstewitz AI4CE 27 30 0 07 Jun 2023
Neural Differential Recurrent Neural Network with Adaptive Time Steps Yixuan Tan Liyan Xie Xiuyuan Cheng AI4TS 11 5 0 02 Jun 2023
Neural Oscillators are Universal S. Lanthaler T. Konstantin Rusch Siddhartha Mishra 32 10 0 15 May 2023
Long-term Forecasting with TiDE: Time-series Dense Encoder Abhimanyu Das Weihao Kong Andrew B. Leach Shaan Mathur Rajat Sen Rose Yu AI4TS 53 239 0 17 Apr 2023
SMPConv: Self-moving Point Representations for Continuous Convolution Sanghyeon Kim Eunbyung Park 3DPC 39 13 0 05 Apr 2023
On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology Francesco Di Giovanni Lorenzo Giusti Federico Barbero Giulia Luise Pietro Lio Michael M. Bronstein 48 112 0 06 Feb 2023
Gated Recurrent Neural Networks with Weighted Time-Delay Feedback N. Benjamin Erichson S. H. Lim Michael W. Mahoney 26 6 0 01 Dec 2022
Flipped Classroom: Effective Teaching for Time Series Forecasting P. Teutsch Patrick Mäder AI4TS 23 8 0 17 Oct 2022
Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks Andrea Ceni ODL 25 3 0 03 Oct 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 107 96 0 26 Sep 2022
Sequence Learning Using Equilibrium Propagation Malyaban Bal Abhronil Sengupta 35 9 0 14 Sep 2022
Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems Manuela Brenner Florian Hess Jonas M. Mikhaeil Leonard Bereska Zahra Monfared Po-Chen Kuo Daniel Durstewitz AI4CE 37 29 0 06 Jul 2022
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time Anand Subramoney Khaleelulla Khan Nazeer Mark Schöne Christian Mayr David Kappel 35 16 0 13 Jun 2022
Graph-Coupled Oscillator Networks T. Konstantin Rusch B. Chamberlain J. Rowbottom S. Mishra M. Bronstein 42 105 0 04 Feb 2022
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers Albert Gu Isys Johnson Karan Goel Khaled Kamal Saab Tri Dao Atri Rudra Christopher Ré 54 553 0 26 Oct 2021
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes David W. Romero Robert-Jan Bruintjes Jakub M. Tomczak Erik J. Bekkers Mark Hoogendoorn Jan van Gemert 80 82 0 15 Oct 2021
On the difficulty of learning chaotic dynamics with RNNs Jonas M. Mikhaeil Zahra Monfared Daniel Durstewitz 59 51 0 14 Oct 2021
Long Expressive Memory for Sequence Modeling T. Konstantin Rusch Siddhartha Mishra N. Benjamin Erichson Michael W. Mahoney AI4TS 14 44 0 10 Oct 2021
Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processing Bing Han Cheng Wang Kaushik Roy 34 7 0 14 Sep 2021
Closed-form Continuous-time Neural Models Ramin Hasani Mathias Lechner Alexander Amini Lucas Liebenwein Aaron Ray Max Tschaikowski G. Teschl Daniela Rus PINN AI4TS 31 82 0 25 Jun 2021
Stateful ODE-Nets using Basis Function Expansions A. Queiruga N. Benjamin Erichson Liam Hodgkinson Michael W. Mahoney 27 16 0 21 Jun 2021
Causal Navigation by Continuous-time Neural Networks Charles J. Vorbach Ramin Hasani Alexander Amini Mathias Lechner Daniela Rus 26 47 0 15 Jun 2021
Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decay H. Chien Javier S. Turek Nicole M. Beckage Vy A. Vo C. Honey Ted Willke 14 15 0 12 May 2021
DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scales Brandon G. Jacques Zoran Tiganj Marc W Howard P. Sederberg 16 5 0 09 Apr 2021
UnICORNN: A recurrent model for learning very long time dependencies T. Konstantin Rusch Siddhartha Mishra 16 60 0 09 Mar 2021