On orthogonality and learning recurrent networks with long term dependencies

31 January 2017

Samuel Kadoury

Papers citing "On orthogonality and learning recurrent networks with long term dependencies"

46 / 46 papers shown

Title
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 74 0 0 28 Jan 2025
Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints Navid Shervani-Tabar Marzieh Alireza Mirhoseini Robert Rosenbaum AAML AI4CE 42 0 0 15 Aug 2024
Initializing Models with Larger Ones Zhiqiu Xu Yanjie Chen Kirill Vishniakov Yida Yin Zhiqiang Shen Trevor Darrell Lingjie Liu Zhuang Liu 33 17 0 30 Nov 2023
Auxiliary Losses for Learning Generalizable Concept-based Models Ivaxi Sheth Samira Ebrahimi Kahou 32 24 0 18 Nov 2023
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold Jun Chen Haishan Ye Mengmeng Wang Tianxin Huang Guangwen Dai Ivor W.Tsang Yong-Jin Liu 23 10 0 21 Aug 2023
Fading memory as inductive bias in residual recurrent networks I. Dubinin Felix Effenberger 43 4 0 27 Jul 2023
Stabilising and accelerating light gated recurrent units for automatic speech recognition Adel Moumen Titouan Parcollet 26 3 0 16 Feb 2023
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold Yanhong Fei Xian Wei Yingjie Liu Zhengyu Li Mingsong Chen 28 6 0 16 Feb 2023
Convolutional unitary or orthogonal recurrent neural networks M. Magnasco 14 2 0 14 Feb 2023
Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks Andrea Ceni ODL 23 3 0 03 Oct 2022
Entangled Residual Mappings Mathias Lechner Ramin Hasani Z. Babaiee Radu Grosu Daniela Rus T. Henzinger Sepp Hochreiter 11 5 0 02 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 28 6 0 12 May 2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy Tianlong Chen Zhenyu (Allen) Zhang Yu Cheng Ahmed Hassan Awadallah Zhangyang Wang ViT 41 37 0 12 Mar 2022
Recency Dropout for Recurrent Recommender Systems Bo-Yu Chang Can Xu Matt Le Jingchen Feng Ya Le Sriraj Badam Ed H. Chi Minmin Chen 25 3 0 26 Jan 2022
Distributed neural network control with dependability guarantees: a compositional port-Hamiltonian approach Luca Furieri C. Galimberti M. Zakwan Giancarlo Ferrari-Trecate 29 20 0 16 Dec 2021
Stable Long-Term Recurrent Video Super-Resolution Benjamin Naoto Chiche Arnaud Woiselle J. Frontera-Pons Jean-Luc Starck SupR 30 6 0 16 Dec 2021
Learning Connectivity with Graph Convolutional Networks for Skeleton-based Action Recognition H. Sahbi GNN 23 27 0 06 Dec 2021
Exploiting a Zoo of Checkpoints for Unseen Tasks Jiaji Huang Qiang Qiu Kenneth Ward Church 27 4 0 05 Nov 2021
Heavy Ball Neural Ordinary Differential Equations Hedi Xia Vai Suliafu H. Ji T. Nguyen Andrea L. Bertozzi Stanley J. Osher Bao Wang 38 56 0 10 Oct 2021
Orthogonal Graph Neural Networks Kai Guo Kaixiong Zhou Xia Hu Yu Li Yi Chang Xin Wang 43 34 0 23 Sep 2021
Acceleration Method for Learning Fine-Layered Optical Neural Networks K. Aoyama H. Sawada 27 1 0 01 Sep 2021
Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity Ryan Henderson Djork-Arné Clevert F. Montanari 33 26 0 11 May 2021
Recurrent neural circuits for contour detection Drew Linsley Junkyung Kim A. Ashok Thomas Serre 24 45 0 29 Oct 2020
Recurrent Quantum Neural Networks Johannes Bausch 21 151 0 25 Jun 2020
Lipschitz Recurrent Neural Networks N. Benjamin Erichson Omri Azencot A. Queiruga Liam Hodgkinson Michael W. Mahoney 30 107 0 22 Jun 2020
Gradient Centralization: A New Optimization Technique for Deep Neural Networks Hongwei Yong Jianqiang Huang Xiansheng Hua Lei Zhang ODL 27 183 0 03 Apr 2020
Concept Whitening for Interpretable Image Recognition Zhi Chen Yijie Bei Cynthia Rudin FAtt 31 314 0 05 Feb 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 24 102 0 04 Feb 2020
Global Convergence of Gradient Descent for Deep Linear Residual Networks Lei Wu Qingcan Wang Chao Ma ODL AI4CE 28 22 0 02 Nov 2019
On Generalization Bounds of a Family of Recurrent Neural Networks Minshuo Chen Xingguo Li T. Zhao 19 70 0 28 Oct 2019
HUBERT Untangles BERT to Improve Transfer across NLP Tasks M. Moradshahi Hamid Palangi M. Lam P. Smolensky Jianfeng Gao 23 16 0 25 Oct 2019
Adaptively Truncating Backpropagation Through Time to Control Gradient Bias Christopher Aicher N. Foti E. Fox MQ 22 32 0 17 May 2019
Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group Mario Lezcano Casado David Martínez-Rubio 21 194 0 24 Jan 2019
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies A. Chandar Chinnadhurai Sankar Eugene Vorontsov Samira Ebrahimi Kahou Yoshua Bengio 21 56 0 22 Jan 2019
Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks Drew Linsley Junkyung Kim D. Berson Thomas Serre 3DV 30 17 0 28 Nov 2018
RePr: Improved Training of Convolutional Filters Aaditya (Adi) Prakash J. Storer D. Florêncio Cha Zhang VLM CVBM 32 57 0 18 Nov 2018
Understanding Recurrent Neural Architectures by Analyzing and Synthesizing Long Distance Dependencies in Benchmark Sequential Datasets Abhijit Mahalunkar John D. Kelleher 24 8 0 06 Oct 2018
On the Learning Dynamics of Deep Neural Networks Rémi Tachet des Combes Mohammad Pezeshki Samira Shabanian Aaron Courville Yoshua Bengio 16 38 0 18 Sep 2018
How Robust are Deep Neural Networks? B. Sengupta Karl J. Friston OOD 25 31 0 30 Apr 2018
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform Kyle E. Helfrich Devin Willmott Q. Ye 45 128 0 29 Jul 2017
Kronecker Recurrent Units C. Jose Moustapha Cissé F. Fleuret ODL 24 45 0 29 May 2017
Unsupervised Perceptual Rewards for Imitation Learning P. Sermanet Kelvin Xu Sergey Levine SSL 19 156 0 20 Dec 2016
Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections Zakaria Mhammedi Andrew D. Hellicar Ashfaqur Rahman James Bailey 19 129 0 01 Dec 2016
Learning Unitary Operators with Help From u(n) Stephanie L. Hyland Gunnar Rätsch 97 41 0 17 Jul 2016
Online Batch Selection for Faster Training of Neural Networks I. Loshchilov Frank Hutter ODL 37 296 0 19 Nov 2015
Training Deep Neural Networks on Noisy Labels with Bootstrapping Scott E. Reed Honglak Lee Dragomir Anguelov Christian Szegedy D. Erhan Andrew Rabinovich NoLa 78 1,012 0 20 Dec 2014