Capacity and Trainability in Recurrent Neural Networks

29 November 2016

Papers citing "Capacity and Trainability in Recurrent Neural Networks"

50 / 95 papers shown

Title
The impact of allocation strategies in subset learning on the expressive power of neural networks Ofir Schlisselberg Ran Darshan 93 0 0 10 Feb 2025
Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks Ann Huang Satpreet H. Singh Kanaka Rajan 24 0 0 04 Oct 2024
UNComp: Uncertainty-Aware Long-Context Compressor for Efficient Large Language Model Inference Jing Xiong Jianghan Shen Fanghua Ye Chaofan Tao Zhongwei Wan ... Xun Wu Chuanyang Zheng Zhijiang Guo Lingpeng Kong Ngai Wong 27 3 0 04 Oct 2024
Exploring RWKV for Memory Efficient and Low Latency Streaming ASR Keyu An Shiliang Zhang 31 4 0 26 Sep 2023
The minimal computational substrate of fluid intelligence Amy Nelson J. Mole Guilherme Pombo Robert J. Gray James K. Ruffle E. Chan Geraint Rees L. Cipolotti P. Nachev 23 0 0 14 Aug 2023
Trainability, Expressivity and Interpretability in Gated Neural ODEs T. Kim T. Can K. Krishnamurthy AI4CE 35 4 0 12 Jul 2023
Adaptive-saturated RNN: Remember more with less instability Khoi Minh Nguyen-Duy Quang-Cuong Pham B. T. Nguyen ODL 15 1 0 24 Apr 2023
Online Evolutionary Neural Architecture Search for Multivariate Non-Stationary Time Series Forecasting Zimeng Lyu Alexander Ororbia Travis J. Desell AI4TS 17 11 0 20 Feb 2023
General-Purpose In-Context Learning by Meta-Learning Transformers Louis Kirsch James Harrison Jascha Narain Sohl-Dickstein Luke Metz 40 72 0 08 Dec 2022
Criteria for Classifying Forecasting Methods Tim Januschowski Jan Gasthaus Bernie Wang David Salinas Valentin Flunkert Michael Bohlke-Schneider Laurent Callot AI4TS 21 173 0 07 Dec 2022
How Does a Deep Learning Model Architecture Impact Its Privacy? A Comprehensive Study of Privacy Attacks on CNNs and Transformers Guangsheng Zhang B. Liu Huan Tian Tianqing Zhu Ming Ding Wanlei Zhou PILM MIACV 20 5 0 20 Oct 2022
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks Kentaro Ohno Sekitoshi Kanai Yasutoshi Ida 18 0 0 04 Oct 2022
Memory-Augmented Graph Neural Networks: A Brain-Inspired Review Guixiang Ma Vy A. Vo Ted Willke Nesreen Ahmed 35 1 0 22 Sep 2022
TeKo: Text-Rich Graph Neural Networks with External Knowledge Zhizhi Yu Di Jin Jianguo Wei Ziyang Liu Yue Shang Yun Xiao Jiawei Han Lingfei Wu 32 4 0 15 Jun 2022
Training neural networks using Metropolis Monte Carlo and an adaptive variant S. Whitelam V. Selin Ian Benlolo Corneel Casert Isaac Tamblyn BDL 11 7 0 16 May 2022
DeepGraviLens: a Multi-Modal Architecture for Classifying Gravitational Lensing Data Nicolò Oreste Pinciroli Vago Piero Fraternali 20 2 0 02 May 2022
ONE-NAS: An Online NeuroEvolution based Neural Architecture Search for Time Series Forecasting Zimeng Lyu Travis J. Desell AI4TS 19 6 0 27 Feb 2022
Intelligent Acoustic Module for Autonomous Vehicles using Fast Gated Recurrent approach Raghav Rawat Shreyash Gupta Shreyas Mohapatra S. P. Mishra Sreesankar Rajagopal 30 2 0 06 Dec 2021
Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning Sanae Lotfi Tiphaine Bonniot de Ruisselet D. Orban Andrea Lodi ODL 17 1 0 29 Nov 2021
Gradients are Not All You Need Luke Metz C. Freeman S. Schoenholz Tal Kachman 28 93 0 10 Nov 2021
Understanding How Encoder-Decoder Architectures Attend Kyle Aitken V. Ramasesh Yuan Cao Niru Maheswaranathan 34 17 0 28 Oct 2021
Multi-layer Perceptron Trainability Explained via Variability Yueyao Yu Yin Zhang 11 2 0 19 May 2021
Is it enough to optimize CNN architectures on ImageNet? Lukas Tuggener Jürgen Schmidhuber Thilo Stadelmann 30 23 0 16 Mar 2021
Gated Ensemble of Spatio-temporal Mixture of Experts for Multi-task Learning in Ride-hailing System M. Rahman S. Rifaat S. N. Sadeek M. Abrar D. Wang 23 5 0 31 Dec 2020
Sequence Generation using Deep Recurrent Networks and Embeddings: A study case in music Sebastian Garcia-Valencia Alejandro Betancourt Juan Guillermo Lalinde Pulido MGen 25 6 0 02 Dec 2020
Continuous Ant-Based Neural Topology Search A. ElSaid Joshua Karns Zimeng Lyu Alexander Ororbia Travis J. Desell 6 3 0 21 Nov 2020
Low-Dimensional Manifolds Support Multiplexed Integrations in Recurrent Neural Networks Arnaud Fanthomme R. Monasson 11 5 0 20 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 670 0 06 Nov 2020
The geometry of integration in text classification RNNs Kyle Aitken V. Ramasesh Ankush Garg Yuan Cao David Sussillo Niru Maheswaranathan AI4CE 20 14 0 28 Oct 2020
Learnability and Complexity of Quantum Samples M. Niu Andrew M. Dai Li Li Augustus Odena Zhengli Zhao Vadim N. Smelyanskyi Hartmut Neven Sergio Boixo 11 12 0 22 Oct 2020
Unfolding recurrence by Green's functions for optimized reservoir computing Sandra Nestler Christian Keup David Dahmen M. Gilson Holger Rauhut M. Helias 11 4 0 13 Oct 2020
RNN Training along Locally Optimal Trajectories via Frank-Wolfe Algorithm Yun Yue Ming Li Venkatesh Saligrama Ziming Zhang 11 4 0 12 Oct 2020
GECKO: Reconciling Privacy, Accuracy and Efficiency in Embedded Deep Learning Vasisht Duddu A. Boutet Virat Shejwalkar GNN 16 4 0 02 Oct 2020
An Experimental Study of Weight Initialization and Weight Inheritance Effects on Neuroevolution Zimeng Lyu A. ElSaid Joshua Karns Mohamed Wiem Mkaouer Travis J. Desell ODL 10 1 0 21 Sep 2020
Demystifying Deep Learning in Predictive Spatio-Temporal Analytics: An Information-Theoretic Framework Qi Tan Yang Liu Jiming Liu AI4TS 23 8 0 14 Sep 2020
Shuffling Recurrent Neural Networks Michael Rotman Lior Wolf BDL 12 35 0 14 Jul 2020
Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval Xun Yang Jianfeng Dong Yixin Cao Xun Wang Meng Wang Tat-Seng Chua 20 137 0 06 Jul 2020
Thalamocortical motor circuit insights for more robust hierarchical control of complex sequences Laureline Logiaco G. S. Escola 12 5 0 23 Jun 2020
Neuroevolutionary Transfer Learning of Deep Recurrent Neural Networks through Network-Aware Adaptation A. ElSaid Joshua Karns Alexander Ororbia Daniel E. Krutz Zimeng Lyu Travis J. Desell 6 0 0 04 Jun 2020
A Tree Architecture of LSTM Networks for Sequential Regression with Missing Data S. O. Sahin Suleyman Serdar Kozat 17 1 0 22 May 2020
Improving Neuroevolution Using Island Extinction and Repopulation Zimeng Lyu Joshua Karns A. ElSaid Travis J. Desell 6 3 0 15 May 2020
Quantitative Analysis of Image Classification Techniques for Memory-Constrained Devices S. Müksch Theo X. Olausson John Wilhelm Pavlos Andreadis 6 1 0 11 May 2020
How recurrent networks implement contextual processing in sentiment analysis Niru Maheswaranathan David Sussillo 22 22 0 17 Apr 2020
TraDE: Transformers for Density Estimation Rasool Fakoor Pratik Chaudhari Jonas W. Mueller Alex Smola 20 30 0 06 Apr 2020
Actor-Transformers for Group Activity Recognition Kirill Gavrilyuk Ryan Sanford Mehrsan Javan Cees G. M. Snoek ViT 19 178 0 28 Mar 2020
CHAMELEON: A Deep Learning Meta-Architecture for News Recommender Systems [Phd. Thesis] Gabriel de Souza Pereira Moreira GNN 23 2 0 29 Dec 2019
Document Structure Extraction using Prior based High Resolution Hierarchical Semantic Segmentation Mausoom Sarkar Milan Aggarwal Arneh Jain Hiresh Gupta Balaji Krishnamurthy SSeg 9 3 0 27 Nov 2019
Capacity, Bandwidth, and Compositionality in Emergent Language Learning Cinjon Resnick Abhinav Gupta Jakob N. Foerster Andrew M. Dai Kyunghyun Cho 18 51 0 24 Oct 2019
On Predictive Information in RNNs Zhe Dong Deniz Oktay Ben Poole Alexander A. Alemi 11 2 0 21 Oct 2019
Probabilistic Deterministic Finite Automata and Recurrent Networks, Revisited Sarah E. Marzen James P. Crutchfield 6 2 0 17 Oct 2019