Using Fast Weights to Attend to the Recent Past

20 October 2016

Jimmy Ba

Papers citing "Using Fast Weights to Attend to the Recent Past"

50 / 63 papers shown

Title
Neural Associative Skill Memories for safer robotics and modelling human sensorimotor repertoires Pranav Mahajan Mufeng Tang T. Ed Li Ioannis Havoutis Ben Seymour 14 0 0 14 May 2025
Binding threshold units with artificial oscillatory neurons V. Fanaskov Ivan Oseledets 62 0 0 06 May 2025
Lightweight Online Adaption for Time Series Foundation Model Forecasts Thomas L. Lee William Toner Rajkarn Singh Artjom Joosem Martin Asenov AI4TS 45 1 0 18 Feb 2025
Key-value memory in the brain Samuel J. Gershman Ila Fiete Kazuki Irie 34 7 0 06 Jan 2025
Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass Tong Chen Hao Fang Patrick Xia Xiaodong Liu Benjamin Van Durme Luke Zettlemoyer Jianfeng Gao Hao Cheng KELM 59 2 0 08 Nov 2024
Brain-like variational inference Hadi Vafaii Dekel Galor Jacob L. Yates DRL 49 0 0 25 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 221 1 0 20 Sep 2024
Self-Contrastive Forward-Forward Algorithm Xing Chen Dongshu Liu Jérémie Laydevant Julie Grollier 46 2 0 17 Sep 2024
Banyan: Improved Representation Learning with Explicit Structure Mattia Opper N. Siddharth 33 1 0 25 Jul 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 71 93 0 05 Jul 2024
Separations in the Representational Capabilities of Transformers and Recurrent Architectures S. Bhattamishra Michael Hahn Phil Blunsom Varun Kanade GNN 44 9 0 13 Jun 2024
Deep Continuous Networks Nergis Tomen S. Pintea Jan van Gemert 94 14 0 02 Feb 2024
Theory of coupled neuronal-synaptic dynamics David G. Clark L. F. Abbott 27 19 0 17 Feb 2023
Adaptive Siamese Tracking with a Compact Latent Network Xingping Dong Jianbing Shen Fatih Porikli Jiebo Luo Ling Shao 35 30 0 02 Feb 2023
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 73 372 0 28 Dec 2022
The Forward-Forward Algorithm: Some Preliminary Investigations Geoffrey E. Hinton 30 260 0 27 Dec 2022
Constructing Organism Networks from Collaborative Self-Replicators Steffen Illium Maximilian Zorn Cristian Lenta Michael Kolle Claudia Linnhoff-Popien Thomas Gabor 19 0 0 20 Dec 2022
Temporal Weights Adam A. Kohan E. Rietman H. Siegelmann 24 0 0 13 Dec 2022
Meta-Learning Fast Weight Language Models Kevin Clark Kelvin Guu Ming-Wei Chang Panupong Pasupat Geoffrey E. Hinton Mohammad Norouzi KELM 32 13 0 05 Dec 2022
Circling Back to Recurrent Models of Language Gábor Melis 40 0 0 03 Nov 2022
Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning Xingping Dong Jianbing Shen Ling Shao 32 7 0 27 Sep 2022
Open Long-Tailed Recognition in a Dynamic World Ziwei Liu Zhongqi Miao Xiaohang Zhan Jiayun Wang Boqing Gong Stella X. Yu VLM 46 18 0 17 Aug 2022
Recurrent Memory Transformer Aydar Bulatov Yuri Kuratov Andrey Kravchenko CLL 13 102 0 14 Jul 2022
Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation Michael Chang Thomas Griffiths Sergey Levine OCL 69 58 0 02 Jul 2022
Short-Term Plasticity Neurons Learning to Learn and Forget Hector Garcia Rodriguez Qinghai Guo Timoleon Moraitis 20 12 0 28 Jun 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 121 17 0 30 May 2022
Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity Thomas Limbacher Ozan Özdenizci Robert Legenstein 21 2 0 23 May 2022
Learning Theory of Mind via Dynamic Traits Attribution D. Nguyen Phuoc Nguyen Hung Le Kien Do Svetha Venkatesh T. Tran 22 5 0 17 Apr 2022
Evolving Modular Soft Robots without Explicit Inter-Module Communication using Local Self-Attention F. Pigozzi Yujin Tang Eric Medvet David R Ha 44 22 0 13 Apr 2022
A Call for Clarity in Beam Search: How It Works and When It Stops Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Dragomir R. Radev Yejin Choi Noah A. Smith 28 6 0 11 Apr 2022
The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention Kazuki Irie Róbert Csordás Jürgen Schmidhuber 16 42 0 11 Feb 2022
A Field Guide to Scientific XAI: Transparent and Interpretable Deep Learning for Bioinformatics Research Thomas P. Quinn Sunil R. Gupta Svetha Venkatesh Vuong Le OOD 19 2 0 13 Oct 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie Imanol Schlag Róbert Csordás Jürgen Schmidhuber 33 57 0 11 Jun 2021
Random Feature Attention Hao Peng Nikolaos Pappas Dani Yogatama Roy Schwartz Noah A. Smith Lingpeng Kong 36 349 0 03 Mar 2021
Linear Transformers Are Secretly Fast Weight Programmers Imanol Schlag Kazuki Irie Jürgen Schmidhuber 46 225 0 22 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 281 179 0 17 Feb 2021
Meta Learning Backpropagation And Improving It Louis Kirsch Jürgen Schmidhuber 58 56 0 29 Dec 2020
Emergent Symbols through Binding in External Memory Taylor Webb I. Sinha Jonathan Cohen 67 65 0 29 Dec 2020
Few-shot Sequence Learning with Transformers Lajanugen Logeswaran Ann Lee Myle Ott Honglak Lee MarcÁurelio Ranzato Arthur Szlam ViT 46 12 0 17 Dec 2020
Continual Learning with Deep Artificial Neurons Blake Camp J. Mandivarapu Rolando Estrada 18 8 0 13 Nov 2020
Hopfield Networks is All You Need Hubert Ramsauer Bernhard Schafl Johannes Lehner Philipp Seidl Michael Widrich ... David P. Kreil Michael K Kopp Günter Klambauer Johannes Brandstetter Sepp Hochreiter 24 415 0 16 Jul 2020
Meta-Learning through Hebbian Plasticity in Random Networks Elias Najarro S. Risi 30 77 0 06 Jul 2020
Learning to Learn with Feedback and Local Plasticity Jack W Lindsey Ashok Litwin-Kumar CLL 34 32 0 16 Jun 2020
Neuroevolution of Self-Interpretable Agents Yujin Tang Duong Nguyen David R Ha 29 111 0 18 Mar 2020
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity Thomas Miconi Aditya Rawal Jeff Clune Kenneth O. Stanley 16 90 0 24 Feb 2020
Self-Attentive Associative Memory Hung Le T. Tran Svetha Venkatesh 22 56 0 10 Feb 2020
Meta-Learning Deep Energy-Based Memory Models Sergey Bartunov Jack W. Rae Simon Osindero Timothy Lillicrap 45 34 0 07 Oct 2019
Meta-Learning with Warped Gradient Descent Sebastian Flennerhag Andrei A. Rusu Razvan Pascanu Francesco Visin Hujun Yin R. Hadsell 8 209 0 30 Aug 2019
LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning Huaiyu Li Weiming Dong Xing Mei Chongyang Ma Feiyue Huang Bao-Gang Hu OffRL 32 98 0 15 May 2019
Large-Scale Long-Tailed Recognition in an Open World Ziwei Liu Zhongqi Miao Xiaohang Zhan Jiayun Wang Boqing Gong Stella X. Yu 48 1,135 0 10 Apr 2019