v1v2 (latest)

A Modern Self-Referential Weight Matrix That Learns to Modify Itself

11 February 2022

Papers citing "A Modern Self-Referential Weight Matrix That Learns to Modify Itself"

37 / 37 papers shown

Title
The CLEAR Benchmark: Continual LEArning on Real-World Imagery Zhiqiu Lin Jia Shi Deepak Pathak Deva Ramanan CLL VLM 192 93 0 17 Jan 2022
HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learning A. Zhmoginov Mark Sandler Max Vladymyrov ViT 70 69 0 11 Jan 2022
The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization Róbert Csordás Kazuki Irie Jürgen Schmidhuber AI4CE 86 57 0 14 Oct 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie Imanol Schlag Róbert Csordás Jürgen Schmidhuber 82 62 0 11 Jun 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 434 2,685 0 04 May 2021
Meta-Learning Bidirectional Update Rules Mark Sandler Max Vladymyrov A. Zhmoginov Nolan Miller Andrew Jackson T. Madams Blaise Agüera y Arcas 76 15 0 10 Apr 2021
Random Feature Attention Hao Peng Nikolaos Pappas Dani Yogatama Roy Schwartz Noah A. Smith Lingpeng Kong 109 362 0 03 Mar 2021
Linear Transformers Are Secretly Fast Weight Programmers Imanol Schlag Kazuki Irie Jürgen Schmidhuber 124 252 0 22 Feb 2021
Meta Learning Backpropagation And Improving It Louis Kirsch Jürgen Schmidhuber 102 57 0 29 Dec 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 186 1,600 0 30 Sep 2020
Meta-Learning through Hebbian Plasticity in Random Networks Elias Najarro S. Risi 93 78 0 06 Jul 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,786 0 29 Jun 2020
Adaptive Reinforcement Learning through Evolving Self-Modifying Neural Networks Samuel Schmidgall KELM CLL 20 7 0 22 May 2020
Addressing Catastrophic Forgetting in Few-Shot Problems Pauching Yap H. Ritter David Barber CLL BDL 60 19 0 30 Apr 2020
Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning Yinbo Chen Zhuang Liu Huijuan Xu Trevor Darrell Xiaolong Wang 205 347 0 09 Mar 2020
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity Thomas Miconi Aditya Rawal Jeff Clune Kenneth O. Stanley 74 90 0 24 Feb 2020
Leveraging Procedural Generation to Benchmark Reinforcement Learning K. Cobbe Christopher Hesse Jacob Hilton John Schulman 81 557 0 03 Dec 2019
TorchBeast: A PyTorch Platform for Distributed RL Heinrich Küttler Nantas Nardelli Thibaut Lavril Marco Selvatici V. Sivakumar Tim Rocktaschel Edward Grefenstette OffRL 65 58 0 08 Oct 2019
Torchmeta: A Meta-Learning library for PyTorch T. Deleu Tobias Würfl Mandana Samiei Joseph Paul Cohen Yoshua Bengio OffRL 74 85 0 14 Sep 2019
Metalearned Neural Memory Tsendsuren Munkhdalai Alessandro Sordoni Tong Wang Adam Trischler KELM 51 62 0 23 Jul 2019
Risks from Learned Optimization in Advanced Machine Learning Systems Evan Hubinger Chris van Merwijk Vladimir Mikulik Joar Skalse Scott Garrabrant 89 154 0 05 Jun 2019
Metalearning with Hebbian Fast Weights Tsendsuren Munkhdalai Adam Trischler VLM FedML 60 37 0 12 Jul 2018
TADAM: Task dependent adaptive metric for improved few-shot learning Boris N. Oreshkin Pau Rodríguez López Alexandre Lacoste 96 1,313 0 23 May 2018
Differentiable plasticity: training plastic neural networks with backpropagation Thomas Miconi Jeff Clune Kenneth O. Stanley AI4CE 64 154 0 06 Apr 2018
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures L. Espeholt Hubert Soyer Rémi Munos Karen Simonyan Volodymyr Mnih ... Vlad Firoiu Tim Harley Iain Dunning Shane Legg Koray Kavukcuoglu 237 1,605 0 05 Feb 2018
Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm Chelsea Finn Sergey Levine SSL 96 223 0 31 Oct 2017
Dynamic Evaluation of Neural Sequence Models Ben Krause Emmanuel Kahembwe Iain Murray Steve Renals 73 135 0 21 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 829 11,943 0 09 Mar 2017
Meta Networks Tsendsuren Munkhdalai Hong-ye Yu GNN AI4CE 101 1,068 0 02 Mar 2017
Learning to reinforcement learn Jane X. Wang Z. Kurth-Nelson Dhruva Tirumala Hubert Soyer Joel Z Leibo Rémi Munos Charles Blundell D. Kumaran M. Botvinick OffRL 97 983 0 17 Nov 2016
RL $^2$ : Fast Reinforcement Learning via Slow Reinforcement Learning Yan Duan John Schulman Xi Chen Peter L. Bartlett Ilya Sutskever Pieter Abbeel OffRL 102 1,028 0 09 Nov 2016
Using Fast Weights to Attend to the Recent Past Jimmy Ba Geoffrey E. Hinton Volodymyr Mnih Joel Z Leibo Catalin Ionescu 70 273 0 20 Oct 2016
Matching Networks for One Shot Learning Oriol Vinyals Charles Blundell Timothy Lillicrap Koray Kavukcuoglu Daan Wierstra VLM 375 7,333 0 13 Jun 2016
Increasing the Action Gap: New Operators for Reinforcement Learning Marc G. Bellemare Georg Ostrovski A. Guez Philip S. Thomas Rémi Munos 74 157 0 15 Dec 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014
Efficient Object Localization Using Convolutional Networks Jonathan Tompson Ross Goroshin Arjun Jain Yann LeCun C. Bregler 3DH 184 1,352 0 16 Nov 2014