General-Purpose In-Context Learning by Meta-Learning Transformers

General-Purpose In-Context Learning by Meta-Learning Transformers

8 December 2022

Jascha Narain Sohl-Dickstein

Papers citing "General-Purpose In-Context Learning by Meta-Learning Transformers"

16 / 66 papers shown

Title
Accelerating Neural Self-Improvement via Bootstrapping Kazuki Irie Jürgen Schmidhuber 27 1 0 02 May 2023
Structured State Space Models for In-Context Reinforcement Learning Chris Xiaoxuan Lu Yannick Schroecker Albert Gu Emilio Parisotto Jakob N. Foerster Satinder Singh Feryal M. P. Behbahani AI4TS 97 82 0 07 Mar 2023
Memory-Based Meta-Learning on Non-Stationary Distributions Tim Genewein Grégoire Delétang Anian Ruoss L. Wenliang Elliot Catt Vincent Dutordoir Jordi Grau-Moya Laurent Orseau Marcus Hutter J. Veness BDL 21 11 0 06 Feb 2023
Learning Functional Transduction Mathieu Chalvidal Thomas Serre Rufin VanRullen AI4CE 35 2 0 01 Feb 2023
Human-Timescale Adaptation in an Open-Ended Task Space Adaptive Agent Team Jakob Bauer Kate Baumli Satinder Baveja Feryal M. P. Behbahani ... Jakub Sygnowski K. Tuyls Sarah York Alexander Zacherl Lei Zhang LM&Ro OffRL AI4CE LRM 35 108 0 18 Jan 2023
Transformers as Algorithms: Generalization and Stability in In-context Learning Yingcong Li M. E. Ildiz Dimitris Papailiopoulos Samet Oymak 20 152 0 17 Jan 2023
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers Damai Dai Yutao Sun Li Dong Y. Hao Shuming Ma Zhifang Sui Furu Wei LRM 23 148 0 20 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 30 429 0 15 Dec 2022
Neural Networks and the Chomsky Hierarchy Grégoire Delétang Anian Ruoss Jordi Grau-Moya Tim Genewein L. Wenliang ... Chris Cundy Marcus Hutter Shane Legg Joel Veness Pedro A. Ortega UQCV 107 130 0 05 Jul 2022
Minimal Neural Network Models for Permutation Invariant Agents J. Pedersen S. Risi 48 3 0 12 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 367 8,495 0 28 Jan 2022
Long-Range Transformers for Dynamic Spatiotemporal Forecasting J. E. Grigsby Zhe Wang Nam Nguyen Yanjun Qi AI4TS 69 87 0 24 Sep 2021
Meta Learning Backpropagation And Improving It Louis Kirsch Jürgen Schmidhuber 51 56 0 29 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 246 4,489 0 23 Jan 2020
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML Aniruddh Raghu M. Raghu Samy Bengio Oriol Vinyals 177 639 0 19 Sep 2019
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 338 11,684 0 09 Mar 2017