In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization

22 February 2024

Papers citing "In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization"

5 / 5 papers shown

Title
Transformers Handle Endogeneity in In-Context Linear Regression Haodong Liang Krishnakumar Balasubramanian Lifeng Lai 44 1 0 02 Oct 2024
Looped Transformers are Better at Learning Learning Algorithms Liu Yang Kangwook Lee Robert D. Nowak Dimitris Papailiopoulos 24 24 0 21 Nov 2023
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression? Jingfeng Wu Difan Zou Zixiang Chen Vladimir Braverman Quanquan Gu Peter L. Bartlett 131 50 0 12 Oct 2023
Meta-learning via Language Model In-context Tuning Yanda Chen Ruiqi Zhong Sheng Zha George Karypis He He 236 157 0 15 Oct 2021
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 365 11,700 0 09 Mar 2017