How Many Pretraining Tasks Are Needed for In-Context Learning of Linear
Regression?

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

12 October 2023

Vladimir Braverman

Quanquan Gu

Peter L. Bartlett

Papers citing "How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?"

13 / 13 papers shown

Title
Understanding In-context Learning of Addition via Activation Subspaces Xinyan Hu Kayo Yin Michael I. Jordan Jacob Steinhardt Lijie Chen 53 0 0 08 May 2025
In-Context Learning with Hypothesis-Class Guidance Ziqian Lin Shubham Kumar Bharti Kangwook Lee 76 0 0 27 Feb 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 54 3 0 21 Feb 2025
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 75 4 0 30 Oct 2024
Context-Scaling versus Task-Scaling in In-Context Learning Amirhesam Abedsoltan Adityanarayanan Radhakrishnan Jingfeng Wu M. Belkin ReLM LRM 40 3 0 16 Oct 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 96 19 0 15 Oct 2024
Spin glass model of in-context learning Yuhao Li Ruoran Bai Haiping Huang LRM 44 0 0 05 Aug 2024
Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification Shang Liu Zhongze Cai Guanting Chen Xiaocheng Li UQCV 46 1 0 24 May 2024
Asymptotic theory of in-context learning by linear attention Yue M. Lu Mary I. Letey Jacob A. Zavatone-Veth Anindita Maiti C. Pehlevan 29 10 0 20 May 2024
Transformers are Provably Optimal In-context Estimators for Wireless Communications Vishnu Teja Kunde Vicram Rajagopalan Chandra Shekhara Kaushik Valmeekam Krishna R. Narayanan S. Shakkottai D. Kalathil J. Chamberland 35 4 0 01 Nov 2023
How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding Yuchen Li Yuan-Fang Li Andrej Risteski 120 61 0 07 Mar 2023
Finite-Sample Analysis of Learning High-Dimensional Single ReLU Neuron Jingfeng Wu Difan Zou Zixiang Chen Vladimir Braverman Quanquan Gu Sham Kakade 90 6 0 03 Mar 2023
Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu Sham Kakade 104 20 0 12 Oct 2021