The Learnability of In-Context Learning

14 March 2023

Papers citing "The Learnability of In-Context Learning"

22 / 22 papers shown

Title
Adversarially Pretrained Transformers may be Universally Robust In-Context Learners Soichiro Kumano Hiroshi Kera Toshihiko Yamasaki AAML 117 0 0 20 May 2025
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning Xuetao Ma Wenbin Jiang Hua Huang LRM 120 4 0 21 Feb 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 128 6 0 21 Feb 2025
K-ON: Stacking Knowledge On the Head Layer of Large Language Model Lingbing Guo Yanzhe Zhang Zhongpu Bo Zhuo Chen Mengshu Sun Qing Cui Wen Zhang Ningyu Zhang 155 0 0 10 Feb 2025
State-space models can learn in-context by gradient descent Neeraj Mohan Sushma Yudou Tian Harshvardhan Mestha Nicolo Colombo David Kappel Anand Subramoney 107 3 0 15 Oct 2024
The Labyrinth of Links: Navigating the Associative Maze of Multi-modal LLMs Hong Li Nanxi Li Yuanjie Chen Jianbin Zhu Qinlu Guo Cewu Lu Yong-Lu Li MLLM 90 1 0 02 Oct 2024
Demystifying Chains, Trees, and Graphs of Thoughts Maciej Besta Florim Memedi Zhenyu Zhang Robert Gerstenberger Guangyuan Piao ... Aleš Kubíček H. Niewiadomski Aidan O'Mahony Onur Mutlu Torsten Hoefler AI4CE LRM 286 33 0 25 Jan 2024
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 116 494 0 15 Dec 2022
In-context Reinforcement Learning with Algorithm Distillation Michael Laskin Luyu Wang Junhyuk Oh Emilio Parisotto Stephen Spencer ... Ethan A. Brooks Maxime Gazeau Himanshu Sahni Satinder Singh Volodymyr Mnih OffRL 66 133 0 25 Oct 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 319 525 0 24 Sep 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 141 513 0 01 Aug 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 286 2,511 0 15 Jun 2022
Data Distributional Properties Drive Emergent In-Context Learning in Transformers Stephanie C. Y. Chan Adam Santoro Andrew Kyle Lampinen Jane X. Wang Aaditya K. Singh Pierre Harvey Richemond J. Mcclelland Felix Hill 155 265 0 22 Apr 2022
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 213 763 0 03 Nov 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 348 1,708 0 15 Oct 2021
The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design Yoav Levine Noam Wies Daniel Jannai D. Navon Yedid Hoshen Amnon Shashua AI4CE 79 37 0 09 Oct 2021
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 217 3,782 0 03 Sep 2021
Do Prompt-Based Models Really Understand the Meaning of their Prompts? Albert Webson Ellie Pavlick LRM 109 373 0 02 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 584 4,084 0 18 Apr 2021
Language Modelling as a Multi-Task Problem Leon Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 78 13 0 27 Jan 2021
A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks Nikunj Saunshi Sadhika Malladi Sanjeev Arora 82 89 0 07 Oct 2020
Colorless green recurrent networks dream hierarchically Kristina Gulordava Piotr Bojanowski Edouard Grave Tal Linzen Marco Baroni 91 505 0 29 Mar 2018