A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks

26 May 2023

Papers citing "A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks"

21 / 21 papers shown

Title
Transformers versus the EM Algorithm in Multi-class Clustering Yihan He Hong-Yu Chen Yuan Cao Jianqing Fan Han Liu 55 1 0 09 Feb 2025
SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization Hongrui Jia Chaoya Jiang Haiyang Xu Wei Ye Mengfan Dong Ming Yan Ji Zhang Fei Huang Shikun Zhang MLLM 101 2 0 17 Nov 2024
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context Zihao Li Yuan Cao Cheng Gao Yihan He Han Liu Jason M. Klusowski Jianqing Fan Mengdi Wang MLT 55 7 0 16 Nov 2024
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 78 4 0 30 Oct 2024
How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression Xingwu Chen Lei Zhao Difan Zou 49 6 0 08 Aug 2024
How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment Heyan Huang Yinghao Li Huashan Sun Yu Bai Yang Gao 50 3 0 17 Jun 2024
Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration Yang Zhang Shixin Yang Chenjia Bai Fei Wu Xiu Li Zhen Wang Xuelong Li LLMAG 36 25 0 23 May 2024
Can large language models explore in-context? Akshay Krishnamurthy Keegan Harris Dylan J. Foster Cyril Zhang Aleksandrs Slivkins LM&Ro LLMAG LRM 131 23 0 22 Mar 2024
In-context Exploration-Exploitation for Reinforcement Learning Zhenwen Dai Federico Tomasi Sina Ghiassian OffRL OnRL 43 3 0 11 Mar 2024
The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains Benjamin L. Edelman Ezra Edelman Surbhi Goel Eran Malach Nikolaos Tsilivis BDL 29 43 0 16 Feb 2024
Linear attention is (maybe) all you need (to understand transformer optimization) Kwangjun Ahn Xiang Cheng Minhak Song Chulhee Yun Ali Jadbabaie S. Sra 33 47 1 02 Oct 2023
Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency Zhihan Liu Hao Hu Shenao Zhang Hongyi Guo Shuqi Ke Boyi Liu Zhaoran Wang LLMAG LRM 36 34 0 29 Sep 2023
Supervised Pretraining Can Learn In-Context Reinforcement Learning Jonathan Lee Annie Xie Aldo Pacchiano Yash Chandak Chelsea Finn Ofir Nachum Emma Brunskill OffRL 35 76 0 26 Jun 2023
Trained Transformers Learn Linear Models In-Context Ruiqi Zhang Spencer Frei Peter L. Bartlett 34 182 0 16 Jun 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 23 17 0 24 May 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 122 97 0 14 Mar 2023
Learning threshold neurons via the "edge of stability" Kwangjun Ahn Sébastien Bubeck Sinho Chewi Y. Lee Felipe Suarez Yi Zhang MLT 40 36 0 14 Dec 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 252 474 0 24 Sep 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 447 8,650 0 28 Jan 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 243 1,927 0 31 Dec 2020
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 466 11,715 0 09 Mar 2017