The Role of Diversity in In-Context Learning for Large Language Models

26 May 2025

Papers citing "The Role of Diversity in In-Context Learning for Large Language Models"

50 / 54 papers shown

Title
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning Xuetao Ma Wenbin Jiang Hua Huang LRM 81 3 0 21 Feb 2025
Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning Donglin Zhan Leonardo F. Toso James Anderson 143 3 0 04 Feb 2025
In-Context Learning with Iterative Demonstration Selection Chengwei Qin Aston Zhang Chong Chen Anirudh Dagar Wenming Ye LRM 116 48 0 31 Dec 2024
Can Models Learn Skill Composition from Examples? Haoyu Zhao Simran Kaur Dingli Yu Anirudh Goyal Sanjeev Arora CoGe MoE 71 5 0 29 Sep 2024
Gemma 2: Improving Open Language Models at a Practical Size Gemma Team Gemma Team Morgane Riviere Shreya Pathak Pier Giuseppe Sessa Cassidy Hardin ... Noah Fiedel Armand Joulin Kathleen Kenealy Robert Dadashi Alek Andreev VLM MoE OSLM 74 772 0 31 Jul 2024
Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving Aniket Didolkar Anirudh Goyal Nan Rosemary Ke Siyuan Guo Michal Valko Timothy Lillicrap Danilo Jimenez Rezende Yoshua Bengio Michael C. Mozer Sanjeev Arora LRM 46 25 0 20 May 2024
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers Qintong Li Leyang Cui Xueliang Zhao Lingpeng Kong Wei Bi LRM 69 54 0 29 Feb 2024
Diversity Measurement and Subset Selection for Instruction Tuning Datasets Peiqi Wang Songlin Yang Zhen Guo Matt Stallone Yoon Kim Polina Golland Yikang Shen 42 11 0 04 Feb 2024
Contributing Dimension Structure of Deep Feature for Coreset Selection Zhijing Wan Zhixiang Wang Yuran Wang Zheng Wang Hongyuan Zhu Shiníchi Satoh 49 7 0 29 Jan 2024
In-context Learning and Gradient Descent Revisited Gilad Deutch Nadav Magar Tomer Bar Natan Guy Dar 43 12 0 13 Nov 2023
Skill-Mix: a Flexible and Expandable Family of Evaluations for AI models Dingli Yu Simran Kaur Arushi Gupta Jonah Brown-Cohen Anirudh Goyal Sanjeev Arora ALM LLMAG 40 40 0 26 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 36 2,102 0 10 Oct 2023
Uncovering mesa-optimization algorithms in Transformers J. Oswald Eyvind Niklasson Maximilian Schlegel Seijin Kobayashi Nicolas Zucchet ... Mark Sandler Blaise Agüera y Arcas Max Vladymyrov Razvan Pascanu João Sacramento 52 58 0 11 Sep 2023
A Theory for Emergence of Complex Skills in Language Models Sanjeev Arora Anirudh Goyal LRM 44 81 0 29 Jul 2023
Lost in the Middle: How Language Models Use Long Contexts Nelson F. Liu Kevin Lin John Hewitt Ashwin Paranjape Michele Bevilacqua Fabio Petroni Percy Liang RALM 74 1,521 0 06 Jul 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 98 1,044 0 31 May 2023
What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization Yufeng Zhang Fengzhuo Zhang Zhuoran Yang Zhaoran Wang BDL 51 69 0 30 May 2023
Coverage-based Example Selection for In-Context Learning Shivanshu Gupta Matt Gardner Sameer Singh 70 46 0 24 May 2023
RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning Alexander Scarlatos Andrew Lan OffRL LRM 57 22 0 23 May 2023
Skill-Based Few-Shot Selection for In-Context Learning Shengnan An Bo Zhou Zeqi Lin Qiang Fu B. Chen Nanning Zheng Weizhu Chen Jian-Guang Lou 48 34 0 23 May 2023
Dr.ICL: Demonstration-Retrieved In-context Learning Man Luo Xin Xu Zhuyun Dai Panupong Pasupat Mehran Kazemi Chitta Baral Vaiva Imbrasaite Vincent Zhao RALM 69 53 0 23 May 2023
What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning Jane Pan Tianyu Gao Howard Chen Danqi Chen 52 116 0 16 May 2023
How Do In-Context Examples Affect Compositional Generalization? Shengnan An Zeqi Lin Qiang Fu B. Chen Nanning Zheng Jian-Guang Lou Dongmei Zhang 55 52 0 08 May 2023
The Closeness of In-Context Learning and Weight Shifting for Softmax Regression Shuai Li Zhao Song Yu Xia Tong Yu Dinesh Manocha 51 40 0 26 Apr 2023
A Latent Space Theory for Emergent Abilities in Large Language Models Hui Jiang LRM 45 36 0 19 Apr 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 155 101 0 14 Mar 2023
Finding Support Examples for In-Context Learning Xiaonan Li Xipeng Qiu 65 95 0 27 Feb 2023
Transformers as Algorithms: Generalization and Stability in In-context Learning Yingcong Li M. E. Ildiz Dimitris Papailiopoulos Samet Oymak 61 163 0 17 Jan 2023
Data Curation Alone Can Stabilize In-context Learning Ting-Yun Chang Robin Jia 34 53 0 20 Dec 2022
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering Zhiyong Wu Yaoxiang Wang Jiacheng Ye Lingpeng Kong 51 131 0 20 Dec 2022
Transformers learn in-context by gradient descent J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 68 470 0 15 Dec 2022
Diverse Demonstrations Improve In-context Compositional Generalization Itay Levy Ben Bogin Jonathan Berant 59 138 0 13 Dec 2022
What learning algorithm is in-context learning? Investigations with linear models Ekin Akyürek Dale Schuurmans Jacob Andreas Tengyu Ma Denny Zhou 57 468 0 28 Nov 2022
Complementary Explanations for Effective In-Context Learning Xi Ye Srini Iyer Asli Celikyilmaz Ves Stoyanov Greg Durrett Ramakanth Pasunuru ReLM LRM 54 89 0 25 Nov 2022
Automatic Chain of Thought Prompting in Large Language Models Zhuosheng Zhang Aston Zhang Mu Li Alexander J. Smola ReLM LRM 110 610 0 07 Oct 2022
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 198 80 0 16 Sep 2022
Selective Annotation Makes Language Models Better Few-Shot Learners Hongjin Su Jungo Kasai Chen Henry Wu Weijia Shi Tianlu Wang ... Rui Zhang Mari Ostendorf Luke Zettlemoyer Noah A. Smith Tao Yu 65 251 0 05 Sep 2022
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 98 479 0 01 Aug 2022
Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations Kang Min Yoo Junyeob Kim Sungmin Cho Hyunsoo Cho Hwiyeol Jo Sang-Woo Lee Sang-goo Lee Taeuk Kim 54 125 0 25 May 2022
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? Sewon Min Xinxi Lyu Ari Holtzman Mikel Artetxe M. Lewis Hannaneh Hajishirzi Luke Zettlemoyer LLMAG LRM 114 1,438 0 25 Feb 2022
Learning To Retrieve Prompts for In-Context Learning Ohad Rubin Jonathan Herzig Jonathan Berant VPVLM RALM 45 679 0 16 Dec 2021
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 141 728 0 03 Nov 2021
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 186 4,175 0 27 Oct 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 208 412 0 10 Sep 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 317 1,152 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 324 1,343 0 17 Jan 2021
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics Swabha Swayamdipta Roy Schwartz Nicholas Lourie Yizhong Wang Hannaneh Hajishirzi Noah A. Smith Yejin Choi 77 444 0 22 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 498 41,106 0 28 May 2020
Dense Passage Retrieval for Open-Domain Question Answering Vladimir Karpukhin Barlas Oğuz Sewon Min Patrick Lewis Ledell Yu Wu Sergey Edunov Danqi Chen Wen-tau Yih RALM 130 3,676 0 10 Apr 2020
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 636 11,979 0 27 Aug 2019