v1v2 (latest)

In-Context Learning with Long-Context Models: An In-Depth Exploration

30 April 2024

Graham Neubig

Papers citing "In-Context Learning with Long-Context Models: An In-Depth Exploration"

50 / 102 papers shown

Title
Assessing the Limits of In-Context Learning beyond Functions using Partially Ordered Relation Debanjan Dutta Faizanuddin Ansari Swagatam Das 25 0 0 16 Jun 2025
$Refract ICL: Rethinking Example Selection in the Era of Million-Token Models$ Refract ICL: Rethinking Example Selection in the Era of Million-Token Models Arjun R. Akula Kazuma Hashimoto Krishna Srinivasan Aditi Chaudhary K. Raman Michael Bendersky 24 0 0 14 Jun 2025
Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching J. Zhang Bei Li Jun Bai Rumei Li Yanmeng Wang Chenghua Lin Wenge Rong 110 0 0 05 Jun 2025
Uncertainty Unveiled: Can Exposure to More In-context Examples Mitigate Uncertainty for Large Language Models? Yifei Wang Yu Sheng Linjing Li D. Zeng 41 0 0 27 May 2025
What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs Sangyeop Kim Yohan Lee Yongwoo Song Kimin Lee AAML 44 0 0 26 May 2025
Efficient Multi-modal Long Context Learning for Training-free Adaptation Zehong Ma Shiliang Zhang Longhui Wei Qi Tian VLM 57 0 0 26 May 2025
Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective Junnan Liu Hongwei Liu Linchen Xiao Shudong Liu Taolin Zhang Zihan Ma Songyang Zhang Kai Chen LRM 133 0 0 26 May 2025
In-context Language Learning for Endangered Languages in Speech Recognition Zhaolin Li Jan Niehues 50 0 0 26 May 2025
MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning Zihan Chen Song Wang Zhen Tan Jundong Li Cong Shen OffRL 242 1 0 22 May 2025
Understanding Differential Transformer Unchains Pretrained Self-Attentions Chaerin Kong Jiho Jang Nojun Kwak 98 0 0 22 May 2025
Bye-bye, Bluebook? Automating Legal Procedure with Large Language Models Matthew Dahl AILaw ELM 103 0 0 05 May 2025
On the generalization of language models from in-context learning and finetuning: a controlled study Andrew Kyle Lampinen Arslan Chaudhry Stephanie Chan Cody Wild Diane Wan Alex Ku Jorg Bornschein Razvan Pascanu Murray Shanahan James L. McClelland 187 5 0 01 May 2025
Self-Generated In-Context Examples Improve LLM Agents for Sequential Decision-Making Tasks Vishnu Sarukkai Zhiqiang Xie Kayvon Fatahalian LLMAG 168 1 0 01 May 2025
A Survey of Scaling in Large Language Model Reasoning Zihan Chen Song Wang Zhen Tan Xingbo Fu Zhenyu Lei Peng Wang Huan Liu Cong Shen Jundong Li LRM 248 2 0 02 Apr 2025
Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts Youxiang Zhu Ruochen Li Danqing Wang Daniel Haehn Xiaohui Liang LRM 136 2 0 30 Mar 2025
A Survey on Transformer Context Extension: Approaches and Evaluation Yijun Liu Jinzheng Yu Yang Xu Zhongyang Li Qingfu Zhu LLMAG 135 3 0 17 Mar 2025
Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention Emily Xiao Chin-Jou Li Yilin Zhang Graham Neubig Amanda Bertsch BDL 116 1 0 11 Mar 2025
Can Large Language Models Unveil the Mysteries? An Exploration of Their Ability to Unlock Information in Complex Scenarios Chao Wang Luning Zhang Ziyi Wang Yang Zhou ELM VLM LRM 126 1 0 27 Feb 2025
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs Jungsoo Park Junmo Kang Gabriel Stanovsky Alan Ritter 134 0 0 26 Feb 2025
Problem-Solving Logic Guided Curriculum In-Context Learning for LLMs Complex Reasoning Xuetao Ma Wenbin Jiang Hua Huang LRM 217 4 0 21 Feb 2025
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 141 7 0 21 Feb 2025
Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection Maximilian Spliethover Tim Knebler Fabian Fumagalli Maximilian Muschalik Barbara Hammer Eyke Hüllermeier Henning Wachsmuth 200 1 0 10 Feb 2025
ARISE: Iterative Rule Induction and Synthetic Data Generation for Text Classification Y. Meena Vaibhav Singh Ayush Maheshwari Amrith Krishna Ganesh Ramakrishnan AI4TS 445 0 0 09 Feb 2025
From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation Xingchen Wan Han Zhou Ruoxi Sun Hootan Nakhost Ke Jiang Sercan Ö. Arık ReLM OffRL LRM 98 4 0 01 Feb 2025
Shared DIFF Transformer Yueyang Cang Yuhang Liu Xiaoteng Zhang Xiangju Wang 58 0 0 29 Jan 2025
NExtLong: Toward Effective Long-Context Training without Long Documents Chaochen Gao Xing Wu Zijia Lin Debing Zhang Songlin Hu SyDa 197 2 0 22 Jan 2025
Better Prompt Compression Without Multi-Layer Perceptrons Edouardo Honig Andrew Lizarraga Zijun Zhang Ying Nian Wu MQ 432 1 0 12 Jan 2025
Revisiting In-Context Learning with Long Context Language Models Jinheon Baek Sun Jae Lee Prakhar Gupta Geunseob Oh Siddharth Dalmia 674 3 0 22 Dec 2024
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning Yansheng Mao Jiaqi Li Fanxu Meng Jing Xiong Zilong Zheng Muhan Zhang LLMAG RALM 185 1 0 18 Dec 2024
VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding Kangsan Kim G. Park Youngwan Lee Woongyeong Yeo Sung Ju Hwang 154 5 0 03 Dec 2024
AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations Gaurav Verma Rachneet Kaur Nishan Srishankar Zhen Zeng T. Balch Manuela Veloso LLMAG 118 6 0 20 Nov 2024
A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classification Sorouralsadat Fatemi Yuheng Hu Maryam Mousavi 107 4 0 04 Nov 2024
Keep Guessing? When Considering Inference Scaling, Mind the Baselines G. Yona Or Honovich Omer Levy Roee Aharoni UQLM LRM 116 0 0 20 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 148 1 0 12 Oct 2024
Differential Transformer Tianzhu Ye Li Dong Yuqing Xia Yutao Sun Yi Zhu Gao Huang Furu Wei 526 0 0 07 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 225 6 0 06 Oct 2024
Accelerating Inference of Networks in the Frequency Domain Chenqiu Zhao Guanfang Dong Anup Basu 126 20 0 06 Oct 2024
Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization Ryan Barron Ves Grantcharov Selma Wanna Maksim E. Eren Manish Bhattarai ... George Tompkins Charles K. Nicholas Kim Ø. Rasmussen Cynthia Matuszek Boian S. Alexandrov RALM 113 9 0 03 Oct 2024
L-CiteEval: Do Long-Context Models Truly Leverage Context for Responding? Zecheng Tang Keyan Zhou Juntao Li Baibei Ji Jianye Hou Min Zhang 87 2 0 03 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 206 48 0 03 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 149 37 0 03 Oct 2024
FlashMask: Efficient and Rich Mask Extension of FlashAttention Guoxia Wang Jinle Zeng Xiyuan Xiao Siming Wu Jiabin Yang Lujing Zheng Zeyu Chen Jiang Bian Dianhai Yu Haifeng Wang 388 3 0 02 Oct 2024
Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Pieces Lilac Atassi 88 0 0 01 Oct 2024
A Controlled Study on Long Context Extension and Generalization in LLMs Yi Lu Jing Nathan Yan Songlin Yang Justin T. Chiu Siyu Ren Fei Yuan Wenting Zhao Zhiyong Wu Alexander M. Rush 128 11 0 18 Sep 2024
A Fused Large Language Model for Predicting Startup Success Abdurahman Maarouf Stefan Feuerriegel Nicolas Pröllochs 90 10 0 05 Sep 2024
Focused Large Language Models are Stable Many-Shot Learners Peiwen Yuan Shaoxiong Feng Yiwei Li Xinglin Wang Y. Zhang Chuyi Tan Boyuan Pan Heda Wang Yao Hu Kan Li 106 5 0 26 Aug 2024
Many-Shot In-Context Learning for Molecular Inverse Design Saeed Moayedpour Alejandro Corrochano-Navarro F. Sahneh Shahriar Noroozizadeh Alexander Koetter ... Hans Matter Christoph Grebner Gerhard Hessler Ziv Bar-Joseph Sven Jager BDL 89 4 0 26 Jul 2024
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack Xiaoyue Xu Qinyuan Ye Xiang Ren 129 10 0 23 Jul 2024
Private prediction for large-scale synthetic text generation Kareem Amin Alex Bie Weiwei Kong Alexey Kurakin Natalia Ponomareva Umar Syed Andreas Terzis Sergei Vassilvitskii SyDa SILM 139 6 0 16 Jul 2024
Training Task Experts through Retrieval Based Distillation Jiaxin Ge Xueying Jia Vijay Viswanathan Hongyin Luo Graham Neubig 88 3 0 07 Jul 2024