Make Your LLM Fully Utilize the Context

25 April 2024

Papers citing "Make Your LLM Fully Utilize the Context"

41 / 41 papers shown

Title
CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability Han Peng Jinhao Jiang Zican Dong Wayne Xin Zhao Lei Fang RALM 32 0 0 15 May 2025
Bye-bye, Bluebook? Automating Legal Procedure with Large Language Models Matthew Dahl AILaw ELM 54 0 0 05 May 2025
ACoRN: Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models Singon Kim Gunho Jung Seong-Whan Lee RALM 33 0 0 17 Apr 2025
The Use of Gaze-Derived Confidence of Inferred Operator Intent in Adjusting Safety-Conscious Haptic Assistance Jeremy D. Webb Michael Bowman Songpo Li Xiaoli Zhang 34 0 0 04 Apr 2025
MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection Yibo Yan Shen Wang Jiahao Huo Philip S. Yu Xuming Hu Qingsong Wen 140 1 0 23 Mar 2025
Tuning LLMs by RAG Principles: Towards LLM-native Memory Jiale Wei Shuchi Wu Ruochen Liu Xiang Ying Jingbo Shang Fangbo Tao RALM 65 0 0 20 Mar 2025
Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrieval Yuwei Zhang Jayanth Srinivasa Gaowen Liu Jingbo Shang LRM LLMAG RALM 95 1 0 12 Mar 2025
Lost-in-the-Middle in Long-Text Generation: Synthetic Dataset, Evaluation Framework, and Mitigation Junhao Zhang Richong Zhang Fanshuang Kong Ziyang Miao Yanhan Ye Yaowei Zheng SyDa 46 0 0 10 Mar 2025
Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling Zhenghua Wang Yiran Ding Changze Lv Zhibo Xu Tianlong Li Tianyuan Shi Xiaoqing Zheng Xuanjing Huang 43 0 0 06 Mar 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-Wei Lee 62 0 0 06 Mar 2025
LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs Jianghao Chen Junhong Wu Yangyifan Xu Junzhe Zhang 65 0 0 04 Mar 2025
LongEval: A Comprehensive Analysis of Long-Text Generation Through a Plan-based Paradigm Siwei Wu Yongqian Li Xingwei Qu Rishi Ravikumar Y. Li Tyler Loakman Shanghaoran Quan Xiaoyong Wei Shanghaoran Quan Xiaoyong Wei R. Batista-Navarro C. Lin 140 2 0 26 Feb 2025
WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale Jiaxi Li Xingxing Zhang Xun Wang Xiaolong Huang Li Dong Liang Wang Si-Qing Chen Wei Lu Furu Wei SyDa 166 0 0 23 Feb 2025
CLIPPER: Compression enables long-context synthetic data generation Chau Minh Pham Yapei Chang Mohit Iyyer SyDa 85 1 0 21 Feb 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs Sumin An Junyoung Sung Wonpyo Park Chanjun Park Paul Hongsuck Seo 100 0 0 10 Feb 2025
Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End" Francesco Sovrano Adam Bauer Alberto Bacchelli 54 0 0 09 Feb 2025
Parallel Key-Value Cache Fusion for Position Invariant RAG Philhoon Oh Jinwoo Shin James Thorne 3DV 155 0 0 13 Jan 2025
From Code to Compliance: Assessing ChatGPT's Utility in Designing an Accessible Webpage -- A Case Study Ammar Ahmed Margarida Fresco Fredrik Forsberg Hallvard Grotli 35 0 0 08 Jan 2025
GeAR: Generation Augmented Retrieval Haoyu Liu Shaohan Huang Jianfeng Liu Yuefeng Zhan H. Sun Weiwei Deng Feng Sun Furu Wei Qi Zhang 40 1 0 06 Jan 2025
Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks Hamed Firooz Maziar Sanjabi Wenlong Jiang Xiaoling Zhai 68 3 0 03 Jan 2025
Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks? Jonathan Roberts Kai Han Samuel Albanie LLMAG 144 0 0 07 Nov 2024
On the Loss of Context-awareness in General Instruction Fine-tuning Yihan Wang Andrew Bai Nanyun Peng Cho-Jui Hsieh 100 1 0 05 Nov 2024
What is Wrong with Perplexity for Long-context Language Modeling? Lizhe Fang Yifei Wang Zhaoyang Liu Chenheng Zhang Stefanie Jegelka Jinyang Gao Bolin Ding Yisen Wang 66 6 0 31 Oct 2024
MDCure: A Scalable Pipeline for Multi-Document Instruction-Following Gabrielle Kaili-May Liu Bowen Shi Avi Caciularu Idan Szpektor Arman Cohan 72 4 0 30 Oct 2024
LongReward: Improving Long-context Large Language Models with AI Feedback J. Zhang Zhongni Hou Xin Lv S. Cao Zhenyu Hou Yilin Niu Lei Hou Yuxiao Dong Ling Feng Juanzi Li OffRL LRM 38 8 0 28 Oct 2024
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer Haotian Tang Yecheng Wu Shang Yang Enze Xie Junsong Chen Junyu Chen Zhuoyang Zhang Han Cai Yaojie Lu Song Han 71 33 0 14 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 72 38 0 03 Oct 2024
GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understanding Yukun Cao Shuo Han Zengyi Gao Zezhong Ding Xike Xie S. Kevin Zhou 40 7 0 05 Sep 2024
LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs Yuhao Wu Ming Shan Hee Zhiqing Hu Roy Ka-Wei Lee RALM 33 7 0 03 Sep 2024
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack Xiaoyue Xu Qinyuan Ye Xiang Ren 50 6 0 23 Jul 2024
CompAct: Compressing Retrieved Documents Actively for Question Answering Chanwoong Yoon Taewhoo Lee Hyeon Hwang Minbyul Jeong Jaewoo Kang KELM RALM MQ 35 12 0 12 Jul 2024
From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data Zheyang Xiong Vasilis Papageorgiou Kangwook Lee Dimitris Papailiopoulos SyDa RALM 34 11 0 27 Jun 2024
AI-native Memory: A Pathway from LLMs Towards AGI Jingbo Shang Zai Zheng Jiale Wei Xiang Ying Felix Tao Mindverse Team LLMAG 43 7 0 26 Jun 2024
Attention Instruction: Amplifying Attention in the Middle via Prompting Meiru Zhang Zaiqiao Meng Nigel Collier 43 4 0 24 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 54 28 0 04 Jun 2024
Mitigate Position Bias in Large Language Models via Scaling a Single Dimension Yijiong Yu Huiqiang Jiang Xufang Luo Qianhui Wu Chin-Yew Lin Dongsheng Li Yuqing Yang Yongfeng Huang L. Qiu 46 9 0 04 Jun 2024
RTL-Repo: A Benchmark for Evaluating LLMs on Large-Scale RTL Design Projects Ahmed Allam Mohamed Shalan 35 14 0 27 May 2024
Just Say the Name: Online Continual Learning with Category Names Only via Data Generation Minhyuk Seo Diganta Misra Seongwon Cho Minjae Lee Jonghyun Choi CLL 37 7 0 16 Mar 2024
Where is the answer? Investigating Positional Bias in Language Model Knowledge Extraction Kuniaki Saito Kihyuk Sohn Chen-Yu Lee Yoshitaka Ushiku 62 2 0 16 Feb 2024
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 695 0 27 Aug 2021
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 243 580 0 12 Mar 2020