v1v2 (latest)

Extending Context Window of Large Language Models via Positional Interpolation

27 June 2023

Papers citing "Extending Context Window of Large Language Models via Positional Interpolation"

50 / 117 papers shown

Title
When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework Zhen Xu Shang Zhu Jue Wang Junlin Wang Ben Athiwaratkun Chi Wang James Zou Ce Zhang LLMAG 17 0 0 19 Jun 2025
From General to Targeted Rewards: Surpassing GPT-4 in Open-Ended Long-Context Generation Zhihan Guo Jiele Wu Wenqian Cui Yifei Zhang Minda Hu Yufei Wang Irwin King ALM LRM 22 0 0 19 Jun 2025
Efficient Serving of LLM Applications with Probabilistic Demand Modeling Yifei Liu Zuo Gan Zhenghao Gan Weiye Wang Chen Chen ... Xusheng Chen Zhenhua Han Yifei Zhu Shixuan Sun Minyi Guo 18 0 0 17 Jun 2025
LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs Xiaoran Liu Zhigeng Liu Zengfeng Huang Qipeng Guo Ziwei He Xipeng Qiu 41 0 0 17 Jun 2025
Multipole Attention for Efficient Long Context Reasoning Coleman Hooper Sebastian Zhao Luca Manolache Sehoon Kim Michael W. Mahoney Y. Shao Kurt Keutzer Amir Gholami OffRL LRM 26 0 0 16 Jun 2025
Long-Short Alignment for Effective Long-Context Modeling in LLMs Tianqi Du Haotian Huang Yifei Wang Yisen Wang 21 0 0 13 Jun 2025
AbsenceBench: Language Models Can't Tell What's Missing Harvey Yiyun Fu Aryan Shrivastava Jared Moore Peter West Chenhao Tan Ari Holtzman RALM 15 0 0 13 Jun 2025
Beyond Benchmarks: A Novel Framework for Domain-Specific LLM Evaluation and Knowledge Mapping Nitin Sharma Thomas Wolfers Çağatay Yıldız ALM 24 0 0 09 Jun 2025
SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models Y. Wu Yushi Bai Zhiqiang Hu Juanzi Li Roy Ka-wei Lee 66 0 0 04 Jun 2025
Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers Woomin Song Sai Muralidhar Jayanthi S. Ronanki Kanthashree Mysore Sathyendra Jinwoo Shin Aram Galstyan Shubham Katiyar S. Bodapati VLM 52 0 0 01 Jun 2025
Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models Boheng Sheng Jiacheng Yao Meicong Zhang Guoxiu He RALM 45 0 0 01 Jun 2025
NexusSum: Hierarchical LLM Agents for Long-Form Narrative Summarization Hyuntak Kim Byung-Hak Kim 35 0 0 30 May 2025
Curse of High Dimensionality Issue in Transformer for Long-context Modeling Shuhai Zhang Zeng You Yaofo Chen Z. Wen Qianyue Wang Zhijie Qiu Yuanqing Li Mingkui Tan 44 0 0 28 May 2025
Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation Xiaochuan Liu Ruihua Song Xiting Wang Xu Chen 42 0 0 26 May 2025
AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science An Luo Xun Xian Jin Du Fangqiao Tian G. Wang ... Jayanth Srinivasa Ashish Kundu Charles Fleming Mingyi Hong Jie Ding 13 0 0 25 May 2025
100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability? Wang Yang Hongye Jin Shaochen Zhong Song Jiang Qifan Wang Vipin Chaudhary Xiaotian Han ELM 55 0 0 25 May 2025
LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions Chaochen Gao Xing Wu Zijia Lin Debing Zhang Songlin Hu SyDa 214 0 0 22 May 2025
SELF: Self-Extend the Context Length With Logistic Growth Function Phat Thanh Dang Saahil Thoppay Wang Yang Qifan Wang Vipin Chaudhary Xiaotian Han 108 0 0 22 May 2025
Scale-invariant Attention Ben Anson Xi Wang Laurence Aitchison LRM 105 0 0 20 May 2025
PSC: Extending Context Window of Large Language Models via Phase Shift Calibration Wenqiao Zhu Chao Xu Lulu Wang Jun Wu 107 1 0 18 May 2025
The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them) Zihao Wang Yibo Jiang Jiahao Yu Heqing Huang 104 0 0 01 May 2025
OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training Juntao Zhao Qi Lu Wei Jia Borui Wan Lei Zuo ... Size Zheng Yanghua Peng H. Lin Xin Liu Chuan Wu AI4CE 139 0 0 14 Apr 2025
Leveraging State Space Models in Long Range Genomics Matvei Popov Aymen Kallala Anirudha Ramesh Narimane Hennouni Shivesh Khaitan Rick Gentry Alain-Sam Cohen Mamba 139 0 0 07 Apr 2025
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts Yifei Yu Qian Zhang Lingfeng Qiao Di Yin Fang Li Jie Wang Zheyu Chen Suncong Zheng Xiaolong Liang Xingwu Sun 95 0 0 07 Apr 2025
HOT: Hadamard-based Optimized Training Seonggon Kim Juncheol Shin Seung-taek Woo Eunhyeok Park 112 0 0 27 Mar 2025
Long-Context Autoregressive Video Modeling with Next-Frame Prediction Yuchao Gu Weijia Mao Mike Zheng Shou VGen 176 11 0 25 Mar 2025
A Survey on Transformer Context Extension: Approaches and Evaluation Yijun Liu Jinzheng Yu Yang Xu Zhongyang Li Qingfu Zhu LLMAG 128 3 0 17 Mar 2025
Context-aware Biases for Length Extrapolation Ali Veisi Hamidreza Amirzadeh Amir Mansourian 165 1 0 11 Mar 2025
InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models Yuchen Yan Yongliang Shen Yuhang Liu Jin Jiang Hao Fei Jian Shao Yueting Zhuang LRM ReLM 144 10 0 09 Mar 2025
LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs Jianghao Chen Junhong Wu Yangyifan Xu J.N. Zhang 105 1 0 04 Mar 2025
LongAttn: Selecting Long-context Training Data via Token-level Attention Longyun Wu Dawei Zhu Guangxiang Zhao Zhuocheng Yu Junfeng Ran Xiangyu Wong Lin Sun Sujian Li 108 2 0 24 Feb 2025
The Role of Sparsity for Length Generalization in Transformers Noah Golowich Samy Jelassi David Brandfonbrener Sham Kakade Eran Malach 83 0 0 24 Feb 2025
Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning Wenhao Zhu Pinzhen Chen Hanxu Hu Shujian Huang Fei Yuan Jiajun Chen Alexandra Birch SyDa 146 4 0 24 Feb 2025
WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale Jiaxi Li Xingxing Zhang Xun Wang Xiaolong Huang Li Dong Liang Wang Si-Qing Chen Wei Lu Furu Wei SyDa 476 1 0 23 Feb 2025
RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers Min Zhao Guande He Yixiao Chen Hongzhou Zhu Chong Li Jun Zhu VGen 132 11 0 21 Feb 2025
LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data Cehao Yang Xueyuan Lin Chengjin Xu Xuhui Jiang Shengjie Ma Aofan Liu Hui Xiong Jian Guo LRM 71 2 0 18 Feb 2025
HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading Cheng Luo Zefan Cai Hanshi Sun Jinqi Xiao Bo Yuan Wen Xiao Junjie Hu Jiawei Zhao Beidi Chen Anima Anandkumar 126 2 0 18 Feb 2025
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning Qifan Yu Zhenyu He Sijie Li Xun Zhou Jun Zhang Jingjing Xu Di He OffRL LRM 141 5 0 12 Feb 2025
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation Zican Dong Junyi Li Jinhao Jiang Mingyu Xu Wayne Xin Zhao Bin Wang Xin Wu VLM 371 5 0 11 Feb 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs Sumin An Junyoung Sung Wonpyo Park Chanjun Park Paul Hongsuck Seo 232 0 0 10 Feb 2025
Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End" Francesco Sovrano Adam Bauer Alberto Bacchelli 112 1 0 09 Feb 2025
Can LLMs Maintain Fundamental Abilities under KV Cache Compression? Xiang Liu Zhenheng Tang Hong Chen Peijie Dong Zeyu Li Xiuze Zhou Bo Li Xuming Hu Xiaowen Chu 475 7 0 04 Feb 2025
Context-Aware Hierarchical Merging for Long Document Summarization Litu Ou Mirella Lapata MoMe 535 1 0 03 Feb 2025
SEAL: Scaling to Emphasize Attention for Long-Context Retrieval Changhun Lee Jun-gyu Jin Jun-gyu Jin Younghyun Cho Eunhyeok Park RALM LRM 119 0 0 25 Jan 2025
LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion Zhan Ling Kang Liu Kai Yan Yue Yang Weijian Lin Ting-Han Fan Lingfeng Shen Zhengyin Du Jiecao Chen ReLM ELM LRM 109 8 0 25 Jan 2025
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass Jianing Yang Alexander Sax Kevin J. Liang Mikael Henaff Hao Tang Ang Cao J. Chai Franziska Meier Matt Feiszli 3DGS 193 31 0 23 Jan 2025
NExtLong: Toward Effective Long-Context Training without Long Documents Chaochen Gao Xing Wu Zijia Lin Debing Zhang Songlin Hu SyDa 186 2 0 22 Jan 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models Thibaut Thonet Jos Rozen Laurent Besacier RALM 225 3 0 20 Jan 2025
Visual RAG: Expanding MLLM visual knowledge without fine-tuning Mirco Bonomo Simone Bianco VLM 111 5 0 18 Jan 2025
Guiding Retrieval using LLM-based Listwise Rankers Mandeep Rathee Sean MacAvaney Avishek Anand KELM LRM 157 6 0 17 Jan 2025