Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2306.15595
Cited By
v1
v2 (latest)
Extending Context Window of Large Language Models via Positional Interpolation
27 June 2023
Shouyuan Chen
Sherman Wong
Liangjian Chen
Yuandong Tian
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Extending Context Window of Large Language Models via Positional Interpolation"
50 / 117 papers shown
Title
When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
Zhen Xu
Shang Zhu
Jue Wang
Junlin Wang
Ben Athiwaratkun
Chi Wang
James Zou
Ce Zhang
LLMAG
17
0
0
19 Jun 2025
From General to Targeted Rewards: Surpassing GPT-4 in Open-Ended Long-Context Generation
Zhihan Guo
Jiele Wu
Wenqian Cui
Yifei Zhang
Minda Hu
Yufei Wang
Irwin King
ALM
LRM
22
0
0
19 Jun 2025
Efficient Serving of LLM Applications with Probabilistic Demand Modeling
Yifei Liu
Zuo Gan
Zhenghao Gan
Weiye Wang
Chen Chen
...
Xusheng Chen
Zhenhua Han
Yifei Zhu
Shixuan Sun
Minyi Guo
18
0
0
17 Jun 2025
LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
Xiaoran Liu
Zhigeng Liu
Zengfeng Huang
Qipeng Guo
Ziwei He
Xipeng Qiu
41
0
0
17 Jun 2025
Multipole Attention for Efficient Long Context Reasoning
Coleman Hooper
Sebastian Zhao
Luca Manolache
Sehoon Kim
Michael W. Mahoney
Y. Shao
Kurt Keutzer
Amir Gholami
OffRL
LRM
26
0
0
16 Jun 2025
Long-Short Alignment for Effective Long-Context Modeling in LLMs
Tianqi Du
Haotian Huang
Yifei Wang
Yisen Wang
21
0
0
13 Jun 2025
AbsenceBench: Language Models Can't Tell What's Missing
Harvey Yiyun Fu
Aryan Shrivastava
Jared Moore
Peter West
Chenhao Tan
Ari Holtzman
RALM
15
0
0
13 Jun 2025
Beyond Benchmarks: A Novel Framework for Domain-Specific LLM Evaluation and Knowledge Mapping
Nitin Sharma
Thomas Wolfers
Çağatay Yıldız
ALM
24
0
0
09 Jun 2025
SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models
Y. Wu
Yushi Bai
Zhiqiang Hu
Juanzi Li
Roy Ka-wei Lee
66
0
0
04 Jun 2025
Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
Woomin Song
Sai Muralidhar Jayanthi
S. Ronanki
Kanthashree Mysore Sathyendra
Jinwoo Shin
Aram Galstyan
Shubham Katiyar
S. Bodapati
VLM
52
0
0
01 Jun 2025
Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models
Boheng Sheng
Jiacheng Yao
Meicong Zhang
Guoxiu He
RALM
45
0
0
01 Jun 2025
NexusSum: Hierarchical LLM Agents for Long-Form Narrative Summarization
Hyuntak Kim
Byung-Hak Kim
35
0
0
30 May 2025
Curse of High Dimensionality Issue in Transformer for Long-context Modeling
Shuhai Zhang
Zeng You
Yaofo Chen
Z. Wen
Qianyue Wang
Zhijie Qiu
Yuanqing Li
Mingkui Tan
44
0
0
28 May 2025
Select, Read, and Write: A Multi-Agent Framework of Full-Text-based Related Work Generation
Xiaochuan Liu
Ruihua Song
Xiting Wang
Xu Chen
42
0
0
26 May 2025
AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science
An Luo
Xun Xian
Jin Du
Fangqiao Tian
G. Wang
...
Jayanth Srinivasa
Ashish Kundu
Charles Fleming
Mingyi Hong
Jie Ding
13
0
0
25 May 2025
100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?
Wang Yang
Hongye Jin
Shaochen Zhong
Song Jiang
Qifan Wang
Vipin Chaudhary
Xiaotian Han
ELM
55
0
0
25 May 2025
LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
Chaochen Gao
Xing Wu
Zijia Lin
Debing Zhang
Songlin Hu
SyDa
214
0
0
22 May 2025
SELF: Self-Extend the Context Length With Logistic Growth Function
Phat Thanh Dang
Saahil Thoppay
Wang Yang
Qifan Wang
Vipin Chaudhary
Xiaotian Han
108
0
0
22 May 2025
Scale-invariant Attention
Ben Anson
Xi Wang
Laurence Aitchison
LRM
105
0
0
20 May 2025
PSC: Extending Context Window of Large Language Models via Phase Shift Calibration
Wenqiao Zhu
Chao Xu
Lulu Wang
Jun Wu
107
1
0
18 May 2025
The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)
Zihao Wang
Yibo Jiang
Jiahao Yu
Heqing Huang
104
0
0
01 May 2025
OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training
Juntao Zhao
Qi Lu
Wei Jia
Borui Wan
Lei Zuo
...
Size Zheng
Yanghua Peng
H. Lin
Xin Liu
Chuan Wu
AI4CE
139
0
0
14 Apr 2025
Leveraging State Space Models in Long Range Genomics
Matvei Popov
Aymen Kallala
Anirudha Ramesh
Narimane Hennouni
Shivesh Khaitan
Rick Gentry
Alain-Sam Cohen
Mamba
139
0
0
07 Apr 2025
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts
Yifei Yu
Qian Zhang
Lingfeng Qiao
Di Yin
Fang Li
Jie Wang
Zheyu Chen
Suncong Zheng
Xiaolong Liang
Xingwu Sun
95
0
0
07 Apr 2025
HOT: Hadamard-based Optimized Training
Seonggon Kim
Juncheol Shin
Seung-taek Woo
Eunhyeok Park
112
0
0
27 Mar 2025
Long-Context Autoregressive Video Modeling with Next-Frame Prediction
Yuchao Gu
Weijia Mao
Mike Zheng Shou
VGen
176
11
0
25 Mar 2025
A Survey on Transformer Context Extension: Approaches and Evaluation
Yijun Liu
Jinzheng Yu
Yang Xu
Zhongyang Li
Qingfu Zhu
LLMAG
128
3
0
17 Mar 2025
Context-aware Biases for Length Extrapolation
Ali Veisi
Hamidreza Amirzadeh
Amir Mansourian
165
1
0
11 Mar 2025
InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models
Yuchen Yan
Yongliang Shen
Yuhang Liu
Jin Jiang
Hao Fei
Jian Shao
Yueting Zhuang
LRM
ReLM
144
10
0
09 Mar 2025
LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs
Jianghao Chen
Junhong Wu
Yangyifan Xu
J.N. Zhang
105
1
0
04 Mar 2025
LongAttn: Selecting Long-context Training Data via Token-level Attention
Longyun Wu
Dawei Zhu
Guangxiang Zhao
Zhuocheng Yu
Junfeng Ran
Xiangyu Wong
Lin Sun
Sujian Li
108
2
0
24 Feb 2025
The Role of Sparsity for Length Generalization in Transformers
Noah Golowich
Samy Jelassi
David Brandfonbrener
Sham Kakade
Eran Malach
83
0
0
24 Feb 2025
Generalizing From Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning
Wenhao Zhu
Pinzhen Chen
Hanxu Hu
Shujian Huang
Fei Yuan
Jiajun Chen
Alexandra Birch
SyDa
146
4
0
24 Feb 2025
WildLong: Synthesizing Realistic Long-Context Instruction Data at Scale
Jiaxi Li
Xingxing Zhang
Xun Wang
Xiaolong Huang
Li Dong
Liang Wang
Si-Qing Chen
Wei Lu
Furu Wei
SyDa
476
1
0
23 Feb 2025
RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers
Min Zhao
Guande He
Yixiao Chen
Hongzhou Zhu
Chong Li
Jun Zhu
VGen
132
11
0
21 Feb 2025
LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data
Cehao Yang
Xueyuan Lin
Chengjin Xu
Xuhui Jiang
Shengjie Ma
Aofan Liu
Hui Xiong
Jian Guo
LRM
71
2
0
18 Feb 2025
HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading
Cheng Luo
Zefan Cai
Hanshi Sun
Jinqi Xiao
Bo Yuan
Wen Xiao
Junjie Hu
Jiawei Zhao
Beidi Chen
Anima Anandkumar
126
2
0
18 Feb 2025
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning
Qifan Yu
Zhenyu He
Sijie Li
Xun Zhou
Jun Zhang
Jingjing Xu
Di He
OffRL
LRM
141
5
0
12 Feb 2025
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation
Zican Dong
Junyi Li
Jinhao Jiang
Mingyu Xu
Wayne Xin Zhao
Bin Wang
Xin Wu
VLM
371
5
0
11 Feb 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs
Sumin An
Junyoung Sung
Wonpyo Park
Chanjun Park
Paul Hongsuck Seo
232
0
0
10 Feb 2025
Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End"
Francesco Sovrano
Adam Bauer
Alberto Bacchelli
112
1
0
09 Feb 2025
Can LLMs Maintain Fundamental Abilities under KV Cache Compression?
Xiang Liu
Zhenheng Tang
Hong Chen
Peijie Dong
Zeyu Li
Xiuze Zhou
Bo Li
Xuming Hu
Xiaowen Chu
475
7
0
04 Feb 2025
Context-Aware Hierarchical Merging for Long Document Summarization
Litu Ou
Mirella Lapata
MoMe
535
1
0
03 Feb 2025
SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
Changhun Lee
Jun-gyu Jin
Jun-gyu Jin
Younghyun Cho
Eunhyeok Park
RALM
LRM
119
0
0
25 Jan 2025
LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion
Zhan Ling
Kang Liu
Kai Yan
Yue Yang
Weijian Lin
Ting-Han Fan
Lingfeng Shen
Zhengyin Du
Jiecao Chen
ReLM
ELM
LRM
109
8
0
25 Jan 2025
Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass
Jianing Yang
Alexander Sax
Kevin J. Liang
Mikael Henaff
Hao Tang
Ang Cao
J. Chai
Franziska Meier
Matt Feiszli
3DGS
193
31
0
23 Jan 2025
NExtLong: Toward Effective Long-Context Training without Long Documents
Chaochen Gao
Xing Wu
Zijia Lin
Debing Zhang
Songlin Hu
SyDa
186
2
0
22 Jan 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models
Thibaut Thonet
Jos Rozen
Laurent Besacier
RALM
225
3
0
20 Jan 2025
Visual RAG: Expanding MLLM visual knowledge without fine-tuning
Mirco Bonomo
Simone Bianco
VLM
111
5
0
18 Jan 2025
Guiding Retrieval using LLM-based Listwise Rankers
Mandeep Rathee
Sean MacAvaney
Avishek Anand
KELM
LRM
157
6
0
17 Jan 2025
1
2
3
Next