Extending Context Window of Large Language Models via Positional
Interpolation

v1v2 (latest)

Extending Context Window of Large Language Models via Positional Interpolation

27 June 2023

ArXiv (abs)PDF HTML

Papers citing "Extending Context Window of Large Language Models via Positional Interpolation"

17 / 117 papers shown

Title
DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent) Zongxin Yang Guikun Chen Xiaodi Li Wenguan Wang Yi Yang LM&Ro LLMAG 181 41 0 16 Jan 2024
Extending LLMs' Context Window with 100 Samples Yikai Zhang Junlong Li Pengfei Liu 89 12 0 13 Jan 2024
AstroLLaMA-Chat: Scaling AstroLLaMA with Conversational and Diverse Datasets Ernest Perkowski Boyao Wang Tuan Dung Nguyen Yuan-Sen Ting Sandor Kruk ... Michael J. Smith Huiling Liu Kevin Schawinski K. Iyer I. Ciucă AI4MH 85 12 0 03 Jan 2024
LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning Hongye Jin Xiaotian Han Jingfeng Yang Zhimeng Jiang Zirui Liu Chia-Yuan Chang Huiyuan Chen Helen Zhou 124 118 0 02 Jan 2024
Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning Xijie Huang Li Lyna Zhang Kwang-Ting Cheng Fan Yang Mao Yang LRM ReLM 94 13 0 14 Dec 2023
Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks Ben Feuer Chinmay Hegde Niv Cohen 115 11 0 17 Nov 2023
LongQLoRA: Efficient and Effective Method to Extend Context Length of Large Language Models Jianxin Yang 43 6 0 08 Nov 2023
CLEX: Continuous Length Extrapolation for Large Language Models Guanzheng Chen Xin Li Zaiqiao Meng Shangsong Liang Li Bing 102 32 0 25 Oct 2023
On Context Utilization in Summarization with Large Language Models Mathieu Ravaut Aixin Sun Nancy F. Chen Shafiq Joty 93 14 0 16 Oct 2023
End-to-end Story Plot Generator Hanlin Zhu Andrew Cohen Danqing Wang Kevin Kaichuang Yang Xiaomeng Yang Jiantao Jiao Yuandong Tian 62 5 0 13 Oct 2023
Qwen Technical Report Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang ... Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou Tianhang Zhu OSLM 369 1,921 0 28 Sep 2023
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models Yukang Chen Shengju Qian Haotian Tang Xin Lai Zhijian Liu Song Han Jiaya Jia 167 170 0 21 Sep 2023
BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model Nolan Dey Daria Soboleva Faisal Al-Khateeb Bowen Yang Ribhu Pathria ... Robert Myers Jacob Robert Steeves Natalia Vassilieva Marvin Tom Joel Hestness MoE 87 16 0 20 Sep 2023
FLM-101B: An Open LLM and How to Train It with $100K Budget$ Xiang Li Yiqun Yao Xin Jiang Xuezhi Fang Xuying Meng ... Li Du Bowen Qin Zheng Zhang Aixin Sun Yequan Wang 149 22 0 07 Sep 2023
YaRN: Efficient Context Window Extension of Large Language Models Bowen Peng Jeffrey Quesnelle Honglu Fan Enrico Shippole OSLM 97 264 0 31 Aug 2023
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models Qingyue Wang Y. Fu Yanan Cao Zhiliang Tian Shi Wang Dacheng Tao LLMAG KELM RALM 178 29 0 29 Aug 2023
Local Large Language Models for Complex Structured Medical Tasks V. Bumgardner Aaron D. Mullen Samuel E. Armstrong Caylin D. Hickey Jeffrey A. Talbert 75 5 0 03 Aug 2023