A Frustratingly Easy Improvement for Position Embeddings via Random
Padding

A Frustratingly Easy Improvement for Position Embeddings via Random Padding

8 May 2023

Dongyan Zhao

ArXiv (abs)PDF HTML

Papers citing "A Frustratingly Easy Improvement for Position Embeddings via Random Padding"

4 / 4 papers shown

Title
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 68 29 0 15 Jan 2024
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey Yunpeng Huang Jingwei Xu Junyu Lai Zixu Jiang Taolue Chen ... Xiaoxing Ma Lijuan Yang Zhou Xin Shupeng Li Penghao Zhao LLMAG KELM 98 66 0 21 Nov 2023
Extending Input Contexts of Language Models through Training on Segmented Sequences Petros Karypis Julian McAuley George Karypis 67 0 0 23 Oct 2023
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models Chi Han Qifan Wang Hao Peng Wenhan Xiong Yu Chen Heng Ji Sinong Wang 155 61 0 30 Aug 2023