v1v2 (latest)

UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs

26 June 2024

Papers citing "UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs"

21 / 21 papers shown

Title
Efficient Streaming Language Models with Attention Sinks Michel Lang Yuandong Tian Beidi Chen Song Han Mike Lewis AI4TS RALM 125 783 0 29 Sep 2023
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models Yukang Chen Shengju Qian Haotian Tang Xin Lai Zhijian Liu Song Han Jiaya Jia 111 164 0 21 Sep 2023
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 134 89 0 19 Sep 2023
YaRN: Efficient Context Window Extension of Large Language Models Bowen Peng Jeffrey Quesnelle Honglu Fan Enrico Shippole OSLM 72 261 0 31 Aug 2023
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding Yushi Bai Xin Lv Jiajie Zhang Hong Lyu Jiankai Tang ... Aohan Zeng Lei Hou Yuxiao Dong Jie Tang Juanzi Li LLMAG RALM 92 600 0 28 Aug 2023
In-context Autoencoder for Context Compression in a Large Language Model Tao Ge Jing Hu Lei Wang Xun Wang Si-Qing Chen Furu Wei RALM 74 79 0 13 Jul 2023
Lost in the Middle: How Language Models Use Long Contexts Nelson F. Liu Kevin Lin John Hewitt Ashwin Paranjape Michele Bevilacqua Fabio Petroni Percy Liang RALM 111 1,629 0 06 Jul 2023
Extending Context Window of Large Language Models via Positional Interpolation Shouyuan Chen Sherman Wong Liangjian Chen Yuandong Tian 158 539 0 27 Jun 2023
Adapting Language Models to Compress Contexts Alexis Chevalier Alexander Wettig Anirudh Ajith Danqi Chen LLMAG 50 187 0 24 May 2023
Learning to Compress Prompts with Gist Tokens Jesse Mu Xiang Lisa Li Noah D. Goodman VLM 105 220 0 17 Apr 2023
The MiniPile Challenge for Data-Efficient Language Models Jean Kaddour MoE ALM 109 45 0 17 Apr 2023
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models David Wingate Mohammad Shoeybi Taylor Sorensen 56 77 0 06 Oct 2022
Recurrent Memory Transformer Aydar Bulatov Yuri Kuratov Andrey Kravchenko CLL 43 109 0 14 Jul 2022
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 186 1,597 0 30 Sep 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,771 0 29 Jun 2020
Compressive Transformers for Long-Range Sequence Modelling Jack W. Rae Anna Potapenko Siddhant M. Jayakumar Timothy Lillicrap RALM VLM KELM 71 652 0 13 Nov 2019
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 725 132,199 0 12 Jun 2017
Unbiasing Truncated Backpropagation Through Time Corentin Tallec Yann Ollivier 59 76 0 23 May 2017
The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings K. Choromanski Mark Rowland Adrian Weller 84 85 0 02 Mar 2017
Unbiased Online Recurrent Optimization Corentin Tallec Yann Ollivier 69 99 0 16 Feb 2017
Training recurrent networks online without backtracking Yann Ollivier Corentin Tallec Guillaume Charpiat 63 39 0 28 Jul 2015