Title
LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures Francisco Aguilera-Martínez Fernando Berzal PILM 55 0 0 02 May 2025
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions Yiming Du Wenyu Huang Danna Zheng Zhaowei Wang Sébastien Montella Mirella Lapata Kam-Fai Wong Jeff Z. Pan KELM MU 83 2 0 01 May 2025
WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference Youhui Zuo Sibo Wei C. Zhang Zhuorui Liu Wenpeng Lu Dawei Song VLM 58 0 0 23 Mar 2025
360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation Hamed Firooz Maziar Sanjabi Adrian Englhardt Aman Gupta Ben Levine ... Xiaoling Zhai Ya Xu Yu Wang Yun Dai Yun Dai ALM 47 3 0 27 Jan 2025
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection Wei Wu Zhuoshi Pan Chao Wang L. Chen Y. Bai Kun Fu Zehua Wang Hui Xiong Hui Xiong LLMAG 39 5 0 05 Nov 2024
An Evolved Universal Transformer Memory Edoardo Cetin Qi Sun Tianyu Zhao Yujin Tang 170 0 0 17 Oct 2024
LongGenBench: Long-context Generation Benchmark Xiang Liu Peijie Dong Xuming Hu Xiaowen Chu RALM 55 8 0 05 Oct 2024
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens Cunxiang Wang Ruoxi Ning Boqi Pan Tonghui Wu Qipeng Guo ... Guangsheng Bao Xiangkun Hu Zheng Zhang Qian Wang Yue Zhang RALM 106 4 0 18 Mar 2024
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 76 78 0 19 Sep 2023
From Sparse to Soft Mixtures of Experts J. Puigcerver C. Riquelme Basil Mustafa N. Houlsby MoE 121 114 0 02 Aug 2023
ZeRO++: Extremely Efficient Collective Communication for Giant Model Training Guanhua Wang Heyang Qin S. A. Jacobs Connor Holmes Samyam Rajbhandari Olatunji Ruwase Feng Yan Lei Yang Yuxiong He VLM 62 57 0 16 Jun 2023
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text Wangchunshu Zhou Yuchen Eleanor Jiang Peng Cui Tiannan Wang Zhenxin Xiao Yifan Hou Ryan Cotterell Mrinmaya Sachan RALM LLMAG 90 59 0 22 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 102 122 0 02 May 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 322 3,021 0 22 Mar 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 253 1,073 0 05 Oct 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 81 122 0 03 Jul 2022
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao Iz Beltagy Giuseppe Carenini Arman Cohan CVBM 83 115 0 16 Oct 2021
SHAPE: Shifted Absolute Position Embedding for Transformers Shun Kiyono Sosuke Kobayashi Jun Suzuki Kentaro Inui 236 45 0 13 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 698 0 27 Aug 2021
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences Zhenhai Zhu Radu Soricut 112 41 0 25 Jul 2021
Combiner: Full Attention Transformer with Sparse Computation Cost Hongyu Ren H. Dai Zihang Dai Mengjiao Yang J. Leskovec Dale Schuurmans Bo Dai 87 77 0 12 Jul 2021
A Simple and Effective Positional Encoding for Transformers Pu-Chin Chen Henry Tsai Srinadh Bhojanapalli Hyung Won Chung Yin-Wen Chang Chun-Sung Ferng 59 62 0 18 Apr 2021
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 204 1,109 0 09 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 227 2,431 0 04 Jan 2021
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer Siyu Ding Junyuan Shang Shuohuan Wang Yu Sun Hao Tian Hua Wu Haifeng Wang 71 52 0 31 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 285 2,017 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 252 580 0 12 Mar 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,826 0 17 Sep 2019