Zebra: Extending Context Window with Layerwise Grouped Local-Global
Attention

Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention

14 December 2023

Dong Yu

ArXiv (abs)PDF HTML

Papers citing "Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention"

4 / 4 papers shown

Title
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 523 3 0 03 Apr 2025
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 478 2 0 20 Sep 2024
Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding Zhenyu Zhang Runjin Chen Shiwei Liu Zhewei Yao Olatunji Ruwase Beidi Chen Xiaoxia Wu Zhangyang Wang 95 36 0 05 Mar 2024
Fast Transformer Decoding: One Write-Head is All You Need Noam M. Shazeer 165 478 0 06 Nov 2019