LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation

20 February 2025

Papers citing "LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation"

3 / 3 papers shown

Title
SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization Huashan Sun Shengyi Liao Yansen Han Yu Bai Yang Gao ... Weizhou Shen Fanqi Wan Ming Yan J.N. Zhang Fei Huang 12 0 0 16 May 2025
CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability Han Peng Jinhao Jiang Zican Dong Wayne Xin Zhao Lei Fang RALM 30 0 0 15 May 2025
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free Z. Qiu Z. Wang Bo Zheng Zeyu Huang Kaiyue Wen ... Fei Huang Suozhi Huang Dayiheng Liu Jingren Zhou Junyang Lin MoE 28 0 0 10 May 2025