Extending Llama-3's Context Ten-Fold Overnight

Extending Llama-3's Context Ten-Fold Overnight

30 April 2024

Peitian Zhang

Zheng Liu

Hongjin Qian

Papers citing "Extending Llama-3's Context Ten-Fold Overnight"

7 / 7 papers shown

Title
SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization Huashan Sun Shengyi Liao Yansen Han Yu Bai Yang Gao ... Weizhou Shen Fanqi Wan Ming Yan Jiaming Zhang Fei Huang 25 0 0 16 May 2025
Two are better than one: Context window extension with multi-grained self-injection Wei Han Pan Zhou Soujanya Poria Shuicheng Yan 29 0 0 25 Oct 2024
LOGO -- Long cOntext aliGnment via efficient preference Optimization Zecheng Tang Zechen Sun Juntao Li Qiaoming Zhu Min Zhang 37 1 0 24 Oct 2024
Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models David Castillo-Bolado Joseph Davidson Finlay Gray Marek Rosa 34 3 0 30 Sep 2024
Attention Overflow: Language Model Input Blur during Long-Context Missing Items Recommendation Damien Sileo LRM RALM 34 0 0 18 Jul 2024
How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensions Julia Kharchenko Tanya Roosta Aman Chadha Chirag Shah 32 16 0 21 Jun 2024
Are Long-LLMs A Necessity For Long-Context Tasks? Hongjin Qian Zheng Liu Peitian Zhang Kelong Mao Yujia Zhou Xu Chen Zhicheng Dou 39 9 0 24 May 2024