LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs

LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs

17 June 2025

ArXiv (abs)PDF HTML

Papers citing "LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs"

16 / 16 papers shown

Title
Beyond Homogeneous Attention: Memory-Efficient LLMs via Fourier-Approximated KV Cache Xiaoran Liu Siyang He Qiqi Wang Ruixiao Li Yuerong Song ... Qun Liu Zengfeng Huang Qipeng Guo Ziwei He Xipeng Qiu VLM 7 0 0 13 Jun 2025
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu Rongzhen Wang Shen Nie Xiaolu Zhang Chunwei Wu ... Jun Zhou Jianfei Chen Yankai Lin Ji-Rong Wen Chongxuan Li 173 2 0 25 May 2025
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning Zebin You Shen Nie Xiaolu Zhang Jun Hu Jun Zhou Zhiwu Lu J. Wen Chongxuan Li MLLM VLM 90 2 0 22 May 2025
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding Runpeng Yu Xinyin Ma Xinchao Wang MLLM 88 2 0 22 May 2025
MMaDA: Multimodal Large Diffusion Language Models Ling Yang Ye Tian Bowen Li Xinchen Zhang Ke Shen Yunhai Tong Mengdi Wang VLM LRM 116 5 0 21 May 2025
dKV-Cache: The Cache for Diffusion Language Models Xinyin Ma Runpeng Yu Gongfan Fang Xinchao Wang DiffM 97 3 0 21 May 2025
Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models Zemin Huang Zhiyang Chen Zijun Wang Tiancheng Li Guo-Jun Qi DiffM LRM AI4CE 87 2 0 15 May 2025
d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning Siyan Zhao Devaansh Gupta Qinqing Zheng Aditya Grover DiffM LRM AI4CE 125 9 0 16 Apr 2025
Thus Spake Long-Context Large Language Model Xiaoran Liu Ruixiao Li Mianqiu Huang Zhigeng Liu Yuerong Song ... Linlin Li Qiang Liu Yaqian Zhou Xuanjing Huang Xipeng Qiu 44 5 0 24 Feb 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 220 54 0 14 Feb 2025
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong Shivam Agarwal Yizhe Zhang Jiacheng Ye Lin Zheng ... Peilin Zhao W. Bi Jiawei Han Hao Peng Dianbo Sui AI4CE 117 27 0 23 Oct 2024
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning Jiacheng Ye Jiahui Gao Shansan Gong Lin Zheng Xin Jiang Zhiyu Li Dianbo Sui DiffM LRM 146 25 0 18 Oct 2024
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data Jingyang Ou Shen Nie Kaiwen Xue Fengqi Zhu Jiacheng Sun Zhenguo Li Chongxuan Li DiffM 117 54 0 06 Jun 2024
YaRN: Efficient Context Window Extension of Large Language Models Bowen Peng Jeffrey Quesnelle Honglu Fan Enrico Shippole OSLM 72 261 0 31 Aug 2023
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 328 774 0 27 Aug 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 284 2,500 0 20 Apr 2021