Scaling up Masked Diffusion Models on Text

24 October 2024

Chongxuan Li

Papers citing "Scaling up Masked Diffusion Models on Text"

50 / 77 papers shown

Title
Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking Pengxiang Li Shilin Yan Joey Tsai Renrui Zhang Ruichuan An Ziyu Guo Xiaowei Gao 61 0 0 26 May 2025
FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities Jin Wang Yao Lai Aoxue Li Shifeng Zhang Jiacheng Sun Ning Kang Chengyue Wu Zhenguo Li Ping Luo 39 0 0 26 May 2025
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu Rongzhen Wang Shen Nie Xiaolu Zhang Chunwei Wu ... Jun Zhou Jianfei Chen Yankai Lin Ji-Rong Wen Chongxuan Li 168 0 0 25 May 2025
Anchored Diffusion Language Model Litu Rout Constantine Caramanis Sanjay Shakkottai 48 0 0 24 May 2025
Bridging Supervised Learning and Reinforcement Learning in Math Reasoning Huayu Chen Kaiwen Zheng Qinsheng Zhang Ganqu Cui Yin Cui Haotian Ye Tsung-Yi Lin Ming-Yu Liu Jun Zhu Haoxiang Wang OffRL LRM 224 2 0 23 May 2025
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning Zebin You Shen Nie Xiaolu Zhang Jun Hu Jun Zhou Zhiwu Lu J. Wen Chongxuan Li MLLM VLM 87 0 0 22 May 2025
Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions Dhruvesh Patel Aishwarya Sahoo Avinash Amballa Tahira Naseem Tim G. J. Rudner Andrew McCallum KELM 96 0 0 09 May 2025
d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning Siyan Zhao Devaansh Gupta Qinqing Zheng Aditya Grover DiffM LRM AI4CE 123 6 0 16 Apr 2025
Unified Multimodal Discrete Diffusion Alexander Swerdlow Mihir Prabhudesai Siddharth Gandhi Deepak Pathak Katerina Fragkiadaki DiffM 114 5 0 26 Mar 2025
Effective and Efficient Masked Image Generation Models Zebin You Jingyang Ou Xiaolu Zhang Jun Hu Jun Zhou Chongxuan Li DiffM VLM 93 3 0 10 Mar 2025
Generalized Interpolating Discrete Diffusion Dimitri von Rutte J. Fluri Yuhui Ding Antonio Orvieto Bernhard Scholkopf Thomas Hofmann DiffM 102 2 0 06 Mar 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 213 47 0 14 Feb 2025
Theoretical Benefit and Limitation of Diffusion Language Model Guhao Feng Yihan Geng Jian Guan Wei Wu Liwei Wang Di He DiffM 138 1 0 13 Feb 2025
Enabling Autoregressive Models to Fill In Masked Tokens Daniel Israel Aditya Grover Guy Van den Broeck AI4CE 148 1 0 09 Feb 2025
Simplified and Generalized Masked Diffusion for Discrete Data Jiaxin Shi Kehang Han Zehao Wang Arnaud Doucet Michalis K. Titsias DiffM 167 100 0 17 Jan 2025
[MASK] is All You Need Vincent Tao Hu Bjorn Ommer DiffM 195 5 0 09 Dec 2024
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong Shivam Agarwal Yizhe Zhang Jiacheng Ye Lin Zheng ... Peilin Zhao W. Bi Jiawei Han Hao Peng Dianbo Sui AI4CE 111 25 0 23 Oct 2024
Diffusion Guided Language Modeling Justin Lovelace Varsha Kishore Yiwei Chen Kilian Q. Weinberger 86 7 0 08 Aug 2024
Discrete Flow Matching Itai Gat Tal Remez Neta Shaul Felix Kreuk Ricky T. Q. Chen Gabriel Synnaeve Yossi Adi Y. Lipman DiffM 95 80 0 22 Jul 2024
Simple and Effective Masked Diffusion Language Models Subham Sekhar Sahoo Marianne Arriola Yair Schiff Aaron Gokaslan Edgar Marroquin Justin T Chiu Alexander M. Rush Volodymyr Kuleshov DiffM 101 110 0 11 Jun 2024
The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More O. Kitouni Niklas Nolte Diane Bouchacourt Adina Williams Mike Rabbat Mark Ibrahim LRM CLL 66 12 0 07 Jun 2024
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data Jingyang Ou Shen Nie Kaiwen Xue Fengqi Zhu Jiacheng Sun Zhenguo Li Chongxuan Li DiffM 117 50 0 06 Jun 2024
Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations Kaiwen Xue Yuhao Zhou Shen Nie Xu Min Xiaolu Zhang Jun Zhou Chongxuan Li DiffM 73 14 0 24 Apr 2024
Understanding Emergent Abilities of Language Models from the Loss Perspective Zhengxiao Du Aohan Zeng Yuxiao Dong Jie Tang UQCV LRM 115 53 0 23 Mar 2024
CLLMs: Consistency Large Language Models Siqi Kou Lanxiang Hu Zhe He Zhijie Deng Hao Zhang 76 31 0 28 Feb 2024
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models Jiacheng Ye Shansan Gong Liheng Chen Lin Zheng Jiahui Gao ... Chuan Wu Xin Jiang Zhenguo Li Wei Bi Lingpeng Kong DiffM LRM AI4CE 104 15 0 12 Feb 2024
TinyLlama: An Open-Source Small Language Model Peiyuan Zhang Guangtao Zeng Tianduo Wang Wei Lu ALM LRM 131 390 0 04 Jan 2024
Implicit Chain of Thought Reasoning via Knowledge Distillation Yuntian Deng Kiran Prasad Roland Fernandez P. Smolensky Vishrav Chaudhary Stuart M. Shieber ReLM LRM 51 51 0 02 Nov 2023
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" Lukas Berglund Meg Tong Max Kaufmann Mikita Balesni Asa Cooper Stickland Tomasz Korbak Owain Evans LRM 110 273 0 21 Sep 2023
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 130 18 0 23 Aug 2023
Bayesian Flow Networks Alex Graves R. Srivastava Timothy James Atkinson Faustino J. Gomez BDL 69 45 0 14 Aug 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 299 11,858 0 18 Jul 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 351 4,312 0 09 Jun 2023
Likelihood-Based Diffusion Language Models Ishaan Gulrajani Tatsunori B. Hashimoto DiffM 76 61 0 30 May 2023
TESS: Text-to-Text Self-Conditioned Simplex Diffusion Rabeeh Karimi Mahabadi Hamish Ivison Jaesung Tae James Henderson Iz Beltagy Matthew E. Peters Arman Cohan 70 27 0 15 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.4K 14,359 0 15 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,247 0 27 Feb 2023
Muse: Text-To-Image Generation via Masked Generative Transformers Huiwen Chang Han Zhang Jarred Barber AJ Maschinot José Lezama ... Kevin Patrick Murphy William T. Freeman Michael Rubinstein Yuanzhen Li Dilip Krishnan DiffM 251 552 0 02 Jan 2023
Score-based Continuous-time Discrete Diffusion Models Haoran Sun Lijun Yu Bo Dai Dale Schuurmans H. Dai DiffM 96 84 0 30 Nov 2022
Continuous diffusion for categorical data Sander Dieleman Laurent Sartran Arman Roshannai Nikolay Savinov Yaroslav Ganin ... Conor Durkan Curtis Hawthorne Rémi Leblond Will Grathwohl J. Adler DiffM 115 106 0 28 Nov 2022
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models Zhengfu He Tianxiang Sun Kuan-Chieh Wang Xuanjing Huang Xipeng Qiu DiffM VLM 66 127 0 28 Nov 2022
Self-conditioned Embedding Diffusion for Text Generation Robin Strudel Corentin Tallec Florent Altché Yilun Du Yaroslav Ganin ... Will Grathwohl Nikolay Savinov Sander Dieleman Laurent Sifre Rémi Leblond DiffM 62 87 0 08 Nov 2022
Concrete Score Matching: Generalized Score Matching for Discrete Data Chenlin Meng Kristy Choi Jiaming Song Stefano Ermon DiffM 216 68 0 02 Nov 2022
SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control Xiaochuang Han Sachin Kumar Yulia Tsvetkov 108 89 0 31 Oct 2022
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models Shansan Gong Mukai Li Jiangtao Feng Zhiyong Wu Lingpeng Kong 93 333 0 17 Oct 2022
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning Ting-Li Chen Ruixiang Zhang Geoffrey E. Hinton DiffM 90 307 0 08 Aug 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 193 3,889 0 26 Jul 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 279 2,480 0 15 Jun 2022
A Continuous Time Framework for Discrete Denoising Models Andrew Campbell Joe Benton Valentin De Bortoli Tom Rainforth George Deligiannidis Arnaud Doucet DiffM 234 164 0 30 May 2022
Diffusion-LM Improves Controllable Text Generation Xiang Lisa Li John Thickstun Ishaan Gulrajani Percy Liang Tatsunori B. Hashimoto AI4CE 235 825 0 27 May 2022