Simple and Effective Masked Diffusion Language Models

11 June 2024

Justin T Chiu

Papers citing "Simple and Effective Masked Diffusion Language Models"

50 / 51 papers shown

Title
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu Rongzhen Wang Shen Nie Xiaolu Zhang Chunwei Wu ... Jun Zhou Jianfei Chen Yankai Lin Ji-Rong Wen Chongxuan Li 168 0 0 25 May 2025
Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking Chen-Hao Chao Wei-Fang Sun Hanwen Liang Chun-Yi Lee Rahul G. Krishnan DiffM 368 0 0 24 May 2025
Discrete Neural Flow Samplers with Locally Equivariant Transformer Zijing Ou Ruixiang Zhang Yingzhen Li 64 0 0 23 May 2025
Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling Tianyu Xie Shuchen Xue Zijin Feng Tianyang Hu Jiacheng Sun Zhenguo Li Cheng Zhang DiffM 759 0 0 23 May 2025
Learning Flexible Forward Trajectories for Masked Molecular Diffusion Hyunjin Seo Taewon Kim Sihyun Yu SungSoo Ahn DiffM AI4CE 121 0 0 22 May 2025
dKV-Cache: The Cache for Diffusion Language Models Xinyin Ma Runpeng Yu Gongfan Fang Xinchao Wang DiffM 93 1 0 21 May 2025
Multi-Objective-Guided Discrete Flow Matching for Controllable Biological Sequence Design Tong Chen Yinuo Zhang Sophia Tang Pranam Chatterjee 176 2 0 11 May 2025
Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions Dhruvesh Patel Aishwarya Sahoo Avinash Amballa Tahira Naseem Tim G. J. Rudner Andrew McCallum KELM 93 0 0 09 May 2025
Guide your favorite protein sequence generative model Junhao Xiong Hunter Nisonoff Maria Lukarska Ishan Gaur Luke M. Oltrogge David F. Savage Jennifer Listgarten DiffM 219 2 0 07 May 2025
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Marianne Arriola Aaron Gokaslan Justin T Chiu Zhihan Yang Zhixuan Qi Jiaqi Han Subham Sekhar Sahoo Volodymyr Kuleshov DiffM 235 16 0 12 Mar 2025
Preference-Based Alignment of Discrete Diffusion Models Umberto Borso Davide Paglieri Jude Wells Tim Rocktaschel 104 3 0 11 Mar 2025
Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator Kaiwen Zheng Yongxin Chen Huayu Chen Guande He Xuan Li Jun Zhu Qinsheng Zhang DiffM 121 3 0 03 Mar 2025
Remasking Discrete Diffusion Models with Inference-Time Scaling Guanghan Wang Yair Schiff Subham Sekhar Sahoo Volodymyr Kuleshov DiffM 129 13 0 01 Mar 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 213 47 0 14 Feb 2025
Simplified and Generalized Masked Diffusion for Discrete Data Jiaxin Shi Kehang Han Zehao Wang Arnaud Doucet Michalis K. Titsias DiffM 167 100 0 17 Jan 2025
OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows Shufan Li Konstantinos Kallidromitis Akash Gokul Zichun Liao Yusuke Kato Kazuki Kozuka Aditya Grover VGen 137 7 0 02 Dec 2024
Beyond Autoregression: Fast LLMs via Self-Distillation Through Time Justin Deschenaux Çağlar Gülçehre 103 4 0 28 Oct 2024
Scaling up Masked Diffusion Models on Text Shen Nie Fengqi Zhu Chao Du Tianyu Pang Qian Liu Guangtao Zeng Min Lin Chongxuan Li AI4CE 130 28 0 24 Oct 2024
Structure Language Models for Protein Conformation Generation Jiarui Lu Xiaoyin Chen Stephen Zhewen Lu Chence Shi Hongyu Guo Yoshua Bengio Xiangbo Shu DiffM 77 3 0 24 Oct 2024
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning Jiacheng Ye Jiahui Gao Shansan Gong Lin Zheng Xin Jiang Zhiyu Li Dianbo Sui DiffM LRM 146 23 0 18 Oct 2024
Probing the Latent Hierarchical Structure of Data via Diffusion Models Antonio Sclocchi Alessandro Favero Noam Itzhak Levi Matthieu Wyart DiffM 86 5 0 17 Oct 2024
Distillation of Discrete Diffusion through Dimensional Correlations Satoshi Hayakawa Yuhta Takida Masaaki Imaizumi Hiromi Wakaki Yuki Mitsufuji DiffM 92 3 0 11 Oct 2024
Think While You Generate: Discrete Diffusion with Planned Denoising Sulin Liu Juno Nam Andrew Campbell Hannes Stärk Yilun Xu Tommi Jaakkola Rafael Gómez-Bombarelli DiffM 116 11 0 08 Oct 2024
Discrete Copula Diffusion Hoang Trung-Dung Oliver Broadrick Mathias Niepert Guy Van den Broeck DiffM 143 12 0 02 Oct 2024
Glauber Generative Model: Discrete Diffusion Models via Binary Classification Harshit Varma Dheeraj M. Nagaraj Karthikeyan Shanmugam VLM 135 3 0 27 May 2024
Categorical Flow Matching on Statistical Manifolds Chaoran Cheng Jiahan Li Jian-wei Peng Ge Liu 103 11 0 26 May 2024
Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling Yair Schiff Chia-Hsiang Kao Aaron Gokaslan Tri Dao Albert Gu Volodymyr Kuleshov Mamba 59 94 0 05 Mar 2024
Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors Wasu Top Piriyakulkij Yingheng Wang Volodymyr Kuleshov DiffM 74 1 0 05 Jan 2024
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 130 18 0 23 Aug 2023
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution Eric N. D. Nguyen Michael Poli Marjan Faizi A. Thomas Callum Birch-Sykes ... Stefano Massaroli Yoshua Bengio Stefano Ermon S. Baccus Christopher Ré MedIm 84 251 0 27 Jun 2023
Scalable Diffusion Models with Transformers William S. Peebles Saining Xie GNN 90 2,301 0 19 Dec 2022
Latent Diffusion for Language Generation Justin Lovelace Varsha Kishore Chao-gang Wan Eliot Shekhtman Kilian Q. Weinberger DiffM 91 79 0 19 Dec 2022
Score-based Continuous-time Discrete Diffusion Models Haoran Sun Lijun Yu Bo Dai Dale Schuurmans H. Dai DiffM 96 84 0 30 Nov 2022
Continuous diffusion for categorical data Sander Dieleman Laurent Sartran Arman Roshannai Nikolay Savinov Yaroslav Ganin ... Conor Durkan Curtis Hawthorne Rémi Leblond Will Grathwohl J. Adler DiffM 113 106 0 28 Nov 2022
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models Zhengfu He Tianxiang Sun Kuan-Chieh Wang Xuanjing Huang Xipeng Qiu DiffM VLM 64 127 0 28 Nov 2022
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning Ting-Li Chen Ruixiang Zhang Geoffrey E. Hinton DiffM 90 307 0 08 Aug 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 324 759 0 27 Aug 2021
Structured Denoising Diffusion Models in Discrete State-Spaces Jacob Austin Daniel D. Johnson Jonathan Ho Daniel Tarlow Rianne van den Berg DiffM 167 941 0 07 Jul 2021
Variational Diffusion Models Diederik P. Kingma Tim Salimans Ben Poole Jonathan Ho DiffM 178 1,121 0 01 Jul 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 278 2,453 0 20 Apr 2021
OmniNet: Omnidirectional Representations from Transformers Yi Tay Mostafa Dehghani V. Aribandi Jai Gupta Philip Pham Zhen Qin Dara Bahri Da-Cheng Juan Donald Metzler 76 28 0 01 Mar 2021
Score-Based Generative Modeling through Stochastic Differential Equations Yang Song Jascha Narain Sohl-Dickstein Diederik P. Kingma Abhishek Kumar Stefano Ermon Ben Poole DiffM SyDa 335 6,480 0 26 Nov 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 424 20,181 0 23 Oct 2019
Generative Modeling by Estimating Gradients of the Data Distribution Yang Song Stefano Ermon SyDa DiffM 258 3,916 0 12 Jul 2019
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model Alex Jinpeng Wang Kyunghyun Cho VLM 79 356 0 11 Feb 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 241 3,728 0 09 Jan 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,159 0 20 Apr 2018
A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents Arman Cohan Franck Dernoncourt Doo Soon Kim Trung Bui Seokhwan Kim W. Chang Nazli Goharian 467 761 0 16 Apr 2018
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 319 2,859 0 26 Sep 2016
Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Narain Sohl-Dickstein Eric A. Weiss Niru Maheswaranathan Surya Ganguli SyDa DiffM 301 6,949 0 12 Mar 2015