v1v2 (latest)

DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models

28 November 2022

Zhengfu He

Tianxiang Sun

Kuan-Chieh Wang

Xuanjing Huang

Xipeng Qiu

DiffM

VLM

ArXiv (abs)PDF HTML

Papers citing "DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models"

44 / 44 papers shown

Title
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu Rongzhen Wang Shen Nie Xiaolu Zhang Chunwei Wu ... Jun Zhou Jianfei Chen Yankai Lin Ji-Rong Wen Chongxuan Li 186 2 0 25 May 2025
Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modeling Tianyu Xie Shuchen Xue Zijin Feng Tianyang Hu Jiacheng Sun Zhenguo Li Cheng Zhang DiffM 772 0 0 23 May 2025
dKV-Cache: The Cache for Diffusion Language Models Xinyin Ma Runpeng Yu Gongfan Fang Xinchao Wang DiffM 99 3 0 21 May 2025
Overcoming Dimensional Factorization Limits in Discrete Diffusion Models through Quantum Joint Distribution Learning Chuangtao Chen Qinglin Zhao Mengchu Zhou Zhimin He Haozhen Situ DiffM 221 0 0 08 May 2025
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Marianne Arriola Aaron Gokaslan Justin T Chiu Zhihan Yang Zhixuan Qi Jiaqi Han Subham Sekhar Sahoo Volodymyr Kuleshov DiffM 265 25 0 12 Mar 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 242 55 0 14 Feb 2025
Theoretical Benefit and Limitation of Diffusion Language Model Guhao Feng Yihan Geng Jian Guan Wei Wu Liwei Wang Di He DiffM 148 1 0 13 Feb 2025
Scaling up Masked Diffusion Models on Text Shen Nie Fengqi Zhu Chao Du Tianyu Pang Qian Liu Guangtao Zeng Min Lin Chongxuan Li AI4CE 187 30 0 24 Oct 2024
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong Shivam Agarwal Yizhe Zhang Jiacheng Ye Lin Zheng ... Peilin Zhao W. Bi Jiawei Han Hao Peng Dianbo Sui AI4CE 134 27 0 23 Oct 2024
Text-Guided Multi-Property Molecular Optimization with a Diffusion Language Model Yida Xiong Kun Li Jiameng Chen Hongzhi Zhang Di Lin Shirui Pan Wenbin Hu 97 3 0 17 Oct 2024
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data Jingyang Ou Shen Nie Kaiwen Xue Fengqi Zhu Jiacheng Sun Zhenguo Li Chongxuan Li DiffM 147 54 0 06 Jun 2024
Glauber Generative Model: Discrete Diffusion Models via Binary Classification Harshit Varma Dheeraj M. Nagaraj Karthikeyan Shanmugam VLM 161 3 0 27 May 2024
IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognition Xiaomeng Yang Zhi Qiao Yu Zhou DiffM 177 1 0 19 Dec 2023
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 161 19 0 23 Aug 2023
Language Models are Few-shot Learners for Prognostic Prediction Zekai Chen Mariann Micsinai Balan Kevin Brown LM&MA 97 90 0 24 Feb 2023
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models Shansan Gong Mukai Li Jiangtao Feng Zhiyong Wu Lingpeng Kong 96 333 0 17 Oct 2022
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps Cheng Lu Yuhao Zhou Fan Bao Jianfei Chen Chongxuan Li Jun Zhu DiffM 241 1,464 0 02 Jun 2022
Diffusion-LM Improves Controllable Text Generation Xiang Lisa Li John Thickstun Ishaan Gulrajani Percy Liang Tatsunori B. Hashimoto AI4CE 242 833 0 27 May 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 466 6,083 0 23 May 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 425 6,921 0 13 Apr 2022
Black-Box Tuning for Language-Model-as-a-Service Tianxiang Sun Yunfan Shao Hong Qian Xuanjing Huang Xipeng Qiu VLM 158 275 0 10 Jan 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 511 15,788 0 20 Dec 2021
Step-unrolled Denoising Autoencoders for Text Generation Nikolay Savinov Junyoung Chung Mikolaj Binkowski Erich Elsen Aaron van den Oord DiffM 125 120 0 13 Dec 2021
Autoregressive Diffusion Models Emiel Hoogeboom Alexey A. Gritsenko Jasmijn Bastings Ben Poole Rianne van den Berg Tim Salimans DiffM 107 155 0 05 Oct 2021
Structured Denoising Diffusion Models in Discrete State-Spaces Jacob Austin Daniel D. Johnson Jonathan Ho Daniel Tarlow Rianne van den Berg DiffM 203 948 0 07 Jul 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 605 4,099 0 18 Apr 2021
Improved Denoising Diffusion Probabilistic Models Alex Nichol Prafulla Dhariwal DiffM 354 3,728 0 18 Feb 2021
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 304 7,500 0 06 Oct 2020
DiffWave: A Versatile Diffusion Model for Audio Synthesis Zhifeng Kong Ming-Yu Liu Jiaji Huang Kexin Zhao Bryan Catanzaro DiffM BDL 169 1,468 0 21 Sep 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 770 18,408 0 19 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 905 42,520 0 28 May 2020
Non-Autoregressive Machine Translation with Latent Alignments Chitwan Saharia William Chan Saurabh Saxena Mohammad Norouzi 74 159 0 16 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 377 1,493 0 18 Mar 2020
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 266 10,880 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 506 20,376 0 23 Oct 2019
Levenshtein Transformer Jiatao Gu Changhan Wang Jake Zhao 127 359 0 27 May 2019
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model Alex Jinpeng Wang Kyunghyun Cho VLM 112 358 0 11 Feb 2019
Language GANs Falling Short Massimo Caccia Lucas Caccia W. Fedus Hugo Larochelle Joelle Pineau Laurent Charlin 224 219 0 06 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,324 0 11 Oct 2018
Texygen: A Benchmarking Platform for Text Generation Models Yaoming Zhu Sidi Lu Lei Zheng Jiaxian Guo Weinan Zhang Jun Wang Yong Yu 115 693 0 06 Feb 2018
Non-Autoregressive Neural Machine Translation Jiatao Gu James Bradbury Caiming Xiong Victor O.K. Li R. Socher 107 798 0 07 Nov 2017
Quasar: Datasets for Question Answering by Search and Reading Bhuwan Dhingra Kathryn Mazaitis William W. Cohen RALM 104 185 0 12 Jul 2017
Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Narain Sohl-Dickstein Eric A. Weiss Niru Maheswaranathan Surya Ganguli SyDa DiffM 312 7,035 0 12 Mar 2015
One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling Ciprian Chelba Tomas Mikolov M. Schuster Qi Ge T. Brants P. Koehn T. Robinson 190 1,109 0 11 Dec 2013