Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning

23 August 2023

Jiasheng Ye

Quanquan Gu

Papers citing "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"

50 / 117 papers shown

Title
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning Zebin You Shen Nie Xiaolu Zhang Jun Hu Jun Zhou Zhiwu Lu J. Wen Chongxuan Li MLLM VLM 85 0 0 22 May 2025
Target Concrete Score Matching: A Holistic Framework for Discrete Diffusion Ruixiang Zhang Shuangfei Zhai Yizhe Zhang James Thornton Zijing Ou Joshua M. Susskind Navdeep Jaitly DiffM 69 2 0 23 Apr 2025
Diffusion Models for Tabular Data: Challenges, Current Progress, and Future Directions Zhong Li Qi Huang Lincen Yang Jiayang Shi Zhao Yang Niki van Stein Thomas Bäck M. Leeuwen DiffM 92 0 0 24 Feb 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 204 47 0 14 Feb 2025
Theoretical Benefit and Limitation of Diffusion Language Model Guhao Feng Yihan Geng Jian Guan Wei Wu Liwei Wang Di He DiffM 138 1 0 13 Feb 2025
Simplified and Generalized Masked Diffusion for Discrete Data Jiaxin Shi Kehang Han Zehao Wang Arnaud Doucet Michalis K. Titsias DiffM 152 100 0 17 Jan 2025
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Though Violet Xiang Charlie Snell Kanishk Gandhi Alon Albalak Anikait Singh ... Dakota Mahan Louis Castricato Jan-Philipp Fränken Nick Haber Chelsea Finn LRM 101 50 0 08 Jan 2025
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference Benjamin Warner Antoine Chaffin Benjamin Clavié Orion Weller Oskar Hallström ... Tom Aarsen Nathan Cooper Griffin Adams Jeremy Howard Iacopo Poli 136 120 0 18 Dec 2024
Predicting Emergent Capabilities by Finetuning Charlie Snell Eric Wallace Dan Klein Sergey Levine ELM LRM 117 5 0 25 Nov 2024
Scaling up Masked Diffusion Models on Text Shen Nie Fengqi Zhu Chao Du Tianyu Pang Qian Liu Guangtao Zeng Min Lin Chongxuan Li AI4CE 130 28 0 24 Oct 2024
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong Shivam Agarwal Yizhe Zhang Jiacheng Ye Lin Zheng ... Peilin Zhao W. Bi Jiawei Han Hao Peng Dianbo Sui AI4CE 109 25 0 23 Oct 2024
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning Jiacheng Ye Jiahui Gao Shansan Gong Lin Zheng Xin Jiang Zhiyu Li Dianbo Sui DiffM LRM 140 23 0 18 Oct 2024
DPLM-2: A Multimodal Diffusion Protein Language Model Xinze Wang Zaixiang Zheng Fei Ye Dongyu Xue Shujian Huang Quanquan Gu 55 16 0 17 Oct 2024
Latent Diffusion Models for Controllable RNA Sequence Generation Kaixuan Huang Yukang Yang Kaidi Fu Yanyi Chu Le Cong Mengdi Wang 76 2 0 15 Sep 2024
Discrete Flow Matching Itai Gat Tal Remez Neta Shaul Felix Kreuk Ricky T. Q. Chen Gabriel Synnaeve Yossi Adi Y. Lipman DiffM 95 80 0 22 Jul 2024
Simple and Effective Masked Diffusion Language Models Subham Sekhar Sahoo Marianne Arriola Yair Schiff Aaron Gokaslan Edgar Marroquin Justin T Chiu Alexander M. Rush Volodymyr Kuleshov DiffM 94 110 0 11 Jun 2024
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data Jingyang Ou Shen Nie Kaiwen Xue Fengqi Zhu Jiacheng Sun Zhenguo Li Chongxuan Li DiffM 110 50 0 06 Jun 2024
Guided Discrete Diffusion for Electronic Health Record Generation Jun Han Zixiang Chen Yongqian Li Yiwen Kou Eran Halperin Robert E. Tillman Quanquan Gu MedIm DiffM 71 7 0 18 Apr 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 83 77 0 11 Mar 2024
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling Jun Zhan Junqi Dai Jiasheng Ye Yunhua Zhou Dong Zhang ... Jie Fu Tao Gui Tianxiang Sun Yugang Jiang Xipeng Qiu MLLM 67 131 0 19 Feb 2024
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models Jiacheng Ye Shansan Gong Liheng Chen Lin Zheng Jiahui Gao ... Chuan Wu Xin Jiang Zhenguo Li Wei Bi Lingpeng Kong DiffM LRM AI4CE 99 15 0 12 Feb 2024
Transfer Learning for Text Diffusion Models Kehang Han Kathleen Kenealy Aditya Barua Noah Fiedel Noah Constant VLM AI4CE 87 4 0 30 Jan 2024
Lumiere: A Space-Time Diffusion Model for Video Generation Omer Bar-Tal Hila Chefer Omer Tov Charles Herrmann Roni Paiss ... T. Michaeli Oliver Wang Deqing Sun Tali Dekel Inbar Mosseri VGen 174 241 0 23 Jan 2024
Fast Sampling via Discrete Non-Markov Diffusion Models Zixiang Chen Huizhuo Yuan Yongqian Li Yiwen Kou Junkai Zhang Quanquan Gu DiffM 53 7 0 14 Dec 2023
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution Aaron Lou Chenlin Meng Stefano Ermon DiffM 79 107 0 25 Oct 2023
Mistral 7B Albert Q. Jiang Alexandre Sablayrolles A. Mensch Chris Bamford Devendra Singh Chaplot ... Teven Le Scao Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LRM 74 2,197 0 10 Oct 2023
Making LLaMA SEE and Draw with SEED Tokenizer Yuying Ge Sijie Zhao Ziyun Zeng Yixiao Ge Chen Li Xintao Wang Ying Shan 69 134 0 02 Oct 2023
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" Lukas Berglund Meg Tong Max Kaufmann Mikita Balesni Asa Cooper Stickland Tomasz Korbak Owain Evans LRM 110 272 0 21 Sep 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 293 11,858 0 18 Jul 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 181 1,145 0 31 May 2023
Likelihood-Based Diffusion Language Models Ishaan Gulrajani Tatsunori B. Hashimoto DiffM 69 61 0 30 May 2023
Scaling Data-Constrained Language Models Niklas Muennighoff Alexander M. Rush Boaz Barak Teven Le Scao Aleksandra Piktus Nouamane Tazi S. Pyysalo Thomas Wolf Colin Raffel ALM 87 216 0 25 May 2023
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities Dong Zhang Shimin Li Xin Zhang Jun Zhan Pengyu Wang Yaqian Zhou Xipeng Qiu AuLLM MLLM 111 334 0 18 May 2023
Tree of Thoughts: Deliberate Problem Solving with Large Language Models Shunyu Yao Dian Yu Jeffrey Zhao Izhak Shafran Thomas Griffiths Yuan Cao Karthik Narasimhan LM&Ro LRM AI4CE 138 1,936 0 17 May 2023
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation Tong Wu Zhihao Fan Xiao Liu Yeyun Gong Yelong Shen ... Juntao Li Zhongyu Wei Jian Guo Nan Duan Weizhu Chen VLM 115 63 0 16 May 2023
Causal Reasoning and Large Language Models: Opening a New Frontier for Causality Emre Kıcıman Robert Osazuwa Ness Amit Sharma Chenhao Tan LRM ELM 107 276 0 28 Apr 2023
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation Fei Huang Pei Ke Minlie Huang AI4CE 61 8 0 24 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 529 4,725 0 17 Apr 2023
Understanding Causality with Large Language Models: Feasibility and Opportunities Cheng Zhang Stefan Bauer Paul N. Bennett Jian-chuan Gao Wenbo Gong ... Joel Jennings Chao Ma Tom Minka Nick Pawlowski James Vaughan LRM ELM 95 59 0 11 Apr 2023
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise Jiaao Chen Aston Zhang Mu Li Alexander J. Smola Diyi Yang DiffM 49 20 0 10 Apr 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,182 0 27 Feb 2023
DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Mingxuan Wang DiffM 78 49 0 20 Feb 2023
A Reparameterized Discrete Diffusion Model for Text Generation Lin Zheng Jianbo Yuan Lei Yu Lingpeng Kong DiffM 91 68 0 11 Feb 2023
Structure-informed Language Models Are Protein Designers Zaixiang Zheng Yifan Deng Dongyu Xue Yi Zhou YE Fei Quanquan Gu 78 95 0 03 Feb 2023
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning Shayne Longpre Le Hou Tu Vu Albert Webson Hyung Won Chung ... Denny Zhou Quoc V. Le Barret Zoph Jason W. Wei Adam Roberts ALM 101 670 0 31 Jan 2023
Specializing Smaller Language Models towards Multi-Step Reasoning Yao Fu Hao-Chun Peng Litu Ou Ashish Sabharwal Tushar Khot ReLM LRM 86 258 0 30 Jan 2023
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language Seyed Mehran Kazemi Najoung Kim Deepti Bhatia Xinyuan Xu Deepak Ramachandran LRM 83 80 0 20 Dec 2022
SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers Hongyi Yuan Zheng Yuan Chuanqi Tan Fei Huang Songfang Huang DiffM 90 69 0 20 Dec 2022
Teaching Small Language Models to Reason Lucie Charlotte Magister Jonathan Mallinson Jakub Adamek Eric Malmi Aliaksei Severyn LRM AI4CE ReLM 171 268 0 16 Dec 2022
Continuous diffusion for categorical data Sander Dieleman Laurent Sartran Arman Roshannai Nikolay Savinov Yaroslav Ganin ... Conor Durkan Curtis Hawthorne Rémi Leblond Will Grathwohl J. Adler DiffM 113 104 0 28 Nov 2022