Video Diffusion Models: A Survey

6 May 2024

ArXiv (abs)PDF HTML Github (44★)

Papers citing "Video Diffusion Models: A Survey"

50 / 68 papers shown

Title
Open-Sora: Democratizing Efficient Video Production for All Zangwei Zheng Xiangyu Peng Tianji Yang Chenhui Shen Shenggui Li Hongxin Liu Yukun Zhou Tianyi Li Yang You VGen 148 241 0 31 Dec 2024
Bespoke Non-Stationary Solvers for Fast Sampling of Diffusion and Flow Models Neta Shaul Uriel Singer Ricky T. Q. Chen Matt Le Ali K. Thabet Albert Pumarola Y. Lipman DiffM 71 4 0 02 Mar 2024
Latte: Latent Diffusion Transformer for Video Generation Xin Ma Yaohui Wang Gengyun Jia Xinyuan Chen Ziqiang Liu Yuan-Fang Li Cunjian Chen Yu Qiao DiffM VGen 240 270 0 05 Jan 2024
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations Hakan Inan Kartikeya Upasani Jianfeng Chi Rashi Rungta Krithika Iyer ... Michael Tontchev Qing Hu Brian Fuller Davide Testuggine Madian Khabsa AI4MH 159 447 0 07 Dec 2023
I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models Shiwei Zhang Jiayu Wang Yingya Zhang Kang Zhao Hangjie Yuan Zhan Qin Xiang Wang Deli Zhao Jingren Zhou DiffM VGen 110 227 0 07 Nov 2023
SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction Xinyuan Chen Yaohui Wang Lingjun Zhang Shaobin Zhuang Xin Ma Jiashuo Yu Yali Wang Dahua Lin Yu Qiao Ziwei Liu VGen DiffM 70 140 0 31 Oct 2023
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation Haoxin Chen Menghan Xia Yin-Yin He Yong Zhang Xiaodong Cun ... Yaofang Liu Qifeng Chen Xintao Wang Chao-Liang Weng Ying Shan DiffM 70 307 0 30 Oct 2023
GAIA-1: A Generative World Model for Autonomous Driving Masane Fuchi Lloyd Russell Hudson Yeo Zak Murez Hiroto Minami Alex Kendall Tomohiro Takagi Gianluca Corrado VGen 89 241 0 29 Sep 2023
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation David Junhao Zhang Jay Zhangjie Wu Jia-Wei Liu Rui Zhao L. Ran Yuchao Gu Difei Gao Mike Zheng Shou DiffM VGen 86 220 0 27 Sep 2023
Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts Yuyang Zhao Enze Xie Lanqing Hong Zhenguo Li G. Lee DiffM VGen 68 33 0 15 May 2023
AADiff: Audio-Aligned Video Synthesis with Text-to-Image Diffusion Seungwoo Lee Chaerin Kong D. Jeon Nojun Kwak DiffM 74 20 0 06 May 2023
Pix2Video: Video Editing using Image Diffusion Duygu Ceylan C. Huang Niloy J. Mitra DiffM VGen 82 260 0 22 Mar 2023
Video-P2P: Video Editing with Cross-attention Control Shaoteng Liu Yuechen Zhang Wenbo Li Zhe Lin Jiaya Jia DiffM VGen 190 217 0 08 Mar 2023
GenAug: Retargeting behaviors to unseen situations via Generative Augmentation Zoey Chen Sho Kiami Abhishek Gupta Vikash Kumar LM&Ro 67 86 0 13 Feb 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 173 4,146 1 10 Feb 2023
Dreamix: Video Diffusion Models are General Video Editors Eyal Molad Eliahu Horwitz Dani Valevski Alex Rav-Acha Yossi Matias Yael Pritch Yaniv Leviathan Yedid Hoshen DiffM VGen 122 187 0 02 Feb 2023
Improving and generalizing flow-based generative models with minibatch optimal transport Alexander Tong Kilian Fatras Nikolay Malkin G. Huguet Yanlei Zhang Jarrid Rector-Brooks Guy Wolf Yoshua Bengio OOD DiffM OT 110 294 0 01 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 426 4,563 0 30 Jan 2023
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation Jay Zhangjie Wu Yixiao Ge Xintao Wang Weixian Lei Yuchao Gu Yufei Shi Wynne Hsu Ying Shan Xiaohu Qie Mike Zheng Shou VGen 114 737 0 22 Dec 2022
Scalable Diffusion Models with Transformers William S. Peebles Saining Xie GNN 93 2,386 0 19 Dec 2022
Is Conditional Generative Modeling all you need for Decision-Making? Anurag Ajay Yilun Du Abhi Gupta J. Tenenbaum Tommi Jaakkola Pulkit Agrawal DiffM 132 394 0 28 Nov 2022
InstructPix2Pix: Learning to Follow Image Editing Instructions Tim Brooks Aleksander Holynski Alexei A. Efros DiffM 207 1,813 0 17 Nov 2022
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 194 3,482 0 16 Oct 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 162 1,528 0 05 Oct 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 81 1,421 0 29 Sep 2022
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow Xingchao Liu Chengyue Gong Qiang Liu OOD 192 1,003 0 07 Sep 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein Kfir Aberman 279 2,861 0 25 Aug 2022
Prompt-to-Prompt Image Editing with Cross Attention Control Amir Hertz Ron Mokady J. Tenenbaum Kfir Aberman Yael Pritch Daniel Cohen-Or DiffM 200 1,773 0 02 Aug 2022
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion Rinon Gal Yuval Alaluf Yuval Atzmon Or Patashnik Amit H. Bermano Gal Chechik Daniel Cohen-Or 161 1,889 0 02 Aug 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 193 3,898 0 26 Jul 2022
Elucidating the Design Space of Diffusion-Based Generative Models Tero Karras M. Aittala Timo Aila S. Laine DiffM 201 1,981 0 01 Jun 2022
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong Ming Ding Wendi Zheng Xinghan Liu Jie Tang DiffM 311 608 0 29 May 2022
Flexible Diffusion Modeling of Long Videos William Harvey Saeid Naderiparizi Vaden Masrani Christian D. Weilbach Frank Wood DiffM BDL VGen 222 295 0 23 May 2022
CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers Ming Ding Wendi Zheng Wenyi Hong Jie Tang VLM 106 331 0 28 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 410 6,897 0 13 Apr 2022
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer Songwei Ge Thomas Hayes Harry Yang Xiaoyue Yin Guan Pang David Jacobs Jia-Bin Huang Devi Parikh ViT 123 223 0 07 Apr 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 204 1,626 0 07 Apr 2022
Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks Sihyun Yu Jihoon Tack Sangwoo Mo Hyunsu Kim Junho Kim Jung-Woo Ha Jinwoo Shin DiffM VGen 102 201 0 21 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 542 4,360 0 28 Jan 2022
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 460 15,665 0 20 Dec 2021
NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion Chenfei Wu Jian Liang Lei Ji Fan Yang Yuejian Fang Daxin Jiang Nan Duan ViT VGen 72 296 0 24 Nov 2021
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 70 194 0 19 Nov 2021
MUSIQ: Multi-scale Image Quality Transformer Junjie Ke Qifei Wang Yilin Wang P. Milanfar Feng Yang 223 666 0 12 Aug 2021
LoRA: Low-Rank Adaptation of Large Language Models J. E. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang Weizhu Chen OffRL AI4TS AI4CE ALM AIMat 477 10,367 0 17 Jun 2021
Cascaded Diffusion Models for High Fidelity Image Generation Jonathan Ho Chitwan Saharia William Chan David J. Fleet Mohammad Norouzi Tim Salimans 164 1,222 0 30 May 2021
A Good Image Generator Is What You Need for High-Resolution Video Synthesis Yu Tian Jian Ren Menglei Chai Kyle Olszewski Xi Peng Dimitris N. Metaxas Sergey Tulyakov VGen 98 186 0 30 Apr 2021
VideoGPT: Video Generation using VQ-VAE and Transformers Wilson Yan Yunzhi Zhang Pieter Abbeel A. Srinivas ViT VGen 308 502 0 20 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 151 1,176 0 01 Apr 2021
Score-Based Generative Modeling through Stochastic Differential Equations Yang Song Jascha Narain Sohl-Dickstein Diederik P. Kingma Abhishek Kumar Stefano Ermon Ben Poole DiffM SyDa 344 6,480 0 26 Nov 2020
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 283 7,384 0 06 Oct 2020