Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs

26 August 2023

Hao Fei

Wei Ji

Papers citing "Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs"

41 / 41 papers shown

Title
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing Hao Fei Shengqiong Wu Hao Zhang Tat-Seng Chua Shuicheng Yan 160 41 0 31 Dec 2024
FlexCache: Flexible Approximate Cache System for Video Diffusion Desen Sun Henry Tian Tim Lu Sihang Liu DiffM 103 0 0 18 Dec 2024
MotionDirector: Motion Customization of Text-to-Video Diffusion Models Rui Zhao Yuchao Gu Jay Zhangjie Wu David Junhao Zhang Jia-Wei Liu Weijia Wu Jussi Keppo Mike Zheng Shou DiffM VGen 75 113 0 12 Oct 2023
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation David Junhao Zhang Jay Zhangjie Wu Jia-Wei Liu Rui Zhao L. Ran Yuchao Gu Difei Gao Mike Zheng Shou DiffM VGen 78 218 0 27 Sep 2023
Conditional Image-to-Video Generation with Latent Flow Diffusion Models Haomiao Ni Changhao Shi Kaican Li Sharon X. Huang Martin Renqiang Min VGen DiffM 66 175 0 24 Mar 2023
Video Probabilistic Diffusion Models in Projected Latent Space Sihyun Yu Kihyuk Sohn Subin Kim Jinwoo Shin VGen DiffM 84 169 0 15 Feb 2023
Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training Ling Yang Zhilin Huang Yang Song Shenda Hong Ge Li Wentao Zhang Tengjiao Wang Guohao Li Ming-Hsuan Yang 66 55 0 21 Nov 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 140 1,528 0 05 Oct 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 81 1,409 0 29 Sep 2022
Diffusion Models for Video Prediction and Infilling Tobias Höppe Arash Mehrjou Stefan Bauer Didrik Nielsen Andrea Dittadi DiffM VGen 88 135 0 15 Jun 2022
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong Ming Ding Wendi Zheng Xinghan Liu Jie Tang DiffM 303 606 0 29 May 2022
Flexible Diffusion Modeling of Long Videos William Harvey Saeid Naderiparizi Vaden Masrani Christian D. Weilbach Frank Wood DiffM BDL VGen 222 295 0 23 May 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 393 6,859 0 13 Apr 2022
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer Songwei Ge Thomas Hayes Harry Yang Xiaoyue Yin Guan Pang David Jacobs Jia-Bin Huang Devi Parikh ViT 113 220 0 07 Apr 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 204 1,612 0 07 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 477 6,240 0 05 Apr 2022
Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks Sihyun Yu Jihoon Tack Sangwoo Mo Hyunsu Kim Junho Kim Jung-Woo Ha Jinwoo Shin DiffM VGen 97 201 0 21 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 814 9,387 0 28 Jan 2022
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2 Ivan Skorokhodov Sergey Tulyakov Mohamed Elhoseiny VGen 82 287 0 29 Dec 2021
A Good Image Generator Is What You Need for High-Resolution Video Synthesis Yu Tian Jian Ren Menglei Chai Kyle Olszewski Xi Peng Dimitris N. Metaxas Sergey Tulyakov VGen 93 186 0 30 Apr 2021
VideoGPT: Video Generation using VQ-VAE and Transformers Wilson Yan Yunzhi Zhang Pieter Abbeel A. Srinivas ViT VGen 301 500 0 20 Apr 2021
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 137 1,558 0 18 Apr 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 136 1,173 0 01 Apr 2021
Image-to-Image Retrieval by Learning Similarity between Scene Graphs Sangwoong Yoon Woo-Young Kang Sungwook Jeon SeongEun Lee C. Han Jonghun Park Eun-Sol Kim 3DH 70 43 0 29 Dec 2020
Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language Songyang Zhang Houwen Peng Jianlong Fu Yijuan Lu Jiebo Luo 67 52 0 04 Dec 2020
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 266 7,356 0 06 Oct 2020
Unbiased Scene Graph Generation from Biased Training Kaihua Tang Yulei Niu Jianqiang Huang Jiaxin Shi Hanwang Zhang CML 81 695 0 27 Feb 2020
Lower Dimensional Kernels for Video Discriminators Emmanuel Kahembwe S. Ramamoorthy 50 51 0 18 Dec 2019
Action Genome: Actions as Composition of Spatio-temporal Scene Graphs Jingwei Ji Ranjay Krishna Li Fei-Fei Juan Carlos Niebles 70 345 0 15 Dec 2019
Markov Decision Process for Video Generation V. Yushchenko Nikita Araslanov Stefan Roth VGen 54 20 0 26 Sep 2019
DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis Minfeng Zhu Pingbo Pan Wei Chen Yi Yang GAN 52 580 0 02 Apr 2019
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation Manoj Kumar Mohammad Babaeizadeh D. Erhan Chelsea Finn Sergey Levine Laurent Dinh Durk Kingma VGen 84 132 0 04 Mar 2019
Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN Masaki Saito Shunta Saito Masanori Koyama Sosuke Kobayashi 81 147 0 22 Nov 2018
Image Generation from Scene Graphs Justin Johnson Agrim Gupta Li Fei-Fei GNN 300 820 0 04 Apr 2018
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks Tao Xu Pengchuan Zhang Qiuyuan Huang Han Zhang Zhe Gan Xiaolei Huang Xiaodong He GAN ViT 108 1,716 0 28 Nov 2017
Graph Attention Networks Petar Velickovic Guillem Cucurull Arantxa Casanova Adriana Romero Pietro Lio Yoshua Bengio GNN 479 20,138 0 30 Oct 2017
Dense-Captioning Events in Videos Ranjay Krishna Kenji Hata F. Ren Li Fei-Fei Juan Carlos Niebles 136 1,244 0 02 May 2017
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling Diego Marcheggiani Ivan Titov GNN NAI 75 832 0 14 Mar 2017
Generating Videos with Scene Dynamics Carl Vondrick Hamed Pirsiavash Antonio Torralba GAN VGen 183 1,468 0 08 Sep 2016
Improved Techniques for Training GANs Tim Salimans Ian Goodfellow Wojciech Zaremba Vicki Cheung Alec Radford Xi Chen GAN 480 9,048 0 10 Jun 2016
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild K. Soomro Amir Zamir M. Shah CLIP VGen 144 6,148 0 03 Dec 2012