VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation

1 September 2023

Errui Ding

Jingdong Wang

VGen

ArXiv PDF HTML

Papers citing "VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation"

50 / 53 papers shown

Title
ACT-R: Adaptive Camera Trajectories for Single View 3D Reconstruction Yun Wang Mingrui Zhao Ali Mahdavi Amiri Hao Zhang 39 0 0 13 May 2025
Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion Evgeniia Vu Andrei Boiarov Dmitry Vetrov VGen 57 0 0 13 Mar 2025
UVE: Are MLLMs Unified Evaluators for AI-Generated Videos? Yuanxin Liu Rui Zhu Shuhuai Ren Jiacong Wang Haoyuan Guo Xu Sun Lu Jiang 237 1 0 13 Mar 2025
VRMDiff: Text-Guided Video Referring Matting Generation of Diffusion Lehan Yang Jincen Song Tianlong Wang Daiqing Qi Weili Shi Yuheng Liu Sheng Li DiffM VOS VGen 81 0 0 11 Mar 2025
HOFAR: High-Order Augmentation of Flow Autoregressive Transformers Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao Song Mingda Wan 77 1 0 11 Mar 2025
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 86 2 0 14 Nov 2024
Loong: Generating Minute-level Long Videos with Autoregressive Language Models Yuqing Wang Tianwei Xiong Daquan Zhou Zhijie Lin Yang Zhao Bingyi Kang Jiashi Feng Xihui Liu VGen 61 24 0 03 Oct 2024
OSV: One Step is Enough for High-Quality Image to Video Generation Xiaofeng Mao Zhengkai Jiang Fu-Yun Wang Wenbing Zhu Hao Chen Mingmin Chi Yabiao Wang Wenhan Luo DiffM VGen 84 9 0 17 Sep 2024
DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes Jianbiao Mei Yukai Ma Xuemeng Yang Licheng Wen Tiantian Wei Min Dou Yukai Ma Min Dou Botian Shi Yong Liu DiffM VGen 82 3 0 06 Sep 2024
MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning Haoning Wu Shaocheng Shen Qiang Hu Xiaoyun Zhang Ya Zhang Yanfeng Wang 54 10 0 20 Aug 2024
Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model Min Zhao Hongzhou Zhu Chendong Xiang Kaiwen Zheng Chongxuan Li Jun Zhu 74 8 0 22 Jun 2024
Training-free Camera Control for Video Generation Chen Hou Guoqiang Wei VGen DiffM 101 32 0 14 Jun 2024
Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality Tianle Zhang Langtian Ma Yuchen Yan Yuchen Zhang Kai Wang ... Wenqi Shao Yang You Yu Qiao Ping Luo Kaipeng Zhang VGen 98 2 0 13 Jun 2024
Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control Zhengfei Kuang Shengqu Cai Hao He Yinghao Xu Hongsheng Li Leonidas Guibas Gordon Wetzstein VGen DiffM 74 31 0 27 May 2024
Predicting Long-horizon Futures by Conditioning on Geometry and Time Tarasha Khurana Deva Ramanan AI4TS 60 0 0 17 Apr 2024
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition Sihyun Yu Weili Nie De-An Huang Boyi Li Jinwoo Shin A. Anandkumar VGen DiffM 52 15 0 21 Mar 2024
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text Roberto Henschel Levon Khachatryan Daniil Hayrapetyan Hayk Poghosyan Vahram Tadevosyan Zhangyang Wang Shant Navasardyan Humphrey Shi DiffM VGen 101 79 0 21 Mar 2024
Generalized Predictive Model for Autonomous Driving Jiazhi Yang Shenyuan Gao Yihang Qiu Li Chen Tianyu Li ... Ping Luo Jun Zhang Andreas Geiger Yu Qiao Hongyang Li VGen 73 60 0 14 Mar 2024
VideoElevator: Elevating Video Generation Quality with Versatile Text-to-Image Diffusion Models Yabo Zhang Yuxiang Wei Xianhui Lin Zheng Hui Peiran Ren Xuansong Xie Xiangyang Ji Wangmeng Zuo VGen 51 6 0 08 Mar 2024
Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation Joseph Cho Fachrina Dewi Puspitasari Sheng Zheng Jingyao Zheng Lik-Hang Lee Tae-Ho Kim Choong Seon Hong Chaoning Zhang EGVM VGen 46 41 0 08 Mar 2024
Context-aware Talking Face Video Generation Meidai Xuanyuan Yuwang Wang Honglei Guo Qionghai Dai DiffM 48 0 0 28 Feb 2024
Diffusion Model-Based Image Editing: A Survey Yi Huang Jiancheng Huang Yifan Liu Mingfu Yan Jiaxi Lv Jianzhuang Liu Wei Xiong He Zhang Liangliang Cao Liangliang Cao EGVM 79 89 0 27 Feb 2024
UniVG: Towards UNIfied-modal Video Generation Ludan Ruan Lei Tian Chuanwei Huang Xu Zhang Xinyan Xiao VGen DiffM 42 3 0 17 Jan 2024
Towards A Better Metric for Text-to-Video Generation Jay Zhangjie Wu Guian Fang Haoning Wu Xintao Wang Yixiao Ge ... Rui Zhao Weisi Lin Wynne Hsu Ying Shan Mike Zheng Shou VGen 46 35 0 15 Jan 2024
Generative Multimodal Models are In-Context Learners Quan-Sen Sun Yufeng Cui Xiaosong Zhang Fan Zhang Qiying Yu ... Yueze Wang Yongming Rao Jingjing Liu Tiejun Huang Xinlong Wang MLLM LRM 50 254 0 20 Dec 2023
Hierarchical Spatio-temporal Decoupling for Text-to-Video Generation Zhiwu Qing Shiwei Zhang Jiayu Wang Xiang Wang Yujie Wei Yingya Zhang Changxin Gao Nong Sang VGen DiffM 39 40 0 07 Dec 2023
DreamVideo: Composing Your Dream Videos with Customized Subject and Motion Yujie Wei Shiwei Zhang Zhiwu Qing Hangjie Yuan Zhiheng Liu Yu Liu Yingya Zhang Jingren Zhou Hongming Shan DiffM VGen 24 90 0 07 Dec 2023
Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models Shengqu Cai Duygu Ceylan Matheus Gadelha C. Huang Tuanfeng Y. Wang Gordon Wetzstein VGen 50 17 0 03 Dec 2023
$ART$\boldsymbol{\cdot}$V: Auto-Regressive Text-to-Video Generation with Diffusion Models$ ART $\boldsymbol{\cdot}$ V: Auto-Regressive Text-to-Video Generation with Diffusion Models Wenming Weng Ruoyu Feng Yanhui Wang Qi Dai Chunyu Wang ... Jianmin Bao Yuhui Yuan Chong Luo Yueyi Zhang Zhiwei Xiong VGen 38 34 0 30 Nov 2023
MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generation Yanhui Wang Jianmin Bao Wenming Weng Ruoyu Feng Dacheng Yin ... Yuhui Yuan Chuanxin Tang Xiaoyan Sun Chong Luo Baining Guo DiffM VGen 68 15 0 30 Nov 2023
Panacea: Panoramic and Controllable Video Generation for Autonomous Driving Yuqing Wen Yucheng Zhao Yingfei Liu Fan Jia Yanhui Wang Chong Luo Chi Zhang Tiancai Wang Xiaoyan Sun Xiangyu Zhang 80 60 0 28 Nov 2023
MotionZero:Exploiting Motion Priors for Zero-shot Text-to-Video Generation Jingkuan Song Litao Guo Lianli Gao Hengtao Shen Jingkuan Song VGen 39 4 0 28 Nov 2023
FusionFrames: Efficient Architectural Aspects for Text-to-Video Generation Pipeline V.Ya. Arkhipkin Zein Shaheen Viacheslav Vasilev E. Dakhova Andrey Kuznetsov Denis Dimitrov DiffM VGen 40 5 0 22 Nov 2023
Breathing Life Into Sketches Using Text-to-Video Priors Rinon Gal Yael Vinker Yuval Alaluf Amit H. Bermano Daniel Cohen-Or Ariel Shamir Gal Chechik VGen DiffM 50 29 0 21 Nov 2023
GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning Jiaxi Lv Yi Huang Mingfu Yan Jiancheng Huang Jianzhuang Liu Yifan Liu Yafei Wen Xiaoxin Chen Shifeng Chen VGen DiffM 39 23 0 21 Nov 2023
MoVideo: Motion-Aware Video Generation with Diffusion Models Christos Sakaridis Yuchen Fan Kai Zhang Radu Timofte Luc Van Gool Rakesh Ranjan DiffM VGen 55 10 0 19 Nov 2023
Make Pixels Dance: High-Dynamic Video Generation Yan Zeng Guoqiang Wei Jiani Zheng Jiaxin Zou Yang Wei Yuchen Zhang Hang Li DiffM VGen 21 93 0 18 Nov 2023
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation Haoxin Chen Menghan Xia Yin-Yin He Yong Zhang Xiaodong Cun ... Yaofang Liu Qifeng Chen Xintao Wang Chao-Liang Weng Ying Shan DiffM 26 288 0 30 Oct 2023
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors Jinbo Xing Menghan Xia Yong Zhang Haoxin Chen Wangbo Yu Hanyuan Liu Xintao Wang Tien-Tsin Wong Ying Shan VGen 52 231 0 18 Oct 2023
A Survey on Video Diffusion Models Zhen Xing Qijun Feng Haoran Chen Qi Dai Hang-Rui Hu Hang Xu Zuxuan Wu Yu-Gang Jiang EGVM VGen 84 119 0 16 Oct 2023
State of the Art on Diffusion Models for Visual Computing Ryan Po Wang Yifan Vladislav Golyanik Kfir Aberman Jonathan T. Barron ... Matthias Nießner Bjorn Ommer Christian Theobalt Peter Wonka Gordon Wetzstein 62 104 0 11 Oct 2023
ID.8: Co-Creating Visual Stories with Generative AI Victor Nikhil Antony Chien-Ming Huang 46 26 0 25 Sep 2023
Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs Hao Fei Shengqiong Wu Wei Ji Hanwang Zhang Tat-Seng Chua VGen DiffM 26 33 0 26 Aug 2023
Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models Chang-rui Liu Haoning Wu Yujie Zhong Xiaoyu Zhang Yanfeng Wang Weidi Xie DiffM VLM 40 39 0 01 Jun 2023
DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation Susung Hong Junyoung Seo Heeseong Shin Sung‐Jin Hong Seung Wook Kim DiffM VGen 36 35 0 23 May 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 324 4,362 0 30 Jan 2023
Muse: Text-To-Image Generation via Masked Generative Transformers Huiwen Chang Han Zhang Jarred Barber AJ Maschinot José Lezama ... Kevin Patrick Murphy William T. Freeman Michael Rubinstein Yuanzhen Li Dilip Krishnan DiffM 203 532 0 02 Jan 2023
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong Ming Ding Wendi Zheng Xinghan Liu Jie Tang DiffM 266 578 0 29 May 2022
IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation Lingtong Kong Boyuan Jiang Donghao Luo Wenqing Chu Xiaoming Huang Ying Tai Chengjie Wang Jie Yang 76 147 0 29 May 2022
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 401 1,607 0 10 Nov 2021