Structure and Content-Guided Video Synthesis with Diffusion Models

6 February 2023

Papers citing "Structure and Content-Guided Video Synthesis with Diffusion Models"

50 / 424 papers shown

Title
EVE: Efficient zero-shot text-based Video Editing with Depth Map Guidance and Temporal Consistency Constraints Yutao Chen Xingning Dong Tian Gan Chunluan Zhou Ming Yang Qingpei Guo DiffM 27 5 0 21 Aug 2023
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization Soumik Mukhopadhyay Saksham Suri R. Gadde Abhinav Shrivastava DiffM 46 20 0 18 Aug 2023
SimDA: Simple Diffusion Adapter for Efficient Video Generation Zhen Xing Qi Dai Hang-Rui Hu Zuxuan Wu Yu-Gang Jiang VGen DiffM 35 81 0 18 Aug 2023
StableVideo: Text-driven Consistency-aware Diffusion Video Editing Wenhao Chai Xun Guo Gaoang Wang Yang Lu VGen DiffM 27 147 0 18 Aug 2023
Edit Temporal-Consistent Videos with Image Diffusion Model Yuan-Zheng Wang Yong Li Xiaoya Zhang Xin Liu Anbo Dai Antoni B. Chan Zhen Cui DiffM 33 6 0 17 Aug 2023
DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory Sheng-Siang Yin Chenfei Wu Jian Liang Jie Shi Houqiang Li Gong Ming Nan Duan VGen 23 130 0 16 Aug 2023
CoDeF: Content Deformation Fields for Temporally Consistent Video Processing Ouyang Hao Qiuyu Wang Yuxi Xiao Qingyan Bai Juntao Zhang Kecheng Zheng Xiaowei Zhou Qifeng Chen Yujun Shen DiffM VGen 46 81 0 15 Aug 2023
ModelScope Text-to-Video Technical Report Jiuniu Wang Hangjie Yuan Dayou Chen Yingya Zhang Xiang Wang Shiwei Zhang VGen DiffM 33 390 0 12 Aug 2023
DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis Zhongjie Duan Lizhou You Chengyu Wang Cen Chen Ziheng Wu Weining Qian Jun Huang DiffM 34 8 0 07 Aug 2023
ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints Elad Richardson Kfir Goldberg Yuval Alaluf Daniel Cohen-Or DiffM 31 11 0 03 Aug 2023
VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet Zhihao Hu Dong Xu DiffM VGen 41 65 0 26 Jul 2023
InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editing Anant Khandelwal DiffM VGen 31 14 0 22 Jul 2023
PreDiff: Precipitation Nowcasting with Latent Diffusion Models Zhihan Gao Xingjian Shi Boran Han Hongya Wang Xiaoyong Jin Danielle C. Maddix Yi Zhu Mu Li Bernie Wang BDL DiffM 43 57 0 19 Jul 2023
TokenFlow: Consistent Diffusion Features for Consistent Video Editing Michal Geyer Omer Bar-Tal Shai Bagon Tali Dekel VGen DiffM 20 251 0 19 Jul 2023
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning Yuwei Guo Ceyuan Yang Anyi Rao Zhengyang Liang Yaohui Wang Yu Qiao Maneesh Agrawala Dahua Lin Bo Dai VGen 31 782 0 10 Jul 2023
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis Dustin Podell Zion English Kyle Lacey A. Blattmann Tim Dockhorn Jonas Muller Joe Penna Robin Rombach 106 2,147 0 04 Jul 2023
Collaborative Score Distillation for Consistent Visual Synthesis Subin Kim Kyungmin Lee June Suk Choi Jongheon Jeong Kihyuk Sohn Jinwoo Shin DiffM 29 21 0 04 Jul 2023
Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation Tserendorj Adiya Jae Shin Yoon Jungeun Lee Sang-hyeon Kim Hwasup Lim DiffM 31 0 0 02 Jul 2023
DisCo: Disentangled Control for Realistic Human Dance Generation Tan Wang Linjie Li Kevin Qinghong Lin Yuanhao Zhai Chung-Ching Lin Zhengyuan Yang Hanwang Zhang Zicheng Liu Lijuan Wang VGen 32 74 0 30 Jun 2023
Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models Simian Luo Chuanhao Yan Chenxu Hu Hang Zhao DiffM 24 79 0 29 Jun 2023
VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing Paul Couairon Clément Rambour Jean-Emmanuel Haugeard Nicolas Thome DiffM VGen 12 29 0 14 Jun 2023
The Age of Synthetic Realities: Challenges and Opportunities J. P. Cardenuto Jing Yang Rafael Padilha Renjie Wan Daniel Moreira Haoliang Li Shiqi Wang Fernanda A. Andaló Sébastien Marcel Anderson de Rezende Rocha DeLMO 42 29 0 09 Jun 2023
Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning Jifeng Hu Yan Sun Sili Huang Siyuan Guo Hechang Chen Li Shen Lichao Sun Yi-Ju Chang Dacheng Tao DiffM OffRL 43 13 0 08 Jun 2023
HeadSculpt: Crafting 3D Head Avatars with Text Xiaoping Han Yukang Cao Kai Han Xiatian Zhu Jiankang Deng Yi-Zhe Song Tao Xiang Kwan-Yee K. Wong DiffM 19 45 0 05 Jun 2023
Stable Diffusion is Unstable Chengbin Du Yanxi Li Zhongwei Qiu Chang Xu DiffM 35 17 0 05 Jun 2023
VideoComposer: Compositional Video Synthesis with Motion Controllability Xiang Wang Hangjie Yuan Shiwei Zhang Dayou Chen Jiuniu Wang Yingya Zhang Yujun Shen Deli Zhao Jingren Zhou VGen DiffM 33 316 0 03 Jun 2023
Probabilistic Adaptation of Text-to-Video Models Mengjiao Yang Yilun Du Bo Dai Dale Schuurmans J. Tenenbaum Pieter Abbeel VGen DiffM 43 24 0 02 Jun 2023
Video Colorization with Pre-trained Text-to-Image Diffusion Models Hanyuan Liu M. Xie Jinbo Xing Chengze Li T. Wong VLM DiffM 19 12 0 02 Jun 2023
Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models Chang-rui Liu Haoning Wu Yujie Zhong Xiaoyu Zhang Yanfeng Wang Weidi Xie DiffM VLM 28 39 0 01 Jun 2023
Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance Jinbo Xing Menghan Xia Yuxin Liu Yuechen Zhang Yong Zhang ... Haoxin Chen Xiaodong Cun Xintao Wang Ying Shan T. Wong VGen DiffM 39 1 0 01 Jun 2023
Control4D: Efficient 4D Portrait Editing with Text Ruizhi Shao Jingxiang Sun Cheng Peng Zerong Zheng Boyao Zhou Hongwen Zhang Yebin Liu DiffM 24 23 0 31 May 2023
SAVE: Spectral-Shift-Aware Adaptation of Image Diffusion Models for Text-driven Video Editing Nazmul Karim Umar Khalid M. Joneidi Chen Chen Nazanin Rahnavard DiffM VGen 19 5 0 30 May 2023
Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising Fu Lee Wang Wenshuo Chen Guanglu Song Han-Jia Ye Yu Liu Hongsheng Li VGen DiffM 50 89 0 29 May 2023
InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions Qian Wang Biao Zhang Michael Birsak Peter Wonka DiffM 30 31 0 29 May 2023
Towards Consistent Video Editing with Text-to-Image Diffusion Models Zicheng Zhang Bonan li Xuecheng Nie Congying Han Tiande Guo Luoqi Liu DiffM 20 24 0 27 May 2023
ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond Min Zhao Rongzheng Wang Fan Bao Chongxuan Li Jun Zhu VGen DiffM 21 4 0 26 May 2023
Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality Schedulers for Diffusion Models Zhongjie Duan Chengyu Wang Cen Chen Jun Huang Weining Qian DiffM 27 12 0 24 May 2023
Vision + Language Applications: A Survey Yutong Zhou N. Shimada VLM 30 6 0 24 May 2023
DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation Susung Hong Junyoung Seo Heeseong Shin Sung‐Jin Hong Seung Wook Kim DiffM VGen 31 34 0 23 May 2023
Control-A-Video: Controllable Text-to-Video Generation with Diffusion Models Weifeng Chen Yatai Ji Jie Wu Hefeng Wu Pan Xie Jiashi Li Xin Xia Xuefeng Xiao Liang Lin VGen 121 6 0 23 May 2023
VDT: General-purpose Video Diffusion Transformers via Mask Modeling Haoyu Lu Guoxing Yang Nanyi Fei Yuqi Huo Zhiwu Lu Ping Luo Mingyu Ding DiffM VGen 28 56 0 22 May 2023
ControlVideo: Training-free Controllable Text-to-Video Generation Yabo Zhang Yuxiang Wei Dongsheng Jiang Xiaopeng Zhang W. Zuo Qi Tian VGen DiffM 48 236 0 22 May 2023
Any-to-Any Generation via Composable Diffusion Zineng Tang Ziyi Yang Chenguang Zhu Michael Zeng Joey Tianyi Zhou VGen DiffM 36 172 0 19 May 2023
Graphologue: Exploring Large Language Model Responses with Interactive Diagrams Peiling Jiang Jude Rayan Steven W. Dow Haijun Xia 32 100 0 19 May 2023
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation Wenjing Wang Huan Yang Zixi Tuo Huiguo He Junchen Zhu Jianlong Fu Jiaying Liu DiffM VGen 48 114 0 18 May 2023
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models Songwei Ge Seungjun Nah Guilin Liu Tyler Poon Andrew Tao Bryan Catanzaro David Jacobs Jia-Bin Huang Ming-Yu Liu Yogesh Balaji DiffM VGen 45 253 0 17 May 2023
Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts Yuyang Zhao Enze Xie Lanqing Hong Zhenguo Li G. Lee DiffM VGen 35 32 0 15 May 2023
Style-A-Video: Agile Diffusion for Arbitrary Text-based Video Style Transfer Nisha Huang Yuxin Zhang Weiming Dong DiffM VGen 29 16 0 09 May 2023
Motion-Conditioned Diffusion Model for Controllable Video Synthesis Tsai-Shien Chen C. Lin Hung-Yu Tseng Nayeon Lee Ming Yang DiffM VGen 81 62 0 27 Apr 2023
Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images Zeyu Lu Di Huang Lei Bai Jingjing Qu Chengzhi Wu Xihui Liu Wanli Ouyang 26 52 0 25 Apr 2023