Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators

23 March 2023

ArXiv (abs)PDF HTML Github (4184★)

Papers citing "Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators"

50 / 78 papers shown

Title
Solving New Tasks by Adapting Internet Video Knowledge Calvin Luo Zilai Zeng Yilun Du Chen Sun 99 6 0 21 Apr 2025
DyST-XL: Dynamic Layout Planning and Content Control for Compositional Text-to-Video Generation Weijie He Mushui Liu YunLong Yu Zhao Wang Chao Wu DiffM VGen 131 0 0 21 Apr 2025
SketchVideo: Sketch-based Video Generation and Editing Feng-Lin Liu Hongbo Fu Xintao Wang Weicai Ye Pengfei Wan Di Zhang Lin Gao DiffM VGen 111 0 0 30 Mar 2025
SyncSDE: A Probabilistic Framework for Diffusion Synchronization Hyunjun Lee Hyunsoo Lee Sookwan Han DiffM 126 0 0 27 Mar 2025
Latent Beam Diffusion Models for Decoding Image Sequences Guilherme Fernandes Vasco Ramos Regev Cohen Idan Szpektor João Magalhães 155 1 0 26 Mar 2025
VideoUFO: A Million-Scale User-Focused Dataset for Text-to-Video Generation Wenhao Wang Yue Yang DiffM VGen 162 1 0 03 Mar 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos Sicheng Xie Haidong Cao Zejia Weng Zhen Xing Shiwei Shen Jiaqi Leng Xipeng Qiu Yanwei Fu Zuxuan Wu Yu Jiang 137 0 0 23 Feb 2025
Animate Your Thoughts: Decoupled Reconstruction of Dynamic Natural Vision from Slow Brain Activity Yizhuo Lu Changde Du Chong Wang Xuanliu Zhu Liuyun Jiang Xujin Li Huiguang He VGen 222 4 0 20 Feb 2025
A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction Yongfan Chen Xiuwen Zhu Tianyu Li EGVM VGen 142 3 0 08 Feb 2025
ConceptMaster: Multi-Concept Video Customization on Diffusion Transformer Models Without Test-Time Tuning Yuzhou Huang Ziyang Yuan Quande Liu Qiulin Wang Xintao Wang Ruimao Zhang Pengfei Wan Di Zhang Kun Gai VGen DiffM 125 16 0 08 Jan 2025
Edicho: Consistent Image Editing in the Wild Qingyan Bai Hao Ouyang Yinghao Xu Qiuyu Wang Ceyuan Yang Ka Leong Cheng Yujun Shen Qifeng Chen DiffM 123 1 0 30 Dec 2024
Map Imagination Like Blind Humans: Group Diffusion Model for Robotic Map Generation Qijin Song Weibang Bai 151 1 0 22 Dec 2024
Label-Efficient Data Augmentation with Video Diffusion Models for Guidewire Segmentation in Cardiac Fluoroscopy Shaoyan Pan Yikang Liu Lin Zhao Eric Z. Chen Xiao Chen Terrence Chen Shanhui Sun VGen MedIm 152 0 0 20 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip Torr VLM ObjD 502 1 0 12 Dec 2024
PCDreamer: Point Cloud Completion Through Multi-view Diffusion Priors Guangshun Wei Yuan Feng Long Ma Chen Wang Yuanfeng Zhou Changjian Li 536 0 0 28 Nov 2024
Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharing Kaifeng Gao Jiaxin Shi Hanwang Zhang Chunping Wang Jun Xiao Long Chen VGen DiffM 185 4 0 25 Nov 2024
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric Zhichao Zhang Wei Sun Xinyue Li Yunhao Li Qihang Ge ... Zhongpeng Ji Fengyu Sun Shangling Jui Xiongkuo Min Guangtao Zhai EGVM 228 1 0 25 Nov 2024
SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation Koichi Namekata Sherwin Bahmani Ziyi Wu Yash Kant Igor Gilitschenski David B. Lindell VGen 148 16 0 07 Nov 2024
MovieCharacter: A Tuning-Free Framework for Controllable Character Video Synthesis Di Qiu Zheng Chen Rui Wang Mingyuan Fan Changqian Yu Junshi Huan Xiang Wen VGen 86 9 0 28 Oct 2024
Preference Diffusion for Recommendation Shuo Liu An Zhang Guoqing Hu Hong Qian Tat-Seng Chua 124 1 0 17 Oct 2024
Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content Qiuheng Wang Yukai Shi Jiarong Ou Ruoxin Chen Ke Lin ... Mingwu Zheng Xin Tao Fei Yang Pengfei Wan Di Zhang VGen 141 33 0 10 Oct 2024
MotionAura: Generating High-Quality and Motion Consistent Videos using Discrete Diffusion Onkar Susladkar Jishu Sen Gupta Chirag Sehgal Sparsh Mittal Rekha Singhal DiffM VGen 80 0 0 10 Oct 2024
MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation T. Pham Tri Ton Chang D. Yoo 82 3 0 03 Oct 2024
Text2PDE: Latent Diffusion Models for Accessible Physics Simulation Anthony Zhou Zijie Li Michael Schneier John R Buchanan Jr Amir Barati Farimani AI4CE DiffM 144 8 0 02 Oct 2024
Replace Anyone in Videos Xiang Wang Shiwei Zhang Haonan Qiu Ruihang Chu Zekun Li Yuanxing Zhang Changxin Gao Yuehuan Wang Chunhua Shen Nong Sang VGen DiffM 114 1 0 30 Sep 2024
Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classification Xinrui Zhou Yuhao Huang Haoran Dou Shijing Chen Ao Chang ... Jie Jessie Ren Ruobing Huang Jun Cheng Wufeng Xue Dong Ni MedIm 354 0 0 25 Sep 2024
TrackGo: A Flexible and Efficient Method for Controllable Video Generation Haitao Zhou Chuang Wang Rui Nie Jinxiao Lin Dongdong Yu Qian Yu Changhu Wang VGen DiffM 144 15 0 21 Aug 2024
Zero-Shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model Cong Cao Huanjing Yue Xin Liu Jingyu Yang DiffM VGen 117 1 0 02 Jul 2024
OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation Kepan Nan Rui Xie Penghao Zhou Tiehan Fan Zhenheng Yang Zhijie Chen Xiang Li Jian Yang Ying Tai 135 92 0 02 Jul 2024
V-LASIK: Consistent Glasses-Removal from Videos Using Synthetic Data Rotem Shalev-Arkushin Aharon Azulay Tavi Halperin Eitan Richardson Amit H. Bermano Ohad Fried DiffM 97 0 0 20 Jun 2024
FRAG: Frequency Adapting Group for Diffusion Video Editing Sunjae Yoon Gwanhyeong Koo Geonwoo Kim Chang D. Yoo DiffM 96 5 0 10 Jun 2024
Enhancing Temporal Consistency in Video Editing by Reconstructing Videos with 3D Gaussian Splatting Inkyu Shin Qihang Yu Xiaohui Shen In So Kweon KuK-Jin Yoon Liang-Chieh Chen VGen DiffM 113 1 0 04 Jun 2024
NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer Meng You Zhiyu Zhu Hui Liu Junhui Hou VGen DiffM 96 25 0 24 May 2024
Looking Backward: Streaming Video-to-Video Translation with Feature Banks Feng Liang Akio Kodaira Chenfeng Xu Masayoshi Tomizuka Kurt Keutzer Diana Marculescu DiffM VGen 148 9 0 24 May 2024
COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Hongxin Zhang Zeyuan Wang Qiushi Lyu Zheyuan Zhang Sunli Chen Tianmin Shu Yilun Du Kwonjoon Lee Yilun Du Chuang Gan 126 17 0 16 Apr 2024
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators Shenghai Yuan Jinfa Huang Yujun Shi Yongqi Xu Ruijie Zhu Bin Lin Xinhua Cheng Li-xin Yuan Jiebo Luo VGen 150 36 0 07 Apr 2024
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text Roberto Henschel Levon Khachatryan Daniil Hayrapetyan Hayk Poghosyan Vahram Tadevosyan Zhangyang Wang Shant Navasardyan Humphrey Shi DiffM VGen 185 88 0 21 Mar 2024
GazeFusion: Saliency-Guided Image Generation Yunxiang Zhang Nan Wu Connor Z. Lin Gordon Wetzstein Qi Sun 79 0 0 16 Mar 2024
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models Chong Mou Xintao Wang Liangbin Xie Yanze Wu Shuai Liu Zhongang Qi Ying Shan Xiaohu Qie DiffM 132 1,030 0 16 Feb 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 182 4,175 1 10 Feb 2023
Structure and Content-Guided Video Synthesis with Diffusion Models Patrick Esser Johnathan Chiu Parmida Atighehchian Jonathan Granskog Anastasis Germanidis DiffM VGen 174 537 0 06 Feb 2023
Dreamix: Video Diffusion Models are General Video Editors Eyal Molad Eliahu Horwitz Dani Valevski Alex Rav-Acha Yossi Matias Yael Pritch Yaniv Leviathan Yedid Hoshen DiffM VGen 124 187 0 02 Feb 2023
Shape-aware Text-driven Layered Video Editing Yao-Chih Lee Ji-Ze Jang Yi-Ting Chen Elizabeth Qiu Jia-Bin Huang VGen DiffM 70 54 0 30 Jan 2023
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation Jay Zhangjie Wu Yixiao Ge Xintao Wang Weixian Lei Yuchao Gu Yufei Shi Wynne Hsu Ying Shan Xiaohu Qie Mike Zheng Shou VGen 123 743 0 22 Dec 2022
InstructPix2Pix: Learning to Follow Image Editing Instructions Tim Brooks Aleksander Holynski Alexei A. Efros DiffM 211 1,830 0 17 Nov 2022
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model Xingqian Xu Zhangyang Wang Eric Zhang Kai Wang Humphrey Shi DiffM 116 198 0 15 Nov 2022
Phenaki: Variable Length Video Generation From Open Domain Textual Description Ruben Villegas Mohammad Babaeizadeh Pieter-Jan Kindermans Hernan Moraldo Han Zhang M. Saffar Santiago Castro Julius Kunze D. Erhan DiffM VGen 133 395 0 05 Oct 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 171 1,541 0 05 Oct 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 83 1,428 0 29 Sep 2022
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation Nataniel Ruiz Yuanzhen Li Varun Jampani Yael Pritch Michael Rubinstein Kfir Aberman 279 2,888 0 25 Aug 2022