GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning

21 November 2023

Jiancheng Huang

Yifan Liu

Shifeng Chen

Papers citing "GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning"

50 / 68 papers shown

Title
MG-Gen: Single Image to Motion Graphics Generation with Layer Decomposition Takahiro Shirakawa Tomoyuki Suzuki Daichi Haraguchi VGen 74 0 0 03 Apr 2025
PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation Qiyao Xue Xiangyu Yin Boyuan Yang Wei Gao DiffM VGen 140 11 0 30 Nov 2024
Diffusion Model-Based Image Editing: A Survey Yi Huang Jiancheng Huang Yifan Liu Mingfu Yan Jiaxi Lv Jianzhuang Liu Wei Xiong He Zhang Liangliang Cao Liangliang Cao EGVM 108 96 0 27 Feb 2024
VBench: Comprehensive Benchmark Suite for Video Generative Models Ziqi Huang Yinan He Jiashuo Yu Fan Zhang Chenyang Si ... Xinyuan Chen Limin Wang Dahua Lin Yu Qiao Ziwei Liu VGen 130 406 0 29 Nov 2023
EvalCrafter: Benchmarking and Evaluating Large Video Generation Models Yaofang Liu Xiaodong Cun Xuebo Liu Xintao Wang Yong Zhang Haoxin Chen Yang Liu Tieyong Zeng Raymond H. F. Chan Ying Shan VGen EGVM 57 134 0 17 Oct 2023
LLM-grounded Video Diffusion Models Long Lian Baifeng Shi Semih Yavuz Ye Liu Boyi Li DiffM 42 54 0 29 Sep 2023
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation David Junhao Zhang Jay Zhangjie Wu Jia-Wei Liu Rui Zhao L. Ran Yuchao Gu Difei Gao Mike Zheng Shou DiffM VGen 76 218 0 27 Sep 2023
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models Yaohui Wang Xinyuan Chen Xin Ma Shangchen Zhou Ziqi Huang ... Chen Change Loy Bo Dai Dahua Lin Yu Qiao Ziwei Liu VGen DiffM 63 226 0 26 Sep 2023
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning Han Lin Abhaysinh Zala Jaemin Cho Joey Tianyi Zhou LM&Ro VGen DiffM 73 78 0 26 Sep 2023
Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator Hanzhuo Huang Yufan Feng Cheng Shi Lan Xu Jingyi Yu Sibei Yang DiffM VGen 54 65 0 25 Sep 2023
VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation Xin Li Wenqing Chu Ye Wu Weihang Yuan Fanglong Liu Qi Zhang Fu Li Haocheng Feng Errui Ding Jingdong Wang VGen 71 53 0 01 Sep 2023
ModelScope Text-to-Video Technical Report Jiuniu Wang Hangjie Yuan Dayou Chen Yingya Zhang Xiang Wang Shiwei Zhang VGen DiffM 66 413 0 12 Aug 2023
DiffSynth: Latent In-Iteration Deflickering for Realistic Video Synthesis Zhongjie Duan Lizhou You Chengyu Wang Cen Chen Ziheng Wu Weining Qian Jun Huang DiffM 37 8 0 07 Aug 2023
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation Yi Wang Yinan He Yizhuo Li Kunchang Li Jiashuo Yu ... Ping Luo Ziwei Liu Yali Wang Limin Wang Yu Qiao VLM VGen 83 260 0 13 Jul 2023
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning Yuwei Guo Ceyuan Yang Anyi Rao Zhengyang Liang Yaohui Wang Yu Qiao Maneesh Agrawala Dahua Lin Bo Dai VGen 69 833 0 10 Jul 2023
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis Dustin Podell Zion English Kyle Lacey A. Blattmann Tim Dockhorn Jonas Muller Joe Penna Robin Rombach 183 2,331 0 04 Jul 2023
Infinite Photorealistic Worlds using Procedural Generation Alexander Raistrick Lahav Lipson Zeyu Ma Lingjie Mei Mingzhe Wang ... Alejandro Newell Hei Law Ankit Goyal Kaiyu Yang Jia Deng VGen 49 76 0 15 Jun 2023
Grounded Text-to-Image Synthesis with Attention Refocusing Quynh Phung Songwei Ge Jia-Bin Huang DiffM 48 111 0 08 Jun 2023
LayoutGPT: Compositional Visual Planning and Generation with Large Language Models Weixi Feng Wanrong Zhu Tsu-Jui Fu Varun Jampani Arjun Reddy Akula Xuehai He Sugato Basu Xinze Wang William Yang Wang MLLM 75 171 0 24 May 2023
DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation Susung Hong Junyoung Seo Heeseong Shin Sung‐Jin Hong Seung Wook Kim DiffM VGen 43 35 0 23 May 2023
LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models Long Lian Boyi Li Adam Yala Trevor Darrell 57 160 0 23 May 2023
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models Songwei Ge Seungjun Nah Guilin Liu Tyler Poon Andrew Tao Bryan Catanzaro David Jacobs Jia-Bin Huang Ming-Yuan Liu Yogesh Balaji DiffM VGen 70 257 0 17 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 169 1,187 0 17 May 2023
AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation Zerui Li Zuo-Liang Zhu Linghao Han Qibin Hou Chunle Guo Ming-Ming Cheng 60 102 0 19 Apr 2023
Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models A. Blattmann Robin Rombach Huan Ling Tim Dockhorn Seung Wook Kim Sanja Fidler Karsten Kreis 3DGS VGen 170 1,062 0 18 Apr 2023
Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation Jie An Songyang Zhang Harry Yang Sonal Gupta Jia-Bin Huang Jiebo Luo Xiaoyue Yin DiffM VGen 55 109 0 17 Apr 2023
Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators Levon Khachatryan A. Movsisyan Vahram Tadevosyan Roberto Henschel Zhangyang Wang Shant Navasardyan Humphrey Shi VGen 66 563 0 23 Mar 2023
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation Sheng-Siang Yin Chenfei Wu Huan Yang Jianfeng Wang Xiaodong Wang ... Gong Ming Lijuan Wang Zicheng Liu Houqiang Li Nan Duan VGen 63 132 0 22 Mar 2023
A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation Hui Tang Kui Jia OOD 51 14 0 16 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.0K 14,179 0 15 Mar 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 103 4,074 1 10 Feb 2023
Structure and Content-Guided Video Synthesis with Diffusion Models Patrick Esser Johnathan Chiu Parmida Atighehchian Jonathan Granskog Anastasis Germanidis DiffM VGen 142 526 0 06 Feb 2023
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation Jay Zhangjie Wu Yixiao Ge Xintao Wang Weixian Lei Yuchao Gu Yufei Shi Wynne Hsu Ying Shan Xiaohu Qie Mike Zheng Shou VGen 93 716 0 22 Dec 2022
Optimizing Prompts for Text-to-Image Generation Y. Hao Zewen Chi Li Dong Furu Wei 76 145 0 19 Dec 2022
MagicVideo: Efficient Video Generation With Latent Diffusion Models Daquan Zhou Weimin Wang Hanshu Yan Weiwei Lv Yizhe Zhu Jiashi Feng DiffM VGen 71 383 0 20 Nov 2022
InstructPix2Pix: Learning to Follow Image Editing Instructions Tim Brooks Aleksander Holynski Alexei A. Efros DiffM 173 1,785 0 17 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 349 2,377 0 09 Nov 2022
Large Language Models Are Human-Level Prompt Engineers Yongchao Zhou Andrei Ioan Muresanu Ziwen Han Keiran Paster Silviu Pitis Harris Chan Jimmy Ba ALM LLMAG 129 878 0 03 Nov 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 100 1,518 0 05 Oct 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 74 1,399 0 29 Sep 2022
Classifier-Free Diffusion Guidance Jonathan Ho Tim Salimans FaML 147 3,858 0 26 Jul 2022
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong Ming Ding Wendi Zheng Xinghan Liu Jie Tang DiffM 299 602 0 29 May 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Chitwan Saharia William Chan Saurabh Saxena Lala Li Jay Whang ... Raphael Gontijo-Lopes Tim Salimans Jonathan Ho David J Fleet Mohammad Norouzi VLM 325 5,978 0 23 May 2022
CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers Ming Ding Wendi Zheng Wenyi Hong Jie Tang VLM 79 328 0 28 Apr 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 339 6,830 0 13 Apr 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 163 1,597 0 07 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 754 12,835 0 04 Mar 2022
Generative Adversarial Networks Gilad Cohen Raja Giryes GAN 223 30,089 0 01 Mar 2022
Pre-Trained Language Models for Interactive Decision-Making Shuang Li Xavier Puig Chris Paxton Yilun Du Clinton Jia Wang ... Anima Anandkumar Jacob Andreas Igor Mordatch Antonio Torralba Yuke Zhu LM&Ro 93 257 0 03 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 490 4,324 0 28 Jan 2022