Title
MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation Weijia Wu Mingyu Liu Zeyu Zhu Xi Xia Haoen Feng Wen Wang Kevin Qinghong Lin Chunhua Shen Mike Zheng Shou DiffM VGen 122 1 0 22 Nov 2024
Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification Sundar Sripada V. S. Minkyu Choi Sahil Shah Harsh Goel Mohammad Omama Sandeep P. Chinchali EGVM 108 2 0 22 Nov 2024
PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulation Zhuoman Liu Weicai Ye Yan Luximon Pengfei Wan Di Zhang VGen AI4CE 117 2 0 21 Nov 2024
Generating 3D-Consistent Videos from Unposed Internet Photos Gene Chou Kai Zhang Sai Bi Hao Tan Zexiang Xu Fujun Luan Bharath Hariharan Noah Snavely 3DGS VGen 89 3 0 20 Nov 2024
LaVin-DiT: Large Vision Diffusion Transformer Zhaoqing Wang Xiaobo Xia Runnan Chen Dongdong Yu Changhu Wang Mingming Gong Tongliang Liu 92 6 0 18 Nov 2024
SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization Jintao Zhang Haofeng Huang Pengle Zhang Jia wei Jun-Jie Zhu Jianfei Chen VLM MQ 65 15 0 17 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 67 8 0 14 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 63 1 0 12 Nov 2024
Grounding Video Models to Actions through Goal Conditioned Exploration Yunhao Luo Yilun Du LM&Ro VGen 85 1 0 11 Nov 2024
Improved Video VAE for Latent Video Diffusion Model Pingyu Wu Kai Zhu Yu Liu Liming Zhao Wei-dong Zhai Yang Cao Zheng-jun Zha VGen DiffM 61 4 0 10 Nov 2024
ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning David Junhao Zhang Roni Paiss Shiran Zada Nikhil Karnad David E. Jacobs Yael Pritch Inbar Mosseri Mike Zheng Shou Neal Wadhwa Nataniel Ruiz DiffM VGen 73 15 0 07 Nov 2024
DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion Wenqiang Sun Shuo Chen F. Liu Zilong Chen Yueqi Duan Jun Zhang Yikai Wang VGen 51 31 0 07 Nov 2024
Taming Rectified Flow for Inversion and Editing Jiangshan Wang Junfu Pu Zhongang Qi Jiayi Guo Yue Ma Nisha Huang Yuxin Chen Xiu Li Ying Shan 47 23 0 07 Nov 2024
TIP-I2V: A Million-Scale Real Text and Image Prompt Dataset for Image-to-Video Generation Wenhao Wang Yuqing Yang VGen 47 3 0 05 Nov 2024
xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism Jiarui Fang Jinzhe Pan Xibo Sun Aoyu Li Jiannan Wang 59 5 0 04 Nov 2024
Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generation Zhenbin Wang Lei Zhang Lituan Wang Minjuan Zhu Zhenwei Zhang VGen MedIm 62 1 0 03 Nov 2024
GameGen-X: Interactive Open-world Game Video Generation Haoxuan Che Xuanhua He Quande Liu C. Jin Hao Chen VGen 66 17 0 01 Nov 2024
MovieCharacter: A Tuning-Free Framework for Controllable Character Video Synthesis Di Qiu Zheng Chen Rui Wang Mingyuan Fan Changqian Yu Junshi Huan Xiang Wen VGen 43 6 0 28 Oct 2024
ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generation Zongyi Li Shujie Hu Shujie Liu Long Zhou Jeongsoo Choi Lingwei Meng Xun Guo Jiajian Li H. Ling Furu Wei VGen DiffM 80 5 0 27 Oct 2024
Allegro: Open the Black Box of Commercial-Level Video Generation Model Yuan Zhou Qiuyue Wang Yuxuan Cai Huan Yang VGen VLM 88 26 0 20 Oct 2024
FrameBridge: Improving Image-to-Video Generation with Bridge Models Yuji Wang Zehua Chen Xiaoyu Chen Jun-Jie Zhu Jianfei Chen DiffM VGen 193 1 0 20 Oct 2024
DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Control Yujie Wei Shiwei Zhang Hangjie Yuan Xiang Wang Haonan Qiu ... F. Liu Zhizhong Huang Jiaxin Ye Yingya Zhang Hongming Shan DiffM VGen 72 14 0 17 Oct 2024
DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation Guosheng Zhao Chaojun Ni Xiaofeng Wang Zheng Zhu Xinming Zhang ... Xinze Chen Boyuan Wang Youyi Zhang Wenjun Mei Xingang Wang VGen 83 24 0 17 Oct 2024
SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation Jaehong Yoon Shoubin Yu Vaidehi Patil Huaxiu Yao Joey Tianyi Zhou 79 16 0 16 Oct 2024
Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free Ziyue Li Dinesh Manocha MoE 74 5 0 14 Oct 2024
Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning Aakanksha Arash Ahmadian Seraphina Goldfarb-Tarrant Beyza Ermis Marzieh Fadaee Sara Hooker MoMe 69 5 0 14 Oct 2024
Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning Etai Littwin Vimal Thilak Anand Gopalakrishnan 57 0 0 14 Oct 2024
Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation Jiahao Cui Hui Li Yao Yao Hao Zhu Hanlin Shang Kaihui Cheng Hang Zhou Siyu Zhu Jingdong Wang DiffM VGen 46 22 0 10 Oct 2024
Progressive Autoregressive Video Diffusion Models Desai Xie Zhan Xu Yicong Hong Hao Tan Difan Liu Feng Liu Arie E. Kaufman Yang Zhou DiffM VGen 61 10 0 10 Oct 2024
Pyramidal Flow Matching for Efficient Video Generative Modeling Yang Jin Zhicheng Sun Ningyuan Li Kun Xu K. Xu ... Nan Zhuang Quzhe Huang Yang Song Yadong Mu Zhouchen Lin VGen 66 66 0 08 Oct 2024
Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation Fanqing Meng Jiaqi Liao Xinyu Tan Wenqi Shao Quanfeng Lu Kaipeng Zhang Yu Cheng Dianqi Li Yu Qiao Ping Luo VGen EGVM 32 24 0 07 Oct 2024
VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning Han Lin Tushar Nagarajan Nicolas Ballas Mido Assran Mojtaba Komeili Joey Tianyi Zhou Koustuv Sinha AI4TS 57 3 0 04 Oct 2024
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration Jintao Zhang Jia wei Pengle Zhang Jun-Jie Zhu Jun Zhu Jianfei Chen VLM MQ 84 19 0 03 Oct 2024
Denoising with a Joint-Embedding Predictive Architecture Dengsheng Chen Jie Hu Xiaoming Wei Enhua Wu DiffM 52 2 0 02 Oct 2024
Replace Anyone in Videos Xiang Wang Shiwei Zhang Haonan Qiu Ruihang Chu Zekun Li Yang Zhang Changxin Gao Yuehuan Wang Chunhua Shen Nong Sang VGen DiffM 69 1 0 30 Sep 2024
Emu3: Next-Token Prediction is All You Need Xinlong Wang Xiaosong Zhang Zhengxiong Luo Quan-Sen Sun Yufeng Cui ... Xi Yang Jingjing Liu Yonghua Lin Tiejun Huang Zhongyuan Wang MLLM 39 159 0 27 Sep 2024
Multi-Modal Generative AI: Multi-modal LLM, Diffusion and Beyond Hong Chen Xin Wang Yuwei Zhou Bin Huang Yipeng Zhang Wei Feng Houlun Chen Zeyang Zhang Siao Tang Wenwu Zhu DiffM 55 7 0 23 Sep 2024
Video-to-Audio Generation with Fine-grained Temporal Semantics Yuchen Hu Yu Gu Chenxing Li Rilin Chen Dong Yu VGen DiffM 29 1 0 23 Sep 2024
The Art of Storytelling: Multi-Agent Generative AI for Dynamic Multimodal Narratives Samee Arif Taimoor Arif Muhammad Saad Haroon Aamina Jamal Khan Agha Ali Raza Awais Athar 34 0 0 17 Sep 2024
DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes Jianbiao Mei Yukai Ma Xuemeng Yang Licheng Wen Tiantian Wei Min Dou Yukai Ma Min Dou Botian Shi Yong Liu DiffM VGen 71 3 0 06 Sep 2024
CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention Gaojie Lin Jianwen Jiang Chao Liang Tianyun Zhong Jiaqi Yang Yanbo Zheng VGen DiffM 69 13 0 03 Sep 2024
FLUX that Plays Music Zhengcong Fei Mingyuan Fan Changqian Yu Junshi Huang 84 7 0 01 Sep 2024
SurGen: Text-Guided Diffusion Model for Surgical Video Generation Joseph Cho Samuel Schmidgall C. Zakka Mrudang Mathur Dhamanpreet Kaur R. Shad W. Hiesinger VGen MedIm 31 6 0 26 Aug 2024
Real-Time Video Generation with Pyramid Attention Broadcast Xuanlei Zhao Xiaolong Jin Kai Wang Yang You VGen DiffM 77 32 0 22 Aug 2024
Kubrick: Multimodal Agent Collaborations for Synthetic Video Generation Liu He Yizhi Song Hejun Huang Pinxin Liu Yunlong Tang Daniel G. Aliaga Xin Zhou DiffM VGen 90 3 0 19 Aug 2024
SkyScript-100M: 1,000,000,000 Pairs of Scripts and Shooting Scripts for Short Drama Jing Tang Quanlu Jia Yuqiang Xie Zeyu Gong Xiang Wen Jiayi Zhang Yalong Guo Guibin Chen Jiangping Yang VGen 38 1 0 18 Aug 2024
Tora: Trajectory-oriented Diffusion Transformer for Video Generation Zhenghao Zhang Junchao Liao Menghao Li Zuozhuo Dai Bingxue Qiu Hao Hu Shaowei Cai Weizhi Wang VGen 46 44 0 31 Jul 2024
Diffusion Feedback Helps CLIP See Better Wenxuan Wang Quan-Sen Sun Fan Zhang Yepeng Tang Jing Liu Xinlong Wang VLM 46 14 0 29 Jul 2024
VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control Sherwin Bahmani Ivan Skorokhodov Aliaksandr Siarohin Willi Menapace Guocheng Qian ... Chaoyang Wang Jiaxu Zou Andrea Tagliasacchi David B. Lindell Sergey Tulyakov VGen DiffM 88 42 0 17 Jul 2024
ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation Shenghai Yuan Jinfa Huang Yongqi Xu Yaoyang Liu Shaofeng Zhang Yujun Shi Ruijie Zhu Xinhua Cheng Jiebo Luo Li Yuan EGVM VGen 77 34 0 26 Jun 2024