TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation

12 June 2024

Weixi Feng

Jiachen Li

Michael Stephen Saxon

Tsu-Jui Fu

Wenhu Chen

William Yang Wang

EGVM

VGen

ArXiv PDF HTML

Papers citing "TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation"

25 / 25 papers shown

Title
Neuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification S P Sharan Minkyu Choi Sahil Shah Harsh Goel Mohammad Omama Sandeep Chinchali EGVM 131 3 0 22 Nov 2024
Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) Michael Stephen Saxon Fatima Jahara Mahsa Khoshnoodi Yujie Lu Aditya Sharma William Y. Wang EGVM 49 10 0 05 Apr 2024
Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences Xiyao Wang Yuhang Zhou Xiaoyu Liu Hongjin Lu Yuancheng Xu ... Taixi Lu Gedas Bertasius Mohit Bansal Huaxiu Yao Furong Huang LRM VLM 119 73 0 19 Jan 2024
Towards A Better Metric for Text-to-Video Generation Jay Zhangjie Wu Guian Fang Haoning Wu Xintao Wang Yixiao Ge ... Rui Zhao Weisi Lin Wynne Hsu Ying Shan Mike Zheng Shou VGen 76 36 0 15 Jan 2024
SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction Xinyuan Chen Yaohui Wang Lingjun Zhang Shaobin Zhuang Xin Ma Jiashuo Yu Yali Wang Dahua Lin Yu Qiao Ziwei Liu VGen DiffM 53 139 0 31 Oct 2023
Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation Jaemin Cho Yushi Hu Roopal Garg Peter Anderson Ranjay Krishna Jason Baldridge Mohit Bansal Jordi Pont-Tuset Su Wang EGVM 50 78 0 27 Oct 2023
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation David Junhao Zhang Jay Zhangjie Wu Jia-Wei Liu Rui Zhao L. Ran Yuchao Gu Difei Gao Mike Zheng Shou DiffM VGen 76 219 0 27 Sep 2023
Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment Royi Rassin Eran Hirsch Daniel Glickman Shauli Ravfogel Yoav Goldberg Gal Chechik DiffM 59 106 0 15 Jun 2023
Faith and Fate: Limits of Transformers on Compositionality Nouha Dziri Ximing Lu Melanie Sclar Xiang Lorraine Li Liwei Jian ... Sean Welleck Xiang Ren Allyson Ettinger Zaïd Harchaoui Yejin Choi ReLM LRM 110 376 0 29 May 2023
Unmasked Teacher: Towards Training-Efficient Video Foundation Models Kunchang Li Yali Wang Yizhuo Li Yi Wang Yinan He Limin Wang Yu Qiao VGen 93 166 0 28 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.3K 14,289 0 15 Mar 2023
Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis Weixi Feng Xuehai He Tsu-Jui Fu Varun Jampani Arjun Reddy Akula P. Narayana Sugato Basu Xinze Wang William Yang Wang CoGe 91 315 0 09 Dec 2022
Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation Tsu-Jui Fu Licheng Yu Ning Zhang Cheng-Yang Fu Jong-Chyi Su William Yang Wang Sean Bell VGen 90 38 0 23 Nov 2022
Imagen Video: High Definition Video Generation with Diffusion Models Jonathan Ho William Chan Chitwan Saharia Jay Whang Ruiqi Gao ... Diederik P. Kingma Ben Poole Mohammad Norouzi David J. Fleet Tim Salimans VGen 105 1,528 0 05 Oct 2022
Make-A-Video: Text-to-Video Generation without Text-Video Data Uriel Singer Adam Polyak Thomas Hayes Xiaoyue Yin Jie An ... Oron Ashual Oran Gafni Devi Parikh Sonal Gupta Yaniv Taigman DiffM VGen 74 1,406 0 29 Sep 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid ... Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu EGVM 178 1,114 0 22 Jun 2022
Compositional Visual Generation with Composable Diffusion Models Nan Liu Shuang Li Yilun Du Antonio Torralba J. Tenenbaum DiffM CoGe 143 519 0 03 Jun 2022
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong Ming Ding Wendi Zheng Xinghan Liu Jie Tang DiffM 299 606 0 29 May 2022
GMFlow: Learning Optical Flow via Global Matching Haofei Xu Jing Zhang Jianfei Cai Hamid Rezatofighi Dacheng Tao 93 359 0 26 Nov 2021
Learning to Compose Visual Relations Nan Liu Shuang Li Yilun Du J. Tenenbaum Antonio Torralba CoGe OCL 60 79 0 17 Nov 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 136 1,172 0 01 Apr 2021
Compositional Video Synthesis with Action Graphs Amir Bar Roei Herzig Xiaolong Wang Anna Rohrbach Gal Chechik Trevor Darrell Amir Globerson 69 44 0 27 Jun 2020
Measuring Compositionality in Representation Learning Jacob Andreas CoGe 60 149 0 19 Feb 2019
Image Generation from Scene Graphs Justin Johnson Agrim Gupta Li Fei-Fei GNN 293 820 0 04 Apr 2018
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild K. Soomro Amir Zamir M. Shah CLIP VGen 141 6,145 0 03 Dec 2012