Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt
Coherence Metrics with T2IScoreScore (TS2)

Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)

5 April 2024

Michael Stephen Saxon

Mahsa Khoshnoodi

William Y. Wang

Papers citing "Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)"

18 / 18 papers shown

Title
CROC: Evaluating and Training T2I Metrics with Pseudo- and Human-Labeled Contrastive Robustness Checks Christoph Leiter Yuki M. Asano M. Keuper Steffen Eger 22 0 0 16 May 2025
Multi-Modal Language Models as Text-to-Image Model Evaluators Jiahui Chen Candace Ross Reyhane Askari Hemmat Koustuv Sinha Melissa Hall M. Drozdzal Adriana Romero-Soriano EGVM 60 0 0 01 May 2025
TikZero: Zero-Shot Text-Guided Graphics Program Synthesis Jonas Belouadi Eddy Ilg M. Keuper Hideki Tanaka Masao Utiyama Raj Dabre Steffen Eger Simone Paolo Ponzetto 52 0 0 14 Mar 2025
What makes a good metric? Evaluating automatic metrics for text-to-image consistency Candace Ross Melissa Hall Adriana Romero Soriano Adina Williams 95 3 0 18 Dec 2024
ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation? Leixin Zhang Steffen Eger Yinjie Cheng Weihe Zhai Jonas Belouadi Christoph Leiter Simone Paolo Ponzetto Fahimeh Moafian Zhixue Zhao MLLM 93 1 0 03 Dec 2024
TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives Maitreya Patel Abhiram Kusumba Sheng Cheng Changhoon Kim Tejas Gokhale Chitta Baral Yezhou Yang CoGe CLIP 62 7 0 04 Nov 2024
Natural Language Inference Improves Compositionality in Vision-Language Models Paola Cascante-Bonilla Yu Hou Yang Trista Cao Hal Daumé III Rachel Rudinger ReLM CoGe VLM 60 3 0 29 Oct 2024
Benchmarks as Microscopes: A Call for Model Metrology Michael Stephen Saxon Ari Holtzman Peter West William Y. Wang Naomi Saphra 39 10 0 22 Jul 2024
TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation Weixi Feng Jiachen Li Michael Stephen Saxon Tsu-jui Fu Wenhu Chen William Yang Wang EGVM VGen 38 9 0 12 Jun 2024
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings Olivia Wiles Chuhan Zhang Isabela Albuquerque Ivana Kajić Su Wang ... Jordi Pont-Tuset Aida Nematzadeh Anant Nawalgaria Jordi Pont-Tuset Aida Nematzadeh EGVM 135 14 0 25 Apr 2024
Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts Michael Stephen Saxon Yiran Luo Sharon Levy Chitta Baral Yezhou Yang William Y. Wang EGVM 33 3 0 17 Mar 2024
Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation Yuval Kirstain Adam Polyak Uriel Singer Shahbuland Matiana Joe Penna Omer Levy EGVM 168 352 0 02 May 2023
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models Jaemin Cho Abhaysinh Zala Joey Tianyi Zhou ViT 145 170 0 08 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 392 4,154 0 28 Jan 2022
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,796 0 24 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 337 3,708 0 11 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020
Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola Jun-Yan Zhu Tinghui Zhou Alexei A. Efros SSeg 212 19,455 0 21 Nov 2016