v1v2 (latest)

Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory

24 May 2023

Papers citing "Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory"

27 / 27 papers shown

Title
SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction Lu Dai Yijie Xu Jinhui Ye Hao Liu Hui Xiong 3DV RALM 192 3 0 03 Mar 2025
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts Helia Hashemi J. Eisner Corby Rosset Benjamin Van Durme Chris Kedzie 138 4 0 03 Jan 2025
4-LEGS: 4D Language Embedded Gaussian Splatting Gal Fiebelman Tamir Cohen Ayellet Morgenstern Peter Hedman Hadar Averbuch-Elor 3DGS 121 1 0 14 Oct 2024
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing Han Jiang Xiaoyuan Yi Zhihua Wei Ziang Xiao Shu Wang Xing Xie ELM ALM 146 8 0 20 Jun 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 197 40 0 02 Feb 2024
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation Elizabeth Clark Shruti Rijhwani Sebastian Gehrmann Joshua Maynez Roee Aharoni Vitaly Nikolaev Thibault Sellam Aditya Siddhant Dipanjan Das Ankur P. Parikh 80 41 0 22 May 2023
On the Effectiveness of Automated Metrics for Text Generation Systems Pius von Daniken Jan Deriu Don Tuggener Mark Cieliebak 61 3 0 24 Oct 2022
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications Kaitlyn Zhou Su Lin Blodgett Adam Trischler Hal Daumé Kaheer Suleman Alexandra Olteanu ELM 142 30 0 13 May 2022
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text Sebastian Gehrmann Elizabeth Clark Thibault Sellam ELM AI4CE 154 193 0 14 Feb 2022
Perturbation CheckLists for Evaluating NLG Evaluation Metrics Ananya B. Sai Tanay Dixit D. Y. Sheth S. Mohan Mitesh M. Khapra AAML 149 58 0 13 Sep 2021
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text Elizabeth Clark Tal August Sofia Serrano Nikita Haduong Suchin Gururangan Noah A. Smith DeLMO 133 415 0 30 Jun 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 139 850 0 22 Jun 2021
GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation Daniel Khashabi Gabriel Stanovsky Jonathan Bragg Nicholas Lourie Jungo Kasai Yejin Choi Noah A. Smith Daniel S. Weld 115 21 0 17 Jan 2021
Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale Ozan Caglayan Pranava Madhyastha Lucia Specia ELM 96 36 0 26 Oct 2020
Defining and Evaluating Fair Natural Language Generation C. Yeo A. Chen 77 24 0 28 Jul 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 139 387 0 26 Jun 2020
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization Yang Gao Wei Zhao Steffen Eger ELM 100 126 0 07 May 2020
BLEURT: Learning Robust Metrics for Text Generation Thibault Sellam Dipanjan Das Ankur P. Parikh 113 1,507 0 09 Apr 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 100 119 0 23 Feb 2020
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance Wei Zhao Maxime Peyrard Fei Liu Yang Gao Christian M. Meyer Steffen Eger 196 602 0 05 Sep 2019
Answers Unite! Unsupervised Metrics for Reinforced Summarization Models Thomas Scialom Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano 76 150 0 04 Sep 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 386 5,872 0 21 Apr 2019
Towards a Better Metric for Evaluating Question Generation Systems Preksha Nema Mitesh M. Khapra 90 108 0 30 Aug 2018
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies Max Grusky Mor Naaman Yoav Artzi 102 559 0 30 Apr 2018
Better Summarization Evaluation with Word Embeddings for ROUGE Jun-Ping Ng Viktoria Abrecht 65 172 0 25 Aug 2015
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 308 4,520 0 20 Nov 2014
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 406 33,573 0 16 Oct 2013