Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications

13 May 2022

Papers citing "Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications"

7 / 7 papers shown

Title
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts Helia Hashemi J. Eisner Corby Rosset Benjamin Van Durme Chris Kedzie 68 1 0 03 Jan 2025
METAL: Towards Multilingual Meta-Evaluation Rishav Hada Varun Gumma Mohamed Ahmed Kalika Bali Sunayana Sitaram ELM 35 2 0 02 Apr 2024
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices Yu Lu Liu Meng Cao Su Lin Blodgett Jackie Chi Kit Cheung Alexandra Olteanu Adam Trischler 26 1 0 18 Nov 2023
Rethinking Model Evaluation as Narrowing the Socio-Technical Gap Q. V. Liao Ziang Xiao ALM ELM 43 29 0 01 Jun 2023
The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation Marzena Karpinska Nader Akoury Mohit Iyyer 218 106 0 14 Sep 2021
Perturbation CheckLists for Evaluating NLG Evaluation Metrics Ananya B. Sai Tanay Dixit D. Y. Sheth S. Mohan Mitesh M. Khapra AAML 108 56 0 13 Sep 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 248 285 0 02 Feb 2021