Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics

21 April 2022

Papers citing "Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics"

31 / 31 papers shown

Title
Summarization Metrics for Spanish and Basque: Do Automatic Scores and LLM-Judges Correlate with Humans? Jeremy Barnes Naiara Perez Alba Bonet-Jover Begoña Altuna 97 2 0 21 Mar 2025
Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge Aparna Elangovan Jongwoo Ko Lei Xu Mahsa Elyasi Ling Liu S. Bodapati Dan Roth 97 6 0 28 Jan 2025
Analyzing and Evaluating Correlation Measures in NLG Meta-Evaluation Mingqi Gao Xinyu Hu Li Lin Xiaojun Wan 71 2 0 28 Jan 2025
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 166 48 0 03 Oct 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 274 22 0 28 Feb 2024
SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization Yixin Liu Peng Liu AILaw 74 261 0 03 Jun 2021
Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning Linzi Xing Wen Xiao Giuseppe Carenini 54 18 0 29 May 2021
The statistical advantage of automatic NLG metrics at the system level Johnny Tian-Zheng Wei Robin Jia 66 23 0 26 May 2021
Improving Factual Consistency of Abstractive Summarization via Question Answering Feng Nan Cicero Nogueira dos Santos Henghui Zhu Patrick Ng Kathleen McKeown Ramesh Nallapati Dejiao Zhang Zhiguo Wang Andrew O. Arnold Bing Xiang HILM 67 86 0 10 May 2021
Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs Jiaao Chen Diyi Yang 76 98 0 16 Apr 2021
RefSum: Refactoring Neural Summarization Yixin Liu Zi-Yi Dou Pengfei Liu 57 40 0 15 Apr 2021
LazyDAgger: Reducing Context Switching in Interactive Imitation Learning Ryan Hoque Ashwin Balakrishna Carl Putterman Michael Luo Daniel S. Brown Daniel Seita Brijen Thananjeyan Ellen R. Novoseller Ken Goldberg 129 76 0 31 Mar 2021
Unsupervised Extractive Summarization using Pointwise Mutual Information Vishakh Padmakumar He He 40 46 0 11 Feb 2021
Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries Daniel Deutsch Dan Roth 104 7 0 23 Oct 2020
GSum: A General Framework for Guided Neural Abstractive Summarization Zi-Yi Dou Pengfei Liu Hiroaki Hayashi Zhengbao Jiang Graham Neubig 66 259 0 15 Oct 2020
Compressive Summarization with Plausibility and Salience Modeling Shrey Desai Jiacheng Xu Greg Durrett 47 18 0 15 Oct 2020
Re-evaluating Evaluation in Text Summarization Manik Bhandari Pranav Narayan Gour A. Ashfaq Pengfei Liu Graham Neubig 153 178 0 14 Oct 2020
Stepwise Extractive Summarization and Planning with Structured Transformers Shashi Narayan Joshua Maynez Jakub Adamek Daniele Pighin Blavz Bratanivc Ryan T. McDonald 65 30 0 06 Oct 2020
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary Daniel Deutsch Tania Bedrax-Weiss Dan Roth 69 113 0 01 Oct 2020
Noisy Self-Knowledge Distillation for Text Summarization Yang Liu S. Shen Mirella Lapata 63 44 0 15 Sep 2020
SummEval: Re-evaluating Summarization Evaluation Alexander R. Fabbri Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher Dragomir R. Radev HILM 100 722 0 24 Jul 2020
Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics Nitika Mathur Tim Baldwin Trevor Cohn 56 247 0 11 Jun 2020
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang Lingfei Wu Lu Wang RALM 83 164 0 03 May 2020
Extractive Summarization as Text Matching Ming Zhong Pengfei Liu Yiran Chen Danqing Wang Xipeng Qiu Xuanjing Huang 143 462 0 19 Apr 2020
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising Ziyi Yang Chenguang Zhu R. Gmyr Michael Zeng Xuedong Huang Eric Darve 79 61 0 03 Jan 2020
Discourse-Aware Neural Extractive Text Summarization Jiacheng Xu Zhe Gan Yu Cheng Jingjing Liu BDL 144 283 0 30 Oct 2019
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance Wei Zhao Maxime Peyrard Fei Liu Yang Gao Christian M. Meyer Steffen Eger 187 602 0 05 Sep 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 357 5,872 0 21 Apr 2019
Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation Ori Shapira David Gabay Yang Gao H. Ronen Ramakanth Pasunuru Joey Tianyi Zhou Yael Amsterdamer Ido Dagan 68 60 0 11 Apr 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,229 0 11 Oct 2018
Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond Ramesh Nallapati Bowen Zhou Cicero Nogueira dos Santos Çağlar Gülçehre Bing Xiang AIMat 281 2,567 0 19 Feb 2016