Better Summarization Evaluation with Word Embeddings for ROUGE

25 August 2015

Papers citing "Better Summarization Evaluation with Word Embeddings for ROUGE"

40 / 40 papers shown

Title
LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning Joy Lim Jia Yin Daniel Zhang-Li Jifan Yu Haoyang Li Shangqing Tu ... Zhiyuan Liu Huiqin Liu Lei Hou Juanzi Li Bin Xu 34 0 0 04 May 2025
Summarization Metrics for Spanish and Basque: Do Automatic Scores and LLM-Judges Correlate with Humans? Jeremy Barnes Naiara Perez Alba Bonet-Jover Begoña Altuna 67 1 0 21 Mar 2025
MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences Genta Indra Winata David Anugraha Lucky Susanto Garry Kuwanto Derry Wijaya 50 9 0 03 Oct 2024
Rethinking Transformer-based Multi-document Summarization: An Empirical Investigation Congbo Ma Wei Emma Zhang Dileepa Pitawela Haojie Zhuang Yanfeng Shu 22 0 0 16 Jul 2024
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment Abhinav Agarwalla Abhay Gupta Alexandre Marques Shubhra Pandit Michael Goin ... Tuan Nguyen Mahmoud Salem Dan Alistarh Sean Lie Mark Kurtz MoE SyDa 45 11 0 06 May 2024
ROUGE-K: Do Your Summaries Have Keywords? Sotaro Takeshita Simone Paolo Ponzetto Kai Eckert 34 0 0 08 Mar 2024
Generative Judge for Evaluating Alignment Junlong Li Shichao Sun Weizhe Yuan Run-Ze Fan Hai Zhao Pengfei Liu ELM ALM 35 80 0 09 Oct 2023
UMSE: Unified Multi-scenario Summarization Evaluation Shen Gao Zhitao Yao Chongyang Tao Preslav Nakov Pengjie Ren Zhaochun Ren Zhumin Chen 43 5 0 26 May 2023
Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method Yiming Wang Zhuosheng Zhang Rui Wang 46 79 0 22 May 2023
Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks Anas Himmi Ekhine Irurozki Nathan Noiry Stéphan Clémençon Pierre Colombo 34 5 0 17 May 2023
SimCSum: Joint Learning of Simplification and Cross-lingual Summarization for Cross-lingual Science Journalism Mehwish Fatima Tim Kolber K. Markert Michael Strube 26 0 0 04 Apr 2023
Lay Text Summarisation Using Natural Language Processing: A Narrative Literature Review Oliver Vinzelberg M. Jenkins Gordon Morison David McMinn Z. Tieges 37 6 0 24 Mar 2023
Curriculum-Guided Abstractive Summarization Sajad Sotudeh Hanieh Deilamsalehy Franck Dernoncourt Nazli Goharian 45 1 0 02 Feb 2023
How Far are We from Robust Long Abstractive Summarization? Huan Yee Koh Jiaxin Ju He Zhang Ming Liu Shirui Pan HILM 38 39 0 30 Oct 2022
Towards Interpretable Summary Evaluation via Allocation of Contextual Embeddings to Reference Text Topics Ben Schaper Christopher Lohse Marcell Streile Andrea Giovannini Richard Osuala 24 1 0 25 Oct 2022
DATScore: Evaluating Translation with Data Augmented Translations Moussa Kamal Eddine Guokan Shang Michalis Vazirgiannis 44 5 0 12 Oct 2022
The Glass Ceiling of Automatic Evaluation in Natural Language Generation Pierre Colombo Maxime Peyrard Nathan Noiry Robert West Pablo Piantanida 52 11 0 31 Aug 2022
Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation Cyril Chhun Pierre Colombo Chloé Clavel Fabian M. Suchanek 58 51 0 24 Aug 2022
SMART: Sentences as Basic Units for Text Evaluation Reinald Kim Amplayo Peter J. Liu Yao-Min Zhao Shashi Narayan 38 21 0 01 Aug 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 83 122 0 03 Jul 2022
Towards Explainable Evaluation Metrics for Natural Language Generation Christoph Leiter Piyawat Lertvittayakumjorn M. Fomicheva Wei Zhao Yang Gao Steffen Eger AAML ELM 40 20 0 21 Mar 2022
DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence Wei Zhao Michael Strube Steffen Eger 27 37 0 26 Jan 2022
Multi-Narrative Semantic Overlap Task: Evaluation and Benchmark Naman Bansal Mousumi Akter Shubhra (Santu) Karmaker 36 0 0 14 Jan 2022
InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation Pierre Colombo Chloe Clave Pablo Piantanida 40 41 0 02 Dec 2021
Better than Average: Paired Evaluation of NLP Systems Maxime Peyrard Wei Zhao Steffen Eger Robert West ELM 19 24 0 20 Oct 2021
FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation Moussa Kamal Eddine Guokan Shang A. Tixier Michalis Vazirgiannis 26 25 0 16 Oct 2021
SummerTime: Text Summarization Toolkit for Non-experts Ansong Ni Zhangir Azerbayev Mutethia Mutuma Troy Feng Yusen Zhang Tao Yu Ahmed Hassan Awadallah Dragomir R. Radev 31 10 0 29 Aug 2021
Towards Human-Free Automatic Quality Evaluation of German Summarization Neslihan Iskender Oleg V. Vasilyev Tim Polzehl John Bohannon Sebastian Möller 34 1 0 13 May 2021
Multi-document Summarization via Deep Learning Techniques: A Survey Congbo Ma W. Zhang Mingyu Guo Hu Wang Quan Z. Sheng 18 126 0 10 Nov 2020
A critical analysis of metrics used for measuring progress in artificial intelligence Kathrin Blagec Georg Dorffner M. Moradi Matthias Samwald 41 33 0 06 Aug 2020
SummEval: Re-evaluating Summarization Evaluation Alexander R. Fabbri Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher Dragomir R. Radev HILM 38 691 0 24 Jul 2020
SueNes: A Weakly Supervised Approach to Evaluating Single-Document Summarization via Negative Sampling F. S. Bao Hebi Li Ge Luo Minghui Qiu Yinfei Yang Youbiao He Cen Chen 24 4 0 13 May 2020
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization Yang Gao Wei Zhao Steffen Eger ELM 27 124 0 07 May 2020
Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization Dongyub Lee M. Shin Taesun Whang Seung Woo Cho Byeongil Ko Daniel Lee EungGyun Kim Jaechoon Jo 41 12 0 29 Apr 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 47 116 0 23 Feb 2020
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models Sandeep Subramanian Raymond Li Jonathan Pilault C. Pal 248 215 0 07 Sep 2019
Answers Unite! Unsupervised Metrics for Reinforced Summarization Models Thomas Scialom Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano 32 149 0 04 Sep 2019
Neural Text Summarization: A Critical Evaluation Wojciech Kry'sciñski N. Keskar Bryan McCann Caiming Xiong R. Socher 22 361 0 23 Aug 2019
A Semantically Motivated Approach to Compute ROUGE Scores Elaheh Shafieibavani M. Ebrahimi R. Wong Fang Chen 16 4 0 20 Oct 2017
A Semantic QA-Based Approach for Text Summarization Evaluation Ping Chen Fei Wu Tong Wang Wei Ding 19 41 0 21 Apr 2017