v1v2 (latest)

Summarization Metrics for Spanish and Basque: Do Automatic Scores and LLM-Judges Correlate with Humans?

21 March 2025

Papers citing "Summarization Metrics for Spanish and Basque: Do Automatic Scores and LLM-Judges Correlate with Humans?"

38 / 38 papers shown

Title
Summarizing Speech: A Comprehensive Survey Fabian Retkowski Maike Züfle Andreas Sudmann Dinah Pfau Jan Niehues Alexander Waibel Alexander H. Waibel 122 0 0 10 Apr 2025
Conditioning LLMs to Generate Code-Switched Text Maite Heredia Gorka Labaka Jeremy Barnes A. Soroa 44 1 0 18 Feb 2025
Atla Selene Mini: A General Purpose Evaluation Model Andrei Alexandru Antonia Calvi Henry Broomfield Jackson Golden Kyle Dai ... Max Bartolo Roman Engeler Sashank Pisupati Toby Drane Young Sun Park ALM ELM AILaw LM&MA LRM 101 6 0 27 Jan 2025
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models Seungone Kim Juyoung Suk Shayne Longpre Bill Yuchen Lin Jamin Shin Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo MoMe ALM ELM 155 205 0 02 May 2024
Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models Aitor Ormazabal Che Zheng Cyprien de Masson dÁutume Dani Yogatama Deyu Fu ... Yazheng Yang Yi Tay Yuqi Wang Zhongkai Zhu Zhihui Xie LRM VLM ReLM 98 52 0 18 Apr 2024
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document Joonho Yang Seunghyun Yoon Byeongjeong Kim Hwanhee Lee HILM 117 7 0 17 Apr 2024
Latxa: An Open Language Model and Evaluation Suite for Basque Julen Etxaniz Oscar Sainz Naiara Pérez Itziar Aldabe German Rigau Eneko Agirre Aitor Ormazabal Mikel Artetxe A. Soroa ELM 72 32 0 29 Mar 2024
The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics Christoph Leiter Juri Opitz Daniel Deutsch Yang Gao Rotem Dror Steffen Eger ALM LRM ELM 103 32 0 30 Oct 2023
Do Multilingual Language Models Think Better in English? Julen Etxaniz Gorka Azkune Aitor Soroa Etxabe Oier López de Lacalle Mikel Artetxe LRM 92 73 0 02 Aug 2023
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation Elizabeth Clark Shruti Rijhwani Sebastian Gehrmann Joshua Maynez Roee Aharoni Vitaly Nikolaev Thibault Sellam Aditya Siddhant Dipanjan Das Ankur P. Parikh 97 41 0 22 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.7K 14,870 0 15 Mar 2023
mFACE: Multilingual Summarization with Factual Consistency Evaluation Roee Aharoni Shashi Narayan Joshua Maynez Jonathan Herzig Elizabeth Clark Mirella Lapata HILM 85 47 0 20 Dec 2022
Language Models are Multilingual Chain-of-Thought Reasoners Freda Shi Mirac Suzgun Markus Freitag Xuezhi Wang Suraj Srivats ... Yi Tay Sebastian Ruder Denny Zhou Dipanjan Das Jason W. Wei ReLM LRM 276 369 0 06 Oct 2022
Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics Daniel Deutsch Rotem Dror Dan Roth 75 45 0 21 Apr 2022
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization Philippe Laban Tobias Schnabel Paul N. Bennett Marti A. Hearst HILM 109 398 0 18 Nov 2021
Evaluating the Efficacy of Summarization Evaluation across Languages Fajri Koto Jey Han Lau Timothy Baldwin 114 19 0 02 Jun 2021
How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluation Julius Steen K. Markert ELM 90 13 0 27 Jan 2021
FFCI: A Framework for Interpretable Automatic Evaluation of Summarization Fajri Koto Timothy Baldwin Jey Han Lau HILM 113 37 0 27 Nov 2020
Re-evaluating Evaluation in Text Summarization Manik Bhandari Pranav Narayan Gour A. Ashfaq Pengfei Liu Graham Neubig 189 178 0 14 Oct 2020
SummEval: Re-evaluating Summarization Evaluation Alexander R. Fabbri Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher Dragomir R. Radev HILM 155 725 0 24 Jul 2020
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization Yang Gao Wei Zhao Steffen Eger ELM 120 126 0 07 May 2020
On Faithfulness and Factuality in Abstractive Summarization Joshua Maynez Shashi Narayan Bernd Bohnet Ryan T. McDonald HILM 108 1,048 0 02 May 2020
MLSUM: The Multilingual Summarization Corpus Thomas Scialom Paul-Alexis Dray Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano 102 177 0 30 Apr 2020
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages Peng Qi Yuhao Zhang Yuhui Zhang Jason Bolton Christopher D. Manning AI4TS 330 1,702 0 16 Mar 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 114 119 0 23 Feb 2020
Evaluating the Factual Consistency of Abstractive Text Summarization Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher HILM 193 748 0 28 Oct 2019
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance Wei Zhao Maxime Peyrard Fei Liu Yang Gao Christian M. Meyer Steffen Eger 271 602 0 05 Sep 2019
Answers Unite! Unsupervised Metrics for Reinforced Summarization Models Thomas Scialom Sylvain Lamprier Benjamin Piwowarski Jacopo Staiano 90 150 0 04 Sep 2019
Neural Text Summarization: A Critical Evaluation Wojciech Kry'sciñski N. Keskar Bryan McCann Caiming Xiong R. Socher 124 368 0 23 Aug 2019
HighRES: Highlight-based Reference-less Evaluation of Summarization Hardy Hardy Shashi Narayan Andreas Vlachos 90 61 0 04 Jun 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 684 5,897 0 21 Apr 2019
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 261 1,689 0 27 Aug 2018
Improving Abstraction in Text Summarization Wojciech Kry'sciñski Romain Paulus Caiming Xiong R. Socher 79 147 0 23 Aug 2018
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies Max Grusky Mor Naaman Yoav Artzi 176 559 0 30 Apr 2018
A Call for Clarity in Reporting BLEU Scores Matt Post 253 3,001 0 23 Apr 2018
Better Summarization Evaluation with Word Embeddings for ROUGE Jun-Ping Ng Viktoria Abrecht 69 172 0 25 Aug 2015
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 395 3,558 0 10 Jun 2015
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 454 4,537 0 20 Nov 2014