xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection

16 October 2023

Papers citing "xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection"

32 / 32 papers shown

Title
LLMs Are Not Scorers: Rethinking MT Evaluation with Generation-Based Methods Hyang Cui LRM 84 0 0 22 May 2025
DeepSeek-R1 vs. o3-mini: How Well can Reasoning LLMs Evaluate MT and Summarization? Daniil Larionov Sotaro Takeshita Ran Zhang Yanran Chen Christoph Leiter Zhipin Wang Christian Greisinger Steffen Eger ReLM ELM LRM 150 1 0 10 Apr 2025
Post-edits Are Preferences Too Nathaniel Berger Stefan Riezler M. Exel Matthias Huck 114 1 0 24 Feb 2025
Automatic Input Rewriting Improves Translation with Large Language Models Dayeon Ki Marine Carpuat 81 0 0 23 Feb 2025
M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation Zhaopeng Feng Jiayuan Su Jiamei Zheng Jiahan Ren Yan Zhang Jian Wu Hongwei Wang Zuozhu Liu ELM 254 1 0 21 Feb 2025
Varco Arena: A Tournament Approach to Reference-Free Benchmarking Large Language Models Seonil Son Ju-Min Oh Heegon Jin Cheolhun Jang Jeongbeom Jeong Kuntae Kim 115 1 0 20 Feb 2025
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models Xu Huang Wenhao Zhu Hanxu Hu Zeang Sheng Lei Li Shujian Huang Fei Yuan ELM 128 4 0 11 Feb 2025
Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study Menglong Cui Pengzhi Gao Wei Liu Jian Luan Bin Wang LRM 97 5 0 04 Feb 2025
mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation Nishat Raihan Antonios Anastasopoulos Marcos Zampieri ELM 105 8 0 28 Jan 2025
A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls Sheikh Shafayat Dongkeun Yoon Woori Jang Jiwoo Choi Alice Oh Seohyon Jung 179 1 0 03 Jan 2025
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings Miguel Moura Ramos Tomás Almeida Daniel Vareta Filipe Azevedo Sweta Agrawal Patrick Fernandes André F. T. Martins 106 4 0 08 Nov 2024
How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMs Ran Zhang Wei Zhao Steffen Eger 123 10 0 24 Oct 2024
Adapters for Altering LLM Vocabularies: What Languages Benefit the Most? HyoJung Han Akiko Eriguchi Haoran Xu Hieu T. Hoang Marine Carpuat Huda Khayrallah VLM 74 3 0 12 Oct 2024
MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences Genta Indra Winata David Anugraha Lucky Susanto Garry Kuwanto Derry Wijaya 122 11 0 03 Oct 2024
AI-Assisted Human Evaluation of Machine Translation Vilém Zouhar Tom Kocmi Mrinmaya Sachan 98 7 0 18 Jun 2024
Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation Boxuan Lyu Hidetaka Kamigaito Kotaro Funakoshi Manabu Okumura 139 0 0 17 Jun 2024
(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts Minghao Wu Jiahao Xu Yulin Yuan Gholamreza Haffari Longyue Wang Weihua Luo Kaifu Zhang LLMAG 171 27 0 20 May 2024
CroissantLLM: A Truly Bilingual French-English Language Model Manuel Faysse Patrick Fernandes Nuno M. Guerreiro António Loison Duarte M. Alves ... François Yvon André F.T. Martins Gautier Viaud C´eline Hudelot Pierre Colombo 114 37 0 01 Feb 2024
Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task Ricardo Rei Nuno M. Guerreiro José P. Pombal Daan van Stigt Marcos Vinícius Treviso Luísa Coheur José G. C. de Souza André F. T. Martins 82 63 0 21 Sep 2023
Reinforced Self-Training (ReST) for Language Modeling Çağlar Gülçehre T. Paine S. Srinivasan Ksenia Konyushkova L. Weerts ... Chenjie Gu Wolfgang Macherey Arnaud Doucet Orhan Firat Nando de Freitas OffRL 123 308 0 17 Aug 2023
Hallucinations in Large Multilingual Translation Models Nuno M. Guerreiro Duarte M. Alves Jonas Waldendorf Barry Haddow Alexandra Birch Pierre Colombo André F.T. Martins VLM HILM LRM 182 152 0 28 Mar 2023
Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors Keqin Bao Boyi Deng Dayiheng Liu Baosong Yang Wenqiang Lei Xiangnan He Derek F.Wong Jun Xie 88 4 0 17 Feb 2023
IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languages Ananya B. Sai Vignesh Nagarajan Tanay Dixit Raj Dabre Anoop Kunchukuttan Pratyush Kumar Mitesh M. Khapra 125 23 0 20 Dec 2022
DEMETR: Diagnosing Evaluation Metrics for Translation Marzena Karpinska N. Raj Katherine Thai Yixiao Song Ankita Gupta Mohit Iyyer 68 38 0 25 Oct 2022
Quality-Aware Decoding for Neural Machine Translation Patrick Fernandes António Farinhas Ricardo Rei José G. C. de Souza Perez Ogayo Graham Neubig André F. T. Martins 96 57 0 02 May 2022
UniTE: Unified Translation Evaluation Boyi Deng Dayiheng Liu Baosong Yang Haibo Zhang Boxing Chen Derek F. Wong Lidia S. Chao 70 41 0 28 Apr 2022
Learning Compact Metrics for MT Amy Pu Hyung Won Chung Ankur P. Parikh Sebastian Gehrmann Thibault Sellam 72 101 0 12 Oct 2021
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation Tom Kocmi C. Federmann Roman Grundkiewicz Marcin Junczys-Dowmunt Hitokazu Matsushita Arul Menezes 86 210 0 22 Jul 2021
The Curious Case of Hallucinations in Neural Machine Translation Vikas Raunak Arul Menezes Marcin Junczys-Dowmunt 208 194 0 14 Apr 2021
Beyond English-Centric Multilingual Machine Translation Angela Fan Shruti Bhosale Holger Schwenk Zhiyi Ma Ahmed El-Kishky ... Vitaliy Liptchinsky Sergey Edunov Edouard Grave Michael Auli Armand Joulin LRM 83 857 0 21 Oct 2020
Language-agnostic BERT Sentence Embedding Fangxiaoyu Feng Yinfei Yang Daniel Cer N. Arivazhagan Wei Wang 174 913 0 03 Jul 2020
Optuna: A Next-generation Hyperparameter Optimization Framework Takuya Akiba Shotaro Sano Toshihiko Yanase Takeru Ohta Masanori Koyama 668 5,839 0 25 Jul 2019