The Devil is in the Errors: Leveraging Large Language Models for
Fine-grained Machine Translation Evaluation

The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation

14 August 2023

Patrick Fernandes

André F. T. Martins

Graham Neubig

ArXiv (abs)PDF HTML

Papers citing "The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation"

17 / 17 papers shown

Title
Same evaluation, more tokens: On the effect of input length for machine translation evaluation using Large Language Models Tobias Domhan Dawei Zhu 96 0 0 03 May 2025
An Illusion of Progress? Assessing the Current State of Web Agents Tianci Xue Weijian Qi Tianneng Shi Chan Hee Song Boyu Gou Basel Alomair Huan Sun Yu Su LLMAG ELM Presented at ResearchTrend Connect \| LLMAG on 21 May 2025 236 13 1 02 Apr 2025
Automatic Input Rewriting Improves Translation with Large Language Models Dayeon Ki Marine Carpuat 81 0 0 23 Feb 2025
M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation Zhaopeng Feng Jiayuan Su Jiamei Zheng Jiahan Ren Yan Zhang Jian Wu Hongwei Wang Zuozhu Liu ELM 251 1 0 21 Feb 2025
When LLMs Struggle: Reference-less Translation Evaluation for Low-resource Languages Archchana Sindhujan Diptesh Kanojia Constantin Orasan Shenbin Qian 91 3 0 08 Jan 2025
Language Imbalance Driven Rewarding for Multilingual Self-improving Wen Yang Junhong Wu Chen Wang Chengqing Zong J.N. Zhang ALM LRM 164 7 0 11 Oct 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 365 337 0 18 Jan 2024
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 147 334 0 02 Jun 2023
T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics Yiwei Qin Weizhe Yuan Graham Neubig Pengfei Liu 61 23 0 12 Dec 2022
Prompting PaLM for Translation: Assessing Strategies and Performance David Vilar Markus Freitag Colin Cherry Jiaming Luo Viresh Ratnakar George F. Foster LRM 101 166 0 16 Nov 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 515 6,293 0 05 Apr 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 823 9,644 0 28 Jan 2022
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 217 3,782 0 03 Sep 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 218 3,987 0 28 Jul 2021
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation Tom Kocmi C. Federmann Roman Grundkiewicz Marcin Junczys-Dowmunt Hitokazu Matsushita Arul Menezes 84 210 0 22 Jul 2021
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 228 6,585 0 05 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 459 20,298 0 23 Oct 2019