A Set of Recommendations for Assessing Human-Machine Parity in Language Translation

3 April 2020

Graham Neubig

Antonio Toral

Papers citing "A Set of Recommendations for Assessing Human-Machine Parity in Language Translation"

21 / 21 papers shown

Title
AI agents may be worth the hype but not the resources (yet): An initial exploration of machine translation quality and costs in three language pairs in the legal and news domains Vicent Briva-Iglesias Gokhan Dogru LLMAG ELM 39 0 0 02 May 2025
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation Yingfeng Luo Tong Zheng Yongyu Mu Yangqiu Song Qinghong Zhang ... Ziqiang Xu Peinan Feng Xiaoqian Liu Tong Xiao Jingbo Zhu AI4CE 215 0 0 09 Mar 2025
A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala Surangika Ranathunga Asanka Ranasinghea Janaka Shamala Ayodya Dandeniyaa Rashmi Galappaththia Malithi Samaraweeraa 76 0 0 03 Dec 2024
Human Evaluation of English--Irish Transformer-Based NMT Séamus Lankford Haithem Afli Andy Way 42 10 0 04 Mar 2024
Large Language Models "Ad Referendum": How Good Are They at Machine Translation in the Legal Domain? Vicent Briva-Iglesias Joao Lucas Cavalheiro Camargo Gokhan Dogru AILaw ELM 38 7 0 12 Feb 2024
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus Yu Jiang Tianyu Liu Shuming Ma Dongdong Zhang Mrinmaya Sachan Ryan Cotterell 27 7 0 18 May 2023
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 42 26 0 15 May 2023
HilMeMe: A Human-in-the-Loop Machine Translation Evaluation Metric Looking into Multi-Word Expressions Lifeng Han 12 2 0 09 Nov 2022
A Bilingual Parallel Corpus with Discourse Annotations Yu Jiang Tianyu Liu Shuming Ma Dongdong Zhang Mrinmaya Sachan Ryan Cotterell 24 1 0 26 Oct 2022
Searching for a higher power in the human evaluation of MT Johnny Tian-Zheng Wei Tom Kocmi C. Federmann 18 6 0 20 Oct 2022
DEEP: DEnoising Entity Pre-training for Neural Machine Translation Junjie Hu Hiroaki Hayashi Kyunghyun Cho Graham Neubig AI4CE 24 21 0 14 Nov 2021
Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers Benjamin Marie Atsushi Fujita Raphaël Rubino ELM 13 103 0 29 Jun 2021
Translation Quality Assessment: A Brief Survey on Manual and Automatic Methods Lifeng Han Gareth J. F. Jones Alan F. Smeaton 36 36 0 05 May 2021
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation Markus Freitag George F. Foster David Grangier Viresh Ratnakar Qijun Tan Wolfgang Macherey 21 373 0 29 Apr 2021
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation Yu Jiang Tianyu Liu Shuming Ma Dongdong Zhang Jian Yang Haoyang Huang Rico Sennrich Ryan Cotterell Mrinmaya Sachan M. Zhou 24 56 0 22 Mar 2021
KoBE: Knowledge-Based Machine Translation Evaluation Zorik Gekhman Roee Aharoni Genady Beryozkin Markus Freitag Wolfgang Macherey 23 15 0 23 Sep 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 19 376 0 26 Jun 2020
Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019 Antonio Toral 19 43 0 12 May 2020
What Question Answering can Learn from Trivia Nerds Jordan L. Boyd-Graber Benjamin Borschinger 24 36 0 31 Oct 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,748 0 26 Sep 2016
Neural versus Phrase-Based Machine Translation Quality: a Case Study L. Bentivogli Arianna Bisazza Mauro Cettolo Marcello Federico 191 328 0 16 Aug 2016