A Challenge Set Approach to Evaluating Machine Translation

24 April 2017

Colin Cherry

Papers citing "A Challenge Set Approach to Evaluating Machine Translation"

31 / 31 papers shown

Title
A Large-Scale Benchmark for Vietnamese Sentence Paraphrases Sang Quang Nguyen Kiet Van Nguyen 62 0 0 11 Feb 2025
Should We Attend More or Less? Modulating Attention for Fairness A. Zayed Gonçalo Mordido Samira Shabanian Sarath Chandar 40 10 0 22 May 2023
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus Yu Jiang Tianyu Liu Shuming Ma Dongdong Zhang Mrinmaya Sachan Ryan Cotterell 27 7 0 18 May 2023
Angler: Helping Machine Translation Practitioners Prioritize Model Improvements Samantha Robertson Zijie J. Wang Dominik Moritz Mary Beth Kery Fred Hohman 38 15 0 12 Apr 2023
Democratizing Neural Machine Translation with OPUS-MT Jörg Tiedemann Mikko Aulamo Daria Bakshandaeva M. Boggia Stig-Arne Gronroos Tommi Nieminen Alessandro Raganato Yves Scherrer Raúl Vázquez Sami Virpioja 18 28 0 04 Dec 2022
MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation Anna Currey Maria Nadejde R. Pappagari Mia C. Mayer Stanislas Lauly Xing Niu B. Hsu Georgiana Dinu 35 32 0 02 Nov 2022
ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics Chantal Amrhein Nikita Moghe Liane Guillou ELM 39 22 0 27 Oct 2022
A Bilingual Parallel Corpus with Discourse Annotations Yu Jiang Tianyu Liu Shuming Ma Dongdong Zhang Mrinmaya Sachan Ryan Cotterell 24 1 0 26 Oct 2022
PreQuEL: Quality Estimation of Machine Translation Outputs in Advance Shachar Don-Yehiya Leshem Choshen Omri Abend 33 10 0 18 May 2022
Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets Philippe Laban Chien-Sheng Wu Wenhao Liu Caiming Xiong 43 5 0 13 May 2022
Sometimes We Want Translationese Prasanna Parthasarathi Koustuv Sinha J. Pineau Adina Williams AAML 22 4 0 15 Apr 2021
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation Yu Jiang Tianyu Liu Shuming Ma Dongdong Zhang Jian Yang Haoyang Huang Rico Sennrich Ryan Cotterell Mrinmaya Sachan M. Zhou 24 58 0 22 Mar 2021
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering Abhilasha Ravichander Siddharth Dalmia Maria Ryskina Florian Metze Eduard H. Hovy A. Black ELM 29 32 0 16 Feb 2021
HateCheck: Functional Tests for Hate Speech Detection Models Paul Röttger B. Vidgen Dong Nguyen Zeerak Talat Helen Z. Margetts J. Pierrehumbert 31 260 0 31 Dec 2020
Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale Ozan Caglayan Pranava Madhyastha Lucia Specia ELM 39 35 0 26 Oct 2020
CausaLM: Causal Model Explanation Through Counterfactual Language Models Amir Feder Nadav Oved Uri Shalit Roi Reichart CML LRM 44 157 0 27 May 2020
Cross-Linguistic Syntactic Evaluation of Word Prediction Models Aaron Mueller Garrett Nicolai Panayiota Petrou-Zeniou N. Talmina Tal Linzen 22 55 0 01 May 2020
Evaluating Models' Local Decision Boundaries via Contrast Sets Matt Gardner Yoav Artzi Victoria Basmova Jonathan Berant Ben Bogin ... Sanjay Subramanian Reut Tsarfaty Eric Wallace Ally Zhang Ben Zhou ELM 43 84 0 06 Apr 2020
Stance Detection Benchmark: How Robust Is Your Stance Detection? Benjamin Schiller Johannes Daxenberger Iryna Gurevych 19 96 0 06 Jan 2020
Neural Machine Translation: A Review and Survey Felix Stahlberg 3DV AI4TS MedIm 28 313 0 04 Dec 2019
Automatically Extracting Challenge Sets for Non local Phenomena in Neural Machine Translation Leshem Choshen Omri Abend 19 18 0 15 Sep 2019
Evaluating Gender Bias in Machine Translation Gabriel Stanovsky Noah A. Smith Luke Zettlemoyer 22 395 0 03 Jun 2019
DiaBLa: A Corpus of Bilingual Spontaneous Written Dialogues for Machine Translation Rachel Bawden S. Rosset Thomas Lavergne Éric Bilinski 116 13 0 30 May 2019
Chinese-Japanese Unsupervised Neural Machine Translation Using Sub-character Level Information Longtu Zhang Mamoru Komachi 20 10 0 01 Mar 2019
Semantic Neural Machine Translation using AMR Linfeng Song D. Gildea Yue Zhang Zhiguo Wang Jinsong Su 22 141 0 19 Feb 2019
Sentence-Level Fluency Evaluation: References Help, But Can Be Spared! Katharina Kann S. Rothe Katja Filippova 22 71 0 24 Sep 2018
Stress Test Evaluation for Natural Language Inference Aakanksha Naik Abhilasha Ravichander Norman M. Sadeh Carolyn Rose Graham Neubig ELM 36 371 0 02 Jun 2018
Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness V. Carmona Jeff Mitchell Sebastian Riedel 29 44 0 11 May 2018
Analyzing Uncertainty in Neural Machine Translation Myle Ott Michael Auli David Grangier MarcÁurelio Ranzato UQLM 43 271 0 28 Feb 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016
Neural versus Phrase-Based Machine Translation Quality: a Case Study L. Bentivogli Arianna Bisazza Mauro Cettolo Marcello Federico 191 328 0 16 Aug 2016