v1v2 (latest)

IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languages

20 December 2022

Mitesh M. Khapra

Papers citing "IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languages"

33 / 33 papers shown

Title
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 392 2,388 0 09 Nov 2022
ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics Chantal Amrhein Nikita Moghe Liane Guillou ELM 86 23 0 27 Oct 2022
DEMETR: Diagnosing Evaluation Metrics for Translation Marzena Karpinska N. Raj Katherine Thai Yixiao Song Ankita Gupta Mohit Iyyer 65 38 0 25 Oct 2022
No Language Left Behind: Scaling Human-Centered Machine Translation Nllb team Marta R. Costa-jussá James Cross Onur cCelebi Maha Elbayad ... Alexandre Mourachko C. Ropers Safiyyah Saleem Holger Schwenk Jeff Wang MoE 226 1,263 0 11 Jul 2022
Perturbation CheckLists for Evaluating NLG Evaluation Metrics Ananya B. Sai Tanay Dixit D. Y. Sheth S. Mohan Mitesh M. Khapra AAML 141 58 0 13 Sep 2021
AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages Machel Reid Junjie Hu Graham Neubig Y. Matsuo 142 33 0 10 Sep 2021
IndicBART: A Pre-trained Model for Indic Natural Language Generation Raj Dabre Himani Shrotriya Anoop Kunchukuttan Ratish Puduppully Mitesh M. Khapra Pratyush Kumar 105 74 0 07 Sep 2021
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation Tom Kocmi C. Federmann Roman Grundkiewicz Marcin Junczys-Dowmunt Hitokazu Matsushita Arul Menezes 84 210 0 22 Jul 2021
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text Elizabeth Clark Tal August Sofia Serrano Nikita Haduong Suchin Gururangan Noah A. Smith DeLMO 109 412 0 30 Jun 2021
The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation Naman Goyal Cynthia Gao Vishrav Chaudhary Peng-Jen Chen Guillaume Wenzek Da Ju Sanjan Krishnan MarcÁurelio Ranzato Francisco Guzman Angela Fan 99 587 0 06 Jun 2021
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation Markus Freitag George F. Foster David Grangier Viresh Ratnakar Qijun Tan Wolfgang Macherey 158 391 0 29 Apr 2021
Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages Gowtham Ramesh Sumanth Doddapaneni Aravinth Bheemaraj Mayank Jobanputra AK Raghavan ... K. Deepak Vivek Raghavan Anoop Kunchukuttan Pratyush Kumar Mitesh Khapra LRM 85 235 0 12 Apr 2021
MuRIL: Multilingual Representations for Indian Languages Simran Khanuja Diksha Bansal Sarvesh Mehtani Savya Khosla Atreyee Dey ... Shachi Dave Shruti Gupta Subhash Chandra Bose Gali Vishnu Subramanian Partha P. Talukdar 80 289 0 19 Mar 2021
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 152 143 0 24 Oct 2020
mT5: A massively multilingual pre-trained text-to-text transformer Linting Xue Noah Constant Adam Roberts Mihir Kale Rami Al-Rfou Aditya Siddhant Aditya Barua Colin Raffel 138 2,555 0 22 Oct 2020
COMET: A Neural Framework for MT Evaluation Ricardo Rei Craig Alan Stewart Ana C. Farinha A. Lavie 114 1,094 0 18 Sep 2020
A Survey of Evaluation Metrics Used for NLG Systems Ananya B. Sai Akash Kumar Mohankumar Mitesh M. Khapra ELM 87 236 0 27 Aug 2020
SummEval: Re-evaluating Summarization Evaluation Alexander R. Fabbri Wojciech Kry'sciñski Bryan McCann Caiming Xiong R. Socher Dragomir R. Radev HILM 97 713 0 24 Jul 2020
Language-agnostic BERT Sentence Embedding Fangxiaoyu Feng Yinfei Yang Daniel Cer N. Arivazhagan Wei Wang 165 908 0 03 Jul 2020
Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics Nitika Mathur Tim Baldwin Trevor Cohn 52 247 0 11 Jun 2020
Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing Brian Thompson Matt Post LRM 53 190 0 30 Apr 2020
BLEURT: Learning Robust Metrics for Text Generation Thibault Sellam Dipanjan Das Ankur P. Parikh 101 1,498 0 09 Apr 2020
Utilizing Language Relatedness to improve Machine Translation: A Case Study on Languages of the Indian Subcontinent Anoop Kunchukuttan P. Bhattacharyya 50 22 0 19 Mar 2020
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad M. Lewis Luke Zettlemoyer AI4CE AIMat 124 1,809 0 22 Jan 2020
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 223 6,565 0 05 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 249 10,829 0 29 Oct 2019
Quantifying the Carbon Emissions of Machine Learning Alexandre Lacoste A. Luccioni Victor Schmidt Thomas Dandres 99 707 0 21 Oct 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 329 5,845 0 21 Apr 2019
Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond Mikel Artetxe Holger Schwenk 3DV 154 1,016 0 26 Dec 2018
Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages V. Rudra Murthy Anoop Kunchukuttan P. Bhattacharyya 93 42 0 01 Nov 2018
Towards a Better Metric for Evaluating Question Generation Systems Preksha Nema Mitesh M. Khapra 80 108 0 30 Aug 2018
A Call for Clarity in Reporting BLEU Scores Matt Post 162 2,994 0 23 Apr 2018
Quantitative Fine-Grained Human Evaluation of Machine Translation Systems: a Case Study on English to Croatian Filip Klubicka Antonio Toral Víctor M. Sánchez-Cartagena 54 55 0 02 Feb 2018