Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement

29 May 2025

Papers citing "Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement"

42 / 42 papers shown

Title
QE4PE: Word-level Quality Estimation for Human Post-Editing Gabriele Sarti Vilém Zouhar Grzegorz Chrupała Ana Guerberof Arenas Malvina Nissim Arianna Bisazza 55 1 0 04 Mar 2025
Open Problems in Machine Unlearning for AI Safety Fazl Barez Tingchen Fu Ameya Prabhu Stephen Casper Amartya Sanyal ... David M. Krueger Sören Mindermann José Hernandez-Orallo Mor Geva Y. Gal MU 66 19 0 10 Jan 2025
ContextCite: Attributing Model Generation to Context Benjamin Cohen-Wang Harshay Shah Kristian Georgiev Aleksander Madry LRM 56 23 0 01 Sep 2024
Confidence Regulation Neurons in Language Models Alessandro Stolfo Ben Wu Wes Gurnee Yonatan Belinkov Xingyi Song Mrinmaya Sachan Neel Nanda 54 15 0 24 Jun 2024
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP Marius Mosbach Vagrant Gautam Tomás Vergara-Browne Dietrich Klakow Mor Geva AI4CE 62 9 0 18 Jun 2024
AI-Assisted Human Evaluation of Machine Translation Vilém Zouhar Tom Kocmi Mrinmaya Sachan 62 7 0 18 Jun 2024
Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translation Tom Kocmi Vilém Zouhar Eleftherios Avramidis Roman Grundkiewicz Marzena Karpinska Maja Popović Mrinmaya Sachan Mariya Shmatova 52 16 0 17 Jun 2024
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification Ekaterina Fadeeva Aleksandr Rubashevskii Artem Shelmanov Sergey Petrakov Haonan Li ... Gleb Kuzmin Alexander Panchenko Timothy Baldwin Preslav Nakov Maxim Panov HILM 61 50 0 07 Mar 2024
VariErr NLI: Separating Annotation Error from Human Label Variation Leon Weber-Genzel Siyao Peng M. Marneffe Barbara Plank 58 11 0 04 Mar 2024
Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains Vilém Zouhar Shuoyang Ding Anna Currey Tatyana Badeka Jenyuan Wang Brian Thompson 56 16 0 28 Feb 2024
Enhanced Hallucination Detection in Neural Machine Translation through Simple Detector Aggregation Anas Himmi Guillaume Staerman Marine Picot Pierre Colombo Nuno M. Guerreiro 232 5 0 20 Feb 2024
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model Ahmet Üstün Viraat Aryabumi Zheng-Xin Yong Wei-Yin Ko Daniel D'souza ... Shayne Longpre Niklas Muennighoff Marzieh Fadaee Julia Kreutzer Sara Hooker ALM ELM SyDa LRM 58 211 0 12 Feb 2024
LM-Polygraph: Uncertainty Estimation for Language Models Ekaterina Fadeeva Roman Vashurin Akim Tsvigun Artem Vazhentsev Sergey Petrakov ... Elizaveta Goncharova Alexander Panchenko Maxim Panov Timothy Baldwin Artem Shelmanov 39 66 0 13 Nov 2023
xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection Nuno M. Guerreiro Ricardo Rei Daan van Stigt Luísa Coheur Pierre Colombo André F.T. Martins 93 126 0 16 Oct 2023
Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness Fran Jelenić Josip Jukić Martin Tutek Mate Puljiz Jan vSnajder OODD 62 6 0 04 Oct 2023
Towards Explainable Evaluation Metrics for Machine Translation Christoph Leiter Piyawat Lertvittayakumjorn M. Fomicheva Wei Zhao Yang Gao Steffen Eger ELM 71 15 0 22 Jun 2023
Explaining How Transformers Use Context to Build Predictions Javier Ferrando Gerard I. Gállego Ioannis Tsiamas Marta R. Costa-jussá 41 33 0 21 May 2023
The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics Ricardo Rei Nuno M. Guerreiro Marcos Vinícius Treviso Luísa Coheur A. Lavie André F.T. Martins 56 16 0 19 May 2023
HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation David Dale Elena Voita Janice Lam Prangthip Hansanti C. Ropers Elahe Kalbassi Cynthia Gao Loïc Barrault Marta R. Costa-jussá HILM 100 29 0 19 May 2023
What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability Mario Giulianelli Joris Baan Wilker Aziz Raquel Fernández Barbara Plank UQLM 50 31 0 19 May 2023
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations Alexander Yom Din Taelin Karidi Leshem Choshen Mor Geva 27 61 0 16 Mar 2023
Eliciting Latent Predictions from Transformers with the Tuned Lens Nora Belrose Zach Furman Logan Smith Danny Halawi Igor V. Ostrovsky Lev McKinney Stella Biderman Jacob Steinhardt 38 213 0 14 Mar 2023
Large Language Models Are State-of-the-Art Evaluators of Translation Quality Tom Kocmi C. Federmann ELM 80 352 0 28 Feb 2023
Inseq: An Interpretability Toolkit for Sequence Generation Models Gabriele Sarti Nils Feldhus Ludwig Sickert Oskar van der Wal Malvina Nissim Arianna Bisazza 42 66 0 27 Feb 2023
Quantifying Context Mixing in Transformers Hosein Mohebbi Willem H. Zuidema Grzegorz Chrupała Afra Alishahi 186 26 0 30 Jan 2023
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better David Dale Elena Voita Loïc Barrault Marta R. Costa-jussá HILM 134 70 0 16 Dec 2022
A Close Look into the Calibration of Pre-trained Language Models Yangyi Chen Lifan Yuan Ganqu Cui Zhiyuan Liu Heng Ji 90 50 0 31 Oct 2022
Calibrating Sequence likelihood Improves Conditional Language Generation Yao-Min Zhao Misha Khalman Rishabh Joshi Shashi Narayan Mohammad Saleh Peter J. Liu UQLM 48 126 0 30 Sep 2022
DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages Gabriele Sarti Arianna Bisazza Ana Guerberof Arenas Antonio Toral 77 8 0 24 May 2022
Measuring the Mixing of Contextual Information in the Transformer Javier Ferrando Gerard I. Gállego Marta R. Costa-jussá 50 51 0 08 Mar 2022
The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results M. Fomicheva Piyawat Lertvittayakumjorn Wei Zhao Steffen Eger Yang Gao ELM 46 40 0 08 Oct 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 67 159 0 17 Jun 2021
The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation Naman Goyal Cynthia Gao Vishrav Chaudhary Peng-Jen Chen Guillaume Wenzek Da Ju Sanjan Krishnan MarcÁurelio Ranzato Francisco Guzman Angela Fan 78 575 0 06 Jun 2021
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation Markus Freitag George F. Foster David Grangier Viresh Ratnakar Qijun Tan Wolfgang Macherey 132 382 0 29 Apr 2021
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset M. Fomicheva Shuo Sun E. Fonseca Chrysoula Zerva Frédéric Blain Vishrav Chaudhary Francisco Guzmán Nina Lopatina Lucia Specia André F. T. Martins 68 68 0 09 Oct 2020
COMET: A Neural Framework for MT Evaluation Ricardo Rei Craig Alan Stewart Ana C. Farinha A. Lavie 100 1,075 0 18 Sep 2020
Unsupervised Quality Estimation for Neural Machine Translation M. Fomicheva Shuo Sun Lisa Yankovskaya Frédéric Blain Francisco Guzmán Mark Fishel Nikolaos Aletras Vishrav Chaudhary Lucia Specia UQLM 75 198 0 21 May 2020
Multilingual Denoising Pre-training for Neural Machine Translation Yinhan Liu Jiatao Gu Naman Goyal Xian Li Sergey Edunov Marjan Ghazvininejad M. Lewis Luke Zettlemoyer AI4CE AIMat 109 1,786 0 22 Jan 2020
How Can We Know What Language Models Know? Zhengbao Jiang Frank F. Xu Jun Araki Graham Neubig KELM 101 1,396 0 28 Nov 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 159 10,720 0 29 Oct 2019
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks Dan Hendrycks Kevin Gimpel UQCV 111 3,420 0 07 Oct 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 526 9,233 0 06 Jun 2015