Towards More Fine-grained and Reliable NLP Performance Prediction

10 February 2021

Zihuiwen Ye

Pengfei Liu

Jinlan Fu

Graham Neubig

ArXiv (abs)PDF HTML Github (5★)

Papers citing "Towards More Fine-grained and Reliable NLP Performance Prediction"

22 / 22 papers shown

Title
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions Emmy Liu Amanda Bertsch Lintang Sutawika Lindia Tjuatja Patrick Fernandes ... Siyang Song Carolin (Haas) Lawrence Aditi Raghunathan Kiril Gashteovski Graham Neubig 275 3 0 05 Mar 2025
Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback Lester James V. Miranda Yizhong Wang Yanai Elazar Sachin Kumar Valentina Pyatkin Faeze Brahman Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 138 12 0 24 Oct 2024
ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models David Anugraha Genta Indra Winata Chenyue Li Patrick Amadeus Irawan En-Shiun Annie Lee 96 8 0 13 Jun 2024
Third-Party Language Model Performance Prediction from Instruction Rahul Nadkarni Yizhong Wang Noah A. Smith ELM LRM 67 0 0 19 Mar 2024
Linking Surface Facts to Large-Scale Knowledge Graphs Gorjan Radevski Kiril Gashteovski Chia-Chien Hung Carolin (Haas) Lawrence Goran Glavaš HILM 60 3 0 23 Oct 2023
How to Estimate Model Transferability of Pre-Trained Speech Models? Zih-Ching Chen Chao-Han Huck Yang Yue Liu Yu Zhang Nanxin Chen Shoufeng Chang Rohit Prabhavalkar Hung-yi Lee Tara N. Sainath 153 9 0 01 Jun 2023
Characterizing and Measuring Linguistic Dataset Drift Tyler A. Chang Kishaloy Halder Neha Ann John Yogarshi Vyas Yassine Benajiba Miguel Ballesteros Dan Roth 71 2 0 26 May 2023
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench Qinyuan Ye Harvey Yiyun Fu Xiang Ren Robin Jia ELM 115 24 0 24 May 2023
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes Louis Clouâtre Prasanna Parthasarathi Payel Das Sarath Chandar 75 3 0 09 Nov 2022
Evidence > Intuition: Transferability Estimation for Encoder Selection Elisa Bassignana Max Müller-Eberstein Mike Zhang Barbara Plank 65 8 0 20 Oct 2022
Predicting Fine-Tuning Performance with Probing Zining Zhu Soroosh Shahtalebi Frank Rudzicz 64 10 0 13 Oct 2022
Assessing Out-of-Domain Language Model Performance from Few Examples Prasann Singhal Jarad Forristal Xi Ye Greg Durrett LRM 56 5 0 13 Oct 2022
Sort by Structure: Language Model Ranking as Dependency Probing Max Müller-Eberstein Rob van der Goot Barbara Plank 73 3 0 10 Jun 2022
Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages Kabir Ahuja Sandipan Dandapat Sunayana Sitaram Monojit Choudhury LRM 99 16 0 12 May 2022
Multi Task Learning For Zero Shot Performance Prediction of Multilingual Models Kabir Ahuja Shanu Kumar Sandipan Dandapat Monojit Choudhury 57 25 0 12 May 2022
Are All the Datasets in Benchmark Necessary? A Pilot Study of Dataset Evaluation for Text Classification Yanghua Xiao Jinlan Fu See-Kiong Ng Pengfei Liu 48 2 0 04 May 2022
DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation Cheonbok Park H. Kim Ioan Calapodescu Hyun Chang Cho Vassilina Nikoulina AI4CE 76 2 0 20 Apr 2022
Disentangling Uncertainty in Machine Translation Evaluation Chrysoula Zerva T. Glushkova Ricardo Rei André F.T. Martins UD UQCV 103 9 0 13 Apr 2022
Predicting the Performance of Multilingual NLP Models A. Srinivasan Sunayana Sitaram T. Ganu Sandipan Dandapat Kalika Bali Monojit Choudhury LRM 65 29 0 17 Oct 2021
Uncertainty-Aware Machine Translation Evaluation T. Glushkova Chrysoula Zerva Ricardo Rei André F.T. Martins UQLM 128 45 0 13 Sep 2021
ExplainaBoard: An Explainable Leaderboard for NLP Pengfei Liu Jinlan Fu Yanghua Xiao Weizhe Yuan Shuaichen Chang Junqi Dai Yixin Liu Zihuiwen Ye Zi-Yi Dou Graham Neubig XAI LRM ELM 87 55 0 13 Apr 2021
Larger-Context Tagging: When and Why Does It Work? Jinlan Fu Liangjing Feng Qi Zhang Xuanjing Huang Pengfei Liu 59 5 0 09 Apr 2021