Influence Scores at Scale for Efficient Language Data Sampling

27 November 2023

Papers citing "Influence Scores at Scale for Efficient Language Data Sampling"

21 / 21 papers shown

Title
BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning Mohsen Fayyaz Ehsan Aghazadeh Ali Modarressi Mohammad Taher Pilehvar Yadollah Yaghoobzadeh Samira Ebrahimi Kahou 46 18 0 10 Nov 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 114 28 0 20 Sep 2022
Beyond neural scaling laws: beating power law scaling via data pruning Ben Sorscher Robert Geirhos Shashank Shekhar Surya Ganguli Ari S. Morcos 100 444 0 29 Jun 2022
Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation Futong Liu Tao R. Lin Martin Jaggi TDI 57 8 0 16 Dec 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 121 462 0 15 Jul 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 87 161 0 17 Jun 2021
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation Vitaly Feldman Chiyuan Zhang TDI 228 470 0 09 Aug 2020
TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation Shaojie Jiang Thomas Wolf Christof Monz Maarten de Rijke 57 12 0 26 Mar 2020
Estimating Training Data Influence by Tracing Gradient Descent G. Pruthi Frederick Liu Mukund Sundararajan Satyen Kale TDI 110 419 0 19 Feb 2020
Adversarial Filters of Dataset Biases Ronan Le Bras Swabha Swayamdipta Chandra Bhagavatula Rowan Zellers Matthew E. Peters Ashish Sabharwal Yejin Choi 124 223 0 10 Feb 2020
Identifying Mislabeled Data using the Area Under the Margin Ranking Geoff Pleiss Tianyi Zhang Ethan R. Elenberg Kilian Q. Weinberger NoLa 94 274 0 28 Jan 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 651 4,925 0 23 Jan 2020
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications Nicholas Carlini Ulfar Erlingsson Nicolas Papernot OOD OODD 68 61 0 29 Oct 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 506 20,376 0 23 Oct 2019
Accelerating Deep Learning by Focusing on the Biggest Losers Angela H. Jiang Daniel L.-K. Wong Giulio Zhou D. Andersen J. Dean ... Gauri Joshi M. Kaminsky M. Kozuch Zachary Chase Lipton Padmanabhan Pillai 82 123 0 02 Oct 2019
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 134 743 0 12 Dec 2018
Learning to Reweight Examples for Robust Deep Learning Mengye Ren Wenyuan Zeng Binh Yang R. Urtasun OOD NoLa 152 1,431 0 24 Mar 2018
A Unified Approach to Interpreting Model Predictions Scott M. Lundberg Su-In Lee FAtt 1.1K 22,090 0 22 May 2017
Understanding Black-box Predictions via Influence Functions Pang Wei Koh Percy Liang TDI 227 2,910 0 14 Mar 2017
Axiomatic Attribution for Deep Networks Mukund Sundararajan Ankur Taly Qiqi Yan OOD FAtt 193 6,027 0 04 Mar 2017
A large annotated corpus for learning natural language inference Samuel R. Bowman Gabor Angeli Christopher Potts Christopher D. Manning 338 4,297 0 21 Aug 2015