What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

9 August 2020

Papers citing "What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation"

50 / 369 papers shown

Title
A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks Yefan Zhou Yiru Shen Yujun Yan Chen Feng Yaoqing Yang 23 0 0 30 Nov 2021
ModelPred: A Framework for Predicting Trained Model from Training Data Yingyan Zeng Jiachen T. Wang Si-An Chen H. Just Ran Jin R. Jia TDI MU 63 3 0 24 Nov 2021
On the Effects of Artificial Data Modification Antonia Marcu Adam Prugel-Bennett 24 3 0 26 Oct 2021
Distinguishing rule- and exemplar-based generalization in learning systems Ishita Dasgupta Erin Grant Thomas Griffiths 67 16 0 08 Oct 2021
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation Orevaoghene Ahia Julia Kreutzer Sara Hooker 161 55 0 06 Oct 2021
SoK: Machine Learning Governance Varun Chandrasekaran Hengrui Jia Anvith Thudi Adelin Travers Mohammad Yaghini Nicolas Papernot 90 16 0 20 Sep 2021
Associative Memories via Predictive Coding Tommaso Salvatori Yuhang Song Yujian Hong Simon Frieder Lei Sha Zhenghua Xu Rafal Bogacz Thomas Lukasiewicz 58 64 0 16 Sep 2021
On Language Models for Creoles Heather Lent Emanuele Bugliarello Miryam de Lhoneux Chen Qiu Anders Søgaard 93 20 0 13 Sep 2021
Excess Capacity and Backdoor Poisoning N. Manoj Avrim Blum SILM AAML 55 24 0 02 Sep 2021
Multi-Label Gold Asymmetric Loss Correction with Single-Label Regulators Cosmin Octavian Pene Amirmasoud Ghiassi Taraneh Younesian Robert Birke Lydia Y. Chen 69 3 0 04 Aug 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 72 31 0 27 Jul 2021
Memorization in Deep Neural Networks: Does the Loss Function matter? Deep Patel P. Sastry TDI 40 8 0 21 Jul 2021
Rethinking Graph Auto-Encoder Models for Attributed Graph Clustering Nairouz Mrabah Mohamed Bouguessa M. Touati Riadh Ksantini 79 63 0 19 Jul 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 111 457 0 15 Jul 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 360 631 0 14 Jul 2021
RoFL: Robustness of Secure Federated Learning Hidde Lycklama Lukas Burkhalter Alexander Viand Nicolas Küchler Anwar Hithnawi FedML 61 56 0 07 Jul 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 76 160 0 17 Jun 2021
On the Robustness of Average Losses for Partial-Label Learning Jiaqi Lv Biao Liu Lei Feng Ning Xu Miao Xu Bo An Gang Niu Xin Geng Masashi Sugiyama 56 35 0 11 Jun 2021
Towards the Memorization Effect of Neural Networks in Adversarial Training Han Xu Xiaorui Liu Wentao Wang Wenbiao Ding Zhongqin Wu Zitao Liu Anil K. Jain Jiliang Tang TDI AAML 104 7 0 09 Jun 2021
White Paper Assistance: A Step Forward Beyond the Shortcut Learning Xuan Cheng Tianshu Xie Xiaomin Wang Jiali Deng Minghui Liu Meilin Liu AAML 34 0 0 08 Jun 2021
On Memorization in Probabilistic Deep Generative Models G. V. D. Burg Christopher K. I. Williams TDI 62 62 0 06 Jun 2021
Exploring Memorization in Adversarial Training Yinpeng Dong Ke Xu Xiao Yang Tianyu Pang Zhijie Deng Hang Su Jun Zhu TDI 51 73 0 03 Jun 2021
Understanding Instance-based Interpretability of Variational Auto-Encoders Zhifeng Kong Kamalika Chaudhuri TDI 53 28 0 29 May 2021
The Curious Case of Hallucinations in Neural Machine Translation Vikas Raunak Arul Menezes Marcin Junczys-Dowmunt 196 194 0 14 Apr 2021
Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond Xuhong Li Haoyi Xiong Xingjian Li Xuanyu Wu Xiao Zhang Ji Liu Jiang Bian Dejing Dou AAML FaML XAI HAI 64 333 0 19 Mar 2021
Understanding Instance-Level Label Noise: Disparate Impacts and Treatments Yang Liu NoLa 31 35 0 10 Feb 2021
Understanding the Interaction of Adversarial Training with Noisy Labels Jianing Zhu Jingfeng Zhang Bo Han Tongliang Liu Gang Niu Hongxia Yang Mohan Kankanhalli Masashi Sugiyama AAML 72 27 0 06 Feb 2021
Investigating Memorization of Conspiracy Theories in Text Generation Sharon Levy Michael Stephen Saxon Wenjie Wang 64 19 0 02 Jan 2021
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging Han Guo Nazneen Rajani Peter Hase Joey Tianyi Zhou Caiming Xiong TDI 96 114 0 31 Dec 2020
How Does a Neural Network's Architecture Impact Its Robustness to Noisy Labels? Jingling Li Mozhi Zhang Keyulu Xu John P. Dickerson Jimmy Ba OOD NoLa 57 19 0 23 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 489 1,923 0 14 Dec 2020
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning? Gavin Brown Mark Bun Vitaly Feldman Adam D. Smith Kunal Talwar 295 99 0 11 Dec 2020
Efficient Estimation of Influence of a Training Instance Sosuke Kobayashi Sho Yokoi Jun Suzuki Kentaro Inui TDI 64 15 0 08 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 111 203 0 01 Dec 2020
Toward Better Generalization Bounds with Locally Elastic Stability Zhun Deng Hangfeng He Weijie J. Su 45 44 0 27 Oct 2020
Learning Task-aware Robust Deep Learning Systems Keji Han Yun Li Xianzhong Long Yao Ge OOD 29 0 0 11 Oct 2020
How benign is benign overfitting? Amartya Sanyal P. Dokania Varun Kanade Philip Torr NoLa AAML 61 58 0 08 Jul 2020
Trade-offs between membership privacy & adversarially robust learning Jamie Hayes SILM 50 3 0 08 Jun 2020
Learn to Forget: Machine Unlearning via Neuron Masking Yang Liu Zhuo Ma Ximeng Liu Jian Liu Zhongyuan Jiang Jianfeng Ma Philip Yu K. Ren MU 61 65 0 24 Mar 2020
Estimating Training Data Influence by Tracing Gradient Descent G. Pruthi Frederick Liu Mukund Sundararajan Satyen Kale TDI 85 410 0 19 Feb 2020
Characterizing Structural Regularities of Labeled Data in Overparameterized Models Ziheng Jiang Chiyuan Zhang Kunal Talwar Michael C. Mozer TDI 59 102 0 08 Feb 2020
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 134 607 0 04 Dec 2019
What Do Compressed Deep Neural Networks Forget? Sara Hooker Aaron Courville Gregory Clark Yann N. Dauphin Andrea Frome 87 185 0 13 Nov 2019
Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications Nicholas Carlini Ulfar Erlingsson Nicolas Papernot OOD OODD 57 61 0 29 Oct 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 80 776 0 26 Jun 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 123 494 0 12 Jun 2019
Differential Privacy Has Disparate Impact on Model Accuracy Eugene Bagdasaryan Vitaly Shmatikov 149 481 0 28 May 2019
Harmless interpolation of noisy data in regression Vidya Muthukumar Kailas Vodrahalli Vignesh Subramanian A. Sahai 74 202 0 21 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 188 743 0 19 Mar 2019
Two models of double descent for weak features M. Belkin Daniel J. Hsu Ji Xu 96 374 0 18 Mar 2019