What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

9 August 2020

Papers citing "What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation"

43 / 193 papers shown

Title
Neural Representations Reveal Distinct Modes of Class Fitting in Residual Convolutional Networks Michal Jamro.z Marcin Kurdziel 27 0 0 01 Dec 2022
On Pitfalls of Measuring Occlusion Robustness through Data Distortion Antonia Marcu 28 0 0 24 Nov 2022
ModelDiff: A Framework for Comparing Learning Algorithms Harshay Shah Sung Min Park Andrew Ilyas A. Madry SyDa 54 26 0 22 Nov 2022
What Images are More Memorable to Machines? Junlin Han Huangying Zhan Jie Hong Pengfei Fang Hongdong Li L. Petersson Ian Reid 38 3 0 14 Nov 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 49 8 0 25 Oct 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks Vikas Raunak Arul Menezes 38 13 0 24 Oct 2022
Bird-Eye Transformers for Text Generation Models Lei Sha Yuhang Song Yordan Yordanov Tommaso Salvatori Thomas Lukasiewicz 30 0 0 08 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 28 48 0 07 Oct 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 44 27 0 20 Sep 2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 31 5 0 19 Sep 2022
Data Isotopes for Data Provenance in DNNs Emily Wenger Xiuyu Li Ben Y. Zhao Vitaly Shmatikov 20 12 0 29 Aug 2022
The Privacy Onion Effect: Memorization is Relative Nicholas Carlini Matthew Jagielski Chiyuan Zhang Nicolas Papernot Andreas Terzis Florian Tramèr PILM MIACV 35 102 0 21 Jun 2022
Selective Classification Via Neural Network Training Dynamics Stephan Rabanser Anvith Thudi Kimia Hamidieh Adam Dziedzic Nicolas Papernot 29 21 0 26 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 29 187 0 22 May 2022
Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees Jonathan Brophy Zayd Hammoudeh Daniel Lowd TDI 27 22 0 30 Apr 2022
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability Ruifei He Shuyang Sun Jihan Yang Song Bai Xiaojuan Qi 34 36 0 10 Mar 2022
Deduplicating Training Data Mitigates Privacy Risks in Language Models Nikhil Kandpal Eric Wallace Colin Raffel PILM MU 54 275 0 14 Feb 2022
Understanding Rare Spurious Correlations in Neural Networks Yao-Yuan Yang Chi-Ning Chou Kamalika Chaudhuri AAML 18 25 0 10 Feb 2022
Datamodels: Predicting Predictions from Training Data Andrew Ilyas Sung Min Park Logan Engstrom Guillaume Leclerc A. Madry TDI 47 131 0 01 Feb 2022
Fortuitous Forgetting in Connectionist Networks Hattie Zhou Ankit Vani Hugo Larochelle Aaron Courville CLL 16 42 0 01 Feb 2022
Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation Zayd Hammoudeh Daniel Lowd TDI 21 28 0 25 Jan 2022
Towards Adversarial Evaluations for Inexact Machine Unlearning Shashwat Goel Ameya Prabhu Amartya Sanyal Ser-Nam Lim Philip Torr Ponnurangam Kumaraguru AAML ELM MU 43 47 0 17 Jan 2022
Reconstructing Training Data with Informed Adversaries Borja Balle Giovanni Cherubin Jamie Hayes MIACV AAML 43 158 0 13 Jan 2022
Membership Inference Attacks From First Principles Nicholas Carlini Steve Chien Milad Nasr Shuang Song Andreas Terzis Florian Tramèr MIACV MIALM 29 642 0 07 Dec 2021
Scaling Up Influence Functions Andrea Schioppa Polina Zablotskaia David Vilar Artem Sokolov TDI 33 90 0 06 Dec 2021
SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learning Vasisht Duddu S. Szyller Nadarajah Asokan 32 12 0 04 Dec 2021
ModelPred: A Framework for Predicting Trained Model from Training Data Yingyan Zeng Jiachen T. Wang Si-An Chen H. Just Ran Jin R. Jia TDI MU 33 2 0 24 Nov 2021
SoK: Machine Learning Governance Varun Chandrasekaran Hengrui Jia Anvith Thudi Adelin Travers Mohammad Yaghini Nicolas Papernot 38 16 0 20 Sep 2021
Associative Memories via Predictive Coding Tommaso Salvatori Yuhang Song Yujian Hong Simon Frieder Lei Sha Zhenghua Xu Rafal Bogacz Thomas Lukasiewicz 29 61 0 16 Sep 2021
On Language Models for Creoles Heather Lent Emanuele Bugliarello Miryam de Lhoneux Chen Qiu Anders Søgaard 39 20 0 13 Sep 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 32 30 0 27 Jul 2021
Memorization in Deep Neural Networks: Does the Loss Function matter? Deep Patel P. Sastry TDI 27 8 0 21 Jul 2021
Rethinking Graph Auto-Encoder Models for Attributed Graph Clustering Nairouz Mrabah Mohamed Bouguessa M. Touati Riadh Ksantini 35 62 0 19 Jul 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 593 0 14 Jul 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 47 156 0 17 Jun 2021
On Memorization in Probabilistic Deep Generative Models G. V. D. Burg Christopher K. I. Williams TDI 25 59 0 06 Jun 2021
The Curious Case of Hallucinations in Neural Machine Translation Vikas Raunak Arul Menezes Marcin Junczys-Dowmunt 44 190 0 14 Apr 2021
Understanding the Interaction of Adversarial Training with Noisy Labels Jianing Zhu Jingfeng Zhang Bo Han Tongliang Liu Gang Niu Hongxia Yang Mohan Kankanhalli Masashi Sugiyama AAML 27 27 0 06 Feb 2021
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging Han Guo Nazneen Rajani Peter Hase Joey Tianyi Zhou Caiming Xiong TDI 41 102 0 31 Dec 2020
Efficient Estimation of Influence of a Training Instance Sosuke Kobayashi Sho Yokoi Jun Suzuki Kentaro Inui TDI 32 15 0 08 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 32 192 0 01 Dec 2020
How benign is benign overfitting? Amartya Sanyal P. Dokania Varun Kanade Philip Torr NoLa AAML 23 57 0 08 Jul 2020
Learn to Forget: Machine Unlearning via Neuron Masking Yang Liu Zhuo Ma Ximeng Liu Jian-wei Liu Zhongyuan Jiang Jianfeng Ma Philip Yu K. Ren MU 22 61 0 24 Mar 2020