What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

9 August 2020

Papers citing "What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation"

50 / 369 papers shown

Title
Bird-Eye Transformers for Text Generation Models Lei Sha Yuhang Song Yordan Yordanov Tommaso Salvatori Thomas Lukasiewicz 44 0 0 08 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 77 53 0 07 Oct 2022
Understanding Influence Functions and Datamodels via Harmonic Analysis Nikunj Saunshi Arushi Gupta M. Braverman Sanjeev Arora TDI 88 18 0 03 Oct 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 103 27 0 20 Sep 2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 49 6 0 19 Sep 2022
Small Transformers Compute Universal Metric Embeddings Anastasis Kratsios Valentin Debarnot Ivan Dokmanić 95 11 0 14 Sep 2022
Data Isotopes for Data Provenance in DNNs Emily Wenger Xiuyu Li Ben Y. Zhao Vitaly Shmatikov 59 12 0 29 Aug 2022
On the Privacy Effect of Data Enhancement via the Lens of Memorization Xiao-Li Li Qiongxiu Li Zhan Hu Xiaolin Hu 62 15 0 17 Aug 2022
A Data-Based Perspective on Transfer Learning Saachi Jain Hadi Salman Alaa Khaddaj Eric Wong Sung Min Park Aleksander Madry 65 38 0 12 Jul 2022
Towards a More Rigorous Science of Blindspot Discovery in Image Classification Models Gregory Plumb Nari Johnson Ángel Alexander Cabrera Ameet Talwalkar 65 6 0 08 Jul 2022
A law of adversarial risk, interpolation, and label noise Daniel Paleka Amartya Sanyal NoLa AAML 52 10 0 08 Jul 2022
Understanding Instance-Level Impact of Fairness Constraints Jialu Wang Xinze Wang Yang Liu TDI FaML 78 33 0 30 Jun 2022
Beyond neural scaling laws: beating power law scaling via data pruning Ben Sorscher Robert Geirhos Shashank Shekhar Surya Ganguli Ari S. Morcos 93 440 0 29 Jun 2022
The Privacy Onion Effect: Memorization is Relative Nicholas Carlini Matthew Jagielski Chiyuan Zhang Nicolas Papernot Andreas Terzis Florian Tramèr PILM MIACV 114 109 0 21 Jun 2022
Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments Jinkun Lin Anqi Zhang Mathias Lécuyer Jinyang Li Aurojit Panda S. Sen TDI FedML 62 53 0 20 Jun 2022
Memorization-Dilation: Modeling Neural Collapse Under Label Noise Duc Anh Nguyen Ron Levie Julian Lienen Gitta Kutyniok Eyke Hüllermeier 45 2 0 11 Jun 2022
How unfair is private learning ? Amartya Sanyal Yaxian Hu Fanny Yang FaML FedML 84 24 0 08 Jun 2022
Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent Da Yu Gautam Kamath Janardhan Kulkarni Tie-Yan Liu Jian Yin Huishuai Zhang 68 22 0 06 Jun 2022
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning Xiang Chen Lei Li Ningyu Zhang Xiaozhuan Liang Shumin Deng Chuanqi Tan Fei Huang Luo Si Huajun Chen VLM 59 53 0 29 May 2022
Long-Tailed Learning Requires Feature Learning T. Laurent J. V. Brecht Xavier Bresson VLM 41 1 0 29 May 2022
Differentially Private Decoding in Large Language Models Jimit Majmudar Christophe Dupuy Charith Peris S. Smaili Rahul Gupta R. Zemel 35 31 0 26 May 2022
Selective Classification Via Neural Network Training Dynamics Stephan Rabanser Anvith Thudi Kimia Hamidieh Adam Dziedzic Nicolas Papernot 79 22 0 26 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 106 192 0 22 May 2022
Unintended memorisation of unique features in neural networks J. Hartley Sotirios A. Tsaftaris 66 1 0 20 May 2022
Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees Jonathan Brophy Zayd Hammoudeh Daniel Lowd TDI 97 22 0 30 Apr 2022
Improved Group Robustness via Classifier Retraining on Independent Splits Thien Hai Nguyen Hongyang R. Zhang Huy Le Nguyen OOD 40 2 0 20 Apr 2022
Extracting Targeted Training Data from ASR Models, and How to Mitigate It Ehsan Amid Om Thakkar A. Narayanan Rajiv Mathews Franccoise Beaufays 35 9 0 18 Apr 2022
Towards Differential Relational Privacy and its use in Question Answering Simone Bombari Alessandro Achille Zijian Wang Yu Wang Yusheng Xie Kunwar Yashraj Singh Srikar Appalaraju Vijay Mahadevan Stefano Soatto 46 1 0 30 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 75 24 1 23 Mar 2022
Mixed Differential Privacy in Computer Vision Aditya Golatkar Alessandro Achille Yu Wang Aaron Roth Michael Kearns Stefano Soatto PICV VLM 67 49 0 22 Mar 2022
Emulating Quantum Dynamics with Neural Networks via Knowledge Distillation Yu Yao C. Cao S. Haas Mahak Agarwal Divya Khanna M. Abram 127 4 0 19 Mar 2022
Adversarial Counterfactual Augmentation: Application in Alzheimer's Disease Classification Tian Xia Pedro Sanchez C. Qin Sotirios A. Tsaftaris OOD MedIm 66 13 0 15 Mar 2022
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability Ruifei He Shuyang Sun Jihan Yang Song Bai Xiaojuan Qi 81 37 0 10 Mar 2022
Taming the Long Tail of Deep Probabilistic Forecasting Jedrzej Kozerawski Mayank Sharan Rose Yu AI4TS 54 7 0 27 Feb 2022
Measuring Unintended Memorisation of Unique Private Features in Neural Networks J. Hartley Sotirios A. Tsaftaris 64 9 0 16 Feb 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 121 623 0 15 Feb 2022
Impact of Pretraining Term Frequencies on Few-Shot Reasoning Yasaman Razeghi Robert L Logan IV Matt Gardner Sameer Singh ReLM LRM 83 155 0 15 Feb 2022
Threats to Pre-trained Language Models: Survey and Taxonomy Shangwei Guo Chunlong Xie Jiwei Li Lingjuan Lyu Tianwei Zhang PILM 47 32 0 14 Feb 2022
Deduplicating Training Data Mitigates Privacy Risks in Language Models Nikhil Kandpal Eric Wallace Colin Raffel PILM MU 98 293 0 14 Feb 2022
Understanding Rare Spurious Correlations in Neural Networks Yao-Yuan Yang Chi-Ning Chou Kamalika Chaudhuri AAML 65 26 0 10 Feb 2022
Datamodels: Predicting Predictions from Training Data Andrew Ilyas Sung Min Park Logan Engstrom Guillaume Leclerc Aleksander Madry TDI 112 140 0 01 Feb 2022
Fortuitous Forgetting in Connectionist Networks Hattie Zhou Ankit Vani Hugo Larochelle Aaron Courville CLL 62 45 0 01 Feb 2022
Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation Zayd Hammoudeh Daniel Lowd TDI 70 29 0 25 Jan 2022
Towards Adversarial Evaluations for Inexact Machine Unlearning Shashwat Goel Ameya Prabhu Amartya Sanyal Ser-Nam Lim Philip Torr Ponnurangam Kumaraguru AAML ELM MU 98 56 0 17 Jan 2022
Reconstructing Training Data with Informed Adversaries Borja Balle Giovanni Cherubin Jamie Hayes MIACV AAML 86 169 0 13 Jan 2022
Counterfactual Memorization in Neural Language Models Chiyuan Zhang Daphne Ippolito Katherine Lee Matthew Jagielski Florian Tramèr Nicholas Carlini 71 135 0 24 Dec 2021
Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation Futong Liu Tao R. Lin Martin Jaggi TDI 47 8 0 16 Dec 2021
Membership Inference Attacks From First Principles Nicholas Carlini Steve Chien Milad Nasr Shuang Song Andreas Terzis Florian Tramèr MIACV MIALM 80 695 0 07 Dec 2021
Scaling Up Influence Functions Andrea Schioppa Polina Zablotskaia David Vilar Artem Sokolov TDI 95 102 0 06 Dec 2021
SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learning Vasisht Duddu S. Szyller Nadarajah Asokan 50 12 0 04 Dec 2021