Does Learning Require Memorization? A Short Tale about a Long Tail

12 June 2019

Papers citing "Does Learning Require Memorization? A Short Tale about a Long Tail"

50 / 336 papers shown

Title
How Does a Deep Learning Model Architecture Impact Its Privacy? A Comprehensive Study of Privacy Attacks on CNNs and Transformers Guangsheng Zhang B. Liu Huan Tian Tianqing Zhu Ming Ding Wanlei Zhou PILM MIACV 20 5 0 20 Oct 2022
Verifiable and Provably Secure Machine Unlearning Thorsten Eisenhofer Doreen Riepel Varun Chandrasekaran Esha Ghosh O. Ohrimenko Nicolas Papernot AAML MU 41 26 0 17 Oct 2022
A Closer Look at the Calibration of Differentially Private Learners Hanlin Zhang Xuechen Li Prithviraj Sen Salim Roukos Tatsunori Hashimoto 16 3 0 15 Oct 2022
Mitigating Unintended Memorization in Language Models via Alternating Teaching Zhe Liu Xuedong Zhang Fuchun Peng 38 3 0 13 Oct 2022
Transformers generalize differently from information stored in context vs in weights Stephanie C. Y. Chan Ishita Dasgupta Junkyung Kim D. Kumaran Andrew Kyle Lampinen Felix Hill 111 46 0 11 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 28 48 0 07 Oct 2022
Understanding Influence Functions and Datamodels via Harmonic Analysis Nikunj Saunshi Arushi Gupta M. Braverman Sanjeev Arora TDI 63 17 0 03 Oct 2022
On the Impossible Safety of Large AI Models El-Mahdi El-Mhamdi Sadegh Farhadkhani R. Guerraoui Nirupam Gupta L. Hoang Rafael Pinot Sébastien Rouault John Stephan 30 31 0 30 Sep 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 44 27 0 20 Sep 2022
Data Provenance via Differential Auditing Xin Mu Ming Pang Feida Zhu 16 1 0 04 Sep 2022
Group Property Inference Attacks Against Graph Neural Networks Xiuling Wang Wendy Hui Wang AAML 29 30 0 02 Sep 2022
Data Isotopes for Data Provenance in DNNs Emily Wenger Xiuyu Li Ben Y. Zhao Vitaly Shmatikov 20 12 0 29 Aug 2022
SNAP: Efficient Extraction of Private Properties with Poisoning Harsh Chaudhari John Abascal Alina Oprea Matthew Jagielski Florian Tramèr Jonathan R. Ullman MIACV 39 30 0 25 Aug 2022
On the Privacy Effect of Data Enhancement via the Lens of Memorization Xiao-Li Li Qiongxiu Li Zhan Hu Xiaolin Hu 35 13 0 17 Aug 2022
A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors Nikhil Ghosh M. Belkin 21 7 0 23 Jul 2022
Efficient Augmentation for Imbalanced Deep Learning Damien Dablain C. Bellinger Bartosz Krawczyk Nitesh V. Chawla 32 7 0 13 Jul 2022
Generalization-Memorization Machines Zhen Wang Y. Shao 27 6 0 08 Jul 2022
A law of adversarial risk, interpolation, and label noise Daniel Paleka Amartya Sanyal NoLa AAML 20 9 0 08 Jul 2022
Memory Population in Continual Learning via Outlier Elimination J. Hurtado Alain Raymond-Sáez Vladimir Araujo Vincenzo Lomonaco Alvaro Soto D. Bacciu 33 9 0 04 Jul 2022
A Theoretical Analysis of the Learning Dynamics under Class Imbalance Emanuele Francazi Marco Baity-Jesi Aurelien Lucchi 22 16 0 01 Jul 2022
Understanding Instance-Level Impact of Fairness Constraints Jialu Wang Qing Guo Yang Liu TDI FaML 25 33 0 30 Jun 2022
Distilling Model Failures as Directions in Latent Space Saachi Jain Hannah Lawrence Ankur Moitra A. Madry 23 90 0 29 Jun 2022
The Privacy Onion Effect: Memorization is Relative Nicholas Carlini Matthew Jagielski Chiyuan Zhang Nicolas Papernot Andreas Terzis Florian Tramèr PILM MIACV 35 102 0 21 Jun 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 40 132 0 15 Jun 2022
To Aggregate or Not? Learning with Separate Noisy Labels Jiaheng Wei Zhaowei Zhu Tianyi Luo Ehsan Amid Abhishek Kumar Yang Liu NoLa 41 37 0 14 Jun 2022
Privacy Leakage in Text Classification: A Data Extraction Approach Adel M. Elmahdy Huseyin A. Inan Robert Sim 27 13 0 09 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 25 74 0 08 Jun 2022
How unfair is private learning ? Amartya Sanyal Yaxian Hu Fanny Yang FaML FedML 33 22 0 08 Jun 2022
Progressive Purification for Instance-Dependent Partial Label Learning Ning Xu Biao Liu Jiaqi Lv Congyu Qiao Xin Geng 42 14 0 02 Jun 2022
Decoupling Knowledge from Memorization: Retrieval-augmented Prompt Learning Xiang Chen Lei Li Ningyu Zhang Xiaozhuan Liang Shumin Deng Chuanqi Tan Fei Huang Luo Si Huajun Chen VLM 25 52 0 29 May 2022
Long-Tailed Learning Requires Feature Learning T. Laurent J. V. Brecht Xavier Bresson VLM 11 1 0 29 May 2022
Contrastive Learning with Boosted Memorization Zhihan Zhou Jiangchao Yao Yanfeng Wang Bo Han Ya Zhang SSL 28 29 0 25 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 29 187 0 22 May 2022
Unintended memorisation of unique features in neural networks J. Hartley Sotirios A. Tsaftaris 30 1 0 20 May 2022
Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher Vikram S Chundawat Ayush K Tarun Murari Mandal Mohan S. Kankanhalli MU 24 124 0 17 May 2022
Provably Confidential Language Modelling Xuandong Zhao Lei Li Yu-Xiang Wang MU 39 15 0 04 May 2022
Memory Bounds for Continual Learning Xi Chen Christos H. Papadimitriou Binghui Peng CLL LRM 29 22 0 22 Apr 2022
Detecting Unintended Memorization in Language-Model-Fused ASR Yifan Jiang Steve Chien Om Thakkar Rajiv Mathews 41 11 0 20 Apr 2022
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets Florian Tramèr Reza Shokri Ayrton San Joaquin Hoang Minh Le Matthew Jagielski Sanghyun Hong Nicholas Carlini MIACV 38 107 0 31 Mar 2022
Towards Differential Relational Privacy and its use in Question Answering Simone Bombari Alessandro Achille Zijian Wang Yu-Xiang Wang Yusheng Xie Kunwar Yashraj Singh Srikar Appalaraju Vijay Mahadevan Stefano Soatto 21 1 0 30 Mar 2022
Understanding out-of-distribution accuracies through quantifying difficulty of test samples Berfin Simsek Melissa Hall Levent Sagun 31 5 0 28 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 17 24 1 23 Mar 2022
Mixed Differential Privacy in Computer Vision Aditya Golatkar Alessandro Achille Yu-Xiang Wang Aaron Roth Michael Kearns Stefano Soatto PICV VLM 20 49 0 22 Mar 2022
Towards understanding deep learning with the natural clustering prior Simon Carbonnelle 21 0 0 15 Mar 2022
Taming the Long Tail of Deep Probabilistic Forecasting Jedrzej Kozerawski Mayank Sharan Rose Yu AI4TS 21 7 0 27 Feb 2022
Does Label Differential Privacy Prevent Label Inference Attacks? Ruihan Wu Jinfu Zhou Kilian Q. Weinberger Chuan Guo 26 15 0 25 Feb 2022
Memorize to Generalize: on the Necessity of Interpolation in High Dimensional Linear Regression Chen Cheng John C. Duchi Rohith Kuditipudi 13 10 0 20 Feb 2022
Interpolation and Regularization for Causal Learning L. C. Vankadara Luca Rendsburg U. V. Luxburg D. Ghoshdastidar CML 31 1 0 18 Feb 2022
Measuring Unintended Memorisation of Unique Private Features in Neural Networks J. Hartley Sotirios A. Tsaftaris 19 7 0 16 Feb 2022
Impact of Pretraining Term Frequencies on Few-Shot Reasoning Yasaman Razeghi Robert L Logan IV Matt Gardner Sameer Singh ReLM LRM 32 150 0 15 Feb 2022