Does Learning Require Memorization? A Short Tale about a Long Tail

12 June 2019

Papers citing "Does Learning Require Memorization? A Short Tale about a Long Tail"

50 / 336 papers shown

Title
When Synthetic Data Met Regulation Georgi Ganev 29 2 0 01 Jul 2023
Gradients Look Alike: Sensitivity is Often Overestimated in DP-SGD Anvith Thudi Hengrui Jia Casey Meehan Ilia Shumailov Nicolas Papernot 33 3 0 01 Jul 2023
On the Cause of Unfairness: A Training Sample Perspective Yuanshun Yao Yang Liu TDI 44 0 0 30 Jun 2023
Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models Adel M. Elmahdy A. Salem SILM 25 6 0 23 Jun 2023
Towards Regulatable AI Systems: Technical Gaps and Policy Opportunities Xudong Shen H. Brown Jiashu Tao Martin Strobel Yao Tong Akshay Narayan Harold Soh Finale Doshi-Velez 32 3 0 22 Jun 2023
Memory-Query Tradeoffs for Randomized Convex Optimization Xinyu Chen Binghui Peng 36 6 0 21 Jun 2023
Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishing Matthieu Meeus Florent Guépin Ana-Maria Cretu Yves-Alexandre de Montjoye 59 23 0 17 Jun 2023
Understanding the Effect of the Long Tail on Neural Network Compression Harvey Dam Vinu Joseph Aditya Bhaskara G. Gopalakrishna Saurav Muralidharan M. Garland 41 2 0 09 Jun 2023
On the Joint Interaction of Models, Data, and Features Yiding Jiang Christina Baek J. Zico Kolter FedML 28 4 0 07 Jun 2023
TMI! Finetuned Models Leak Private Information from their Pretraining Data John Abascal Stanley Wu Alina Oprea Jonathan R. Ullman 33 16 0 01 Jun 2023
Adaptive Conformal Regression with Jackknife+ Rescaled Scores N. Deutschmann Mattia Rigotti María Rodríguez Martínez 26 10 0 31 May 2023
DPFormer: Learning Differentially Private Transformer on Long-Tailed Data Youlong Ding Xueyang Wu Hongya Wang Weike Pan 39 0 0 28 May 2023
Feature Collapse T. Laurent J. V. Brecht Xavier Bresson 33 3 0 25 May 2023
How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features Simone Bombari Marco Mondelli AAML 37 4 0 20 May 2023
Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization Zimeng Qiu Quanqi Hu Zhuoning Yuan Denny Zhou Lijun Zhang Tianbao Yang 34 17 0 19 May 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 128 1,152 0 17 May 2023
Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models Boxi Cao Qiaoyu Tang Hongyu Lin Shanshan Jiang Bin Dong Xianpei Han Jiawei Chen Tianshu Wang Le Sun CLL KELM 13 17 0 16 May 2023
Incentivising the federation: gradient-based metrics for data selection and valuation in private decentralised training Dmitrii Usynin Daniel Rueckert Georgios Kaissis FedML 28 2 0 04 May 2023
Do SSL Models Have Déjà Vu? A Case of Unintended Memorization in Self-supervised Learning Casey Meehan Florian Bordes Pascal Vincent Kamalika Chaudhuri Chuan Guo 36 18 0 26 Apr 2023
An Evaluation on Large Language Model Outputs: Discourse and Memorization Adrian de Wynter Xun Wang Alex Sokolov Qilong Gu Si-Qing Chen ELM 87 32 0 17 Apr 2023
AI Model Disgorgement: Methods and Choices Alessandro Achille Michael Kearns Carson Klingenberg Stefano Soatto MU 36 11 0 07 Apr 2023
Can Membership Inferencing be Refuted? Zhifeng Kong A. Chowdhury Kamalika Chaudhuri MIALM MIACV 29 6 0 07 Mar 2023
Near Optimal Memory-Regret Tradeoff for Online Learning Binghui Peng A. Rubinstein CLL 34 10 0 03 Mar 2023
Data-Copying in Generative Models: A Formal Framework Robi Bhattacharjee S. Dasgupta Kamalika Chaudhuri TDI 20 6 0 25 Feb 2023
On Differential Privacy and Adaptive Data Analysis with Bounded Space Itai Dinur Uri Stemmer David P. Woodruff Samson Zhou 24 12 0 11 Feb 2023
Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines Eugene Bagdasaryan Vitaly Shmatikov AAML 24 2 0 09 Feb 2023
Information Theoretic Lower Bounds for Information Theoretic Upper Bounds Roi Livni 21 14 0 09 Feb 2023
Bag of Tricks for Training Data Extraction from Language Models Weichen Yu Tianyu Pang Qian Liu Chao Du Bingyi Kang Yan Huang Min Lin Shuicheng Yan 26 47 0 09 Feb 2023
ResMem: Learn what you can and memorize the rest Zitong Yang Michal Lukasik Vaishnavh Nagarajan Zong-xiao Li A. S. Rawat Manzil Zaheer A. Menon Surinder Kumar VLM 43 8 0 03 Feb 2023
Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation Noel Loo Ramin Hasani Mathias Lechner Alexander Amini Daniela Rus DD 42 5 0 02 Feb 2023
Pathologies of Predictive Diversity in Deep Ensembles Taiga Abe E. Kelly Buchanan Geoff Pleiss John P. Cunningham UQCV 40 13 0 01 Feb 2023
Analyzing Leakage of Personally Identifiable Information in Language Models Nils Lukas A. Salem Robert Sim Shruti Tople Lukas Wutschitz Santiago Zanella Béguelin PILM 24 212 0 01 Feb 2023
Extracting Training Data from Diffusion Models Nicholas Carlini Jamie Hayes Milad Nasr Matthew Jagielski Vikash Sehwag Florian Tramèr Borja Balle Daphne Ippolito Eric Wallace DiffM 63 572 0 30 Jan 2023
Context-Aware Differential Privacy for Language Modeling M. H. Dinh Ferdinando Fioretto 30 2 0 28 Jan 2023
Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining Florian Tramèr Gautam Kamath Nicholas Carlini SILM 49 67 0 13 Dec 2022
Training Data Influence Analysis and Estimation: A Survey Zayd Hammoudeh Daniel Lowd TDI 29 82 0 09 Dec 2022
Leveraging Unlabeled Data to Track Memorization Mahsa Forouzesh Hanie Sedghi Patrick Thiran NoLa TDI 34 4 0 08 Dec 2022
Skellam Mixture Mechanism: a Novel Approach to Federated Learning with Differential Privacy Ergute Bao Yizheng Zhu X. Xiao Yifan Yang Beng Chin Ooi B. Tan Khin Mi Mi Aung FedML 31 19 0 08 Dec 2022
Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models Gowthami Somepalli Vasu Singla Micah Goldblum Jonas Geiping Tom Goldstein 26 305 0 07 Dec 2022
CrossSplit: Mitigating Label Noise Memorization through Data Splitting Jihye Kim A. Baratin Yan Zhang Simon Lacoste-Julien NoLa 18 7 0 03 Dec 2022
Neural Representations Reveal Distinct Modes of Class Fitting in Residual Convolutional Networks Michal Jamro.z Marcin Kurdziel 27 0 0 01 Dec 2022
Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes Yiqiao Jin Xiting Wang Y. Hao Yizhou Sun Xing Xie 38 11 0 24 Nov 2022
What Images are More Memorable to Machines? Junlin Han Huangying Zhan Jie Hong Pengfei Fang Hongdong Li L. Petersson Ian Reid 38 3 0 14 Nov 2022
SA-DPSGD: Differentially Private Stochastic Gradient Descent based on Simulated Annealing Jie Fu Zhili Chen Xinpeng Ling 27 0 0 14 Nov 2022
Unintended Memorization and Timing Attacks in Named Entity Recognition Models Rana Salal Ali Benjamin Zi Hao Zhao Hassan Jameel Asghar Tham Nguyen Ian D. Wood Dali Kaafar AAML 28 3 0 04 Nov 2022
Characterizing Datapoints via Second-Split Forgetting Pratyush Maini Saurabh Garg Zachary Chase Lipton J. Zico Kolter 23 34 0 26 Oct 2022
Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano Chuan Guo Alexandre Sablayrolles Maziar Sanjabi FedML 29 17 0 24 Oct 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks Vikas Raunak Arul Menezes 38 13 0 24 Oct 2022
Lexical Generalization Improves with Larger Models and Longer Training Elron Bandel Yoav Goldberg Yanai Elazar 64 6 0 23 Oct 2022
Improving Data Quality with Training Dynamics of Gradient Boosting Decision Trees M. Ponti L. Oliveira Mathias Esteban Valentina Garcia J. Román Luis Argerich TDI 30 4 0 20 Oct 2022