Does Learning Require Memorization? A Short Tale about a Long Tail

12 June 2019

Papers citing "Does Learning Require Memorization? A Short Tale about a Long Tail"

50 / 336 papers shown

Title
Traces of Memorisation in Large Language Models for Code Ali Al-Kaswan M. Izadi A. van Deursen ELM 39 14 0 18 Dec 2023
A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models Aysan Esmradi Daniel Wankit Yip C. Chan AAML 42 11 0 18 Dec 2023
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding Talfan Evans Shreya Pathak Hamza Merzic Jonathan Schwarz Ryutaro Tanno Olivier J. Hénaff 18 16 0 08 Dec 2023
SoK: Unintended Interactions among Machine Learning Defenses and Risks Vasisht Duddu S. Szyller Nadarajah Asokan AAML 47 2 0 07 Dec 2023
Understanding (Un)Intended Memorization in Text-to-Image Generative Models Ali Naseh Jaechul Roh Amir Houmansadr DiffM 33 6 0 06 Dec 2023
How Low Can You Go? Surfacing Prototypical In-Distribution Samples for Unsupervised Anomaly Detection Felix Meissen Johannes Getzner Alexander Ziller Georgios Kaissis Daniel Rueckert OODD AI4TS 21 0 0 06 Dec 2023
Calibrated Language Models Must Hallucinate Adam Tauman Kalai Santosh Vempala HILM 30 76 0 24 Nov 2023
DPSUR: Accelerating Differentially Private Stochastic Gradient Descent Using Selective Update and Release Jie Fu Qingqing Ye Haibo Hu Zhili Chen Lulu Wang Kuncan Wang Xun Ran 21 14 0 23 Nov 2023
Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation? Tim Hartill Joshua Bensemann Michael Witbrock Patricia Riddle KELM 30 0 0 21 Nov 2023
On Retrieval Augmentation and the Limitations of Language Model Training Ting-Rui Chiang Xinyan Velocity Yu Joshua Robinson Ollie Liu Isabelle G. Lee Dani Yogatama RALM 28 0 0 16 Nov 2023
Privacy Threats in Stable Diffusion Models Thomas Cilloni Charles Fleming Charles Walter 33 3 0 15 Nov 2023
On Using Distribution-Based Compositionality Assessment to Evaluate Compositional Generalisation in Machine Translation Anssi Moisio Mathias Creutz M. Kurimo CoGe 19 1 0 14 Nov 2023
Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration Wenjie Fu Huandong Wang Chen Gao Guanghua Liu Yong Li Tao Jiang MIALM 32 43 0 10 Nov 2023
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation Verna Dankers Ivan Titov Dieuwke Hupkes 35 5 0 09 Nov 2023
SoK: Memorisation in machine learning Dmitrii Usynin Moritz Knolle Georgios Kaissis 25 1 0 06 Nov 2023
DP-DCAN: Differentially Private Deep Contrastive Autoencoder Network for Single-cell Clustering Huifa Li Jie Fu Zhili Chen Xiaomin Yang Haitao Liu Xinpeng Ling 27 1 0 06 Nov 2023
Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearning Zheyuan Liu Guangyao Dou Yijun Tian Chunhui Zhang Eli Chien Ziwei Zhu MU 26 16 0 28 Oct 2023
In-Context Learning Dynamics with Random Binary Sequences Eric J. Bigelow Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka T. Ullman 34 4 0 26 Oct 2023
Detecting Pretraining Data from Large Language Models Weijia Shi Anirudh Ajith Mengzhou Xia Yangsibo Huang Daogao Liu Terra Blevins Danqi Chen Luke Zettlemoyer MIALM 33 164 0 25 Oct 2023
SoK: Memorization in General-Purpose Large Language Models Valentin Hartmann Anshuman Suri Vincent Bindschaedler David E. Evans Shruti Tople Robert West KELM LLMAG 24 20 0 24 Oct 2023
Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models Matthieu Meeus Shubham Jain Marek Rei Yves-Alexandre de Montjoye MIALM 31 30 0 23 Oct 2023
Fundamental Limits of Membership Inference Attacks on Machine Learning Models Eric Aubinais Elisabeth Gassiat Pablo Piantanida MIACV 50 2 0 20 Oct 2023
Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework Imdad Ullah Najm Hassan S. Gill Basem Suleiman T. Ahanger Zawar Shah Junaid Qadir S. Kanhere 40 16 0 19 Oct 2023
A Comprehensive Study of Privacy Risks in Curriculum Learning Joann Qiongna Chen Xinlei He Zheng Li Yang Zhang Zhou Li 54 2 0 16 Oct 2023
On the Over-Memorization During Natural, Robust and Catastrophic Overfitting Runqi Lin Chaojian Yu Bo Han Tongliang Liu 35 7 0 13 Oct 2023
Impact of Co-occurrence on Factual Knowledge of Large Language Models Cheongwoong Kang Jaesik Choi KELM 35 17 0 12 Oct 2023
Why Train More? Effective and Efficient Membership Inference via Memorization Jihye Choi Shruti Tople Varun Chandrasekaran Somesh Jha TDI FedML 26 2 0 12 Oct 2023
What do larger image classifiers memorise? Michal Lukasik Vaishnavh Nagarajan A. S. Rawat A. Menon Sanjiv Kumar 38 5 0 09 Oct 2023
Anonymous Learning via Look-Alike Clustering: A Precise Analysis of Model Generalization Adel Javanmard Vahab Mirrokni 15 2 0 06 Oct 2023
Scaling Laws for Associative Memories Vivien A. Cabannes Elvis Dohmatob A. Bietti 24 19 0 04 Oct 2023
On Memorization and Privacy Risks of Sharpness Aware Minimization Young In Kim Pratiksha Agrawal J. Royset Rajiv Khanna FedML 30 3 0 30 Sep 2023
Leave-one-out Distinguishability in Machine Learning Jiayuan Ye Anastasia Borovykh Soufiane Hayou Reza Shokri 39 9 0 29 Sep 2023
On the Computational Entanglement of Distant Features in Adversarial Machine Learning Yen-Lung Lai Xingbo Dong Zhe Jin AAML 13 0 0 27 Sep 2023
MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection Kemal Oksuz Selim Kuzucu Tom Joy P. Dokania MoE 24 5 0 26 Sep 2023
Cluster-based pruning techniques for audio data Boris Bergsma Marta Brzezinska O. Yazyev Milos Cernak 24 2 0 21 Sep 2023
Least Squares Maximum and Weighted Generalization-Memorization Machines Shuai Wang Zhen Wang Yuanfu Shao 26 1 0 31 Aug 2023
Late Stopping: Avoiding Confidently Learning from Mislabeled Examples Suqin Yuan Lei Feng Tongliang Liu NoLa 43 11 0 26 Aug 2023
A Probabilistic Fluctuation based Membership Inference Attack for Diffusion Models Wenjie Fu Huandong Wang Chen Gao Guanghua Liu Yong Li Tao Jiang DiffM 32 11 0 23 Aug 2023
Unlocking Accuracy and Fairness in Differentially Private Image Classification Leonard Berrada Soham De J. Shen Jamie Hayes Robert Stanforth David Stutz Pushmeet Kohli Samuel L. Smith Borja Balle 27 13 0 21 Aug 2023
Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening Jack Foster Stefan Schoepf Alexandra Brintrup MU 27 85 0 15 Aug 2023
Samplable Anonymous Aggregation for Private Federated Data Analysis Kunal Talwar Shan Wang Audra McMillan Vojta Jina Vitaly Feldman ... Congzheng Song Karl Tarbe Sebastian Vogt L. Winstrom Shundong Zhou FedML 38 13 0 27 Jul 2023
Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development Danilo Brajovic Niclas Renner Vincent Philipp Goebels Philipp Wagner Benjamin Frész M. Biller Mara Klaeb Janika Kutz Jens Neuhuettler Marco F. Huber 24 9 0 21 Jul 2023
Long-Tail Theory under Gaussian Mixtures Arman Bolatov Maxat Tezekbayev Igor Melnykov Artur Pak Vassilina Nikoulina Z. Assylbekov 17 1 0 20 Jul 2023
Rethinking Backdoor Attacks Alaa Khaddaj Guillaume Leclerc Aleksandar Makelov Kristian Georgiev Hadi Salman Andrew Ilyas A. Madry SILM 34 28 0 19 Jul 2023
Can Neural Network Memorization Be Localized? Pratyush Maini Michael C. Mozer Hanie Sedghi Zachary Chase Lipton J. Zico Kolter Chiyuan Zhang TDI 49 46 0 18 Jul 2023
Privacy-Utility Trade-offs in Neural Networks for Medical Population Graphs: Insights from Differential Privacy and Graph Structure Tamara T. Mueller Maulik Chevli Ameya Daigavane Daniel Rueckert Georgios Kaissis 33 0 0 13 Jul 2023
Memorization Through the Lens of Curvature of Loss Function Around Samples Isha Garg Deepak Ravikumar Kaushik Roy TDI 16 10 0 11 Jul 2023
Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation Zhexin Zhang Jiaxin Wen Minlie Huang 38 30 0 10 Jul 2023
T-MARS: Improving Visual Representations by Circumventing Text Feature Learning Pratyush Maini Sachin Goyal Zachary Chase Lipton J. Zico Kolter Aditi Raghunathan VLM 45 33 0 06 Jul 2023
Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses G. Buzaglo Niv Haim Gilad Yehudai Gal Vardi Yakir Oz Yaniv Nikankin Michal Irani 34 10 0 04 Jul 2023