Does Learning Require Memorization? A Short Tale about a Long Tail

12 June 2019

Papers citing "Does Learning Require Memorization? A Short Tale about a Long Tail"

50 / 336 papers shown

Title
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers Yibo Jiang Goutham Rajendran Pradeep Ravikumar Bryon Aragam CLL KELM 37 6 0 26 Jun 2024
Early learning of the optimal constant solution in neural networks and humans Jirko Rubruck Jan P. Bauer Andrew M. Saxe Christopher Summerfield 33 1 0 25 Jun 2024
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon USVSN Sai Prashanth Alvin Deng Kyle O'Brien Jyothir S V Mohammad Aflah Khan ... Jacob Ray Fuehne Stella Biderman Tracy Ke Katherine Lee Naomi Saphra 63 12 0 25 Jun 2024
Evaluating $n$ -Gram Novelty of Language Models Using Rusty-DAWG William Merrill Noah A. Smith Yanai Elazar ELM TDI 50 9 0 18 Jun 2024
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 37 33 0 17 Jun 2024
Potion: Towards Poison Unlearning Stefan Schoepf Jack Foster Alexandra Brintrup AAML MU 55 7 0 13 Jun 2024
Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition Eleni Triantafillou Peter Kairouz Fabian Pedregosa Jamie Hayes M. Kurmanji ... Lisheng Sun-Hosoya Sergio Escalera Gintare Karolina Dziugaite Peter Triantafillou Isabelle M Guyon MU 56 14 0 13 Jun 2024
A Survey on Machine Unlearning: Techniques and New Emerged Privacy Risks Hengzhu Liu Ping Xiong Tianqing Zhu Philip S. Yu 37 6 0 10 Jun 2024
Causal Estimation of Memorisation Profiles Pietro Lesci Clara Meister Thomas Hofmann Andreas Vlachos Tiago Pimentel 51 5 0 06 Jun 2024
Memorization in deep learning: A survey Jiaheng Wei Yanjun Zhang Leo Yu Zhang Ming Ding Chao Chen Kok-Leong Ong Jun Zhang Yang Xiang 47 6 0 06 Jun 2024
Reconstructing training data from document understanding models Jérémie Dentan Arnaud Paran A. Shabou AAML SyDa 49 1 0 05 Jun 2024
Measuring Stochastic Data Complexity with Boltzmann Influence Functions Nathan Ng Roger C. Grosse Marzyeh Ghassemi 40 0 0 04 Jun 2024
Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models Dominik Hintersdorf Lukas Struppek Kristian Kersting Adam Dziedzic Franziska Boenisch 34 11 0 04 Jun 2024
What makes unlearning hard and what to do about it Kairan Zhao M. Kurmanji George-Octavian Barbulescu Eleni Triantafillou Peter Triantafillou MU 60 17 0 03 Jun 2024
Delving into Differentially Private Transformer Youlong Ding Xueyang Wu Yining Meng Yonggang Luo Hao Wang Weike Pan 39 5 0 28 May 2024
Privacy-Aware Visual Language Models Laurens Samson Nimrod Barazani S. Ghebreab Yukiyasu Asano PILM VLM 50 1 0 27 May 2024
On Fairness of Low-Rank Adaptation of Large Models Zhoujie Ding Ken Ziyu Liu Pura Peetathawatchai Berivan Isik Sanmi Koyejo 48 4 0 27 May 2024
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 73 5 0 26 May 2024
Data Reconstruction: When You See It and When You Don't Edith Cohen Haim Kaplan Yishay Mansour Shay Moran Kobbi Nissim Uri Stemmer Eliad Tsfadia AAML 42 2 0 24 May 2024
A Multi-Perspective Analysis of Memorization in Large Language Models Bowen Chen Namgi Han Yusuke Miyao 46 1 0 19 May 2024
On the Foundations of Earth and Climate Foundation Models Xiao Xiang Zhu Zhitong Xiong Yi Wang Adam J. Stewart Konrad Heidler Yuanyuan Wang Zhenghang Yuan Thomas Dujardin Qingsong Xu Yilei Shi AI4Cl AI4CE 34 21 0 07 May 2024
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models George-Octavian Barbulescu Peter Triantafillou MU 38 16 0 06 May 2024
Exploring prompts to elicit memorization in masked language model-based named entity recognition Yuxi Xia Anastasiia Sedova Pedro Henrique Luz de Araujo Vasiliki Kougia Lisa Nussbaumer Benjamin Roth 28 1 0 05 May 2024
U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AI Tanja Sarcevic Alicja Karlowicz Rudolf Mayer Ricardo A. Baeza-Yates Andreas Rauber 49 6 0 22 Apr 2024
When Do We Not Need Larger Vision Models? Baifeng Shi Ziyang Wu Maolin Mao Xin Wang Trevor Darrell VLM LRM 54 41 0 19 Mar 2024
RAFT: Adapting Language Model to Domain Specific RAG Tianjun Zhang Shishir G. Patil Naman Jain Sheng Shen Matei A. Zaharia Ion Stoica Joseph E. Gonzalez RALM 37 179 0 15 Mar 2024
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models Frederik Kunstner Robin Yadav Alan Milligan Mark Schmidt Alberto Bietti 39 25 0 29 Feb 2024
Unveiling Privacy, Memorization, and Input Curvature Links Deepak Ravikumar Efstathia Soufleri Abolfazl Hashemi Kaushik Roy 54 5 0 28 Feb 2024
Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems Zhenting Qi Hanlin Zhang Eric Xing Sham Kakade Hima Lakkaraju SILM 44 18 0 27 Feb 2024
SpanSeq: Similarity-based sequence data splitting method for improved development and assessment of deep learning projects A. F. Florensa J. J. A. Armenteros Henrik Nielsen F. Aarestrup P. Clausen 27 0 0 22 Feb 2024
Fairness Without Harm: An Influence-Guided Active Sampling Approach Jinlong Pang Jialu Wang Zhaowei Zhu Yuanshun Yao Chen Qian Yang Liu TDI 48 2 0 20 Feb 2024
DE-COP: Detecting Copyrighted Content in Language Models Training Data André V. Duarte Xuandong Zhao Arlindo L. Oliveira Lei Li 34 33 0 15 Feb 2024
Copyright Traps for Large Language Models Matthieu Meeus Igor Shilov Manuel Faysse Yves-Alexandre de Montjoye 36 18 0 14 Feb 2024
Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization Idan Attias Gintare Karolina Dziugaite Mahdi Haghifam Roi Livni Daniel M. Roy 30 6 0 14 Feb 2024
Systematic Assessment of Tabular Data Synthesis Algorithms Yuntao Du Ninghui Li 35 4 0 09 Feb 2024
Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening Stefan Schoepf Jack Foster Alexandra Brintrup MU 36 6 0 06 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Jindong Wang AI4CE 56 12 0 02 Feb 2024
An Information Theoretic Approach to Machine Unlearning Jack Foster Kyle Fogarty Stefan Schoepf Cengiz Öztireli Alexandra Brintrup MU 49 4 0 02 Feb 2024
No Free Prune: Information-Theoretic Barriers to Pruning at Initialization Tanishq Kumar Kevin Luo Mark Sellke 35 3 0 02 Feb 2024
LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fields Joshua A. Vita Amit Samanta Fei Zhou Vincenzo Lordi 25 2 0 01 Feb 2024
Do LLMs Dream of Ontologies? Marco Bombieri Paolo Fiorini Simone Paolo Ponzetto M. Rospocher CLL 32 2 0 26 Jan 2024
Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective Xuechen Zhang Mingchen Li Jiasi Chen Christos Thrampoulidis Samet Oymak 37 3 0 25 Jan 2024
Memorization in Self-Supervised Learning Improves Downstream Generalization Wenhao Wang Muhammad Ahmad Kaleem Adam Dziedzic Michael Backes Nicolas Papernot Franziska Boenisch SSL 25 9 0 19 Jan 2024
Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation Tong Xie Haoyu Li Andrew Bai Cho-Jui Hsieh TDI 36 2 0 17 Jan 2024
lpNTK: Better Generalisation with Less Data via Sample Interaction During Learning Shangmin Guo Yi Ren Stefano V.Albrecht Kenny Smith 34 3 0 16 Jan 2024
Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity Claudio Novelli F. Casolari Philipp Hacker Giorgio Spedicato Luciano Floridi AILaw SILM 50 44 0 14 Jan 2024
Investigating Data Contamination for Pre-training Language Models Minhao Jiang Ken Ziyu Liu Ming Zhong Rylan Schaeffer Siru Ouyang Jiawei Han Sanmi Koyejo 33 64 0 11 Jan 2024
Effective pruning of web-scale datasets based on complexity of concept clusters Amro Abbas E. Rusak Kushal Tirumala Wieland Brendel Kamalika Chaudhuri Ari S. Morcos VLM CLIP 34 22 0 09 Jan 2024
Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective Haoyi Xiong Xuhong Li Xiaofei Zhang Jiamin Chen Xinhao Sun Yuchen Li Zeyi Sun Jundong Li XAI 40 8 0 09 Jan 2024
Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models Rita Frieske Bertram E. Shi 34 7 0 03 Jan 2024