The Pitfalls of Memorization: When Memorization Hurts Generalization

10 December 2024

Papers citing "The Pitfalls of Memorization: When Memorization Hurts Generalization"

37 / 37 papers shown

Title
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges Nayoung Lee Ziyang Cai Avi Schwarzschild Kangwook Lee Dimitris Papailiopoulos ReLM VLM LRM AI4CE 137 7 0 03 Feb 2025
Rethinking LLM Memorization through the Lens of Adversarial Compression Avi Schwarzschild Zhili Feng Pratyush Maini Zachary Chase Lipton J. Zico Kolter 125 56 0 23 Apr 2024
Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization Idan Attias Gintare Karolina Dziugaite Mahdi Haghifam Roi Livni Daniel M. Roy 83 7 0 14 Feb 2024
Feedback-guided Data Synthesis for Imbalanced Classification Reyhane Askari Hemmat Mohammad Pezeshki Florian Bordes M. Drozdzal Adriana Romero Soriano SyDa 77 21 0 29 Sep 2023
Avoiding spurious correlations via logit correction Sheng Liu Xu Zhang Nitesh Sekhar Yue Wu Prateek Singhal C. Fernandez‐Granda 90 32 0 02 Dec 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 108 10 0 25 Oct 2022
Strong Memory Lower Bounds for Learning Natural Models Gavin Brown Mark Bun Adam D. Smith 78 12 0 09 Jun 2022
Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations Polina Kirichenko Pavel Izmailov A. Wilson OOD 91 339 0 06 Apr 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 124 630 0 15 Feb 2022
Simple data balancing achieves competitive worst-group-accuracy Badr Youbi Idrissi Martín Arjovsky Mohammad Pezeshki David Lopez-Paz 118 183 0 27 Oct 2021
Just Train Twice: Improving Group Robustness without Training Group Information Emmy Liu Behzad Haghgoo Annie S. Chen Aditi Raghunathan Pang Wei Koh Shiori Sagawa Percy Liang Chelsea Finn OOD 110 563 0 19 Jul 2021
On the geometry of generalization and memorization in deep neural networks Cory Stephenson Suchismita Padhy Abhinav Ganesh Yue Hui Hanlin Tang SueYeon Chung TDI AI4CE 85 74 0 30 May 2021
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning? Gavin Brown Mark Bun Vitaly Feldman Adam D. Smith Kunal Talwar 311 100 0 11 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 136 268 0 18 Nov 2020
Understanding the Failure Modes of Out-of-Distribution Generalization Vaishnavh Nagarajan Anders Andreassen Behnam Neyshabur OOD OODD 71 177 0 29 Oct 2020
Environment Inference for Invariant Learning Elliot Creager J. Jacobsen R. Zemel OOD 64 384 0 14 Oct 2020
Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans M. Roberts D. Driggs Matthew Thorpe J. Gilbey Michael Yeung ... Kang Zhang S. Stranks James H. F. Rudd Evis Sala Carola-Bibiane Schönlieb OOD 69 774 0 14 Aug 2020
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation Vitaly Feldman Chiyuan Zhang TDI 225 470 0 09 Aug 2020
Balanced Meta-Softmax for Long-Tailed Visual Recognition Jiawei Ren Cunjun Yu Shunan Sheng Xiao Ma Haiyu Zhao Shuai Yi Hongsheng Li 204 573 0 21 Jul 2020
Long-tail learning via logit adjustment A. Menon Sadeep Jayasumana A. S. Rawat Himanshu Jain Andreas Veit Sanjiv Kumar 125 713 0 14 Jul 2020
The Pitfalls of Simplicity Bias in Neural Networks Harshay Shah Kaustav Tamuly Aditi Raghunathan Prateek Jain Praneeth Netrapalli AAML 72 364 0 13 Jun 2020
An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa Aditi Raghunathan Pang Wei Koh Percy Liang 195 383 0 09 May 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 221 2,064 0 16 Apr 2020
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 123 945 0 04 Dec 2019
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization Shiori Sagawa Pang Wei Koh Tatsunori B. Hashimoto Percy Liang OOD 108 1,249 0 20 Nov 2019
Decoupling Representation and Classifier for Long-Tailed Recognition Bingyi Kang Saining Xie Marcus Rohrbach Zhicheng Yan Albert Gordo Jiashi Feng Yannis Kalantidis OODD 180 1,223 0 21 Oct 2019
Invariant Risk Minimization Martín Arjovsky Léon Bottou Ishaan Gulrajani David Lopez-Paz OOD 203 2,246 0 05 Jul 2019
Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss Kaidi Cao Colin Wei Adrien Gaidon Nikos Arechiga Tengyu Ma 131 1,609 0 18 Jun 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 142 502 0 12 Jun 2019
REPAIR: Removing Representation Bias by Dataset Resampling Yi Li Nuno Vasconcelos FaML 79 287 0 16 Apr 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 249 1,660 0 28 Dec 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,324 0 11 Oct 2018
Learning to Reweight Examples for Robust Deep Learning Mengye Ren Wenyuan Zeng Binh Yang R. Urtasun OOD NoLa 152 1,431 0 24 Mar 2018
A Closer Look at Memorization in Deep Networks Devansh Arpit Stanislaw Jastrzebski Nicolas Ballas David M. Krueger Emmanuel Bengio ... Tegan Maharaj Asja Fischer Aaron Courville Yoshua Bengio Simon Lacoste-Julien TDI 131 1,829 0 16 Jun 2017
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 524 4,497 0 18 Apr 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
Deep Learning Face Attributes in the Wild Ziwei Liu Ping Luo Xiaogang Wang Xiaoou Tang CVBM 257 8,433 0 28 Nov 2014