On the Optimal Memorization Power of ReLU Neural Networks

7 October 2021

Gal Vardi

Papers citing "On the Optimal Memorization Power of ReLU Neural Networks"

26 / 26 papers shown

Title
Minimum width for universal approximation using squashable activation functions Jonghyun Shin Namjun Kim Geonho Hwang Sejun Park 33 0 0 10 Apr 2025
Generalizability of Memorization Neural Networks Lijia Yu Xiao-Shan Gao Lijun Zhang Yibo Miao 33 1 0 01 Nov 2024
On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding Kevin Xu Issei Sato 39 3 0 02 Oct 2024
Deep Neural Networks: Multi-Classification and Universal Approximation Martín Hernández Enrique Zuazua 34 2 0 10 Sep 2024
On the Complexity of Neural Computation in Superposition Micah Adler Nir Shavit 115 3 0 05 Sep 2024
Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks Jy-yong Sohn Dohyun Kwon Seoyeon An Kangwook Lee 40 0 0 01 Aug 2024
Empirical Capacity Model for Self-Attention Neural Networks Aki Härmä M. Pietrasik Anna Wilbik 36 1 0 22 Jul 2024
Expressive Power of ReLU and Step Networks under Floating-Point Operations Yeachan Park Geonho Hwang Wonyeol Lee Sejun Park 14 2 0 26 Jan 2024
One Fits All: Universal Time Series Analysis by Pretrained LM and Specially Designed Adaptors Tian Zhou Peisong Niu Xue Wang Liang Sun Rong Jin AI4TS 68 2 0 24 Nov 2023
From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport Quentin Bouniot I. Redko Anton Mallasto Charlotte Laclau Karol Arndt Oliver Struckmeier Markus Heinonen Ville Kyrki Samuel Kaski 54 2 0 17 Oct 2023
Memorization with neural nets: going beyond the worst case S. Dirksen Patrick Finke Martin Genzel 37 0 0 30 Sep 2023
Minimum width for universal approximation using ReLU networks on compact domain Namjun Kim Chanho Min Sejun Park VLM 29 10 0 19 Sep 2023
On the training and generalization of deep operator networks Sanghyun Lee Yeonjong Shin 11 19 0 02 Sep 2023
Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators? T. Kajitsuka Issei Sato 31 16 0 26 Jul 2023
Memorization Capacity of Multi-Head Attention in Transformers Sadegh Mahdavi Renjie Liao Christos Thrampoulidis 26 22 0 03 Jun 2023
On the Query Complexity of Training Data Reconstruction in Private Learning Prateeti Mukherjee Satyanarayana V. Lokam 19 0 0 29 Mar 2023
Memorization Capacity of Neural Networks with Conditional Computation Erdem Koyuncu 35 4 0 20 Mar 2023
One Fits All:Power General Time Series Analysis by Pretrained LM Tian Zhou Peisong Niu Xue Wang Liang Sun Rong Jin AI4TS 30 380 0 23 Feb 2023
Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data Jonathan W. Siegel 11 2 0 02 Feb 2023
Small Transformers Compute Universal Metric Embeddings Anastasis Kratsios Valentin Debarnot Ivan Dokmanić 59 11 0 14 Sep 2022
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power Binghui Li Jikai Jin Han Zhong J. Hopcroft Liwei Wang OOD 82 27 0 27 May 2022
$Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete$ Training Fully Connected Neural Networks is $\exists\mathbb{R}$ -Complete Daniel Bertschinger Christoph Hertrich Paul Jungeblut Tillmann Miltzow Simon Weber OffRL 57 30 0 04 Apr 2022
Width is Less Important than Depth in ReLU Neural Networks Gal Vardi Gilad Yehudai Ohad Shamir 3DV 13 9 0 08 Feb 2022
Just Least Squares: Binary Compressive Sampling with Low Generative Intrinsic Dimension Yuling Jiao Dingwei Li Min Liu Xiliang Lu Yuanyuan Yang 21 2 0 29 Nov 2021
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 47 95 0 25 Jul 2020
Benefits of depth in neural networks Matus Telgarsky 148 602 0 14 Feb 2016