Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks

31 October 2023

Papers citing "Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks"

45 / 45 papers shown

Title
Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano Chuan Guo Alexandre Sablayrolles Maziar Sanjabi FedML 53 17 0 24 Oct 2022
Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions Sitan Chen Sinho Chewi Jungshian Li Yuanzhi Li Adil Salim Anru R. Zhang DiffM 199 276 0 22 Sep 2022
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos Volkan Cevher 75 21 0 15 Sep 2022
When Does Differentially Private Learning Not Suffer in High Dimensions? Xuechen Li Daogao Liu Tatsunori Hashimoto Huseyin A. Inan Janardhan Kulkarni Y. Lee Abhradeep Thakurta 63 57 0 01 Jul 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 83 141 0 15 Jun 2022
A Blessing of Dimensionality in Membership Inference through Regularization Jasper Tan Daniel LeJeune Blake Mason Hamid Javadi Richard G. Baraniuk 60 19 0 27 May 2022
Differentially Private Learning with Margin Guarantees Raef Bassily M. Mohri A. Suresh 51 10 0 21 Apr 2022
Optimal Membership Inference Bounds for Adaptive Composition of Sampled Gaussian Mechanisms Saeed Mahloujifar Alexandre Sablayrolles Graham Cormode S. Jha 59 22 0 12 Apr 2022
Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference Jasper Tan Blake Mason Hamid Javadi Richard G. Baraniuk FedML 69 20 0 02 Feb 2022
Reconstructing Training Data with Informed Adversaries Borja Balle Giovanni Cherubin Jamie Hayes MIACV AAML 86 171 0 13 Jan 2022
Covariate Shift in High-Dimensional Random Feature Regression Nilesh Tripuraneni Ben Adlam Jeffrey Pennington OOD 45 24 0 16 Nov 2021
When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint Y. Freund Yi-An Ma Tong Zhang 68 16 0 05 Oct 2021
Differentially Private Stochastic Optimization: New Results in Convex and Non-Convex Settings Raef Bassily Cristóbal Guzmán Michael Menart 84 56 0 12 Jul 2021
On the Convergence and Calibration of Deep Learning with Differential Privacy Zhiqi Bu Hua Wang Zongyu Dai Qi Long 72 31 0 15 Jun 2021
What can linearized neural networks actually say about generalization? Guillermo Ortiz-Jiménez Seyed-Mohsen Moosavi-Dezfooli P. Frossard 73 45 0 12 Jun 2021
A Universal Law of Robustness via Isoperimetry Sébastien Bubeck Mark Sellke 50 218 0 26 May 2021
$Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry$ Private Stochastic Convex Optimization: Optimal Rates in $\ell_1$ Geometry Hilal Asi Vitaly Feldman Tomer Koren Kunal Talwar 40 94 0 02 Mar 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 57 83 0 21 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 504 1,943 0 14 Dec 2020
Faster Differentially Private Samplers via Rényi Divergence Analysis of Discretized Langevin MCMC Arun Ganesh Kunal Talwar FedML 49 41 0 27 Oct 2020
Phase diagram for two-layer ReLU neural networks at infinite-width limit Yaoyu Zhang Zhi-Qin John Xu Zheng Ma Yaoyu Zhang 67 61 0 15 Jul 2020
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses Raef Bassily Vitaly Feldman Cristóbal Guzmán Kunal Talwar MLT 54 198 0 12 Jun 2020
Neural Kernels Without Tangents Vaishaal Shankar Alex Fang Wenshuo Guo Sara Fridovich-Keil Ludwig Schmidt Jonathan Ragan-Kelley Benjamin Recht 49 91 0 04 Mar 2020
Private Stochastic Convex Optimization with Optimal Rates Raef Bassily Vitaly Feldman Kunal Talwar Abhradeep Thakurta 78 246 0 27 Aug 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 92 391 0 30 May 2019
Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices Santosh Vempala Andre Wibisono 90 269 0 20 Mar 2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee Lechao Xiao S. Schoenholz Yasaman Bahri Roman Novak Jascha Narain Sohl-Dickstein Jeffrey Pennington 211 1,106 0 18 Feb 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 205 973 0 24 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 839 0 19 Dec 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 264 1,466 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 211 1,135 0 09 Nov 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 227 1,275 0 04 Oct 2018
Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences Borja Balle Gilles Barthe Marco Gaboardi 84 392 0 04 Jul 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 271 3,213 0 20 Jun 2018
Membership Inference Attacks against Machine Learning Models Reza Shokri M. Stronati Congzheng Song Vitaly Shmatikov SLR MIALM MIACV 269 4,152 0 18 Oct 2016
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,155 0 01 Jul 2016
On-Average KL-Privacy and its equivalence to Generalization for Max-Entropy Mechanisms Yu Wang Jing Lei S. Fienberg 48 48 0 08 May 2016
Algorithmic Stability for Adaptive Data Analysis Raef Bassily Kobbi Nissim Adam D. Smith Thomas Steinke Uri Stemmer Jonathan R. Ullman 96 268 0 08 Nov 2015
Train faster, generalize better: Stability of stochastic gradient descent Moritz Hardt Benjamin Recht Y. Singer 116 1,242 0 03 Sep 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 332 18,647 0 06 Feb 2015
Privacy and Statistical Risk: Formalisms and Minimax Bounds Rina Foygel Barber John C. Duchi PILM 67 92 0 15 Dec 2014
Private Empirical Risk Minimization Beyond the Worst Case: The Effect of the Constraint Set Geometry Kunal Talwar Abhradeep Thakurta Li Zhang 73 56 0 20 Nov 2014
Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds Raef Bassily Adam D. Smith Abhradeep Thakurta FedML 138 371 0 27 May 2014
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 153 576 0 08 Dec 2012
Rényi Divergence and Kullback-Leibler Divergence T. Erven P. Harremoes 84 1,341 0 12 Jun 2012