Stronger generalization bounds for deep nets via a compression approach

14 February 2018

Papers citing "Stronger generalization bounds for deep nets via a compression approach"

50 / 440 papers shown

Title
Fantastic Generalization Measures are Nowhere to be Found Michael C. Gastpar Ido Nachum Jonathan Shafer T. Weinberger 29 13 0 24 Sep 2023
Input margins can predict generalization too Coenraad Mouton Marthinus W. Theunissen Marelie Hattingh Davel AAML UQCV AI4CE 23 3 0 29 Aug 2023
Modify Training Directions in Function Space to Reduce Generalization Error Yi Yu Wenlian Lu Boyu Chen 27 0 0 25 Jul 2023
Text Descriptions are Compressive and Invariant Representations for Visual Learning Zhili Feng Anna Bair J. Zico Kolter VLM 24 6 0 10 Jul 2023
Generalization Guarantees via Algorithm-dependent Rademacher Complexity Sarah Sachs T. Erven Liam Hodgkinson Rajiv Khanna Umut Simsekli 25 3 0 04 Jul 2023
Sparsity-aware generalization theory for deep neural networks Ramchandran Muthukumar Jeremias Sulam MLT 24 4 0 01 Jul 2023
Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok Pascal Junior Tikeng Notsawo Hattie Zhou Mohammad Pezeshki Irina Rish G. Dumas 25 23 0 23 Jun 2023
Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD Yijun Wan Melih Barsbey Milad Sefidgaran Umut Simsekli 35 1 0 13 Jun 2023
On Size-Independent Sample Complexity of ReLU Networks Mark Sellke 17 5 0 03 Jun 2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression Runtian Zhai Bing Liu Andrej Risteski Zico Kolter Pradeep Ravikumar SSL 38 9 0 01 Jun 2023
Generalization Bounds for Magnitude-Based Pruning via Sparse Matrix Sketching E. Guha Prasanjit Dubey X. Huo MLT 34 1 0 30 May 2023
On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long Sequences A. F. Pour H. Ashtiani 24 0 0 28 May 2023
Pruning at Initialization -- A Sketching Perspective Noga Bar Raja Giryes 26 1 0 27 May 2023
Pruning Pre-trained Language Models with Principled Importance and Self-regularization Siyu Ren Kenny Q. Zhu 30 2 0 21 May 2023
Mode Connectivity in Auction Design Christoph Hertrich Yixin Tao László A. Végh 24 1 0 18 May 2023
Deep ReLU Networks Have Surprisingly Simple Polytopes Fenglei Fan Wei Huang Xiang-yu Zhong Lecheng Ruan T. Zeng Huan Xiong Fei Wang 67 5 0 16 May 2023
How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression? Jetze T. Schuurmans Kim Batselier Julian F. P. Kooij 21 1 0 09 May 2023
Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective Yuxin Dong Tieliang Gong Hao Chen Chen Li 23 4 0 02 May 2023
Fundamental Tradeoffs in Learning with Prior Information Anirudha Majumdar 32 0 0 26 Apr 2023
AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs Shengrui Li Xueting Han Jing Bai AI4CE 18 10 0 19 Apr 2023
K-means Clustering Based Feature Consistency Alignment for Label-free Model Evaluation Shuyu Miao Lin Zheng Qingbin Liu and Hong Jin 39 5 0 17 Apr 2023
The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning Micah Goldblum Marc Finzi K. Rowan A. Wilson UQCV FedML 29 38 0 11 Apr 2023
Empirical Design in Reinforcement Learning Andrew Patterson Samuel Neumann Martha White Adam White 17 22 0 03 Apr 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 45 167 0 28 Mar 2023
VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin Hang Hua Ming Chen Yikang Li Jenhao Hsiao C. Ho Jiebo Luo 36 30 0 21 Mar 2023
Bayes Complexity of Learners vs Overfitting Grzegorz Gluch R. Urbanke UQCV BDL 11 0 0 13 Mar 2023
Generalization analysis of an unfolding network for analysis-based Compressed Sensing Vicky Kouni Yannis Panagakis MLT 28 0 0 09 Mar 2023
Transformed Low-Rank Parameterization Can Help Robust Generalization for Tensor Neural Networks Andong Wang Chong Li Mingyuan Bai Zhong Jin Guoxu Zhou Qianchuan Zhao OOD AAML 18 5 0 01 Mar 2023
Learning to Generalize Provably in Learning to Optimize Junjie Yang Tianlong Chen Mingkang Zhu Fengxiang He Dacheng Tao Yitao Liang Zhangyang Wang 34 6 0 22 Feb 2023
Towards Optimal Compression: Joint Pruning and Quantization Ben Zandonati Glenn Bucagu Adrian Alan Pol M. Pierini Olya Sirkin Tal Kopetz MQ 25 2 0 15 Feb 2023
Task-Specific Skill Localization in Fine-tuned Language Models A. Panigrahi Nikunj Saunshi Haoyu Zhao Sanjeev Arora MoMe 34 68 0 13 Feb 2023
Koopman-based generalization bound: New aspect for full-rank weights Yuka Hashimoto Sho Sonoda Isao Ishikawa Atsushi Nitanda Taiji Suzuki 11 2 0 12 Feb 2023
Pruning Deep Neural Networks from a Sparsity Perspective Enmao Diao G. Wang Jiawei Zhan Yuhong Yang Jie Ding Vahid Tarokh 27 30 0 11 Feb 2023
Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusion Haotian Ju Dongyue Li Aneesh Sharma Hongyang R. Zhang 31 40 0 09 Feb 2023
Generalization Bounds with Data-dependent Fractal Dimensions Benjamin Dupuis George Deligiannidis Umut cSimcsekli AI4CE 39 12 0 06 Feb 2023
Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation Weijian Deng Yumin Suh Stephen Gould Liang Zheng UQCV 29 12 0 02 Feb 2023
Gradient Descent in Neural Networks as Sequential Learning in RKBS A. Shilton Sunil R. Gupta Santu Rana Svetha Venkatesh MLT 19 1 0 01 Feb 2023
Norm-based Generalization Bounds for Compositionally Sparse Neural Networks Tomer Galanti Mengjia Xu Liane Galanti T. Poggio 38 9 0 28 Jan 2023
A Stability Analysis of Fine-Tuning a Pre-Trained Model Z. Fu Anthony Man-Cho So Nigel Collier 23 3 0 24 Jan 2023
Stretched and measured neural predictions of complex network dynamics V. Vasiliauskaite Nino Antulov-Fantulin 33 1 0 12 Jan 2023
Effects of Data Geometry in Early Deep Learning Saket Tiwari George Konidaris 82 7 0 29 Dec 2022
Lifelong Reinforcement Learning with Modulating Masks Eseoghene Ben-Iwhiwhu Saptarshi Nath Praveen K. Pilly Soheil Kolouri Andrea Soltoggio CLL OffRL 34 20 0 21 Dec 2022
Improved Convergence Guarantees for Shallow Neural Networks A. Razborov ODL 27 1 0 05 Dec 2022
The Effect of Data Dimensionality on Neural Network Prunability Zachary Ankner Alex Renda Gintare Karolina Dziugaite Jonathan Frankle Tian Jin 36 5 0 01 Dec 2022
PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization Sanae Lotfi Marc Finzi Sanyam Kapoor Andres Potapczynski Micah Goldblum A. Wilson BDL MLT AI4CE 29 51 0 24 Nov 2022
Finding Skill Neurons in Pre-trained Transformer-based Language Models Xiaozhi Wang Kaiyue Wen Zhengyan Zhang Lei Hou Zhiyuan Liu Juanzi Li MILM MoE 27 50 0 14 Nov 2022
Do highly over-parameterized neural networks generalize since bad solutions are rare? Julius Martinetz T. Martinetz 30 1 0 07 Nov 2022
Instance-Dependent Generalization Bounds via Optimal Transport Songyan Hou Parnian Kassraie Anastasis Kratsios Andreas Krause Jonas Rothfuss 22 6 0 02 Nov 2022
Auxiliary task discovery through generate-and-test Banafsheh Rafiee Sina Ghiassian Jun Jin R. Sutton Jun Luo Adam White 21 0 0 25 Oct 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 49 8 0 25 Oct 2022