Generalized Leverage Score Sampling for Neural Networks

21 September 2020

Papers citing "Generalized Leverage Score Sampling for Neural Networks"

29 / 29 papers shown

Title
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent Zhao-quan Song Chiwun Yang 29 9 0 17 Oct 2023
Is Solving Graph Neural Tangent Kernel Equivalent to Training Graph Neural Network? Lianke Qin Zhao-quan Song Baocheng Sun 23 6 0 14 Sep 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 20 25 0 14 Sep 2023
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao-quan Song Chiwun Yang 34 29 0 23 Aug 2023
Convergence of Two-Layer Regression with Nonlinear Units Yichuan Deng Zhao-quan Song Shenghao Xie 21 7 0 16 Aug 2023
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum Amnon Geifman Daniel Barzilai Ronen Basri Meirav Galun 29 5 0 26 Jul 2023
Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification Lianke Qin Zhao-quan Song Yuanyuan Yang 25 9 0 13 Jul 2023
Query Complexity of Active Learning for Function Family With Nearly Orthogonal Basis Xiangyi Chen Zhao-quan Song Baochen Sun Junze Yin Danyang Zhuo 34 3 0 06 Jun 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 71 31 0 25 Nov 2022
A Sublinear Adversarial Training Algorithm Yeqi Gao Lianke Qin Zhao-quan Song Yitan Wang GAN 31 25 0 10 Aug 2022
Federated Adversarial Learning: A Framework with Convergence Analysis Xiaoxiao Li Zhao-quan Song Jiaming Yang FedML 27 19 0 07 Aug 2022
Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis Alexander Munteanu Simon Omlor Zhao-quan Song David P. Woodruff 27 15 0 26 Jun 2022
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao-quan Song Licheng Zhang Ruizhe Zhang 23 63 0 14 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao-quan Song Zheng Yu Danyang Zhuo 26 6 0 04 Dec 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao-quan Song Shuo Yang Ruizhe Zhang 35 49 0 09 Oct 2021
Fast Sketching of Polynomial Kernels of Polynomial Degree Zhao-quan Song David P. Woodruff Zheng Yu Lichen Zhang 13 40 0 21 Aug 2021
Scaling Neural Tangent Kernels via Sketching and Random Features A. Zandieh Insu Han H. Avron N. Shoham Chaewon Kim Jinwoo Shin 11 31 0 15 Jun 2021
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis Baihe Huang Xiaoxiao Li Zhao-quan Song Xin Yang FedML 23 16 0 11 May 2021
Random Features for the Neural Tangent Kernel Insu Han H. Avron N. Shoham Chaewon Kim Jinwoo Shin 19 9 0 03 Apr 2021
Learning with Neural Tangent Kernels in Near Input Sparsity Time A. Zandieh 4 0 0 01 Apr 2021
Even your Teacher Needs Guidance: Ground-Truth Targets Dampen Regularization Imposed by Self-Distillation Kenneth Borup L. Andersen 22 14 0 25 Feb 2021
Metric Transforms and Low Rank Matrices via Representation Theory of the Real Hyperrectangle Josh Alman T. Chu Gary Miller Shyam Narayanan Mark Sellke Zhao-quan Song 6 1 0 23 Nov 2020
Algorithms and Hardness for Linear Algebra on Geometric Graphs Josh Alman T. Chu Aaron Schild Zhao-quan Song 49 29 0 04 Nov 2020
MixCon: Adjusting the Separability of Data Representations for Harder Data Recovery Xiaoxiao Li Yangsibo Huang Binghui Peng Zhao-quan Song K. Li MIACV 20 1 0 22 Oct 2020
A Modular Analysis of Provable Acceleration via Polyak's Momentum: Training a Wide ReLU Network and a Deep Linear Network Jun-Kun Wang Chi-Heng Lin Jacob D. Abernethy 8 23 0 04 Oct 2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 28 86 0 30 Sep 2020
Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regime Atsushi Nitanda Taiji Suzuki 6 41 0 22 Jun 2020
An Improved Cutting Plane Method for Convex Optimization, Convex-Concave Games and its Applications Haotian Jiang Y. Lee Zhao-quan Song Sam Chiu-wai Wong 14 106 0 08 Apr 2020
Sharp analysis of low-rank kernel matrix approximations Francis R. Bach 83 277 0 09 Aug 2012