Training (Overparametrized) Neural Networks in Near-Linear Time

20 June 2020

Papers citing "Training (Overparametrized) Neural Networks in Near-Linear Time"

27 / 27 papers shown

Title
Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform Josh Alman Zhao Song 2 0 0 17 May 2025
Video Latent Flow Matching: Optimal Polynomial Projections for Video Interpolation and Extrapolation Yang Cao Zhao Song Chiwun Yang VGen 46 2 0 01 Feb 2025
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao Song 61 12 0 03 Jan 2025
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao Song Chiwun Yang 40 29 0 23 Aug 2023
Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification Lianke Qin Zhao Song Yuanyuan Yang 25 9 0 13 Jul 2023
Learning the Positions in CountSketch Yi Li Honghao Lin Simin Liu A. Vakilian David P. Woodruff 29 18 0 11 Jun 2023
Fast Submodular Function Maximization Lianke Qin Zhao Song Yitan Wang 23 10 0 15 May 2023
Fast and Efficient Matching Algorithm with Deadline Instances Zhao Song Weixin Wang Chenbo Yin Junze Yin 14 7 0 15 May 2023
$A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee$ A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee Zhao Song Mingquan Ye Junze Yin Licheng Zhang 14 11 0 01 Feb 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao Song Ruizhe Zhang Danyang Zhuo 77 31 0 25 Nov 2022
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability Zhao Song Yitan Wang Zheng Yu Licheng Zhang FedML 23 28 0 15 Oct 2022
Dynamic Tensor Product Regression Aravind Reddy Zhao Song Licheng Zhang 45 21 0 08 Oct 2022
Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis Alexander Munteanu Simon Omlor Zhao Song David P. Woodruff 30 15 0 26 Jun 2022
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao Song Licheng Zhang Ruizhe Zhang 23 64 0 14 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao Song Zheng Yu Danyang Zhuo 29 6 0 04 Dec 2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao Song Atri Rudra Christopher Ré 33 75 0 30 Nov 2021
Breaking the Linear Iteration Cost Barrier for Some Well-known Conditional Gradient Methods Using MaxIP Data-structures Anshumali Shrivastava Zhao Song Zhaozhuo Xu 27 28 0 30 Nov 2021
Dynamic Network-Assisted D2D-Aided Coded Distributed Learning Nikita Zeulin O. Galinina N. Himayat Sergey D. Andreev R. Heath 25 5 0 26 Nov 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao Song Shuo Yang Ruizhe Zhang 38 49 0 09 Oct 2021
Improved architectures and training algorithms for deep operator networks Sizhuang He Hanwen Wang P. Perdikaris AI4CE 52 105 0 04 Oct 2021
Fast Sketching of Polynomial Kernels of Polynomial Degree Zhao Song David P. Woodruff Zheng Yu Lichen Zhang 18 40 0 21 Aug 2021
Oblivious sketching for logistic regression Alexander Munteanu Simon Omlor David P. Woodruff UQCV 46 17 0 14 Jul 2021
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis Baihe Huang Xiaoxiao Li Zhao Song Xin Yang FedML 25 16 0 11 May 2021
Lifelong Learning with Sketched Structural Regularization Haoran Li A. Krishnan Jingfeng Wu Soheil Kolouri Praveen K. Pilly Vladimir Braverman CLL 27 17 0 17 Apr 2021
Quantum-Inspired Algorithms from Randomized Numerical Linear Algebra Nadiia Chepurko K. Clarkson L. Horesh Honghao Lin David P. Woodruff 21 24 0 09 Nov 2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 28 86 0 30 Sep 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 21 42 0 21 Sep 2020