Learning Parities with Neural Networks

18 February 2020

Papers citing "Learning Parities with Neural Networks"

50 / 55 papers shown

Title
Mastering NIM and Impartial Games with Weak Neural Networks: An AlphaZero-inspired Multi-Frame Approach Søren Riis 16 0 0 10 Nov 2024
From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency Kaiyue Wen Huaqing Zhang Hongzhou Lin Jingzhao Zhang MoE LRM 61 2 0 07 Oct 2024
Why Larger Language Models Do In-context Learning Differently? Zhenmei Shi Junyi Wei Zhuoyan Xu Yingyu Liang 37 18 0 30 May 2024
A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts Mohammed Nowaz Rabbani Chowdhury Meng Wang K. E. Maghraoui Naigang Wang Pin-Yu Chen Christopher Carothers MoE 36 4 0 26 May 2024
Matching the Statistical Query Lower Bound for k-sparse Parity Problems with Stochastic Gradient Descent Yiwen Kou Zixiang Chen Quanquan Gu Sham Kakade 42 0 0 18 Apr 2024
Half-Space Feature Learning in Neural Networks Mahesh Lorik Yadav H. G. Ramaswamy Chandrashekar Lakshminarayanan MLT 27 0 0 05 Apr 2024
Simulating Weighted Automata over Sequences and Trees with Transformers Michael Rizvi M. Lizaire Clara Lacroce Guillaume Rabusseau AI4CE 45 0 0 12 Mar 2024
How Do Nonlinear Transformers Learn and Generalize in In-Context Learning? Hongkang Li Meng Wang Songtao Lu Xiaodong Cui Pin-Yu Chen MLT 40 14 0 23 Feb 2024
RedEx: Beyond Fixed Representation Methods via Convex Optimization Amit Daniely Mariano Schain Gilad Yehudai 13 0 0 15 Jan 2024
Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks Eszter Székely Lorenzo Bardone Federica Gerace Sebastian Goldt 32 2 0 22 Dec 2023
Exploring Parity Challenges in Reinforcement Learning through Curriculum Learning with Noisy Labels Bei Zhou Søren Riis 20 2 0 08 Dec 2023
Feature emergence via margin maximization: case studies in algebraic tasks Depen Morwani Benjamin L. Edelman Costin-Andrei Oncescu Rosie Zhao Sham Kakade 34 14 0 13 Nov 2023
SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem Margalit Glasgow MLT 74 13 0 26 Sep 2023
Auto-Regressive Next-Token Predictors are Universal Learners Eran Malach LRM 19 36 0 13 Sep 2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 43 8 0 07 Sep 2023
Six Lectures on Linearized Neural Networks Theodor Misiakiewicz Andrea Montanari 34 12 0 25 Aug 2023
Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks Liam Collins Hamed Hassani Mahdi Soltanolkotabi Aryan Mokhtari Sanjay Shakkottai 31 10 0 13 Jul 2023
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs Emmanuel Abbe Elisabetta Cornacchia Aryo Lotfi 28 11 0 29 Jun 2023
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks Mohammed Nowaz Rabbani Chowdhury Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen MoE 26 17 0 07 Jun 2023
Most Neural Networks Are Almost Learnable Amit Daniely Nathan Srebro Gal Vardi 18 0 0 25 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 36 13 0 11 May 2023
SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics Emmanuel Abbe Enric Boix-Adserà Theodor Misiakiewicz FedML MLT 79 72 0 21 Feb 2023
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity Hongkang Li M. Wang Sijia Liu Pin-Yu Chen ViT MLT 35 56 0 12 Feb 2023
Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks Shuai Zhang M. Wang Pin-Yu Chen Sijia Liu Songtao Lu Miaoyuan Liu MLT 19 16 0 06 Feb 2023
A Mathematical Model for Curriculum Learning for Parities Elisabetta Cornacchia Elchanan Mossel 34 10 0 31 Jan 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 33 48 0 30 Jan 2023
Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification Yuxuan Du Yibo Yang Dacheng Tao Min-hsiu Hsieh 36 22 0 29 Dec 2022
Spectral Evolution and Invariance in Linear-width Neural Networks Zhichao Wang A. Engel Anand D. Sarwate Ioana Dumitriu Tony Chiang 40 14 0 11 Nov 2022
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 32 155 0 19 Oct 2022
Spectral Regularization Allows Data-frugal Learning over Combinatorial Spaces Amirali Aghazadeh Nived Rajaraman Tony Tu Kannan Ramchandran 17 2 0 05 Oct 2022
Feature selection with gradient descent on two-layer networks in low-rotation regimes Matus Telgarsky MLT 28 16 0 04 Aug 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 27 123 0 18 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 17 112 0 30 Jun 2022
Limitations of the NTK for Understanding Generalization in Deep Learning Nikhil Vyas Yamini Bansal Preetum Nakkiran 20 31 0 20 Jun 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 18 6 0 10 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 24 10 0 08 Jun 2022
Impartial Games: A Challenge for Reinforcement Learning Bei Zhou Søren Riis 24 6 0 25 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 31 121 0 03 May 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Quantum machine learning beyond kernel methods Sofiene Jerbi Lukas J. Fiderer Hendrik Poulsen Nautrup Jonas M. Kubler H. Briegel Vedran Dunjko 11 159 0 25 Oct 2021
The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networks James B. Simon Madeline Dickens Dhruva Karkada M. DeWeese 40 27 0 08 Oct 2021
On the Power of Differentiable Learning versus PAC and SQ Learning Emmanuel Abbe Pritish Kamath Eran Malach Colin Sandon Nathan Srebro MLT 71 23 0 09 Aug 2021
Video Super-Resolution Transformer Jie Cao Yawei Li K. Zhang Luc Van Gool ViT 28 166 0 12 Jun 2021
Properties of the After Kernel Philip M. Long 11 29 0 21 May 2021
Noether: The More Things Change, the More Stay the Same Grzegorz Gluch R. Urbanke 14 17 0 12 Apr 2021
Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networks Yuval Belfer Amnon Geifman Meirav Galun Ronen Basri 10 17 0 07 Apr 2021
Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels Eran Malach Pritish Kamath Emmanuel Abbe Nathan Srebro 11 39 0 01 Mar 2021
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed Maria Refinetti Sebastian Goldt Florent Krzakala Lenka Zdeborová 17 73 0 23 Feb 2021
The Connection Between Approximation, Depth Separation and Learnability in Neural Networks Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 13 20 0 31 Jan 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 55 20 0 07 Jan 2021