An initial alignment between neural network and target is needed for
gradient descent to learn

An initial alignment between neural network and target is needed for gradient descent to learn

25 February 2022

Elisabetta Cornacchia

Christopher Marquis

Papers citing "An initial alignment between neural network and target is needed for gradient descent to learn"

11 / 11 papers shown

Title
When can transformers reason with abstract symbols? Enric Boix-Adserà Omid Saremi Emmanuel Abbe Samy Bengio Etai Littwin Josh Susskind LRM NAI 31 12 0 15 Oct 2023
Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs Emmanuel Abbe Elisabetta Cornacchia Aryo Lotfi 28 11 0 29 Jun 2023
Expand-and-Cluster: Parameter Recovery of Neural Networks Flavio Martinelli Berfin Simsek W. Gerstner Johanni Brea 26 4 0 25 Apr 2023
SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics Emmanuel Abbe Enric Boix-Adserà Theodor Misiakiewicz FedML MLT 79 72 0 21 Feb 2023
A Mathematical Model for Curriculum Learning for Parities Elisabetta Cornacchia Elchanan Mossel 34 10 0 31 Jan 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 36 48 0 30 Jan 2023
On the non-universality of deep learning: quantifying the cost of symmetry Emmanuel Abbe Enric Boix-Adserà FedML MLT 30 18 0 05 Aug 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 27 123 0 18 Jul 2022
Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs Etienne Boursier Loucas Pillaud-Vivien Nicolas Flammarion ODL 19 58 0 02 Jun 2022
Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures Emmanuel Abbe Samy Bengio Elisabetta Cornacchia Jon M. Kleinberg Aryo Lotfi M. Raghu Chiyuan Zhang MLT 13 10 0 26 May 2022
On the Power of Differentiable Learning versus PAC and SQ Learning Emmanuel Abbe Pritish Kamath Eran Malach Colin Sandon Nathan Srebro MLT 74 23 0 09 Aug 2021