Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods

9 January 2021

Papers citing "Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods"

50 / 60 papers shown

Title
Revisiting Locally Supervised Learning: an Alternative to End-to-end Training Yulin Wang Zanlin Ni Shiji Song Le Yang Gao Huang 60 84 0 26 Jan 2021
Interlocking Backpropagation: Improving depthwise model-parallelism Aidan Gomez Oscar Key Kuba Perlin Stephen Gou Nick Frosst J. Dean Y. Gal 33 19 0 08 Oct 2020
Deriving Differential Target Propagation from Iterating Approximate Inverses Yoshua Bengio 69 25 0 29 Jul 2020
Biological credit assignment through dynamic inversion of feedforward networks William F. Podlaski C. Machens 44 19 0 10 Jul 2020
A Theoretical Framework for Target Propagation Alexander Meulemans Francesco S. Carzaniga Johan A. K. Suykens João Sacramento Benjamin Grewe AAML 54 78 0 25 Jun 2020
Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks Roman Pogodin P. Latham 113 35 0 12 Jun 2020
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias Axel Laborieux M. Ernoult B. Scellier Yoshua Bengio Julie Grollier D. Querlioz 46 73 0 06 Jun 2020
Modularizing Deep Learning via Pairwise Learning With Kernels Shiyu Duan Shujian Yu José C. Príncipe MoMe 55 20 0 12 May 2020
Two Routes to Scalable Credit Assignment without Weight Symmetry D. Kunin Aran Nayebi Javier Sagastuy-Breña Surya Ganguli Jonathan M. Bloom Daniel L. K. Yamins 110 33 0 28 Feb 2020
Local Propagation in Constraint-based Neural Network G. Marra Matteo Tiezzi S. Melacci Alessandro Betti Marco Maggini Marco Gori 16 10 0 18 Feb 2020
Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks D. Obeid Hugo Ramambason Cengiz Pehlevan FedML 34 20 0 11 Oct 2019
Spike-based causal inference for weight alignment Jordan Guerguiev Konrad Paul Kording Blake A. Richards CML 53 23 0 03 Oct 2019
Gated Linear Networks William H. Guss Tor Lattimore David Budden Avishkar Bhoopchand Christopher Mattern ... Ruslan Salakhutdinov Jianan Wang Peter Toth Simon Schmitt Marcus Hutter AI4CE 92 41 0 30 Sep 2019
The HSIC Bottleneck: Deep Learning without Back-Propagation Kurt Wan-Duo Ma J. P. Lewis W. Kleijn BDL 65 130 0 05 Aug 2019
Principled Training of Neural Networks with Direct Feedback Alignment Julien Launay Iacopo Poli Florent Krzakala 41 35 0 11 Jun 2019
Learning to solve the credit assignment problem B. Lansdell P. Prakash Konrad Paul Kording 46 52 0 03 Jun 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity J.N. Zhang Tianxing He S. Sra Ali Jadbabaie 72 459 0 28 May 2019
Putting An End to End-to-End: Gradient-Isolated Learning of Representations Sindy Löwe Peter O'Connor Bastiaan S. Veeling SSL 106 144 0 28 May 2019
Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization Takuya Shimada Han Bao Issei Sato Masashi Sugiyama 46 41 0 26 Apr 2019
A Theoretical Analysis of Contrastive Unsupervised Representation Learning Sanjeev Arora H. Khandeparkar M. Khodak Orestis Plevrakis Nikunj Saunshi SSL 96 776 0 25 Feb 2019
Decoupled Greedy Learning of CNNs Eugene Belilovsky Michael Eickenberg Edouard Oyallon 46 116 0 23 Jan 2019
Training Neural Networks with Local Error Signals Arild Nøkland L. Eidnes 73 227 0 20 Jan 2019
Greedy Layerwise Learning Can Scale to ImageNet Eugene Belilovsky Michael Eickenberg Edouard Oyallon 109 180 0 29 Dec 2018
Feedback alignment in deep convolutional networks Theodore H. Moskovitz Ashok Litwin-Kumar L. F. Abbott 54 60 0 12 Dec 2018
Fenchel Lifted Networks: A Lagrange Relaxation of Neural Network Training Fangda Gu Armin Askari L. Ghaoui 53 39 0 20 Nov 2018
Biologically-plausible learning algorithms can scale to large datasets Y. Chitour Honglin Chen Zhenyu Liao T. Poggio 67 76 0 08 Nov 2018
Lifted Proximal Operator Machines Jia Li Cong Fang Zhouchen Lin ODL 42 36 0 05 Nov 2018
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations Alexander Ororbia A. Mali C. Lee Giles Daniel Kifer 59 63 0 17 Oct 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.4K 94,511 0 11 Oct 2018
Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures Sergey Bartunov Adam Santoro Blake A. Richards Luke Marris Geoffrey E. Hinton Timothy Lillicrap 82 243 0 12 Jul 2018
Unsupervised Learning by Competing Hidden Units Dmitry Krotov J. Hopfield SSL 58 168 0 26 Jun 2018
Beyond Backprop: Online Alternating Minimization with Auxiliary Variables A. Choromańska Benjamin Cowen Yara Rizk Ronny Luss Mattia Rigotti ... Brian Kingsbury Paolo Diachille V. Gurev Ravi Tejwani Djallel Bouneffouf 48 53 0 24 Jun 2018
Biologically Motivated Algorithms for Propagating Local Target Representations Alexander Ororbia A. Mali 110 88 0 26 May 2018
Lifted Neural Networks Armin Askari Geoffrey Negiar Rajiv Sambharya L. Ghaoui 92 37 0 03 May 2018
A Provably Correct Algorithm for Deep Learning that Actually Works Eran Malach Shai Shalev-Shwartz MLT 102 31 0 26 Mar 2018
A Proximal Block Coordinate Descent Algorithm for Deep Neural Network Training Tim Tsz-Kit Lau Jinshan Zeng Baoyuan Wu Yuan Yao ODL 46 33 0 24 Mar 2018
Global Convergence of Block Coordinate Descent in Deep Learning Jinshan Zeng Tim Tsz-Kit Lau Shaobo Lin Yuan Yao 49 77 0 01 Mar 2018
On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation Shiyu Duan Shujian Yu Yunmei Chen José C. Príncipe 34 16 0 11 Feb 2018
Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks Ziming Zhang M. Brand 37 70 0 20 Nov 2017
Deep supervised learning using local errors Hesham Mostafa V. Ramesh Gert Cauwenberghs 58 114 0 17 Nov 2017
Learning Deep ResNet Blocks Sequentially using Boosting Theory Furong Huang Jordan T. Ash John Langford Robert Schapire 58 111 0 15 Jun 2017
Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks? Cengiz Pehlevan Anirvan M. Sengupta D. Chklovskii 35 79 0 23 Mar 2017
Understanding Synthetic Gradients and Decoupled Neural Interfaces Wojciech M. Czarnecki G. Swirszcz Max Jaderberg Simon Osindero Oriol Vinyals Koray Kavukcuoglu 57 82 0 01 Mar 2017
Direct Feedback Alignment Provides Learning in Deep Neural Networks Arild Nøkland ODL 81 454 0 06 Sep 2016
Decoupled Neural Interfaces using Synthetic Gradients Max Jaderberg Wojciech M. Czarnecki Simon Osindero Oriol Vinyals Alex Graves David Silver Koray Kavukcuoglu 75 356 0 18 Aug 2016
Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention Yang Liu Chengjie Sun Mehdi Alizadeh Xiaolong Wang 52 274 0 30 May 2016
Training Neural Networks Without Gradients: A Scalable ADMM Approach Gavin Taylor R. Burmeister Zheng Xu Bharat Singh Ankit B. Patel Tom Goldstein ODL 50 275 0 06 May 2016
Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation B. Scellier Yoshua Bengio 66 489 0 16 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.9K 193,426 0 10 Dec 2015
How Important is Weight Symmetry in Backpropagation? Q. Liao Joel Z Leibo T. Poggio 59 170 0 17 Oct 2015