Understanding Synthetic Gradients and Decoupled Neural Interfaces

Understanding Synthetic Gradients and Decoupled Neural Interfaces

1 March 2017

Wojciech M. Czarnecki

Koray Kavukcuoglu

Papers citing "Understanding Synthetic Gradients and Decoupled Neural Interfaces"

19 / 19 papers shown

Title
Deep Incubation: Training Large Models by Divide-and-Conquering Zanlin Ni Yulin Wang Jiangwei Yu Haojun Jiang Yu Cao Gao Huang VLM 18 11 0 08 Dec 2022
Block-wise Training of Residual Networks via the Minimizing Movement Scheme Skander Karkar Ibrahim Ayed Emmanuel de Bézenac Patrick Gallinari 33 1 0 03 Oct 2022
Hebbian Deep Learning Without Feedback Adrien Journé Hector Garcia Rodriguez Qinghai Guo Timoleon Moraitis AAML 31 49 0 23 Sep 2022
Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Network Jian-Hui Chen Cheng-Lin Liu Zuoren Wang 26 0 0 01 Aug 2022
A Taxonomy of Recurrent Learning Rules Guillermo Martín-Sánchez Sander M. Bohté S. Otte 29 3 0 23 Jul 2022
Signal Propagation: A Framework for Learning and Inference In a Forward Pass Adam A. Kohan E. Rietman H. Siegelmann 24 26 0 04 Apr 2022
Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass Giorgia Dellaferrera Gabriel Kreiman 29 53 0 27 Jan 2022
Target Propagation via Regularized Inversion Vincent Roulet Zaïd Harchaoui BDL AAML 27 4 0 02 Dec 2021
On Training Implicit Models Zhengyang Geng Xinyu Zhang Shaojie Bai Yisen Wang Zhouchen Lin 72 69 0 09 Nov 2021
Learning Multi-Layered GBDT Via Back Propagation Zhendong Zhang AI4CE 33 0 0 24 Sep 2021
SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks Timoleon Moraitis Dmitry Toichkin Adrien Journé Yansong Chua Qinghai Guo AAML BDL 68 28 0 12 Jul 2021
Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods Shiyu Duan José C. Príncipe MQ 38 3 0 09 Jan 2021
Reservoir Transformers Sheng Shen Alexei Baevski Ari S. Morcos Kurt Keutzer Michael Auli Douwe Kiela 35 17 0 30 Dec 2020
A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks O. Marschall Kyunghyun Cho Cristina Savin FedML 36 72 0 05 Jul 2019
Decoupled Greedy Learning of CNNs Eugene Belilovsky Michael Eickenberg Edouard Oyallon 6 114 0 23 Jan 2019
Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery T. Stepleton Razvan Pascanu Will Dabney Siddhant M. Jayakumar Hubert Soyer Rémi Munos 16 4 0 13 May 2018
Decoupled Parallel Backpropagation with Convergence Guarantee Zhouyuan Huo Bin Gu Qian Yang Heng-Chiao Huang 23 97 0 27 Apr 2018
Deep Reinforcement Learning: An Overview Yuxi Li OffRL VLM 104 1,503 0 25 Jan 2017
Decoupled Neural Interfaces using Synthetic Gradients Max Jaderberg Wojciech M. Czarnecki Simon Osindero Oriol Vinyals Alex Graves David Silver Koray Kavukcuoglu 35 353 0 18 Aug 2016