Beyond Backpropagation: Optimization with Multi-Tangent Forward
Gradients

Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients

23 October 2024

Katharina Flügel

ArXiv (abs)PDF HTML

Papers citing "Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients"

12 / 12 papers shown

Title
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 434 2,685 0 04 May 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,430 0 22 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 541 42,591 0 03 Dec 2019
Decoupled Greedy Learning of CNNs Eugene Belilovsky Michael Eickenberg Edouard Oyallon 57 117 0 23 Jan 2019
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets P. Chrabaszcz I. Loshchilov Frank Hutter SSeg OOD 163 649 0 27 Jul 2017
Understanding Synthetic Gradients and Decoupled Neural Interfaces Wojciech M. Czarnecki G. Swirszcz Max Jaderberg Simon Osindero Oriol Vinyals Koray Kavukcuoglu 71 82 0 01 Mar 2017
Direct Feedback Alignment Provides Learning in Deep Neural Networks Arild Nøkland ODL 99 459 0 06 Sep 2016
Decoupled Neural Interfaces using Synthetic Gradients Max Jaderberg Wojciech M. Czarnecki Simon Osindero Oriol Vinyals Alex Graves David Silver Koray Kavukcuoglu 87 358 0 18 Aug 2016
Automatic differentiation in machine learning: a survey A. G. Baydin Barak A. Pearlmutter Alexey Radul J. Siskind PINN AI4CE ODL 168 2,816 0 20 Feb 2015
Towards Biologically Plausible Deep Learning Yoshua Bengio Dong-Hyun Lee J. Bornschein Thomas Mesnard Zhouhan Lin DRL OOD 82 352 0 14 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014