Learning Activation Functions to Improve Deep Neural Networks

21 December 2014

Matthew Hoffman

Papers citing "Learning Activation Functions to Improve Deep Neural Networks"

49 / 49 papers shown

Title
Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptrons Farhad Pourkamali-Anaraki 35 5 0 16 Sep 2024
Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures Michael Unser Alexis Goujon Stanislas Ducotterd 29 2 0 23 Aug 2024
Nonlinearity Enhanced Adaptive Activation Functions David Yevick 22 1 0 29 Mar 2024
Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems Biao Yuan Ana Heitor He Wang Xiaohui Chen AI4CE PINN 34 1 0 08 Jan 2024
Learning Specialized Activation Functions for Physics-informed Neural Networks Honghui Wang Lu Lu Shiji Song Gao Huang PINN AI4CE 16 11 0 08 Aug 2023
ENN: A Neural Network with DCT Adaptive Activation Functions Marc Martinez-Gost Ana I. Pérez-Neira M. Lagunas AAML 11 6 0 02 Jul 2023
When Deep Learning Meets Polyhedral Theory: A Survey Joey Huchette Gonzalo Muñoz Thiago Serra Calvin Tsay AI4CE 94 32 0 29 Apr 2023
Empirical study of the modulus as activation function in computer vision applications Iván Vallés-Pérez E. Soria-Olivas M. Martínez-Sober Antonio J. Serrano Joan Vila-Francés J. Gómez-Sanchís 13 15 0 15 Jan 2023
Efficient Activation Function Optimization through Surrogate Modeling G. Bingham Risto Miikkulainen 16 2 0 13 Jan 2023
Improving Lipschitz-Constrained Neural Networks by Learning Activation Functions Stanislas Ducotterd Alexis Goujon Pakshal Bohra Dimitris Perdios Sebastian Neumayer M. Unser 35 12 0 28 Oct 2022
How important are activation functions in regression and classification? A survey, performance comparison, and future directions Ameya Dilip Jagtap George Karniadakis AI4CE 31 71 0 06 Sep 2022
Neural Networks with A La Carte Selection of Activation Functions Moshe Sipper 13 7 0 24 Jun 2022
Evolution of Activation Functions for Deep Learning-Based Image Classification Raz Lapid Moshe Sipper 19 11 0 24 Jun 2022
On the Number of Regions of Piecewise Linear Neural Networks Alexis Goujon Arian Etemadi M. Unser 44 13 0 17 Jun 2022
Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark S. Dubey S. Singh B. B. Chaudhuri 41 641 0 29 Sep 2021
PowerLinear Activation Functions with application to the first layer of CNNs Kamyar Nasiri Kamaledin Ghiasi-Shirazi 11 0 0 20 Aug 2021
Comparison of different convolutional neural network activation functions and methods for building ensembles L. Nanni Gianluca Maguolo S. Brahnam M. Paci 16 8 0 29 Mar 2021
Privacy and Trust Redefined in Federated Machine Learning Pavlos Papadopoulos Will Abramson A. Hall Nikolaos Pitropakis William J. Buchanan 33 42 0 29 Mar 2021
Improving epidemic testing and containment strategies using machine learning Laura Natali Saga Helgadottir O. Maragò Giovanni Volpe 24 6 0 23 Nov 2020
Parametric Flatten-T Swish: An Adaptive Non-linear Activation Function For Deep Learning Hock Hung Chieng Noorhaniza Wahid P. Ong 21 6 0 06 Nov 2020
Hyperparameter Transfer Across Developer Adjustments Daniel Stoll Jörg Franke Diane Wagner Simon Selg Frank Hutter 27 12 0 25 Oct 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 31 79 0 17 Sep 2020
SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness Mohammadamin Tavakoli Forest Agostinelli Pierre Baldi AAML FAtt 36 39 0 16 Jun 2020
Scalable Partial Explainability in Neural Networks via Flexible Activation Functions S. Sun Chen Li Zhuangkun Wei Antonios Tsourdos Weisi Guo FAtt 29 2 0 10 Jun 2020
Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis Lazhar Khelifi M. Mignotte 28 259 0 10 Jun 2020
Sherpa: Robust Hyperparameter Optimization for Machine Learning L. Hertel Julian Collado Peter Sadowski J. Ott Pierre Baldi 86 103 0 08 May 2020
A Survey on Activation Functions and their relation with Xavier and He Normal Initialization Leonid Datta AI4CE 15 68 0 18 Mar 2020
Efficient Continual Learning in Neural Networks with Embedding Regularization Jary Pomponi Simone Scardapane Vincenzo Lomonaco A. Uncini CLL 30 41 0 09 Sep 2019
Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks Alejandro Molina P. Schramowski Kristian Kersting ODL 23 77 0 15 Jul 2019
Ensemble of Convolutional Neural Networks Trained with Different Activation Functions Gianluca Maguolo L. Nanni Stefano Ghidoni 18 62 0 07 May 2019
Optimization Problems for Machine Learning: A Survey Claudio Gambella Bissan Ghaddar Joe Naoum-Sawaya AI4CE 30 178 0 16 Jan 2019
Deep Asymmetric Networks with a Set of Node-wise Variant Activation Functions Jinhyeok Jang Hyunjoong Cho Jaehong Kim Jaeyeon Lee Seungjoon Yang 18 2 0 11 Sep 2018
Neural Network Encapsulation Hongyang Li Xiaoyang Guo Bo Dai Wanli Ouyang Xiaogang Wang 21 51 0 11 Aug 2018
Adaptive Blending Units: Trainable Activation Functions for Deep Neural Networks L. R. Sütfeld Flemming Brieger Holger Finger S. Füllhase G. Pipa 20 28 0 26 Jun 2018
Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding Dong Liu Ke Sun Zhangyang Wang Runsheng Liu Zhengjun Zha 24 12 0 28 Feb 2018
A representer theorem for deep neural networks M. Unser 27 98 0 26 Feb 2018
Complex-valued Neural Networks with Non-parametric Activation Functions Simone Scardapane S. Van Vaerenbergh Amir Hussain A. Uncini 23 81 0 22 Feb 2018
Learning Combinations of Activation Functions Franco Manessi A. Rozza AI4CE 26 54 0 29 Jan 2018
Stochastic Training of Neural Networks via Successive Convex Approximations Simone Scardapane P. Di Lorenzo 16 9 0 15 Jun 2017
Evolving Parsimonious Networks by Mixing Activation Functions Alexander Hagg Maximilian Mensing A. Asteroth 24 33 0 21 Mar 2017
Activation Ensembles for Deep Neural Networks Mark Harmon Diego Klabjan 23 35 0 24 Feb 2017
Training Skinny Deep Neural Networks with Iterative Hard Thresholding Methods Xiaojie Jin Xiao-Tong Yuan Jiashi Feng Shuicheng Yan 8 78 0 19 Jul 2016
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks Ludovic Trottier Philippe Giguère B. Chaib-draa 36 199 0 30 May 2016
Deep Residual Networks with Exponential Linear Unit Anish Shah Eashan Kadam Hena Shah Sameer Shinde Sandip Shingade 33 120 0 14 Apr 2016
Multi-Bias Non-linear Activation in Deep Neural Networks Hongyang Li Wanli Ouyang Xiaogang Wang 15 64 0 03 Apr 2016
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 44 2,336 0 30 Mar 2016
Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks Devansh Arpit Yingbo Zhou Bhargava U. Kota V. Govindaraju 13 126 0 04 Mar 2016
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 29 18,444 0 06 Feb 2015
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 266 7,636 0 03 Jul 2012