Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark

29 September 2021

Papers citing "Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark"

44 / 94 papers shown

Title
Adaptive Activation Functions for Predictive Modeling with Sparse Experimental Data Farhad Pourkamali-Anaraki Tahamina Nasrin Robert E. Jensen Amy M. Peterson Christopher J. Hansen 19 7 0 08 Feb 2024
MelNet: A Real-Time Deep Learning Algorithm for Object Detection Yashar Azadvatan Murat Kurt ObjD 14 4 0 31 Jan 2024
Segmenting Cardiac Muscle Z-disks with Deep Neural Networks Mihaela Croitor Ibrahim Nishant Ravikumar Alistair P. Curd Joanna Leng Oliver Umney Michelle Peckham 6 0 0 24 Jan 2024
DVL Calibration using Data-driven Methods Zeev Yampolsky Itzik Klein 9 1 0 23 Jan 2024
HOSC: A Periodic Activation Function for Preserving Sharp Features in Implicit Neural Representations Danzel Serrano Jakub Szymkowiak Przemyslaw Musialski 13 4 0 20 Jan 2024
Speech Swin-Transformer: Exploring a Hierarchical Transformer with Shifted Windows for Speech Emotion Recognition Yong Wang Cheng Lu Hailun Lian Yan Zhao Bjorn Schuller Yuan Zong Wenming Zheng 21 10 0 19 Jan 2024
Source-Free Cross-Modal Knowledge Transfer by Unleashing the Potential of Task-Irrelevant Data Jinjin Zhu Yucheng Chen Lin Wang 25 2 0 10 Jan 2024
Brain Tumor Segmentation Based on Deep Learning, Attention Mechanisms, and Energy-Based Uncertainty Prediction Zachary Schwehr Sriman Achanta 26 2 0 31 Dec 2023
An extended asymmetric sigmoid with Perceptron (SIGTRON) for imbalanced linear classification Hyenkyun Woo 15 0 0 26 Dec 2023
Unsupervised Random Quantum Networks for PDEs Josh Dees Antoine Jacquier Sylvain Laizet 16 2 0 21 Dec 2023
DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Grounding Thong Nguyen Xiaobao Wu Xinshuai Dong Cong-Duy Nguyen See-Kiong Ng Anh Tuan Luu 29 7 0 05 Dec 2023
Improved weight initialization for deep and narrow feedforward neural network Hyunwoo Lee Yunho Kim Seungyeop Yang Hayoung Choi ODL 12 3 0 07 Nov 2023
Toward Flare-Free Images: A Survey Yousef Kotp Marwan Torki 43 3 0 22 Oct 2023
From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport Quentin Bouniot I. Redko Anton Mallasto Charlotte Laclau Karol Arndt Oliver Struckmeier Markus Heinonen Ville Kyrki Samuel Kaski 54 2 0 17 Oct 2023
A predict-and-optimize approach to profit-driven churn prevention Nuria Gómez-Vargas Sebastián Maldonado Carla Vairetti 18 2 0 10 Oct 2023
The Geometric Structure of Fully-Connected ReLU Layers Jonatan Vallin K. Larsson M. Larson 16 2 0 05 Oct 2023
A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization Yu Ding Junzhou Yu Chunzhi Gu Shangce Gao Chao Zhang 16 3 0 14 Sep 2023
A physics-informed and attention-based graph learning approach for regional electric vehicle charging demand prediction Haohao Qu Haoxuan Kuang Jun Li Linlin You GNN 19 19 0 11 Sep 2023
Time-Frequency Transformer: A Novel Time Frequency Joint Learning Method for Speech Emotion Recognition Yong Wang Cheng Lu Yuan Zong Hailun Lian Yan Zhao Sunan Li ViT 14 4 0 28 Aug 2023
Gaussian Graph with Prototypical Contrastive Learning in E-Commerce Bundle Recommendation Zhao-Yang Liu Liucheng Sun Chenwei Weng Qijin Chen Chengfu Huo 13 1 0 25 Jul 2023
Rational Neural Network Controllers M. Newton A. Papachristodoulou OOD AAML 37 1 0 12 Jul 2023
Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation using Compact Implicit Layers Bar Lerer Ido Ben-Yair Eran Treister AI4CE 18 3 0 30 Jun 2023
HRTF upsampling with a generative adversarial network using a gnomonic equiangular projection Aidan O. T. Hogg Mads Jenkins Helena E. Liu Isaac Squires Samuel J. Cooper L. Picinali 28 13 0 09 Jun 2023
SMRVIS: Point cloud extraction from 3-D ultrasound for non-destructive testing L. Tang 3DPC 27 0 0 07 Jun 2023
ErfReLU: Adaptive Activation Function for Deep Neural Network Ashish Rajanand Pradeep Singh 15 11 0 02 Jun 2023
Initial Guessing Bias: How Untrained Networks Favor Some Classes Emanuele Francazi Aurélien Lucchi M. Baity-Jesi AI4CE 20 3 0 01 Jun 2023
GELU Activation Function in Deep Learning: A Comprehensive Mathematical Analysis and Performance Minhyeok Lee 13 29 0 20 May 2023
Learning Activation Functions for Sparse Neural Networks Mohammad Loni Aditya Mohan Mehdi Asadi Marius Lindauer 21 4 0 18 May 2023
Orthogonal Transforms in Neural Networks Amount to Effective Regularization Krzysztof Zajkac Wojciech Sopot Paweł Wachel 18 0 0 10 May 2023
Generalizable End-to-End Deep Learning Frameworks for Real-Time Attitude Estimation Using 6DoF Inertial Measurement Units Arman Asgharpoor Golroudbari M. Sabour 14 13 0 13 Feb 2023
Synaptic Stripping: How Pruning Can Bring Dead Neurons Back To Life Tim Whitaker L. D. Whitley CVBM 12 2 0 11 Feb 2023
Empirical study of the modulus as activation function in computer vision applications Iván Vallés-Pérez E. Soria-Olivas M. Martínez-Sober Antonio J. Serrano Joan Vila-Francés J. Gómez-Sanchís 11 15 0 15 Jan 2023
SignReLU neural network and its approximation ability Jianfei Li Han Feng Ding-Xuan Zhou 17 3 0 19 Oct 2022
Transformers with Learnable Activation Functions Haishuo Fang Ji-Ung Lee N. Moosavi Iryna Gurevych AI4CE 17 7 0 30 Aug 2022
Convolutional Neural Network (CNN) to reduce construction loss in JPEG compression caused by Discrete Fourier Transform (DFT) S. Kunwar 11 0 0 26 Aug 2022
Deep Analysis of Visual Product Reviews Chandranath Adak S. Chattopadhyay Muhammad Saqib 9 2 0 19 Jul 2022
$A New Multifractal-based Deep Learning Model for Text Mining$ A New Multifractal-based Deep Learning Model for Text Mining Zhenhua Wang Ming Ren Dong Gao AI4CE 6 16 0 27 Nov 2021
Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks Tomasz Szandała 59 274 0 15 Oct 2020
Dynamic ReLU Yinpeng Chen Xiyang Dai Mengchen Liu Dongdong Chen Lu Yuan Zicheng Liu 177 162 0 22 Mar 2020
Non-linear Neurons with Human-like Apical Dendrite Activations Mariana-Iuliana Georgescu Radu Tudor Ionescu Nicolae-Cătălin Ristea N. Sebe 8 19 0 02 Feb 2020
Learning Neural Activations F. Minhas Amina Asif 19 2 0 27 Dec 2019
diffGrad: An Optimization Method for Convolutional Neural Networks S. Dubey Soumendu Chakraborty S. K. Roy Snehasis Mukherjee S. Singh B. B. Chaudhuri ODL 83 183 0 12 Sep 2019
LiSHT: Non-Parametric Linearly Scaled Hyperbolic Tangent Activation Function for Neural Networks S. K. Roy Suvojit Manna S. Dubey B. B. Chaudhuri 11 49 0 01 Jan 2019
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 950 20,561 0 17 Apr 2017