Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark

29 September 2021

Papers citing "Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark"

50 / 94 papers shown

Title
An AI-driven framework for the prediction of personalised health response to air pollution Nazanin Zounemat Kermani Sadjad Naderi Claire H. Dilliway C. Heaney Shrreya Behll ... Marc Chadeau-Hyam Fangxin Fang Ian M. Adcock Kian Fan Chung Christopher C. Pain 20 0 0 15 May 2025
Mask-PINNs: Regulating Feature Distributions in Physics-Informed Neural Networks Feilong Jiang Xiaonan Hou Jianqiao Ye Min Xia OOD PINN 42 0 0 09 May 2025
Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations Davide Sartor Alberto Sinigaglia Gian Antonio Susto 34 0 0 05 May 2025
Online Multivariate Regularized Distributional Regression for High-dimensional Probabilistic Electricity Price Forecasting Simon Hirsch 42 0 0 03 Apr 2025
Retrospective: A CORDIC Based Configurable Activation Function for NN Applications Omkar Kokane Gopal Raut Salim Ullah Mukul Lokhande Adam Teman Akash Kumar Santosh Kumar Vishvakarma 55 0 0 18 Mar 2025
Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild Damien Teney Liangze Jiang Florin Gogianu Ehsan Abbasnejad 160 0 0 13 Mar 2025
Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMs Yiheng Yang Yujie Wang Chi Ma Lei Yu Emmanuele Chersoni Chu-Ren Huang 76 0 0 26 Feb 2025
Quadrotor Neural Dead Reckoning in Periodic Trajectories Shira Massas Itzik Klein 77 0 0 25 Feb 2025
The Curious Case of Arbitrariness in Machine Learning Prakhar Ganesh Afaf Taik G. Farnadi 59 2 0 28 Jan 2025
Grounding Text-to-Image Diffusion Models for Controlled High-Quality Image Generation Ahmad Süleyman Göksel Biricik 50 2 0 15 Jan 2025
Flex-PE: Flexible and SIMD Multi-Precision Processing Element for AI Workloads Mukul Lokhande Gopal Raut Santosh Kumar Vishvakarma 70 1 0 16 Dec 2024
CESAR: A Convolutional Echo State AutoencodeR for High-Resolution Wind Forecasting Matthew Bonas Paolo Giani Paola Crippa Stefano Castruccio 67 0 0 13 Dec 2024
Quantum Pointwise Convolution: A Flexible and Scalable Approach for Neural Network Enhancement An Ning Tai-Yue Li Nan-Yow Chen 3DPC 61 0 0 02 Dec 2024
Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis Boming Miao C. Li X. U. Wang Andi Zhang Rui Sun Zizhe Wang Yao Zhu DiffM 76 0 0 25 Nov 2024
User-Aware Multilingual Abusive Content Detection in Social Media Mohammad Zia Ur Rehman Somya Mehta Kuldeep Singh Kunal Kaushik Nagendra Kumar 23 14 0 26 Oct 2024
Exploiting Memory-aware Q-distribution Prediction for Nuclear Fusion via Modern Hopfield Network Qingchuan Ma Shiao Wang Tong Zheng Xiaodong Dai Y. Wang Qingquan Yang Xiao Wang 21 1 0 11 Oct 2024
KPCA-CAM: Visual Explainability of Deep Computer Vision Models using Kernel PCA Sachin Karmani Thanushon Sivakaran Gaurav Prasad Mehmet Ali Wenbo Yang Sheyang Tang FAtt 16 3 0 30 Sep 2024
Zorro: A Flexible and Differentiable Parametric Family of Activation Functions That Extends ReLU and GELU Matias Roodschild Jorge Gotay-Sardiñas V. Jimenez A. Will 20 0 0 28 Sep 2024
Cauchy activation function and XNet Xin Li Zhihong Xia Hongkun Zhang 34 4 0 28 Sep 2024
SLIDE: A machine-learning based method for forced dynamic response estimation of multibody systems Peter Manzl Alexander Humer Qasim Khadim Johannes Gerstmayr 28 1 0 26 Sep 2024
Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning Tan-Hanh Pham Godwyll Aikins Tri Truong Kim-Doang Nguyen 29 4 0 22 Sep 2024
Hard-Label Cryptanalytic Extraction of Neural Network Models Yi Chen Xiaoyang Dong Jian Guo Yantian Shen Anyu Wang Xiaoyun Wang AAML MIACV MLAU 35 1 0 18 Sep 2024
Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptrons Farhad Pourkamali-Anaraki 35 5 0 16 Sep 2024
Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features Thomas Schnake Farnoush Rezaei Jafaria Jonas Lederer Ping Xiong Shinichi Nakajima Stefan Gugler G. Montavon Klaus-Robert Müller 40 3 0 30 Aug 2024
Activation function optimization method: Learnable series linear units (LSLUs) Chuan Feng Xi Lin Shiping Zhu Hongkang Shi Maojie Tang Hua Huang 27 0 0 28 Aug 2024
Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures Michael Unser Alexis Goujon Stanislas Ducotterd 26 2 0 23 Aug 2024
DSReLU: A Novel Dynamic Slope Function for Superior Model Training Archisman Chakraborti B. B. Chaudhuri 17 0 0 17 Aug 2024
Activations Through Extensions: A Framework To Boost Performance Of Neural Networks Chandramouli Kamanchi Sumanta Mukherjee K. Sampath Pankaj Dayama Arindam Jati Vijay Ekambaram Dzung Phan 25 0 0 07 Aug 2024
Deep Learning in Medical Image Classification from MRI-based Brain Tumor Images Xiaoyi Liu Zhuoyue Wang 32 17 0 01 Aug 2024
What comes after transformers? -- A selective survey connecting ideas in deep learning Johannes Schneider AI4CE 35 2 0 01 Aug 2024
UNIT: Backdoor Mitigation via Automated Neural Distribution Tightening Shuyang Cheng Guangyu Shen Kaiyuan Zhang Guanhong Tao Shengwei An Hanxi Guo Shiqing Ma Xiangyu Zhang AAML 25 0 0 16 Jul 2024
SwishReLU: A Unified Approach to Activation Functions for Enhanced Deep Neural Networks Performance Jamshaid Ul Rahman Rubiqa Zulfiqar Asad Khan Nimra 23 4 0 11 Jul 2024
Trainable Highly-expressive Activation Functions Irit Chelly Shahaf E. Finder Shira Ifergane O. Freifeld 29 4 0 10 Jul 2024
DiGRAF: Diffeomorphic Graph-Adaptive Activation Function Krishna Sri Ipsit Mantri Xinzhi Wang Carola-Bibiane Schönlieb Bruno Ribeiro Beatrice Bevilacqua Moshe Eliasof GNN 49 1 0 02 Jul 2024
RepAct: The Re-parameterizable Adaptive Activation Function Xian Wu Qingchuan Tao Shuang Wang 34 0 0 28 Jun 2024
A Survey on Failure Analysis and Fault Injection in AI Systems Guangba Yu Gou Tan Haojia Huang Zhenyu Zhang Pengfei Chen Roberto Natella Zibin Zheng 34 3 0 28 Jun 2024
Improving Quaternion Neural Networks with Quaternionic Activation Functions Johannes Pöppelbaum Andreas Schwung 26 1 0 24 Jun 2024
BrowNNe: Brownian Nonlocal Neurons & Activation Functions Sriram Nagaraj Truman Hickok 23 0 0 21 Jun 2024
Development of a Dual-Input Neural Model for Detecting AI-Generated Imagery Jonathan Gallagher William Pugsley 32 0 0 19 Jun 2024
The Cost of Arbitrariness for Individuals: Examining the Legal and Technical Challenges of Model Multiplicity Prakhar Ganesh Ihsan Ibrahim Daldaban Ignacio Cofone G. Farnadi 54 2 0 28 May 2024
Improving the Explain-Any-Concept by Introducing Nonlinearity to the Trainable Surrogate Model Mounes Zaval Sedat Ozer LRM 20 0 0 20 May 2024
Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation SS Sidharth Keerthana AR R. Gokul Anas KP 56 74 0 12 May 2024
Your Network May Need to Be Rewritten: Network Adversarial Based on High-Dimensional Function Graph Decomposition Xiaoyan Su Yinghao Zhu Run Li AAML 29 0 0 04 May 2024
M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training L. Meegahapola Hamza Hassoune D. Gática-Pérez 34 9 0 26 Apr 2024
Spatially Optimized Compact Deep Metric Learning Model for Similarity Search Md. Farhadul Islam Md. Tanzim Reza Meem Arafat Manab Mohammad Rakibul Hasan Mahin Sarah Zabeen Jannatun Noor 30 0 0 09 Apr 2024
Runtime Monitoring and Fault Detection for Neural Network-Controlled Systems Jianglin Lan Siyuan Zhan Ronald J. Patton Xianxian Zhao 16 1 0 24 Mar 2024
Image Classification with Rotation-Invariant Variational Quantum Circuits Paul San Sebastian Mikel Canizo Román Orús 32 4 0 22 Mar 2024
Bridging scales in multiscale bubble growth dynamics with correlated fluctuations using neural operator learning Minglei Lu Chensen Lin Martian Maxey George Karniadakis Zhen Li AI4CE 33 1 0 20 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 94 18 0 04 Mar 2024
Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions Vladimír Kunc 12 1 0 14 Feb 2024