A survey on modern trainable activation functions

2 May 2020

Papers citing "A survey on modern trainable activation functions"

50 / 65 papers shown

Title
Preserving Plasticity in Continual Learning with Adaptive Linearity Injection Seyed Roozbeh Razavi Rohani Khashayar Khajavi Wesley Chung Mo Chen Sharan Vaswani CLL AI4CE 25 0 0 14 May 2025
Kolmogorov-Arnold Network for Transistor Compact Modeling Rodion Novkin Hussam Amrouch 46 0 0 19 Mar 2025
Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild Damien Teney Liangze Jiang Florin Gogianu Ehsan Abbasnejad 169 0 0 13 Mar 2025
Designing DNNs for a trade-off between robustness and processing performance in embedded devices Jon Gutiérrez-Zaballa Koldo Basterretxea Javier Echanobe AAML 101 2 0 04 Dec 2024
Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptrons Farhad Pourkamali-Anaraki 35 5 0 16 Sep 2024
Controlled Learning of Pointwise Nonlinearities in Neural-Network-Like Architectures Michael Unser Alexis Goujon Stanislas Ducotterd 29 2 0 23 Aug 2024
What comes after transformers? -- A selective survey connecting ideas in deep learning Johannes Schneider AI4CE 40 2 0 01 Aug 2024
A Survey on Universal Approximation Theorems Midhun T. Augustine 31 4 0 17 Jul 2024
Don't Fear Peculiar Activation Functions: EUAF and Beyond Qianchao Wang Shijun Zhang Dong Zeng Zhaoheng Xie Hengtao Guo Feng-Lei Fan Tieyong Zeng 39 3 0 12 Jul 2024
Trainable Highly-expressive Activation Functions Irit Chelly Shahaf E. Finder Shira Ifergane O. Freifeld 31 4 0 10 Jul 2024
DiGRAF: Diffeomorphic Graph-Adaptive Activation Function Krishna Sri Ipsit Mantri Xinzhi Wang Carola-Bibiane Schönlieb Bruno Ribeiro Beatrice Bevilacqua Moshe Eliasof GNN 49 1 0 02 Jul 2024
Improving Quaternion Neural Networks with Quaternionic Activation Functions Johannes Pöppelbaum Andreas Schwung 26 1 0 24 Jun 2024
fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions Alireza Afzal Aghaei 40 47 0 11 Jun 2024
Semiring Activation in Neural Networks B. Smets Peter D. Donker Jim W. Portegies LLMSV 21 0 0 29 May 2024
Expanded Gating Ranges Improve Activation Functions Allen Hao Huang AI4CE 14 1 0 25 May 2024
Function Extrapolation with Neural Networks and Its Application for Manifolds Guy Hay N. Sharon 26 0 0 17 May 2024
Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey Guoping Xu Xiaxia Wang Xinglong Wu Xuesong Leng Yongchao Xu 3DPC 36 8 0 02 May 2024
Efficient optimal dispersed Haar-like filters for face detection Zeinab Sedaghatjoo Hossein Hosseinzadeh A. Shirzadi 29 0 0 16 Apr 2024
Nonlinearity Enhanced Adaptive Activation Functions David Yevick 25 1 0 29 Mar 2024
Can physical information aid the generalization ability of Neural Networks for hydraulic modeling? Gianmarco Guglielmo A. Montessori J. Tucny M. Rocca P. Prestininzi AI4CE 23 3 0 13 Mar 2024
Exploring the Relationship: Transformative Adaptive Activation Functions in Comparison to Other Activation Functions Vladimír Kunc 24 1 0 14 Feb 2024
Graph Structure Inference with BAM: Introducing the Bilinear Attention Mechanism Philipp Froehlich Heinz Koeppl GNN 29 1 0 12 Feb 2024
Adaptive Activation Functions for Predictive Modeling with Sparse Experimental Data Farhad Pourkamali-Anaraki Tahamina Nasrin Robert E. Jensen Amy M. Peterson Christopher J. Hansen 24 7 0 08 Feb 2024
HgbNet: predicting hemoglobin level/anemia degree from EHR data Zhuo Zhi M. Elbadawi Adam Daneshmend Mine Orlu Abdul Basit Andreas Demosthenous Miguel R. D. Rodrigues 35 3 0 22 Jan 2024
HOSC: A Periodic Activation Function for Preserving Sharp Features in Implicit Neural Representations Danzel Serrano Jakub Szymkowiak Przemyslaw Musialski 13 5 0 20 Jan 2024
Improved weight initialization for deep and narrow feedforward neural network Hyunwoo Lee Yunho Kim Seungyeop Yang Hayoung Choi ODL 25 3 0 07 Nov 2023
Toward Flare-Free Images: A Survey Yousef Kotp Marwan Torki 43 3 0 22 Oct 2023
Privacy-Preserving Encrypted Low-Dose CT Denoising Ziyuan Yang Huijie Huangfu Maosong Ran Zhiwen Wang Hui Yu Yi Zhang 31 0 0 13 Oct 2023
A Machine Learning-oriented Survey on Tiny Machine Learning Luigi Capogrosso Federico Cunico D. Cheng Franco Fummi Marco Cristani SyDa MU 32 33 0 21 Sep 2023
Hidden Classification Layers: Enhancing linear separability between classes in neural networks layers Andrea Apicella Francesco Isgrò R. Prevete 8 0 0 09 Jun 2023
A Unified Concept-Based System for Local, Global, and Misclassification Explanations Fatemeh Aghaeipoor D. Asgarian Mohammad Sabokrou FAtt 27 0 0 06 Jun 2023
A Bio-Inspired Chaos Sensor Model Based on the Perceptron Neural Network: Machine Learning Concept and Application for Computational Neuro-Science Andrei Velichko P. Boriskov M. Belyaev V. Putrolaynen 7 5 0 03 Jun 2023
ErfReLU: Adaptive Activation Function for Deep Neural Network Ashish Rajanand Pradeep Singh 15 11 0 02 Jun 2023
Stuttering Detection Using Speaker Representations and Self-supervised Contextual Embeddings S. A. Sheikh Md. Sahidullah F. Hirsch Slim Ouni 30 6 0 01 Jun 2023
GELU Activation Function in Deep Learning: A Comprehensive Mathematical Analysis and Performance Minhyeok Lee 18 29 0 20 May 2023
Learning Activation Functions for Sparse Neural Networks Mohammad Loni Aditya Mohan Mehdi Asadi Marius Lindauer 21 4 0 18 May 2023
Improving Classification Neural Networks by using Absolute activation function (MNIST/LeNET-5 example) Oleg I.Berngardt 14 2 0 23 Apr 2023
Bayesian optimization for sparse neural networks with trainable activation functions M. Fakhfakh Lotfi Chaari 15 2 0 10 Apr 2023
A Survey of Deep Learning: From Activations to Transformers Johannes Schneider Michalis Vlachos ViT MedIm AI4TS AI4CE 50 9 0 01 Feb 2023
Efficient Activation Function Optimization through Surrogate Modeling G. Bingham Risto Miikkulainen 16 2 0 13 Jan 2023
Evaluating CNN with Oscillatory Activation Function Jeevanshi Sharma 13 1 0 13 Nov 2022
A deep scalable neural architecture for soil properties estimation from spectral information Flavio Piccoli M. Rossini R. Colombo Raimondo Schettini Paolo Napoletano 11 4 0 26 Oct 2022
A connection between probability, physics and neural networks Sascha Ranftl PINN 17 9 0 26 Sep 2022
Deep Learning Models for Detecting Malware Attacks Pascal Maniriho A. N. Mahmood M. Chowdhury AAML 16 5 0 08 Sep 2022
How important are activation functions in regression and classification? A survey, performance comparison, and future directions Ameya Dilip Jagtap George Karniadakis AI4CE 31 71 0 06 Sep 2022
Transmission Neural Networks: From Virus Spread Models to Neural Networks Shuang Gao P. Caines GNN MedIm 11 1 0 07 Aug 2022
End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge S. A. Sheikh Md. Sahidullah F. Hirsch Slim Ouni SSL 14 9 0 20 Jul 2022
Neural Networks with A La Carte Selection of Activation Functions Moshe Sipper 13 7 0 24 Jun 2022
Energy reconstruction for large liquid scintillator detectors with machine learning techniques: aggregated features approach Arsenii Gavrikov Y. Malyshkin F. Ratnikov 6 7 0 17 Jun 2022
Neural Network Architecture Beyond Width and Depth Zuowei Shen Haizhao Yang Shijun Zhang 3DV MDE 33 13 0 19 May 2022