A General Survey on Attention Mechanisms in Deep Learning

27 March 2022

Papers citing "A General Survey on Attention Mechanisms in Deep Learning"

36 / 36 papers shown

Title
GeloVec: Higher Dimensional Geometric Smoothing for Coherent Visual Feature Extraction in Image Segmentation Boris Kriuk Matey Yordanov 35 0 0 02 May 2025
Bringing together invertible UNets with invertible attention modules for memory-efficient diffusion models Karan Jain Mohammad Nayeem Teli MedIm 24 0 0 15 Apr 2025
Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection Bo Leng Ran Yu Wei Han Lu Xiong Zhuoren Li Hailong Huang 41 0 0 25 Mar 2025
A HEART for the environment: Transformer-Based Spatiotemporal Modeling for Air Quality Prediction Norbert Bodendorfer 62 1 0 26 Feb 2025
Demographic Attributes Prediction from Speech Using WavLM Embeddings Yuchen Yang Thomas Thebaud Najim Dehak 49 0 0 17 Feb 2025
AoI-Sensitive Data Forwarding with Distributed Beamforming in UAV-Assisted IoT Zifan Lang Guixia Liu Geng Sun Jiahui Li Zemin Sun Jiacheng Wang Victor C. M. Leung 80 0 0 13 Feb 2025
Towards Fault Tolerance in Multi-Agent Reinforcement Learning Yuchen Shi Huaxin Pei Liang Feng Yi Zhang D. Yao 70 0 0 30 Nov 2024
RotCAtt-TransUNet++: Novel Deep Neural Network for Sophisticated Cardiac Segmentation Quoc-Bao Nguyen-Le Tuan-Hy Le Anh-Triet Do Quoc-Huy Trinh ViT MedIm 32 0 0 09 Sep 2024
Attention Heads of Large Language Models: A Survey Zifan Zheng Yezhaohui Wang Yuxin Huang Shichao Song Mingchuan Yang Bo Tang Feiyu Xiong Zhiyu Li LRM 58 21 0 05 Sep 2024
BiLSTM and Attention-Based Modulation Classification of Realistic Wireless Signals Rohit Udaiwal N. Baishya Yash Gupta B. R. Manoj 32 0 0 14 Aug 2024
What comes after transformers? -- A selective survey connecting ideas in deep learning Johannes Schneider AI4CE 35 2 0 01 Aug 2024
Attention Beats Linear for Fast Implicit Neural Representation Generation Shuyi Zhang Ke Liu Jingjun Gu Xiaoxu Cai Zhihua Wang Jiajun Bu Haishuai Wang 42 1 0 22 Jul 2024
A Survey of Transformer Enabled Time Series Synthesis Alexander Sommers Logan Cummins Sudip Mittal Shahram Rahimi Maria Seale Joseph Jaboure Thomas Arnold AI4TS 37 2 0 04 Jun 2024
RNNs, CNNs and Transformers in Human Action Recognition: A Survey and a Hybrid Model Khaled Alomar Halil Ibrahim Aysel Xiaohao Cai MedIm ViT 35 7 0 02 Jun 2024
Applications of interpretable deep learning in neuroimaging: a comprehensive review Lindsay Munroe Mariana da Silva Faezeh Heidari I. Grigorescu Simon Dahan E. C. Robinson Maria Deprez Po-Wah So AI4CE 41 5 0 30 May 2024
A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversation Iveta Becková Stefan Pócos Giulia Belgiovine Marco Matarese A. Sciutti Carlo Mazzola Carlo Mazzola 40 0 0 20 May 2024
Developing An Attention-Based Ensemble Learning Framework for Financial Portfolio Optimisation Zhenglong Li Vincent Tam 35 0 0 13 Apr 2024
Machine Learning and Data Analysis Using Posets: A Survey Arnauld Mesinga Mwafise AI4CE 33 0 0 03 Apr 2024
Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights Moein Heidari Reza Azad Sina Ghorbani Kolahi René Arimond Leon Niggemeier ... Afshin Bozorgpour Ehsan Khodapanah Aghdam A. Kazerouni I. Hacihaliloglu Dorit Merhof 41 7 0 28 Mar 2024
Revisiting Generative Adversarial Networks for Binary Semantic Segmentation on Imbalanced Datasets Lei Xu M. Gabbouj GAN 20 1 0 03 Feb 2024
SocraSynth: Multi-LLM Reasoning with Conditional Statistics Edward Y. Chang LLMAG LRM 25 7 0 19 Jan 2024
Brain Tumor Segmentation Based on Deep Learning, Attention Mechanisms, and Energy-Based Uncertainty Prediction Zachary Schwehr Sriman Achanta 26 2 0 31 Dec 2023
WATUNet: A Deep Neural Network for Segmentation of Volumetric Sweep Imaging Ultrasound Donya Khaledyan T. Marini Avice Oconnell Steven Meng J. Kan Galen Brennan Yu T Zhao Timothy M. Baran Kevin J. Parker 16 2 0 17 Nov 2023
A Machine Learning-oriented Survey on Tiny Machine Learning Luigi Capogrosso Federico Cunico D. Cheng Franco Fummi Marco Cristani SyDa MU 24 33 0 21 Sep 2023
FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning T. Shaik Xiaohui Tao Lin Li Haoran Xie Taotao Cai Xiaofeng Zhu Qingyuan Li MU 26 13 0 19 Sep 2023
Empowering Precision Medicine: AI-Driven Schizophrenia Diagnosis via EEG Signals: A Comprehensive Review from 2002-2023 M. Jafari Delaram Sadeghi A. Shoeibi Hamid Alinejad-Rokny Amin Beheshti David López-García Zhaolin Chen U. Acharya Juan M Gorriz 23 19 0 14 Sep 2023
Track Mix Generation on Music Streaming Services using Transformers Walid Bendada Théo Bontempelli M. Morlon Benjamin Chapus Thibault Cador Thomas Bouabça Guillaume Salha-Galvan 28 8 0 06 Jul 2023
Assessment of few-hits machine learning classification algorithms for low energy physics in liquid argon detectors R. Moretti Michele Grossi M. Biassoni Andrea Giachero Marco Rossi D. Guffanti Danilo Labranca F. Terranova S. Vallecorsa 17 4 0 16 May 2023
Neural Attention Forests: Transformer-Based Forest Improvement A. Konstantinov Lev V. Utkin A. Lukashin Vladimir Mulukha 29 4 0 12 Apr 2023
A Survey of Deep Learning: From Activations to Transformers Johannes Schneider Michalis Vlachos ViT MedIm AI4TS AI4CE 46 9 0 01 Feb 2023
GIPA: A General Information Propagation Algorithm for Graph Learning Houyi Li Zhihong Chen Zhao Li Qinkai Zheng Peng Zhang Shuigeng Zhou 43 0 0 19 Jan 2023
Encoder-Decoder Model for Suffix Prediction in Predictive Monitoring Efrén Rama-Maneiro Pablo Monteagudo-Lago J. Vidal Manuel Lama 22 1 0 29 Nov 2022
LARF: Two-level Attention-based Random Forests with a Mixture of Contamination Models A. Konstantinov Lev V. Utkin 30 0 0 11 Oct 2022
Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design Hongxiang Fan Thomas C. P. Chau Stylianos I. Venieris Royson Lee Alexandros Kouris Wayne Luk Nicholas D. Lane Mohamed S. Abdelfattah 32 56 0 20 Sep 2022
Attention and Self-Attention in Random Forests Lev V. Utkin A. Konstantinov 32 3 0 09 Jul 2022
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 216 7,923 0 17 Aug 2015