A Survey of Transformers

8 June 2021

Tianyang Lin

Yuxin Wang

Xiangyang Liu

Xipeng Qiu

ViT

ArXiv PDF HTML

Papers citing "A Survey of Transformers"

50 / 347 papers shown

Title
Reinforcement Learning as an Improvement Heuristic for Real-World Production Scheduling Arthur Muller Lukas Vollenkemper OffRL 13 0 0 18 Sep 2024
RoboMorph: In-Context Meta-Learning for Robot Dynamics Modeling Manuel Bianchi Bazzi Asad Ali Shahid Christopher Agia J. I. Alora Marco Forgione Dario Piga Francesco Braghin Marco Pavone L. Roveda LM&Ro AI4CE 30 0 0 18 Sep 2024
Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics Wenqing Zhang Junming Huang Ruotong Wang Changsong Wei Wenqian Huang Yuxin Qiao Mamba 32 10 0 13 Sep 2024
A Survey of Anomaly Detection in In-Vehicle Networks Ovgu Ozdemir M. Tuğberk İşyapar Pınar Karagöz Klaus Werner Schmidt Demet Demir N. A. Karagoz 25 0 0 11 Sep 2024
Improving Anomalous Sound Detection via Low-Rank Adaptation Fine-Tuning of Pre-Trained Audio Models Xinhu Zheng Anbai Jiang Bing Han Yanmin Qian Pingyi Fan Jia Liu Wei-Qiang Zhang 25 2 0 11 Sep 2024
DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Models Maryam Akhavan Aghdam Hongpeng Jin Yanzhao Wu MoE 23 3 0 10 Sep 2024
The Role of Fibration Symmetries in Geometric Deep Learning Osvaldo Velarde Lucas Parra Paolo Boldi Hernan Makse FedML AI4CE 39 2 0 28 Aug 2024
Towards reliable respiratory disease diagnosis based on cough sounds and vision transformers Qian Wang Zhaoyang Bu Jiaxuan Mao Wenyu Zhu Jingya Zhao Wei Du Guochao Shi Min Zhou Si Chen Jieming Qu MedIm 39 0 0 28 Aug 2024
Legilimens: Practical and Unified Content Moderation for Large Language Model Services Jialin Wu Jiangyi Deng Shengyuan Pang Yanjiao Chen Jiayang Xu Xinfeng Li Wenyuan Xu 37 6 0 28 Aug 2024
Predictability and Causality in Spanish and English Natural Language Generation Andrea Busto-Castiñeira Francisco J. González Castaño Silvia García-Méndez Francisco de Arriba-Pérez CML 51 1 0 26 Aug 2024
LLM-3D Print: Large Language Models To Monitor and Control 3D Printing Yayati Jadhav P. Pak Amir Barati Farimani AI4CE 86 8 0 26 Aug 2024
OccMamba: Semantic Occupancy Prediction with State Space Models Heng Li Yuenan Hou Xiaohan Xing Xiao Sun Xiao Sun Yanyong Zhang Mamba 50 4 0 19 Aug 2024
HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration Pratyush Dhingra J. Doppa P. Pande 30 1 0 06 Aug 2024
radarODE: An ODE-Embedded Deep Learning Model for Contactless ECG Reconstruction from Millimeter-Wave Radar Yizheng Wu Jun Cen Xingyi Li Rui Yang Yutao Yue Guo-Shing Lin 41 3 0 03 Aug 2024
Empowering Clinicians with Medical Decision Transformers: A Framework for Sepsis Treatment A. Rahman Pranav Agarwal R. Noumeir P. Jouvet Vincent Michalski Samira Ebrahimi Kahou OffRL 24 0 0 28 Jul 2024
Domain Adaptation of Visual Policies with a Single Demonstration Weiyao Wang Gregory D. Hager 38 0 0 23 Jul 2024
Deep multimodal saliency parcellation of cerebellar pathways: linking microstructure and individual function through explainable multitask learning Ari Tchetchenian L. Zekelman Yuqian Chen J. Rushmore Fan Zhang ... N. Makris Yogesh Rathi Erik H. W. Meijering Yang Song L. O’Donnell 38 0 0 21 Jul 2024
Rethinking Transformer-based Multi-document Summarization: An Empirical Investigation Congbo Ma Wei Emma Zhang Dileepa Pitawela Haojie Zhuang Yanfeng Shu 19 0 0 16 Jul 2024
Relation DETR: Exploring Explicit Position Relation Prior for Object Detection Xiuquan Hou Mei-qin Liu Senlin Zhang Ping Wei Badong Chen Xuguang Lan ViT 50 15 0 16 Jul 2024
Graph Transformers: A Survey Ahsan Shehzad Feng Xia Shagufta Abid Ciyuan Peng Shuo Yu Dongyu Zhang Karin Verspoor AI4CE 34 9 0 13 Jul 2024
Toto: Time Series Optimized Transformer for Observability Ben Cohen E. Khwaja Kan Wang Charles Masson Elise Ramé Youssef Doubli Othmane Abou-Amal AI4TS 38 3 0 10 Jul 2024
Large Language Model-Augmented Auto-Delineation of Treatment Target Volume in Radiation Therapy Praveenbalaji Rajendran Yong Yang Thomas R. Niedermayr Michael Gensheimer Beth Beadle Quynh Le Lei Xing Xianjin Dai 37 2 0 10 Jul 2024
Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoT Tianheng Ling Chao Qian Gregor Schiele AI4TS MQ 24 1 0 06 Jul 2024
Using LLMs to label medical papers according to the CIViC evidence model Markus Hisch Xing David Wang 47 0 0 05 Jul 2024
On the Anatomy of Attention Nikhil Khatri Tuomas Laakkonen Jonathon Liu Vincent Wang-Ma'scianica 3DV 48 1 0 02 Jul 2024
Papez: Resource-Efficient Speech Separation with Auditory Working Memory Hyunseok Oh Juheon Yi Youngki Lee 19 2 0 01 Jul 2024
Transformer-based Image and Video Inpainting: Current Challenges and Future Directions Omar Elharrouss Rafat Damseh Abdelkader Nasreddine Belkacem E. Badidi Abderrahmane Lakas ViT 32 2 0 28 Jun 2024
When Search Engine Services meet Large Language Models: Visions and Challenges Haoyi Xiong Jiang Bian Yuchen Li Xuhong Li Jundong Li Shuaiqiang Wang Dawei Yin Sumi Helal 53 28 0 28 Jun 2024
Unveiling and Controlling Anomalous Attention Distribution in Transformers Ruiqing Yan Xingbo Du Haoyu Deng Linghan Zheng Qiuzhuang Sun Jifang Hu Yuhang Shao Penghao Jiang Jinrong Jiang Lian Zhao 38 1 0 26 Jun 2024
V-RECS, a Low-Cost LLM4VIS Recommender with Explanations, Captioning and Suggestions L. Podo M. Angelini Paola Velardi 45 1 0 21 Jun 2024
Elliptical Attention Stefan K. Nielsen Laziz U. Abdullaev R. Teo Tan M. Nguyen 23 3 0 19 Jun 2024
Delay Embedding Theory of Neural Sequence Models Mitchell Ostrow Adam J. Eisen Ila Fiete AI4TS 29 2 0 17 Jun 2024
Efficient Multi-View Fusion and Flexible Adaptation to View Missing in Cardiovascular System Signals Qihan Hu Daomiao Wang Hong Wu Jian Liu Cuiwei Yang 38 0 0 13 Jun 2024
Towards Generalized Hydrological Forecasting using Transformer Models for 120-Hour Streamflow Prediction B. Demiray Ibrahim Demir AI4TS 26 1 0 11 Jun 2024
Deep learning for precipitation nowcasting: A survey from the perspective of time series forecasting Sojung An Tae-Jin Oh Eunha Sohn Donghyun Kim AI4TS 54 7 0 07 Jun 2024
Iteration Head: A Mechanistic Study of Chain-of-Thought Vivien A. Cabannes Charles Arnal Wassim Bouaziz Alice Yang Francois Charton Julia Kempe LRM 27 7 0 04 Jun 2024
RNNs, CNNs and Transformers in Human Action Recognition: A Survey and a Hybrid Model Khaled Alomar Halil Ibrahim Aysel Xiaohao Cai MedIm ViT 43 7 0 02 Jun 2024
SoK: Leveraging Transformers for Malware Analysis Pradip Kunwar Kshitiz Aryal Maanak Gupta Mahmoud Abdelsalam Elisa Bertino 90 0 0 27 May 2024
Activator: GLU Activation Function as the Core Component of a Vision Transformer Abdullah Nazhat Abdullah Tarkan Aydin ViT 43 0 0 24 May 2024
Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges Jonas Becker Jan Philip Wahle Bela Gipp Terry Ruas 28 9 0 24 May 2024
Spectraformer: A Unified Random Feature Framework for Transformer Duke Nguyen Aditya Joshi Flora D. Salim 34 0 0 24 May 2024
Attention as an RNN Leo Feng Frederick Tung Hossein Hajimirsadeghi Mohamed Osama Ahmed Yoshua Bengio Greg Mori GNN AI4TS 53 8 0 22 May 2024
Continuous Sign Language Recognition with Adapted Conformer via Unsupervised Pretraining Neena Aloysius M. Geetha Prema Nedungadi SLR 21 2 0 20 May 2024
Large Language Models for Medicine: A Survey Yanxin Zheng Wensheng Gan Zefeng Chen Zhenlian Qi Qian Liang Philip S. Yu LM&MA 23 15 0 20 May 2024
Your Transformer is Secretly Linear Anton Razzhigaev Matvey Mikhalchuk Elizaveta Goncharova Nikolai Gerasimenko Ivan V. Oseledets Denis Dimitrov Andrey Kuznetsov 32 4 0 19 May 2024
Representation Learning of Daily Movement Data Using Text Encoders Alexander Capstick Tianyu Cui Yu Chen Payam Barnaghi AI4TS 28 2 0 07 May 2024
Transformer models classify random numbers Rishabh Goel YiZi Xiao Ramin Ramezani 27 1 0 06 May 2024
What makes Models Compositional? A Theoretical View: With Supplement Parikshit Ram Tim Klinger Alexander G. Gray CoGe 36 6 0 02 May 2024
FRAME: A Modular Framework for Autonomous Map-merging: Advancements in the Field Nikolaos Stathoulopoulos B. Lindqvist A. Koval Ali-akbar Agha-mohammadi G. Nikolakopoulos 3DPC 29 5 0 27 Apr 2024
Deep Models for Multi-View 3D Object Recognition: A Review M. Alzahrani Muhammad Usman S. Jarraya Saeed Anwar Tarek Helmy 16 4 0 23 Apr 2024