A Survey of Transformers

8 June 2021

Tianyang Lin

Yuxin Wang

Xiangyang Liu

Xipeng Qiu

ViT

ArXiv PDF HTML

Papers citing "A Survey of Transformers"

50 / 347 papers shown

Title
F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks Xiangxiang Gao Wei-wei Zhu Jiasheng Gao Congrui Yin VLM 26 12 0 21 May 2023
Less is More! A slim architecture for optimal language translation Luca Herranz-Celotti E. Rrapaj 28 0 0 18 May 2023
HICO-DET-SG and V-COCO-SG: New Data Splits for Evaluating the Systematic Generalization Performance of Human-Object Interaction Detection Models Kenta Takemoto Moyuru Yamada Tomotake Sasaki H. Akima 37 0 0 17 May 2023
Assessment of few-hits machine learning classification algorithms for low energy physics in liquid argon detectors R. Moretti Michele Grossi M. Biassoni Andrea Giachero Marco Rossi D. Guffanti Danilo Labranca F. Terranova S. Vallecorsa 27 4 0 16 May 2023
Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives Qiushi Sun Chengcheng Han Nuo Chen Renyu Zhu Jing Gong Xiang Li Ming Gao VLM 27 8 0 14 May 2023
ChatGPT-Like Large-Scale Foundation Models for Prognostics and Health Management: A Survey and Roadmaps Yanfang Li Huan Wang Muxia Sun LM&MA AI4TS AI4CE 29 46 0 10 May 2023
The emergence of clusters in self-attention dynamics Borjan Geshkovski Cyril Letrouit Yury Polyanskiy Philippe Rigollet 22 46 0 09 May 2023
Mlinear: Rethink the Linear Model for Time-series Forecasting Wei Li Xiangxu Meng Chuhao Chen Jianing Chen AI4TS 23 6 0 08 May 2023
NER-to-MRC: Named-Entity Recognition Completely Solving as Machine Reading Comprehension Yuxiang Zhang Junjie Wang Xinyu Zhu Tetsuya Sakai Hayato Yamana 24 2 0 06 May 2023
SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish Dmytro Kalpakchi Johan Boye SyDa 24 3 0 27 Apr 2023
NoiseTrans: Point Cloud Denoising with Transformers Guangzhe Hou G. Qin Minghui Sun Yanhua Liang Jie Yan Zhonghan Zhang 3DPC ViT 20 2 0 24 Apr 2023
A Review of Deep Learning for Video Captioning Moloud Abdar Meenakshi Kollati Swaraja Kuraparthi Farhad Pourpanah Daniel J. McDuff ... Shuicheng Yan Abduallah A. Mohamed Abbas Khosravi Erik Cambria Fatih Porikli 3DV 32 21 0 22 Apr 2023
DropDim: A Regularization Method for Transformer Networks Hao Zhang Dan Qu Kejia Shao Xu Yang 28 12 0 20 Apr 2023
Transformer-Based Visual Segmentation: A Survey Xiangtai Li Henghui Ding Haobo Yuan Wenwei Zhang Jiangmiao Pang Guangliang Cheng Kai-xiang Chen Ziwei Liu Chen Change Loy ViT MedIm 42 132 0 19 Apr 2023
Prak: An automatic phonetic alignment tool for Czech V. Hanzl Adléta Hanzlová 22 0 0 17 Apr 2023
Experts' cognition-driven safe noisy labels learning for precise segmentation of residual tumor in breast cancer Yongquan Yang Jie Chen Yani Wei Mohammad H. Alobaidi Hong Bu NoLa 45 1 0 13 Apr 2023
SELFormer: Molecular Representation Learning via SELFIES Language Models Atakan Yüksel Erva Ulusoy Atabey Ünlü Tunca Dogan 25 55 0 10 Apr 2023
Transformer Utilization in Medical Image Segmentation Networks Saikat Roy Gregor Koehler Michael Baumgartner Constantin Ulrich Jens Petersen Fabian Isensee Klaus Maier-Hein ViT MedIm 25 2 0 09 Apr 2023
Fairness through Aleatoric Uncertainty Anique Tahir Lu Cheng Huan Liu 45 11 0 07 Apr 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 41 0 07 Apr 2023
Automatic ICD-10 Code Association: A Challenging Task on French Clinical Texts Yakini Tchouka Jean-François Couchot David Laiymani Philippe Selles Azzedine Rahmani 25 3 0 06 Apr 2023
Classification of integers based on residue classes via modern deep learning algorithms Dangwei Wu Jing Yang Mian Umair Ahsan Kai Wang 36 1 0 03 Apr 2023
TabRet: Pre-training Transformer-based Tabular Models for Unseen Columns Soma Onishi Kenta Oono Kohei Hayashi LMTD 29 13 0 28 Mar 2023
It is all Connected: A New Graph Formulation for Spatio-Temporal Forecasting Lars Odegaard Bentsen N. Warakagoda R. Stenbro P. Engelstad AI4TS 15 1 0 23 Mar 2023
Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset Thanh-Dung Le P. Jouvet R. Noumeir MoE MedIm 72 5 0 22 Mar 2023
Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control Nathan Leroux Jan Finkbeiner Emre Neftci 30 9 0 21 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 42 47 0 21 Mar 2023
Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review Sibo Cheng César Quilodrán-Casas Said Ouala A. Farchi Che Liu ... Weiping Ding Yike Guo A. Carrassi Marc Bocquet Rossella Arcucci AI4CE 32 124 0 18 Mar 2023
Transformers and Ensemble methods: A solution for Hate Speech Detection in Arabic languages Angel Felipe Magnossão de Paula Imene Bensalem Paolo Rosso Wajdi Zaghouani 18 5 0 17 Mar 2023
Diffusing Gaussian Mixtures for Generating Categorical Data Florence Regol Mark J. Coates DiffM 33 5 0 08 Mar 2023
A Message Passing Perspective on Learning Dynamics of Contrastive Learning Yifei Wang Qi Zhang Tianqi Du Jiansheng Yang Zhouchen Lin Yisen Wang SSL 32 18 0 08 Mar 2023
A Survey on Long Text Modeling with Transformers Zican Dong Tianyi Tang Lunyi Li Wayne Xin Zhao VLM 21 54 0 28 Feb 2023
Multi-Layer Attention-Based Explainability via Transformers for Tabular Data Andrea Trevino Gavito Diego Klabjan J. Utke LMTD 23 3 0 28 Feb 2023
A comparative assessment of deep learning models for day-ahead load forecasting: Investigating key accuracy drivers Sotiris Pelekis Ioannis-Konstantinos Seisopoulos Evangelos Spiliotis Theodosios Pountridis Evangelos Karakolis S. Mouzakitis D. Askounis 23 16 0 23 Feb 2023
MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notes Ankur Samanta Mark Karlov Meghna Ravikumar Christian McIntosh Clarke Jayakumar Rajadas Kaveh Hassani 26 1 0 21 Feb 2023
ChatGPT: Jack of all trades, master of none Jan Kocoñ Igor Cichecki Oliwier Kaszyca Mateusz Kochanek Dominika Szydło ... Maciej Piasecki Lukasz Radliñski Konrad Wojtasik Stanislaw Wo'zniak Przemyslaw Kazienko AI4MH 37 527 0 21 Feb 2023
Mask-guided BERT for Few Shot Text Classification Wenxiong Liao Zheng Liu Haixing Dai Zihao Wu Yiyang Zhang ... Dajiang Zhu Tianming Liu Sheng Li Xiang Li Hongmin Cai VLM 47 39 0 21 Feb 2023
Energy Transformer Benjamin Hoover Yuchen Liang Bao Pham Rameswar Panda Hendrik Strobelt Duen Horng Chau Mohammed J. Zaki Dmitry Krotov ViT 28 42 0 14 Feb 2023
From paintbrush to pixel: A review of deep neural networks in AI-generated art Anne-Sofie Maerten Derya Soydaner 37 23 0 14 Feb 2023
Transformer models: an introduction and catalog X. Amatriain Ananth Sankar Jie Bing Praveen Kumar Bodigutla Timothy J. Hazen Michaeel Kazi 24 50 0 12 Feb 2023
Local spectral attention for full-band speech enhancement Zhongshu Hou Qi Hu Kai-Jyun Chen Jing Lu 28 0 0 11 Feb 2023
On Achieving Privacy-Preserving State-of-the-Art Edge Intelligence Daphnee Chabal Dolly Sapra Z. Mann 22 3 0 10 Feb 2023
Attending to Graph Transformers Luis Muller Mikhail Galkin Christopher Morris Ladislav Rampášek 49 86 0 08 Feb 2023
PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer Zitong Yu Yuming Shen Jingang Shi Hengshuang Zhao Yawen Cui Jiehua Zhang Philip H. S. Torr Guoying Zhao ViT MedIm 29 80 0 07 Feb 2023
Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognition Binqian Xu Xiangbo Shu 15 55 0 05 Feb 2023
Predicting Visit Cost of Obstructive Sleep Apnea using Electronic Healthcare Records with Transformer Zhaoyang Chen Lina Siltala-Li Mikko Lassila Pekka Malo Eeva Vilkkumaa T. Saaresranta A. Virkki 16 4 0 28 Jan 2023
Enhancing Face Recognition with Latent Space Data Augmentation and Facial Posture Reconstruction Soroush Hashemifar Abdolreza Marefat Javad Hassannataj Joloudari H. Hassanpour CVBM 31 11 0 27 Jan 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin Tim Dettmers Michael Diskin Alexander Borzunov MoE 30 31 0 27 Jan 2023
Deep Quantum Error Correction Yoni Choukroun Lior Wolf 27 8 0 27 Jan 2023
NLP as a Lens for Causal Analysis and Perception Mining to Infer Mental Health on Social Media Muskan Garg Chandni Saxena Usman Naseem Bonnie J. Dorr 36 1 0 26 Jan 2023