A Survey of Transformers

8 June 2021

Tianyang Lin

Yuxin Wang

Xiangyang Liu

Xipeng Qiu

ViT

ArXiv PDF HTML

Papers citing "A Survey of Transformers"

50 / 347 papers shown

Title
SugarViT -- Multi-objective Regression of UAV Images with Vision Transformers and Deep Label Distribution Learning Demonstrated on Disease Severity Prediction in Sugar Beet Maurice Günder F. R. I. Yamati Abel Andree Barreta Alcántara Anne-Katrin Mahlein R. Sifa Christian Bauckhage 15 2 0 06 Nov 2023
What Formal Languages Can Transformers Express? A Survey Lena Strobl William Merrill Gail Weiss David Chiang Dana Angluin AI4CE 20 48 0 01 Nov 2023
The Expressibility of Polynomial based Attention Scheme Zhao-quan Song Guangyi Xu Junze Yin 32 5 0 30 Oct 2023
Knolling bot 2.0: Enhancing Object Organization with Self-supervised Graspability Estimation Yuhang Hu Zhizhuo Zhang Hod Lipson 19 0 0 30 Oct 2023
A Deep Learning Approach to Teeth Segmentation and Orientation from Panoramic X-rays M. K. Dhar Mou Deb D. Madhab Zeyun Yu 25 1 0 26 Oct 2023
Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition Hillary Ngai Rohan Agrawal Neeraj Gaur Ronny Huang Parisa Haghani P. M. Mengibar MoMe 44 0 0 17 Oct 2023
A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization Alexander Ororbia Mary Alexandria Kelly AI4CE 27 2 0 14 Oct 2023
Transformer Choice Net: A Transformer Neural Network for Choice Prediction Hanzhao Wang Xiaocheng Li K. Talluri 29 2 0 12 Oct 2023
CacheGen: KV Cache Compression and Streaming for Fast Language Model Serving Yuhan Liu Hanchen Li Yihua Cheng Siddhant Ray Yuyang Huang ... Ganesh Ananthanarayanan Michael Maire Henry Hoffmann Ari Holtzman Junchen Jiang 50 41 0 11 Oct 2023
RetSeg: Retention-based Colorectal Polyps Segmentation Network Khaled Elkarazle V. Raman Caslon Chua P. Then MedIm ViT 30 1 0 09 Oct 2023
Enhancing Pre-Trained Language Models with Sentence Position Embeddings for Rhetorical Roles Recognition in Legal Opinions Anas Belfathi Nicolas Hernandez Laura Monceaux AILaw 29 3 0 08 Oct 2023
EMOFM: Ensemble MLP mOdel with Feature-based Mixers for Click-Through Rate Prediction Yujian Betterest Li Kai Wu 22 0 0 06 Oct 2023
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4 Katikapalli Subramanyam Kalyan LM&MA AI4CE LRM AILaw ELM 34 224 0 04 Oct 2023
Evolutionary Neural Architecture Search for Transformer in Knowledge Tracing Shangshang Yang Xiaoshan Yu Ye Tian Xueming Yan Haiping Ma Xingyi Zhang ViT KELM AI4Ed 18 2 0 02 Oct 2023
On the Non-Associativity of Analog Computations Lisa Kuhn Bernhard Klein Holger Fröning 11 0 0 25 Sep 2023
GPT-MolBERTa: GPT Molecular Features Language Model for molecular property prediction Suryanarayanan Balaji Rishikesh Magar Yayati Jadhav and Amir Barati Farimani 18 13 0 20 Sep 2023
A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches Maisha Maliha Golnaz Habibi Mohammed Atiquzzaman 14 2 0 17 Sep 2023
Empowering Precision Medicine: AI-Driven Schizophrenia Diagnosis via EEG Signals: A Comprehensive Review from 2002-2023 M. Jafari Delaram Sadeghi A. Shoeibi Hamid Alinejad-Rokny Amin Beheshti David López-García Zhaolin Chen U. Acharya Juan M Gorriz 23 20 0 14 Sep 2023
TFBEST: Dual-Aspect Transformer with Learnable Positional Encoding for Failure Prediction Rohan Mohapatra Saptarshi Sengupta 17 3 0 06 Sep 2023
Efficient Social Choice via NLP and Sampling Lior Ashkenazy Nimrod Talmon 8 0 0 04 Sep 2023
Passing Heatmap Prediction Based on Transformer Model and Tracking Data Yisheng Pei Varuna De Silva Mike Caine 25 0 0 04 Sep 2023
Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages Shunjie Wang Shane Steinert-Threlkeld 25 4 0 02 Sep 2023
Deep learning in medical image registration: introduction and survey Ahmad Hammoudeh Stéphane Dupont MedIm 19 4 0 01 Sep 2023
Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations Théo Deschamps-Berger L. Lamel Laurence Devillers 19 2 0 28 Aug 2023
Cognitive Effects in Large Language Models Jonathan Shaki Sarit Kraus Michael Wooldridge 11 7 0 28 Aug 2023
Fast Training of NMT Model with Data Sorting Daniela N. Rim Kimera Richard Heeyoul Choi 14 0 0 16 Aug 2023
PETformer: Long-term Time Series Forecasting via Placeholder-enhanced Transformer Shengsheng Lin Weiwei Lin Wentai Wu Song Wang Yongxiang Wang AI4TS 40 18 0 09 Aug 2023
Performance Evaluation of Swin Vision Transformer Model using Gradient Accumulation Optimization Technique Sanad Aburass O. Dorgham ViT 13 9 0 31 Jul 2023
Visual Prompt Flexible-Modal Face Anti-Spoofing Zitong Yu Rizhao Cai Yawen Cui Ajian Liu Changsheng Chen 38 6 0 26 Jul 2023
E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning Cheng Han Qifan Wang Yiming Cui Zhiwen Cao Wenguan Wang Siyuan Qi Dongfang Liu VPVLM VLM 25 47 0 25 Jul 2023
Word Sense Disambiguation as a Game of Neurosymbolic Darts Tiansi Dong R. Sifa 15 2 0 25 Jul 2023
Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework Jingxuan Wei Cheng Tan Zhangyang Gao Linzhuang Sun Siyuan Li Bihui Yu R. Guo Stan Z. Li LRM 34 8 0 24 Jul 2023
Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach Cheng Wen Baosheng Yu Rao Fu Dacheng Tao 3DPC 29 1 0 22 Jul 2023
Transferability of Convolutional Neural Networks in Stationary Learning Tasks Damian Owerko Charilaos I. Kanatsoulis Jennifer Bondarchuk Donald J. Bucci Alejandro Ribeiro BDL 29 0 0 21 Jul 2023
Art Authentication with Vision Transformers Ludovica Schaerf Carina Popovici Eric Postma ViT 14 9 0 06 Jul 2023
Transformers in Healthcare: A Survey Subhash Nerella S. Bandyopadhyay Jiaqing Zhang Miguel Contreras Scott Siegel ... Jessica Sena B. Shickel A. Bihorac Kia Khezeli Parisa Rashidi MedIm AI4CE 21 25 0 30 Jun 2023
GPatcher: A Simple and Adaptive MLP Model for Alleviating Graph Heterophily Shuaicheng Zhang Haohui Wang Si Zhang Dawei Zhou 21 2 0 25 Jun 2023
Exploring the Landscape of Ubiquitous In-home Health Monitoring: A Comprehensive Survey Farhad Pourpanah Ali Etemad 26 4 0 22 Jun 2023
LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System Ashish Bastola Hongya Wang Judsen Hembree Pooja Yadav Zihao Gong Emma Dixon Abolfazl Razi Nathan J. Mcneese AI4MH 23 14 0 21 Jun 2023
Building Blocks for a Complex-Valued Transformer Architecture Florian Eilers Xiaoyi Jiang ViT 32 7 0 16 Jun 2023
Reducing Computational Costs in Sentiment Analysis: Tensorized Recurrent Networks vs. Recurrent Networks Gabriel Lopez Anna Nguyen Joe Kaul 16 1 0 16 Jun 2023
Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant Xianbiao Qi Jianan Wang Lei Zhang 15 0 0 15 Jun 2023
Solving Large-scale Spatial Problems with Convolutional Neural Networks Damian Owerko Charilaos I. Kanatsoulis Alejandro Ribeiro 22 2 0 14 Jun 2023
CD-CTFM: A Lightweight CNN-Transformer Network for Remote Sensing Cloud Detection Fusing Multiscale Features Wenhang Ge Xubing Yang Li Zhang 35 10 0 12 Jun 2023
A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks Saidul Islam Hanae Elmekki Ahmed Elsebai Jamal Bentahar Najat Drawel Gaith Rjoub Witold Pedrycz ViT MedIm 24 171 0 11 Jun 2023
An Overview on Generative AI at Scale with Edge-Cloud Computing Yun Cheng Wang Jintang Xue Chengwei Wei C.-C. Jay Kuo 24 30 0 02 Jun 2023
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers Sotiris Anagnostidis Dario Pavllo Luca Biggio Lorenzo Noci Aurelien Lucchi Thomas Hofmann 39 53 0 25 May 2023
Memory Efficient Neural Processes via Constant Memory Attention Block Leo Feng Frederick Tung Hossein Hajimirsadeghi Yoshua Bengio Mohamed Osama Ahmed 28 5 0 23 May 2023
Training Transitive and Commutative Multimodal Transformers with LoReTTa Manuel Tran Yashin Dicente Cid Amal Lahiani Fabian J. Theis Tingying Peng Eldad Klaiman 26 2 0 23 May 2023
Stock and market index prediction using Informer network Yu-Hsin Lu Hailong Zhang Qiwen Guo AIFin AI4TS 12 5 0 22 May 2023