v1v2v3 (latest)

Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding

5 June 2021

Papers citing "Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding"

33 / 33 papers shown

Title
Deep Learning without Global Optimization by Random Fourier Neural Networks Owen Davis Gianluca Geraci Mohammad Motamed BDL 108 0 0 16 Jul 2024
UVTM: Universal Vehicle Trajectory Modeling with ST Feature Domain Generation Yan Lin Jilin Hu Shengnan Guo Bin Yang Christian S. Jensen Youfang Lin Huaiyu Wan 85 0 0 11 Feb 2024
Relative Positional Encoding for Transformers with Linear Complexity Antoine Liutkus Ondřej Cífka Shih-Lun Wu Umut Simsekli Yi-Hsuan Yang Gaël Richard 62 48 0 18 May 2021
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference Ben Graham Alaaeldin El-Nouby Hugo Touvron Pierre Stock Armand Joulin Hervé Jégou Matthijs Douze ViT 93 791 0 02 Apr 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,793 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 667 41,369 0 22 Oct 2020
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements Yongqian Li Gang Li Luheng He Jingjie Zheng Hong Li Zhiwei Guan 68 110 0 08 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 163 2,747 0 05 Jun 2020
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Peter Hawkins Jared Davis David Belanger Lucy J. Colwell Adrian Weller 78 85 0 05 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 432 13,094 0 26 May 2020
Learning to Encode Position for Transformer with Continuous Dynamical Model Xuanqing Liu Hsiang-Fu Yu Inderjit Dhillon Cho-Jui Hsieh 67 110 0 13 Mar 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 199 2,327 0 13 Jan 2020
Encoding word order in complex embeddings Benyou Wang Donghao Zhao Christina Lioma Qiuchi Li Peng Zhang J. Simonsen 56 113 0 27 Dec 2019
Self-Attention with Structural Position Representations Xing Wang Zhaopeng Tu Longyue Wang Shuming Shi MILM 51 72 0 01 Sep 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 234 8,444 0 19 Jun 2019
Stand-Alone Self-Attention in Vision Models Prajit Ramachandran Niki Parmar Ashish Vaswani Irwan Bello Anselm Levskaya Jonathon Shlens VLM SLR ViT 101 1,215 0 13 Jun 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 129 1,915 0 23 Apr 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 137 1,015 0 22 Apr 2019
Recurrent Space-time Graph Neural Networks Andrei Liviu Nicolicioiu Iulia Duta Marius Leordeanu GNN 81 45 0 11 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 253 3,745 0 09 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
But How Does It Work in Theory? Linear SVM with Random Features Yitong Sun A. Gilbert Ambuj Tewari VLM 56 64 0 12 Sep 2018
Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network A. Sherstinsky 95 3,729 0 09 Aug 2018
Self-Attention with Relative Position Representations Peter Shaw Jakob Uszkoreit Ashish Vaswani 177 2,295 0 06 Mar 2018
Image Transformer Niki Parmar Ashish Vaswani Jakob Uszkoreit Lukasz Kaiser Noam M. Shazeer Alexander Ku Dustin Tran ViT 138 1,684 0 15 Feb 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 730 132,199 0 12 Jun 2017
Convolutional Sequence to Sequence Learning Jonas Gehring Michael Auli David Grangier Denis Yarats Yann N. Dauphin AIMat 171 3,288 0 08 May 2017
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 405 7,968 0 17 Aug 2015
Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels H. Avron Vikas Sindhwani Jiyan Yang Michael W. Mahoney 87 166 0 29 Dec 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 575 27,325 0 01 Sep 2014
Fastfood: Approximate Kernel Expansions in Loglinear Time Quoc V. Le Tamás Sarlós Alex Smola 95 444 0 13 Aug 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 424 43,777 0 01 May 2014
Compact Random Feature Maps Raffay Hamid Ying Xiao Alex Gittens D. DeCoste 73 87 0 17 Dec 2013