Self-Attention with Relative Position Representations

6 March 2018

Papers citing "Self-Attention with Relative Position Representations"

50 / 411 papers shown

Title
Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration M. Rath A. P. Condurache 25 8 0 08 Feb 2022
Structure-Aware Transformer for Graph Representation Learning Dexiong Chen Leslie O’Bray Karsten M. Borgwardt 36 237 0 07 Feb 2022
WebFormer: The Web-page Transformer for Structure Information Extraction Qifan Wang Yi Fang Anirudh Ravula Fuli Feng Xiaojun Quan Dongfang Liu ViT 149 65 0 01 Feb 2022
GRPE: Relative Positional Encoding for Graph Transformer Wonpyo Park Woonggi Chang Donggeon Lee Juntae Kim Seung-won Hwang 41 75 0 30 Jan 2022
Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes C. Qu Xiaoyu Tan Siqiao Xue Xiaoming Shi James Y. Zhang Hongyuan Mei OffRL 30 17 0 29 Jan 2022
Generative Cooperative Networks for Natural Language Generation Sylvain Lamprier Thomas Scialom Antoine Chaffin Vincent Claveau Ewa Kijak Jacopo Staiano Benjamin Piwowarski GAN 54 13 0 28 Jan 2022
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition Chao-Yuan Wu Yanghao Li K. Mangalam Haoqi Fan Bo Xiong Jitendra Malik Christoph Feichtenhofer ViT 48 198 0 20 Jan 2022
Continual Transformers: Redundancy-Free Attention for Online Inference Lukas Hedegaard Arian Bakhtiarnia Alexandros Iosifidis CLL 27 11 0 17 Jan 2022
Video Transformers: A Survey Javier Selva A. S. Johansen Sergio Escalera Kamal Nasrollahi T. Moeslund Albert Clapés ViT 22 103 0 16 Jan 2022
Assemble Foundation Models for Automatic Code Summarization Jian Gu P. Salza H. Gall 36 34 0 13 Jan 2022
Reverse-Engineering Information Presentations: Recovering Hierarchical Grouping from Layouts of Visual Elements Danqing Shi Weiwei Cui Danqing Huang Haidong Zhang Nan Cao 13 15 0 13 Jan 2022
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks Lei Cheng Ruslan Khalitov Tong Yu Zhirong Yang 25 32 0 06 Jan 2022
PyramidTNT: Improved Transformer-in-Transformer Baselines with Pyramid Architecture Kai Han Jianyuan Guo Yehui Tang Yunhe Wang ViT 34 22 0 04 Jan 2022
S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversation Chen Liang Chong Yang Jing Xu Juyang Huang Yongliang Wang Yang Dong 31 24 0 23 Dec 2021
Learning Positional Embeddings for Coordinate-MLPs Sameera Ramasinghe Simon Lucey 27 10 0 21 Dec 2021
Masked Feature Prediction for Self-Supervised Visual Pre-Training Chen Wei Haoqi Fan Saining Xie Chaoxia Wu Alan Yuille Christoph Feichtenhofer ViT 100 655 0 16 Dec 2021
Towards More Efficient Insertion Transformer with Fractional Positional Encoding Zhisong Zhang Yizhe Zhang W. Dolan 46 0 0 12 Dec 2021
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection Yanghao Li Chaoxia Wu Haoqi Fan K. Mangalam Bo Xiong Jitendra Malik Christoph Feichtenhofer ViT 75 678 0 02 Dec 2021
SwinTrack: A Simple and Strong Baseline for Transformer Tracking Liting Lin Heng Fan Zhipeng Zhang Yong-mei Xu Haibin Ling ViT 37 303 0 02 Dec 2021
Systematic Generalization with Edge Transformers Leon Bergen Timothy J. O'Donnell Dzmitry Bahdanau 15 46 0 01 Dec 2021
Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-sentence Dependency Graph Liyan Xu Xuchao Zhang Bo Zong Yanchi Liu Wei Cheng Jingchao Ni Haifeng Chen Liang Zhao Jinho Choi 42 4 0 01 Dec 2021
KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers Avi Chawla Nidhi Mulay Vikas Bishnoi Gaurav Dhama ViT 19 2 0 30 Nov 2021
A Contextual Latent Space Model: Subsequence Modulation in Melodic Sequence Taketo Akama BDL 30 3 0 23 Nov 2021
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing Pengcheng He Jianfeng Gao Weizhu Chen 53 1,120 0 18 Nov 2021
PESTO: Switching Point based Dynamic and Relative Positional Encoding for Code-Mixed Languages Mohsin Ali K. S. Teja Sumanth Manduru Parth Patwa Amitava Das 20 3 0 12 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 77 330 0 11 Nov 2021
A Chinese Multi-type Complex Questions Answering Dataset over Wikidata Jianyun Zou Min Yang Lichao Zhang Yechen Xu Qifan Pan ... Ran Qin Shushu Wang Yifan He Songfang Huang Zhou Zhao 30 8 0 11 Nov 2021
MNet-Sim: A Multi-layered Semantic Similarity Network to Evaluate Sentence Similarity Manuela Nayantara Jeyaraj D. Kasthurirathna 11 3 0 09 Nov 2021
PatchFormer: An Efficient Point Transformer with Patch Attention Zhang Cheng Haocheng Wan Xinyi Shen Zizhao Wu 3DPC 24 65 0 30 Oct 2021
Dispensed Transformer Network for Unsupervised Domain Adaptation Yunxiang Li Jingxiong Li Ruilong Dan Shuai Wang Kai Jin ... Qianni Zhang Huiyu Zhou Qun Jin Li Wang Yaqi Wang OOD MedIm 23 4 0 28 Oct 2021
Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction Zhengkai Tu Connor W. Coley 30 90 0 19 Oct 2021
Multi-View Stereo Network with attention thin volume Zihang Wan 3DV 28 1 0 16 Oct 2021
The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization Róbert Csordás Kazuki Irie Jürgen Schmidhuber AI4CE 33 55 0 14 Oct 2021
Study of positional encoding approaches for Audio Spectrogram Transformers L. Pepino Pablo Riera Luciana Ferrer ViT 28 6 0 13 Oct 2021
Relative Molecule Self-Attention Transformer Lukasz Maziarka Dawid Majchrowski Tomasz Danel Piotr Gaiñski Jacek Tabor Igor T. Podolak Pawel M. Morkisz Stanislaw Jastrzebski MedIm 40 34 0 12 Oct 2021
Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning Chongjian Ge Youwei Liang Yibing Song Jianbo Jiao Jue Wang Ping Luo ViT 24 36 0 11 Oct 2021
ATISS: Autoregressive Transformers for Indoor Scene Synthesis Despoina Paschalidou Amlan Kar Maria Shugrina Karsten Kreis Andreas Geiger Sanja Fidler 3DV ViT 38 148 0 07 Oct 2021
Learning to Iteratively Solve Routing Problems with Dual-Aspect Collaborative Transformer Yining Ma Jingwen Li Zhiguang Cao Wen Song Le Zhang Zhenghua Chen Jing Tang 83 129 0 06 Oct 2021
Ripple Attention for Visual Perception with Sub-quadratic Complexity Lin Zheng Huijie Pan Lingpeng Kong 28 3 0 06 Oct 2021
Molformer: Motif-based Transformer on 3D Heterogeneous Molecular Graphs Fang Wu Dragomir R. Radev Huabin Xing ViT 36 54 0 04 Oct 2021
GT U-Net: A U-Net Like Group Transformer Network for Tooth Root Segmentation Yunxiang Li Shuai Wang Jun Wang G. Zeng Wenjun Liu Qianni Zhang Qun Jin Yaqi Wang ViT MedIm 31 47 0 30 Sep 2021
Multiplicative Position-aware Transformer Models for Language Understanding Zhiheng Huang Davis Liang Peng Xu Bing Xiang 9 1 0 27 Sep 2021
Modeling Dynamic Attributes for Next Basket Recommendation Yong-Guang Chen Jia Li Chenghao Liu Chenxi Li M. Anderle Julian McAuley Caiming Xiong 38 18 0 23 Sep 2021
The NiuTrans Machine Translation Systems for WMT21 Yuhao Zhang Tao Zhou Bin Wei Runzhe Cao Yongyu Mu ... Weiqiao Shan Yinqiao Li Bei Li Tong Xiao Jingbo Zhu 32 17 0 22 Sep 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 91 152 0 17 Sep 2021
CodeQA: A Question Answering Dataset for Source Code Comprehension Chenxiao Liu Xiaojun Wan 37 27 0 17 Sep 2021
The NiuTrans System for WNGT 2020 Efficiency Task Chi Hu Bei Li Ye Lin Yinqiao Li Yanyang Li Chenglong Wang Tong Xiao Jingbo Zhu 25 7 0 16 Sep 2021
The NiuTrans System for the WMT21 Efficiency Task Chenglong Wang Chi Hu Yongyu Mu Zhongxiang Yan Siming Wu ... Hang Cao Bei Li Ye Lin Tong Xiao Jingbo Zhu 29 2 0 16 Sep 2021
SPARQLing Database Queries from Intermediate Question Decompositions Irina Saparina A. Osokin 21 14 0 13 Sep 2021
SHAPE: Shifted Absolute Position Embedding for Transformers Shun Kiyono Sosuke Kobayashi Jun Suzuki Kentaro Inui 236 45 0 13 Sep 2021