Self-Attention with Relative Position Representations

6 March 2018

Papers citing "Self-Attention with Relative Position Representations"

50 / 411 papers shown

Title
Domain Adversarial Training on Conditional Variational Auto-Encoder for Controllable Music Generation Jingwei Zhao Gus Xia Ye Wang DRL 39 6 0 15 Sep 2022
Beat Transformer: Demixed Beat and Downbeat Tracking with Dilated Self-Attention Jingwei Zhao Gus Xia Ye Wang 40 18 0 15 Sep 2022
SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers Bowen Qin Lihan Wang Binyuan Hui Bowen Li Xiangpeng Wei Binhua Li Fei Huang Luo Si Min Yang Yongbin Li 45 9 0 14 Sep 2022
Local-Aware Global Attention Network for Person Re-Identification Based on Body and Hand Images N. L. Baisa CVBM 32 3 0 11 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 33 109 0 31 Aug 2022
K-Order Graph-oriented Transformer with GraAttention for 3D Pose and Shape Estimation Weixi Zhao Weiqiang Wang ViT 3DPC 27 2 0 24 Aug 2022
Efficient Attention-free Video Shift Transformers Adrian Bulat Brais Martínez Georgios Tzimiropoulos ViT 29 1 0 23 Aug 2022
Accelerating Vision Transformer Training via a Patch Sampling Schedule Bradley McDanel C. Huynh ViT 30 1 0 19 Aug 2022
Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages Paul Soulos Sudha Rao Caitlin Smith Eric Rosen Asli Celikyilmaz ... Coleman Haley Roland Fernandez Hamid Palangi Jianfeng Gao P. Smolensky 32 6 0 11 Aug 2022
PointConvFormer: Revenge of the Point-based Convolution Wenxuan Wu Li Fuxin Qi Shan 3DPC 25 30 0 04 Aug 2022
Innovations in Neural Data-to-text Generation: A Survey Mandar Sharma Ajay K. Gogineni Naren Ramakrishnan 32 10 0 25 Jul 2022
Learning a Dual-Mode Speech Recognition Model via Self-Pruning Chunxi Liu Yuan Shangguan Haichuan Yang Yangyang Shi Raghuraman Krishnamoorthi Ozlem Kalinli SSL 29 7 0 25 Jul 2022
Learning Object Placement via Dual-path Graph Completion Siyuan Zhou Liu Liu Li Niu Liqing Zhang 36 24 0 23 Jul 2022
Multi-branch Cascaded Swin Transformers with Attention to k-space Sampling Pattern for Accelerated MRI Reconstruction Mevan Ekanayake Kamlesh Pawar Mehrtash Harandi Gary Egan Zhaolin Chen ViT MedIm 32 7 0 18 Jul 2022
Position Prediction as an Effective Pretraining Strategy Shuangfei Zhai Navdeep Jaitly Jason Ramapuram Dan Busbridge Tatiana Likhomanenko Joseph Y. Cheng Walter A. Talbott Chen Huang Hanlin Goh J. Susskind ViT 46 23 0 15 Jul 2022
Boosting Multi-Modal E-commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization Meng-yang Liu Chao Zhu Hongyu Gao Weibo Gu Hongfa Wang Wei Liu Xu-Cheng Yin 24 2 0 15 Jul 2022
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation Qihang Yu Huiyu Wang Dahun Kim Siyuan Qiao Maxwell D. Collins Yukun Zhu Hartwig Adam Alan Yuille Liang-Chieh Chen ViT MedIm 32 90 0 17 Jun 2022
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang Chong Li Biao Wang Xihan Wei Lei Zhang M. Keuper Xia Hua ViT 37 15 0 15 Jun 2022
Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection Y. Zhang Yingwei Pan Ting Yao Rui Huang Tao Mei C. Chen ViT 38 68 0 13 Jun 2022
How to Dissect a Muppet: The Structure of Transformer Embedding Spaces Timothee Mickus Denis Paperno Mathieu Constant 29 19 0 07 Jun 2022
Preparing an Endangered Language for the Digital Age: The Case of Judeo-Spanish A. Oktem Rodolfo Zevallos Yasmin Moslem Günes Öztürk Karen Sarhon 26 0 0 31 May 2022
Transformer with Tree-order Encoding for Neural Program Generation Klaudia Thellmann Bernhard Stadler Ricardo Usbeck Jens Lehmann 27 1 0 30 May 2022
Do we really need temporal convolutions in action segmentation? Dazhao Du Bing-Huang Su Yu Li Zhongang Qi Hui Xiong Ying Shan ViT 29 16 0 26 May 2022
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation Ta-Chung Chi Ting-Han Fan Peter J. Ramadge Alexander I. Rudnicky 47 65 0 20 May 2022
Trading Positional Complexity vs. Deepness in Coordinate Networks Jianqiao Zheng Sameera Ramasinghe Xueqian Li Simon Lucey 31 18 0 18 May 2022
Zero-shot Code-Mixed Offensive Span Identification through Rationale Extraction Manikandan Ravikiran Bharathi Raja Chakravarthi 22 3 0 12 May 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu L. Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 31 181 0 06 May 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 44 149 0 27 Apr 2022
DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation Wei Chen Yeyun Gong Song Wang Bolun Yao Weizhen Qi ... Bartuer Zhou Yi Mao Weizhu Chen Biao Cheng Nan Duan VLM 32 47 0 27 Apr 2022
TANet: Thread-Aware Pretraining for Abstractive Conversational Summarization Ze Yang Liran Wang Zhoujin Tian Wei Wu Zhoujun Li 30 4 0 09 Apr 2022
Compositional Generalization and Decomposition in Neural Program Synthesis Kensen Shi Joey Hong Manzil Zaheer Pengcheng Yin Charles Sutton 40 5 0 07 Apr 2022
Heterogeneous Target Speech Separation Hyunjae Cho Wonbin Jung Junhyeok Lee Paris Smaragdis Sanghyun Woo 46 26 0 07 Apr 2022
Video Diffusion Models Jonathan Ho Tim Salimans Alexey A. Gritsenko William Chan Mohammad Norouzi David J. Fleet DiffM VGen 62 1,523 0 07 Apr 2022
Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection Yuxin Fang Shusheng Yang Shijie Wang Yixiao Ge Ying Shan Xinggang Wang 31 55 0 06 Apr 2022
A Complementary Joint Training Approach Using Unpaired Speech and Text for Low-Resource Automatic Speech Recognition Ye Du Jie Zhang Qiu-shi Zhu Lirong Dai Ming Wu Xin Fang Zhouwang Yang 34 2 0 05 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 34 17 0 04 Apr 2022
Pre-Training Transformer Decoder for End-to-End ASR Model with Unpaired Speech Data Junyi Ao Zi-Hua Zhang Long Zhou Shujie Liu Haizhou Li Tom Ko Lirong Dai Jinyu Li Yao Qian Furu Wei SSL 25 19 0 31 Mar 2022
VPTR: Efficient Transformers for Video Prediction Xi Ye Guillaume-Alexandre Bilodeau ViT 32 18 0 29 Mar 2022
Seq-2-Seq based Refinement of ASR Output for Spoken Name Capture Karan Singla S. Jalalvand Yeon-Jun Kim Ryan Price Daniel Pressel S. Bangalore 18 2 0 29 Mar 2022
ANNA: Enhanced Language Representation for Question Answering Changwook Jun Hansol Jang Myoseop Sim Hyun Kim Jooyoung Choi Kyungkoo Min Kyunghoon Bae 31 6 0 28 Mar 2022
Lane detection with Position Embedding Jun Xie Jiacheng Han Dezhen Qi F. Chen Kaer Huang Jia Shuai 35 4 0 23 Mar 2022
Uncertainty Estimation for Language Reward Models Adam Gleave G. Irving UQLM 39 31 0 14 Mar 2022
S $^2$ SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers Binyuan Hui Ruiying Geng Lihan Wang Bowen Qin Bowen Li Jian Sun Yongbin Li 28 55 0 14 Mar 2022
XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding Zhangxuan Gu Changhua Meng Ke Wang Jun Lan Weiqiang Wang Ming Gu Liqing Zhang 39 77 0 14 Mar 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs Xiaohan Ding Xinming Zhang Yi Zhou Jungong Han Guiguang Ding Jian Sun VLM 49 528 0 13 Mar 2022
HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging Xiaowan Hu Yuanhao Cai Jing Lin Haoqian Wang X. Yuan Yulun Zhang Radu Timofte Luc Van Gool 37 134 0 04 Mar 2022
LGT-Net: Indoor Panoramic Room Layout Estimation with Geometry-Aware Transformer Network Zhigang Jiang Zhongzheng Xiang Jinhua Xu Mingbi Zhao ViT 3DV 27 34 0 03 Mar 2022
A Unified Query-based Paradigm for Point Cloud Understanding Zetong Yang Li Jiang Yanan Sun Bernt Schiele Jiaya Jia 3DPC 25 38 0 02 Mar 2022
TableFormer: Robust Transformer Modeling for Table-Text Encoding Jingfeng Yang Aditya Gupta Shyam Upadhyay Luheng He Rahul Goel Shachi Paul LMTD 27 113 0 01 Mar 2022
FastRPB: a Scalable Relative Positional Encoding for Long Sequence Tasks Maksim Zubkov Daniil Gavrilov 27 0 0 23 Feb 2022