Self-Attention with Relative Position Representations

6 March 2018

Papers citing "Self-Attention with Relative Position Representations"

50 / 411 papers shown

Title
RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL Haoyang Li Jing Zhang Cuiping Li Hong Chen 32 174 0 12 Feb 2023
Investigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters Sébastien Montella Alexis Nasr Johannes Heinecke Frédéric Béchet L. Rojas-Barahona 29 2 0 12 Feb 2023
Single Cells Are Spatial Tokens: Transformers for Spatial Transcriptomic Data Imputation Haifang Wen Wenzhuo Tang Wei Jin Jiayuan Ding Renming Liu Xinnan Dai Feng Shi Lulu Shang Jiliang Tang Yuying Xie 29 9 0 06 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 37 8 0 03 Feb 2023
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness Shuaichen Chang Jun Wang Mingwen Dong Lin Pan Henghui Zhu ... William Yang Wang Zhiguo Wang Vittorio Castelli Patrick Ng Bing Xiang OOD 44 34 0 21 Jan 2023
Screen Correspondence: Mapping Interchangeable Elements between UIs Jason Wu Amanda Swearngin Xiaoyi Zhang Jeffrey Nichols Jeffrey P. Bigham 43 7 0 20 Jan 2023
PromptShots at the FinNLP-2022 ERAI Tasks: Pairwise Comparison and Unsupervised Ranking Peratham Wiriyathammabhum 8 4 0 16 Jan 2023
Structured Case-based Reasoning for Inference-time Adaptation of Text-to-SQL parsers Abhijeet Awasthi Soumen Chakrabarti Sunita Sarawagi 33 5 0 10 Jan 2023
Towards Autoformalization of Mathematics and Code Correctness: Experiments with Elementary Proofs Garett Cunningham Razvan C. Bunescu D. Juedes LRM 23 16 0 05 Jan 2023
Modeling the Rhythm from Lyrics for Melody Generation of Pop Song Daiyu Zhang Ju-Chiang Wang Katerina Kosta Jordan B. L. Smith Shicen Zhou 26 3 0 03 Jan 2023
Boosting Neural Networks to Decompile Optimized Binaries Ying Cao Ruigang Liang Kai Chen Peiwei Hu 31 17 0 03 Jan 2023
Swin MAE: Masked Autoencoders for Small Datasets Zián Xu Yin Dai Fayu Liu Weibin Chen Yue Liu Li-Li Shi Sheng Liu Yuhang Zhou SyDa MedIm ViT 36 28 0 28 Dec 2022
Optimizing Deep Transformers for Chinese-Thai Low-Resource Translation Wenjie Hao Hongfei Xu Lingling Mu Hongying Zan MoE 33 4 0 24 Dec 2022
A Length-Extrapolatable Transformer Yutao Sun Li Dong Barun Patra Shuming Ma Shaohan Huang Alon Benhaim Vishrav Chaudhary Xia Song Furu Wei 30 115 0 20 Dec 2022
Human-in-the-loop Evaluation for Early Misinformation Detection: A Case Study of COVID-19 Treatments Ethan Mendes Yang Chen Wei-ping Xu Alan Ritter 16 14 0 19 Dec 2022
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li Jiahao Yu Ce Zhang Gao Huang Yu Tong ViT 27 6 0 16 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 125 36 0 15 Dec 2022
Video Prediction by Efficient Transformers Xi Ye Guillaume-Alexandre Bilodeau ViT 44 33 0 12 Dec 2022
Position Embedding Needs an Independent Layer Normalization Runyi Yu Zhennan Wang Yinhuai Wang Kehan Li Yian Zhao Jian Zhang Guoli Song Jie Chen 31 1 0 10 Dec 2022
A Domain-Knowledge-Inspired Music Embedding Space and a Novel Attention Mechanism for Symbolic Music Modeling Z. Guo J. Kang Dorien Herremans 14 18 0 02 Dec 2022
Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges K. T. Baghaei Amirreza Payandeh Pooya Fayyazsanavi Shahram Rahimi Zhiqian Chen Somayeh Bakhtiari Ramezani FaML AI4TS 38 6 0 27 Nov 2022
NQE: N-ary Query Embedding for Complex Query Answering over Hyper-Relational Knowledge Graphs Haoran Luo E. Haihong Yuhao Yang Gengxian Zhou Yikai Guo Tianyu Yao Zichen Tang X. Lin Kaiyang Wan 30 10 0 24 Nov 2022
On the Typicality of Musical Sequences Mathias Rose Bjare Stefan Lattner 22 3 0 23 Nov 2022
Unifying Tracking and Image-Video Object Detection Peirong Liu Rui Wang Pengchuan Zhang Omid Poursaeed Yipin Zhou Xuefei Cao Sreya . Dutta Roy Ashish Shah Ser-Nam Lim 26 0 0 20 Nov 2022
CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow Philippe Weinzaepfel Thomas Lucas Vincent Leroy Yohann Cabon Vaibhav Arora Romain Brégier G. Csurka L. Antsfeld Boris Chidlovskii Jérôme Revaud ViT 29 83 0 18 Nov 2022
Hypergraph Transformer for Skeleton-based Action Recognition Yuxuan Zhou Zhi-Qi Cheng Chong Li Yanwen Fang Yifeng Geng Xuansong Xie M. Keuper ViT 32 52 0 17 Nov 2022
Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation Yiyue Hu Lei Zhang Nan Mu Leijun Liu ViT MedIm 22 1 0 17 Nov 2022
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale Yuxin Fang Wen Wang Binhui Xie Quan-Sen Sun Ledell Yu Wu Xinggang Wang Tiejun Huang Xinlong Wang Yue Cao VLM CLIP 87 679 0 14 Nov 2022
ParCNetV2: Oversized Kernel with Enhanced Attention Ruihan Xu Haokui Zhang Wenze Hu Shiliang Zhang Xiaoyu Wang ViT 32 6 0 14 Nov 2022
Continuous Soft Pseudo-Labeling in ASR Tatiana Likhomanenko R. Collobert Navdeep Jaitly Samy Bengio VLM 27 3 0 11 Nov 2022
Structured State Space Decoder for Speech Recognition and Synthesis Koichi Miyazaki Masato Murata Tomoki Koriyama 34 12 0 31 Oct 2022
Diverse Parallel Data Synthesis for Cross-Database Adaptation of Text-to-SQL Parsers Abhijeet Awasthi Ashutosh Sathe Sunita Sarawagi 41 4 0 29 Oct 2022
Grafting Vision Transformers Jong Sung Park Kumara Kahatapitiya Donghyun Kim Shivchander Sudalairaj Quanfu Fan Michael S. Ryoo ViT 29 2 0 28 Oct 2022
Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future Guo-Jun Qi M. Shah SSL 23 8 0 23 Oct 2022
The Curious Case of Absolute Position Embeddings Koustuv Sinha Amirhossein Kazemnejad Siva Reddy J. Pineau Dieuwke Hupkes Adina Williams 87 15 0 23 Oct 2022
Is Encoder-Decoder Redundant for Neural Machine Translation? Yingbo Gao Christian Herold Zijian Yang Hermann Ney 27 4 0 21 Oct 2022
Revisiting Checkpoint Averaging for Neural Machine Translation Yingbo Gao Christian Herold Zijian Yang Hermann Ney MoMe 29 11 0 21 Oct 2022
Amos: An Adam-style Optimizer with Adaptive Weight Decay towards Model-Oriented Scale Ran Tian Ankur P. Parikh ODL 23 6 0 21 Oct 2022
Categorizing Semantic Representations for Neural Machine Translation Yongjing Yin Yafu Li Fandong Meng Jie Zhou Yue Zhang 24 6 0 13 Oct 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
Melody Infilling with User-Provided Structural Context Chih-Pin Tan A. Su Yi-Hsuan Yang 36 3 0 06 Oct 2022
Point Cloud Recognition with Position-to-Structure Attention Transformers Zhenghu Ding James Hou Zhuowen Tu ViT 3DPC 38 1 0 05 Oct 2022
MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models Chenglin Yang Siyuan Qiao Qihang Yu Xiaoding Yuan Yukun Zhu Alan Yuille Hartwig Adam Liang-Chieh Chen ViT MoE 39 59 0 04 Oct 2022
E-Branchformer: Branchformer with Enhanced merging for speech recognition Kwangyoun Kim Felix Wu Yifan Peng Jing Pan Prashant Sridhar Kyu Jeong Han Shinji Watanabe 61 105 0 30 Sep 2022
Protein structure generation via folding diffusion Kevin E. Wu Kevin Kaichuang Yang Rianne van den Berg James Zou Alex X. Lu Ava P. Amini DiffM 35 192 0 30 Sep 2022
Verifiable and Energy Efficient Medical Image Analysis with Quantised Self-attentive Deep Neural Networks Rakshith Sathish S. Khare Debdoot Sheet 42 4 0 30 Sep 2022
Dynamic Graph Message Passing Networks for Visual Recognition Li Zhang Mohan Chen Anurag Arnab Xiangyang Xue Philip Torr GNN 31 1 0 20 Sep 2022
Parameter-Efficient Conformers via Sharing Sparsely-Gated Experts for End-to-End Speech Recognition Ye Bai Jie Li W. Han Hao Ni Kaituo Xu Zhuo Zhang Cheng Yi Xiaorui Wang MoE 26 1 0 17 Sep 2022
LogGD:Detecting Anomalies from System Logs by Graph Neural Networks Yongzhen Xie Hongyu Zhang M. Babar AI4TS 12 20 0 16 Sep 2022
Graph-to-Text Generation with Dynamic Structure Pruning Liang Li Ruiying Geng Bowen Li Can Ma Yinliang Yue Binhua Li Yongbin Li 55 2 0 15 Sep 2022