Self-Attention with Relative Position Representations

6 March 2018

Papers citing "Self-Attention with Relative Position Representations"

50 / 411 papers shown

Title
A 2D Semantic-Aware Position Encoding for Vision Transformers Xi Chen Shiyang Zhou Muqi Huang Jiaxu Feng Yun Xiong ... Yujie Zhang Huishuai Bao Sijia Peng Chong Li Feng Shi ViT 31 0 0 14 May 2025
Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait Feng Liu Nicholas Chimitt Lanqing guo Jitesh Jain Aditya Kane ... Arun Ross Humphrey Shi Zhangyang Wang A. Jain Xiaoming Liu CVBM 32 1 0 07 May 2025
User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems Jianling Wang Yifan Liu Yinghao Sun Xuejian Ma Yueqi Wang ... Onkar Dalal Ed Chi Lichan Hong Ningren Han Haokai Lu 31 0 0 07 Apr 2025
Charm: The Missing Piece in ViT fine-tuning for Image Aesthetic Assessment Fatemeh Behrad Tinne Tuytelaars Johan Wagemans ViT 42 0 0 03 Apr 2025
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives Shuyu Li Shulei Ji Zihao Wang Songruoyao Wu Jiaxing Yu Kaipeng Zhang MGen VGen 73 1 0 01 Apr 2025
Direction-Aware Diagonal Autoregressive Image Generation Yijia Xu Jianzhong Ju Jian Luan J. Cui 57 0 0 14 Mar 2025
Distributional Scaling Laws for Emergent Capabilities Rosie Zhao Tian Qin David Alvarez-Melis Sham Kakade Naomi Saphra LRM 39 1 0 24 Feb 2025
Positional Encoding in Transformer-Based Time Series Models: A Survey Habib Irani Vangelis Metsis AI4TS 53 0 0 17 Feb 2025
AttentionSmithy: A Modular Framework for Rapid Transformer Development and Customization Caleb Cranney Jesse G. Meyer 85 0 0 13 Feb 2025
Generative Retrieval for Book search Yubao Tang Ruqing Zhang J. Guo Maarten de Rijke Shihao Liu S. Wang Dawei Yin Xueqi Cheng RALM 31 0 0 19 Jan 2025
Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding Ziyang Chen Mingxiao Li Z. Chen Nan Du Xiaolong Li Yuexian Zou 53 0 0 19 Jan 2025
Holistic Semantic Representation for Navigational Trajectory Generation Ji Cao Tongya Zheng Qinghong Guo Yu Wang Junshu Dai Shunyu Liu Jie Yang Jie Song Mingli Song 36 0 0 06 Jan 2025
Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniques Lijie Tao Han Zhang Haizhao Jing Yu Liu Kelu Yao Guoting Wei Xizhe Xue 37 0 0 03 Jan 2025
Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention Zhenyu Guo Wenguang Chen 46 0 0 01 Jan 2025
Investigating Length Issues in Document-level Machine Translation Ziqian Peng Rachel Bawden François Yvon 69 1 0 23 Dec 2024
Does Self-Attention Need Separate Weights in Transformers? Md. Kowsher Nusrat Jahan Prottasha Chun-Nam Yu O. Garibay Niloofar Yousefi 241 0 0 30 Nov 2024
Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric Zhichao Zhang Wei Sun Xinyue Li Yunhao Li Qihang Ge ... Zhongpeng Ji Fengyu Sun Shangling Jui Xiongkuo Min Guangtao Zhai EGVM 122 1 0 25 Nov 2024
LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generation Mufei Li Viraj Shitole Eli Chien Changhai Man Zhaodong Wang Srinivas Sridharan Ying Zhang Tushar Krishna P. Li 41 0 0 04 Nov 2024
Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study Shawn Tan Songlin Yang Aaron Courville Rameswar Panda Yikang Shen 30 4 0 23 Oct 2024
In-context learning and Occam's razor Eric Elmoznino Tom Marty Tejas Kasetty Léo Gagnon Sarthak Mittal Mahan Fathi Dhanya Sridhar Guillaume Lajoie 44 1 0 17 Oct 2024
TULIP: Token-length Upgraded CLIP Ivona Najdenkoska Mohammad Mahdi Derakhshani Yuki M. Asano Nanne van Noord Marcel Worring Cees G. M. Snoek VLM 48 3 0 13 Oct 2024
Temporal Graph Memory Networks For Knowledge Tracing Seif Gad Sherif M. Abdelfattah Ghodai M. Abdelrahman AI4Ed 26 0 0 23 Sep 2024
Mastering Chess with a Transformer Model Daniel Monroe The Leela Chess Zero Team 44 3 0 18 Sep 2024
Hear Your Face: Face-based voice conversion with F0 estimation Jaejun Lee Yoori Oh Injune Hwang Kyogu Lee CVBM 29 2 0 19 Aug 2024
Rethinking Attention Module Design for Point Cloud Analysis Chengzhi Wu Kaige Wang Zeyun Zhong Hao Fu Junwei Zheng Jiaming Zhang Julius Pfrommer Jürgen Beyerer 3DPC 51 1 0 27 Jul 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 39 0 0 03 Jul 2024
Pathformer: Recursive Path Query Encoding for Complex Logical Query Answering Chongzhi Zhang Zhiping Peng Junhao Zheng Linghao Wang Ruifeng Shi Qianli Ma 43 1 0 21 Jun 2024
Attention-Based Deep Reinforcement Learning for Qubit Allocation in Modular Quantum Architectures Enrico Russo M. Palesi Davide Patti G. Ascia V. Catania 50 3 0 17 Jun 2024
Generative Inverse Design of Crystal Structures via Diffusion Models with Transformers Izumi Takahara Kiyou Shibata Teruyasu Mizoguchi DiffM AI4CE 36 2 0 13 Jun 2024
A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation Zhengrui Ma Qingkai Fang Shaolei Zhang Shoutao Guo Yang Feng Min Zhang 53 9 0 11 Jun 2024
DeformTime: Capturing Variable Dependencies with Deformable Attention for Time Series Forecasting Yuxuan Shu Vasileios Lampos AI4TS AI4CE 70 0 0 11 Jun 2024
An Effective-Efficient Approach for Dense Multi-Label Action Detection Faegheh Sardari Armin Mustafa Philip J. B. Jackson Adrian Hilton 37 0 0 10 Jun 2024
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks Mahdi Sabbaghi George Pappas Hamed Hassani Surbhi Goel 45 4 0 04 Jun 2024
Anomaly Detection in Dynamic Graphs: A Comprehensive Survey Ocheme Anthony Ekle William Eberle AI4TS 42 10 0 31 May 2024
Are queries and keys always relevant? A case study on Transformer wave functions Riccardo Rende Luciano Loris Viteritti 29 5 0 29 May 2024
Steerable Transformers Soumyabrata Kundu Risi Kondor ViT LLMSV 38 1 0 24 May 2024
Positional encoding is not the same as context: A study on positional encoding for sequential recommendation Alejo López-Ávila Jinhua Du Abbas Shimary Ze Li 46 1 0 16 May 2024
Semantically Consistent Video Inpainting with Conditional Diffusion Models Dylan Green William Harvey Saeid Naderiparizi Matthew Niedoba Yunpeng Liu ... Vasileios Lioutas Setareh Dabiri Adam Scibior Berend Zwartsenberg Frank Wood DiffM 38 1 0 30 Apr 2024
Contrastive Learning Method for Sequential Recommendation based on Multi-Intention Disentanglement Zeyu Hu Yuzhi Xiao Tao Huang Xuanrong Huo DRL 47 0 0 28 Apr 2024
EEGEncoder: Advancing BCI with Transformer-Based Motor Imagery Classification Wangdan Liao Weidong Wang 22 4 0 23 Apr 2024
Global Contrastive Training for Multimodal Electronic Health Records with Language Supervision Yingbo Ma Suraj Kolla Zhenhong Hu Dhruv Kaliraman Victoria Nolan ... Jeremy A. Balch Tyler J. Loftus Parisa Rashidi A. Bihorac B. Shickel AI4TS 33 1 0 10 Apr 2024
CSA-Trans: Code Structure Aware Transformer for AST Saeyoon Oh Shin Yoo 44 1 0 07 Apr 2024
A Morphology-Based Investigation of Positional Encodings Poulami Ghosh Shikhar Vashishth Raj Dabre Pushpak Bhattacharyya 34 1 0 06 Apr 2024
Equipping Sketch Patches with Context-Aware Positional Encoding for Graphic Sketch Representation Sicong Zang Zhijun Fang 42 0 0 26 Mar 2024
Semantic-Enhanced Representation Learning for Road Networks with Temporal Dynamics Yile Chen Xiucheng Li Gao Cong Zhifeng Bao Cheng Long 21 2 0 18 Mar 2024
Schema-Aware Multi-Task Learning for Complex Text-to-SQL Yangjun Wu Han Wang 34 0 0 09 Mar 2024
SPAFormer: Sequential 3D Part Assembly with Transformers Boshen Xu Sipeng Zheng Qin Jin 44 2 0 09 Mar 2024
Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance Liting Lin Heng Fan Zhipeng Zhang Yaowei Wang Yong-mei Xu Haibin Ling 52 26 0 08 Mar 2024
FViT: A Focal Vision Transformer with Gabor Filter Yulong Shi Mingwei Sun Yongshuai Wang Rui Wang 60 4 0 17 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 134 371 0 09 Feb 2024