Papers citing 'Position Information in Transformers: An Overview'

Title
Explicit Context-Driven Neural Acoustic Modeling for High-Fidelity RIR Generation Chen Si Qianyi Wu Chaitanya Amballa Romit Roy Choudhury 0 0 0 18 Sep 2025
What can we learn from signals and systems in a transformer? Insights for probabilistic modeling and inference architecture Heng-Sheng Chang P. Mehta AI4TS 4 0 0 27 Aug 2025
What do language models model? Transformers, automata, and the format of thought Colin Klein 16 0 0 26 Aug 2025
Fractal Language Modelling by Universal Sequence Maps (USM) Jonas S. Almeida Daniel E Russ Susana Vinga Ines Duarte Lee Mason Praphulla M. S. Bhawsar Aaron Ge Arlindo L. Oliveira J. Balasubramanian 16 0 0 08 Aug 2025
Enhancing Temporal Sensitivity of Large Language Model for Recommendation with Counterfactual Tuning Y. Liu Zhengyi Yang Jiancan Wu Xiang Wang OffRL AI4TS 61 0 0 03 Jul 2025
Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability Yarden Bakish Itamar Zimerman Hila Chefer Lior Wolf 65 1 0 02 Jun 2025
CACTI: Leveraging Copy Masking and Contextual Information to Improve Tabular Data Imputation Aditya Gorla Ryan Wang Zhengtong Liu Ulzee An Sriram Sankararaman 82 0 0 02 Jun 2025
Equivariant Spherical Transformer for Efficient Molecular Modeling Junyi An Xinyu Lu Chao Qu Yunfei Shi Peijia Lin Qianwei Tang Licheng Xu Fenglei Cao Yuan Qi 127 0 0 29 May 2025
Stronger Enforcement of Instruction Hierarchy via Augmented Intermediate Representations Sanjay Kariyappa G. E. Suh 102 0 0 25 May 2025
PaTH Attention: Position Encoding via Accumulating Householder Transformations Songlin Yang Yikang Shen Kaiyue Wen Shawn Tan Mayank Mishra Liliang Ren Rameswar Panda Yoon Kim 142 3 0 22 May 2025
Dual Filter: A Mathematical Framework for Inference using Transformer-like Architectures Heng-Sheng Chang P. Mehta 121 1 0 01 May 2025
The effect of the number of parameters and the number of local feature patches on loss landscapes in distributed quantum neural networks Yoshiaki Kawase 156 0 0 27 Apr 2025
Distilling semantically aware orders for autoregressive image generation Rishav Pramanik Antoine Poupon Juan A. Rodriguez Masih Aminbeidokhti David Vazquez Christopher Pal Zhaozheng Yin M. Pedersoli 127 0 0 23 Apr 2025
Beyond Semantics: Rediscovering Spatial Awareness in Vision-Language Models Jianing Qi Jiawei Liu Hao Tang Zhigang Zhu 212 5 0 21 Mar 2025
Are Transformers Truly Foundational for Robotics? James A. R. Marshall Andrew B. Barron AI4CE 187 1 0 25 Nov 2024
Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers Akhilesh Kakolu Ramarao Kevin Tang Dinah Baer-Henney 196 1 0 28 Oct 2024
Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting Jingjing Xu Caesar Wu Yuan-Fang Li Grégoire Danoy Pascal Bouvry AI4TS 134 2 0 29 Jul 2024
Shared Imagination: LLMs Hallucinate Alike Yilun Zhou Caiming Xiong Silvio Savarese Chien-Sheng Wu HILM 69 3 0 23 Jul 2024
Transformers with Stochastic Competition for Tabular Data Modelling Andreas Voskou Charalambos Christoforou S. Chatzis LMTD 117 2 0 18 Jul 2024
An Effective-Efficient Approach for Dense Multi-Label Action Detection Faegheh Sardari Armin Mustafa Philip J. B. Jackson Adrian Hilton 189 0 0 10 Jun 2024
Contextual Position Encoding: Learning to Count What's Important O. Yu. Golovneva Tianlu Wang Jason Weston Sainbayar Sukhbaatar 154 39 0 29 May 2024
Reference Neural Operators: Learning the Smooth Dependence of Solutions of PDEs on Geometric Deformations Ze Cheng Zhongkai Hao Xiaoqiang Wang Jianing Huang Youjia Wu Xudan Liu Yiru Zhao Songming Liu Hang Su AI4CE 86 4 0 27 May 2024
Positional Knowledge is All You Need: Position-induced Transformer (PiT) for Operator Learning Junfeng Chen Kailiang Wu 179 6 0 15 May 2024
Test-Time Augmentation for Traveling Salesperson Problem Ryo Ishiyama Takahiro Shirakawa Seiichi Uchida Shinnosuke Matsuo 102 0 0 08 May 2024
Learning with 3D rotations, a hitchhiker's guide to SO(3) A. R. Geist Jonas Frey Mikel Zobro Anna Levina Georg Martius 3DH SSL 131 30 0 17 Apr 2024
Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda Johannes Schneider 164 48 0 15 Apr 2024
A Morphology-Based Investigation of Positional Encodings Poulami Ghosh Shikhar Vashishth Raj Dabre Pushpak Bhattacharyya 96 4 0 06 Apr 2024
MEP: Multiple Kernel Learning Enhancing Relative Positional Encoding Length Extrapolation Weiguo Gao 94 1 0 26 Mar 2024
Materials science in the era of large language models: a perspective Ge Lei Ronan Docherty Samuel J. Cooper 106 25 0 11 Mar 2024
Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Records Yingbo Ma Suraj Kolla Dhruv Kaliraman Victoria Nolan Zhenhong Hu ... T. Ozrazgat-Baslanti Tyler J. Loftus Parisa Rashidi A. Bihorac B. Shickel AI4TS 118 1 0 06 Mar 2024
Knowledge of Pretrained Language Models on Surface Information of Tokens Tatsuya Hiraoka Naoaki Okazaki 104 2 0 15 Feb 2024
Accelerating Material Property Prediction using Generically Complete Isometry Invariants Jonathan Balasingham Viktor Zamaraev V. Kurlin 129 7 0 22 Jan 2024
SymTC: A Symbiotic Transformer-CNN Net for Instance Segmentation of Lumbar Spine MRI Jiasong Chen Linchen Qian Linhai Ma Timur Urakov Weiyong Gu Liang Liang MedIm 163 9 0 17 Jan 2024
Code Simulation Challenges for Large Language Models Emanuele La Malfa Christoph Weinhuber Orazio Torre Fangru Lin Samuele Marro Anthony Cohn Nigel Shadbolt Michael Wooldridge LLMAG LRM 121 10 0 17 Jan 2024
Graph Language Models Moritz Plenz Anette Frank KELM AI4CE 138 7 0 13 Jan 2024
Algebraic Positional Encodings Konstantinos Kogkalidis Jean-Philippe Bernardy Vikas Garg 66 4 0 26 Dec 2023
Graph Neural Networks with Diverse Spectral Filtering Jingwei Guo Kaizhu Huang Xinping Yi Rui Zhang 210 14 0 14 Dec 2023
Prompt Cache: Modular Attention Reuse for Low-Latency Inference Caihua Li Guojun Chen Seung-seob Lee Nikhil Sarda Anurag Khandelwal Lin Zhong 133 101 0 07 Nov 2023
Transformers as Graph-to-Graph Models James Henderson Alireza Mohammadshahi Andrei Catalin Coman Lesly Miculicich GNN 96 6 0 27 Oct 2023
The Locality and Symmetry of Positional Encodings Lihu Chen Gaël Varoquaux Fabian M. Suchanek 106 1 0 19 Oct 2023
From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers Shaoxiong Duan Yining Shi Wei Xu 160 13 0 18 Oct 2023
DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions Haochen Wang Junsong Fan Yuxi Wang Kaiyou Song Tong Wang Zhaoxiang Zhang 132 21 0 07 Sep 2023
PAT: Position-Aware Transformer for Dense Multi-Label Action Detection Faegheh Sardari A. Mustafa Philip J. B. Jackson A. Hilton ViT 120 6 0 09 Aug 2023
A Survey of Techniques for Optimizing Transformer Inference Krishna Teja Chitty-Venkata Sparsh Mittal M. Emani V. Vishwanath Arun Somani 159 92 0 16 Jul 2023
Pseudo-rigid body networks: learning interpretable deformable object dynamics from partial observations Shamil Mamedov A. R. Geist Jan Swevers Sebastian Trimpe AI4CE 71 3 0 16 Jul 2023
Monotonic Location Attention for Length Generalization Jishnu Ray Chowdhury Cornelia Caragea LLMAG 100 9 0 31 May 2023
Improving Position Encoding of Transformers for Multivariate Time Series Classification Navid Mohammadi Foumani Chang Wei Tan Geoffrey I. Webb Mahsa Salehi AI4TS 116 102 0 26 May 2023
Causal Decision Transformer for Recommender Systems via Offline Reinforcement Learning Siyu Wang Xiaocong Chen Dietmar Jannach Lina Yao CML OffRL 143 30 0 17 Apr 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 172 119 0 20 Mar 2023
Universal Morphology Control via Contextual Modulation Zheng Xiong Jacob Beck Shimon Whiteson 123 16 0 22 Feb 2023