Why bother with geometry? On the relevance of linear decompositions of Transformer embeddings

10 October 2023

Papers citing "Why bother with geometry? On the relevance of linear decompositions of Transformer embeddings"

17 / 17 papers shown

Title
Explaining How Transformers Use Context to Build Predictions Javier Ferrando Gerard I. Gállego Ioannis Tsiamas Marta R. Costa-jussá 53 36 0 21 May 2023
Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions Byung-Doh Oh William Schuler 56 2 0 17 May 2023
Analyzing Transformers in Embedding Space Guy Dar Mor Geva Ankit Gupta Jonathan Berant 60 92 0 06 Sep 2022
How to Dissect a Muppet: The Structure of Transformer Embedding Spaces Timothee Mickus Denis Paperno Mathieu Constant 69 23 0 07 Jun 2022
Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer Javier Ferrando Gerard I. Gállego Belen Alastruey Carlos Escolano Marta R. Costa-jussá 170 46 0 23 May 2022
GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformers Ali Modarressi Mohsen Fayyaz Yadollah Yaghoobzadeh Mohammad Taher Pilehvar ViT 67 35 0 06 May 2022
Measuring the Mixing of Contextual Information in the Transformer Javier Ferrando Gerard I. Gállego Marta R. Costa-jussá 87 54 0 08 Mar 2022
All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality William Timkey Marten van Schijndel 280 116 0 09 Sep 2021
Of Non-Linearity and Commutativity in BERT Sumu Zhao Damian Pascual Gino Brunner Roger Wattenhofer 87 17 0 12 Jan 2021
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 163 843 0 29 Dec 2020
Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation Elena Voita Rico Sennrich Ivan Titov 59 86 0 21 Oct 2020
The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT Jörg Tiedemann 221 171 0 13 Oct 2020
Learning to Deceive with Attention-Based Explanations Danish Pruthi Mansi Gupta Bhuwan Dhingra Graham Neubig Zachary Chase Lipton 80 193 0 17 Sep 2019
Attention is not not Explanation Sarah Wiegreffe Yuval Pinter XAI AAML FAtt 120 914 0 13 Aug 2019
Is Attention Interpretable? Sofia Serrano Noah A. Smith 108 684 0 09 Jun 2019
Attention is not Explanation Sarthak Jain Byron C. Wallace FAtt 145 1,328 0 26 Feb 2019
Marian: Fast Neural Machine Translation in C++ Marcin Junczys-Dowmunt Roman Grundkiewicz Tomasz Dwojak Hieu T. Hoang Kenneth Heafield ... Ulrich Germann Alham Fikri Aji Nikolay Bogoychev André F. T. Martins Alexandra Birch 92 718 0 01 Apr 2018