The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives

3 September 2019

Papers citing "The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives"

33 / 133 papers shown

Title
Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT Zekai Chen Dingshuo Chen Xiao Zhang Zixuan Yuan Xiuzhen Cheng AI4TS 28 331 0 08 Apr 2021
Exploring the Role of BERT Token Representations to Explain Sentence Probing Results Hosein Mohebbi Ali Modarressi Mohammad Taher Pilehvar MILM 27 23 0 03 Apr 2021
Neural Machine Translation: A Review of Methods, Resources, and Tools Zhixing Tan Shuo Wang Zonghan Yang Gang Chen Xuancheng Huang Maosong Sun Yang Liu 3DV AI4TS 19 105 0 31 Dec 2020
Understanding Pure Character-Based Neural Machine Translation: The Case of Translating Finnish into English Gongbo Tang Rico Sennrich Joakim Nivre 30 7 0 06 Nov 2020
Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases Ryan Steed Aylin Caliskan SSL 27 156 0 28 Oct 2020
Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation Yongchang Hao Shilin He Wenxiang Jiao Zhaopeng Tu Michael Lyu Xing Wang 98 28 0 24 Oct 2020
PyMT5: multi-mode translation of natural language and Python code with transformers Colin B. Clement Dawn Drain Jonathan Timcheck Alexey Svyatkovskiy Neel Sundaresan 27 152 0 07 Oct 2020
BERT Knows Punta Cana is not just beautiful, it's gorgeous: Ranking Scalar Adjectives with Contextualised Representations Aina Garí Soler Marianna Apidianaki 17 19 0 06 Oct 2020
Which *BERT? A Survey Organizing Contextualized Encoders Patrick Xia Shijie Wu Benjamin Van Durme 26 50 0 02 Oct 2020
DeLighT: Deep and Light-weight Transformer Sachin Mehta Marjan Ghazvininejad Srini Iyer Luke Zettlemoyer Hannaneh Hajishirzi VLM 25 32 0 03 Aug 2020
Neural Language Generation: Formulation, Methods, and Evaluation Cristina Garbacea Qiaozhu Mei 45 30 0 31 Jul 2020
Building Interpretable Interaction Trees for Deep NLP Models Die Zhang Huilin Zhou Hao Zhang Xiaoyi Bao Da Huo Ruizhao Chen Xu Cheng Mengyue Wu Quanshi Zhang FAtt 16 3 0 29 Jun 2020
Emergence of Separable Manifolds in Deep Language Representations Jonathan Mamou Hang Le Miguel Angel del Rio Cory Stephenson Hanlin Tang Yoon Kim SueYeon Chung AAML AI4CE 22 38 0 01 Jun 2020
Staying True to Your Word: (How) Can Attention Become Explanation? Martin Tutek Jan Snajder 8 27 0 19 May 2020
Similarity Analysis of Contextual Word Representation Models John M. Wu Yonatan Belinkov Hassan Sajjad Nadir Durrani Fahim Dalvi James R. Glass 51 73 0 03 May 2020
Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance? Abhilasha Ravichander Yonatan Belinkov Eduard H. Hovy 39 123 0 02 May 2020
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking Nicola De Cao M. Schlichtkrull Wilker Aziz Ivan Titov 25 89 0 30 Apr 2020
Asking without Telling: Exploring Latent Ontologies in Contextual Representations Julian Michael Jan A. Botha Ian Tenney 31 42 0 29 Apr 2020
What Happens To BERT Embeddings During Fine-tuning? Amil Merchant Elahe Rahimtoroghi Ellie Pavlick Ian Tenney 20 176 0 29 Apr 2020
Assessing the Bilingual Knowledge Learned by Neural Machine Translation Models Shilin He Xing Wang Shuming Shi M. Lyu Zhaopeng Tu 27 5 0 28 Apr 2020
Word Interdependence Exposes How LSTMs Compose Representations Naomi Saphra Adam Lopez 39 3 0 27 Apr 2020
Quantifying the Contextualization of Word Representations with Semantic Class Probing Mengjie Zhao Philipp Dufter Yadollah Yaghoobzadeh Hinrich Schütze 25 27 0 25 Apr 2020
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings Masoud Jalili Sabet Philipp Dufter François Yvon Hinrich Schütze 23 226 0 18 Apr 2020
On the Effect of Dropping Layers of Pre-trained Transformer Models Hassan Sajjad Fahim Dalvi Nadir Durrani Preslav Nakov 31 132 0 08 Apr 2020
Evaluating Multimodal Representations on Visual Semantic Textual Similarity Oier López de Lacalle Ander Salaberria Aitor Soroa Etxabe Gorka Azkune Eneko Agirre 19 2 0 04 Apr 2020
Information-Theoretic Probing with Minimum Description Length Elena Voita Ivan Titov 23 270 0 27 Mar 2020
A Survey of Deep Learning for Scientific Discovery M. Raghu Erica Schmidt OOD AI4CE 38 120 0 26 Mar 2020
Felix: Flexible Text Editing Through Tagging and Insertion Jonathan Mallinson Aliaksei Severyn Eric Malmi Guillermo Garrido 26 76 0 24 Mar 2020
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers Hongfei Xu Josef van Genabith Qiuhui Liu Deyi Xiong 20 3 0 21 Mar 2020
A Primer in BERTology: What we know about how BERT works Anna Rogers Olga Kovaleva Anna Rumshisky OffRL 35 1,458 0 27 Feb 2020
Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation Alessandro Raganato Yves Scherrer Jörg Tiedemann 32 92 0 24 Feb 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 118 19,493 0 23 Oct 2019
Transfusion: Understanding Transfer Learning for Medical Imaging M. Raghu Chiyuan Zhang Jon M. Kleinberg Samy Bengio MedIm 30 972 0 14 Feb 2019