A Multiscale Visualization of Attention in the Transformer Model

12 June 2019

Papers citing "A Multiscale Visualization of Attention in the Transformer Model"

50 / 101 papers shown

Title
LISA: Learning Interpretable Skill Abstractions from Language Divyansh Garg Skanda Vaidyanath Kuno Kim Jiaming Song Stefano Ermon LM&Ro OffRL 158 29 0 28 Feb 2022
Do Transformers know symbolic rules, and would we know if they did? Tommi Gröndahl Yu-Wen Guo Nirmal Asokan 33 0 0 19 Feb 2022
Punctuation restoration in Swedish through fine-tuned KB-BERT J. Nilsson 21 0 0 14 Feb 2022
Pre-Trained Language Models for Interactive Decision-Making Shuang Li Xavier Puig Chris Paxton Yilun Du Clinton Jia Wang ... Anima Anandkumar Jacob Andreas Igor Mordatch Antonio Torralba Yuke Zhu LM&Ro 50 250 0 03 Feb 2022
A Survey on Gender Bias in Natural Language Processing Karolina Stañczak Isabelle Augenstein 30 111 0 28 Dec 2021
Is "My Favorite New Movie" My Favorite Movie? Probing the Understanding of Recursive Noun Phrases Qing Lyu Hua Zheng Daoxin Li Li Zhang Marianna Apidianaki Chris Callison-Burch 24 4 0 15 Dec 2021
Discovering Explanatory Sentences in Legal Case Decisions Using Pre-trained Language Models Jaromír Šavelka Kevin D. Ashley ELM AILaw 37 10 0 14 Dec 2021
LMdiff: A Visual Diff Tool to Compare Language Models Hendrik Strobelt Benjamin Hoover Arvind Satyanarayan Sebastian Gehrmann VLM 37 19 0 02 Nov 2021
GenNI: Human-AI Collaboration for Data-Backed Text Generation Hendrik Strobelt J. Kinley Robert Krueger Johanna Beyer Hanspeter Pfister Alexander M. Rush 31 23 0 19 Oct 2021
Detecting Gender Bias in Transformer-based Models: A Case Study on BERT Bingbing Li Hongwu Peng Rajat Sainju Junhuan Yang Lei Yang Yueying Liang Weiwen Jiang Binghui Wang Hang Liu Caiwen Ding 32 12 0 15 Oct 2021
BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models Kangjie Chen Yuxian Meng Xiaofei Sun Shangwei Guo Tianwei Zhang Jiwei Li Chun Fan SILM 34 106 0 06 Oct 2021
Understanding and Overcoming the Challenges of Efficient Transformer Quantization Yelysei Bondarenko Markus Nagel Tijmen Blankevoort MQ 25 133 0 27 Sep 2021
Automated and Explainable Ontology Extension Based on Deep Learning: A Case Study in the Chemical Domain A. Memariani Martin Glauer Fabian Neuhaus Till Mossakowski Janna Hastings 36 5 0 19 Sep 2021
Puzzle Solving without Search or Human Knowledge: An Unnatural Language Approach David Noever Ryerson Burdick ReLM 171 7 0 07 Sep 2021
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP Raymond Li Wen Xiao Lanjun Wang Hyeju Jang Giuseppe Carenini ViT 31 23 0 31 Aug 2021
Multilingual Multi-Aspect Explainability Analyses on Machine Reading Comprehension Models Yiming Cui Weinan Zhang Wanxiang Che Ting Liu Zhigang Chen Shijin Wang LRM 25 9 0 26 Aug 2021
An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers Lu Wang Munif Ishad Mujib Jake Williams G. Demiris Jina Huh-Yoo AI4MH 32 32 0 28 Jul 2021
Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoff Michael J. Naylor C. French Samantha R. Terker Uday Kamath 44 10 0 12 Jul 2021
Elbert: Fast Albert with Confidence-Window Based Early Exit Keli Xie Siyuan Lu Meiqi Wang Zhongfeng Wang 19 20 0 01 Jul 2021
Semantic-aware Binary Code Representation with BERT Hyungjoon Koo Soyeon Park Daejin Choi Taesoo Kim 27 23 0 10 Jun 2021
Do Models Learn the Directionality of Relations? A New Evaluation: Relation Direction Recognition Shengfei Lyu Xingyu Wu Jinlong Li Qiuju Chen Huanhuan Chen 35 5 0 19 May 2021
"Subverting the Jewtocracy": Online Antisemitism Detection Using Multimodal Deep Learning Mohit Chandra D. Pailla Himanshu Bhatia AadilMehdi J. Sanchawala Manish Gupta Manish Shrivastava Ponnurangam Kumaraguru 19 38 0 13 Apr 2021
The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation Yuankai Qi Zizheng Pan Yicong Hong Ming-Hsuan Yang Anton Van Den Hengel Qi Wu LM&Ro 34 68 0 09 Apr 2021
VisQA: X-raying Vision and Language Reasoning in Transformers Theo Jaunet Corentin Kervadec Romain Vuillemot G. Antipov M. Baccouche Christian Wolf 19 26 0 02 Apr 2021
Synthesis of Compositional Animations from Textual Descriptions Anindita Ghosh N. Cheema Cennet Oguz Christian Theobalt P. Slusallek 31 171 0 26 Mar 2021
Dodrio: Exploring Transformer Models with Interactive Visualization Zijie J. Wang Robert Turko Duen Horng Chau 40 35 0 26 Mar 2021
GPT Understands, Too Xiao Liu Yanan Zheng Zhengxiao Du Ming Ding Yujie Qian Zhilin Yang Jie Tang VLM 87 1,147 0 18 Mar 2021
LazyFormer: Self Attention with Lazy Update Chengxuan Ying Guolin Ke Di He Tie-Yan Liu 25 15 0 25 Feb 2021
A scalable approach for developing clinical risk prediction applications in different hospitals Hong Sun K. Depraetere Laurent Meesseman J. D. Roo Martijn Vanbiervliet J. Baerdemaeker Herman Muys V. Dossow N. Hulde Ralph Szymanowsky 15 18 0 21 Jan 2021
Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing Xi Lin R. Socher Caiming Xiong LMTD 20 207 0 23 Dec 2020
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction Seyone Chithrananda Gabriel Grand Bharath Ramsundar AI4CE 37 389 0 19 Oct 2020
The elephant in the interpretability room: Why use attention as explanation when we have saliency methods? Jasmijn Bastings Katja Filippova XAI LRM 59 174 0 12 Oct 2020
Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task Dongyeop Kang Eduard H. Hovy LRM 42 24 0 11 Oct 2020
Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders Jue Wang Wei Lu 26 225 0 08 Oct 2020
Transformer-GCRF: Recovering Chinese Dropped Pronouns with General Conditional Random Fields Jingxuan Yang Kerui Xu Jun Xu Si Li Sheng Gao Jun Guo Ji-Rong Wen Nianwen Xue 24 5 0 07 Oct 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 66 1,527 0 30 Sep 2020
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models Joseph F DeRose Jiayao Wang M. Berger 17 83 0 03 Sep 2020
BERTology Meets Biology: Interpreting Attention in Protein Language Models Jesse Vig Ali Madani Lav Varshney Caiming Xiong R. Socher Nazneen Rajani 29 289 0 26 Jun 2020
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Peter Hawkins Jared Davis David Belanger Lucy J. Colwell Adrian Weller 39 84 0 05 Jun 2020
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction L. Rasmy Yang Xiang Z. Xie Cui Tao Degui Zhi AI4MH LM&MA 29 659 0 22 May 2020
IMoJIE: Iterative Memory-Based Joint Open Information Extraction Keshav Kolluru Samarth Aggarwal Vipul Rathore Mausam Soumen Chakrabarti VLM 27 72 0 17 May 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 246 1,454 0 18 Mar 2020
ProGen: Language Modeling for Protein Generation Ali Madani Bryan McCann Nikhil Naik N. Keskar N. Anand Raphael R. Eguchi Po-Ssu Huang R. Socher 34 275 0 08 Mar 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 47 1,214 0 25 Feb 2020
Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks Carlos Aspillaga Andrés Carvallo Vladimir Araujo ELM 47 31 0 14 Feb 2020
End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models John Giorgi Xindi Wang Nicola Sahar W. Shin Gary D. Bader Bo Wang 11 36 0 20 Dec 2019
Knowledge Guided Named Entity Recognition for BioMedical Text Pratyay Banerjee Kuntal Kumar Pal M. Devarakonda Chitta Baral 24 0 0 10 Nov 2019
Generalizing Natural Language Analysis through Span-relation Representations Zhengbao Jiang Wenyuan Xu Jun Araki Graham Neubig 30 60 0 10 Nov 2019
Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings Dhruva Sahrawat Debanjan Mahata Mayank Kulkarni Haimin Zhang Rakesh Gosangi Amanda Stent Agniv Sharma Yaman Kumar Singla R. Shah Roger Zimmermann 14 30 0 19 Oct 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 82 1,651 0 22 Aug 2019