BERT Rediscovers the Classical NLP Pipeline

15 May 2019

Papers citing "BERT Rediscovers the Classical NLP Pipeline"

50 / 296 papers shown

Title
Layer-wise Analysis of a Self-supervised Speech Representation Model Ankita Pasad Ju-Chieh Chou Karen Livescu SSL 26 288 0 10 Jul 2021
A Novel Deep Reinforcement Learning Based Stock Direction Prediction using Knowledge Graph and Community Aware Sentiments Anil Berk Altuner Zeynep Hilal Kilimci AIFin 17 15 0 02 Jul 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 24 93 0 30 Jun 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning Colin Wei Sang Michael Xie Tengyu Ma 24 97 0 17 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 43 815 0 14 Jun 2021
Causal Abstractions of Neural Networks Atticus Geiger Hanson Lu Thomas Icard Christopher Potts NAI CML 17 218 0 06 Jun 2021
LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond Daniel Loureiro A. Jorge Jose Camacho-Collados 33 26 0 26 May 2021
A comparative evaluation and analysis of three generations of Distributional Semantic Models Alessandro Lenci Magnus Sahlgren Patrick Jeuniaux Amaru Cuba Gyllensten Martina Miliani 29 50 0 20 May 2021
Compositional Processing Emerges in Neural Networks Solving Math Problems Jacob Russin Roland Fernandez Hamid Palangi Eric Rosen Nebojsa Jojic P. Smolensky Jianfeng Gao 14 13 0 19 May 2021
How is BERT surprised? Layerwise detection of linguistic anomalies Bai Li Zining Zhu Guillaume Thomas Yang Xu Frank Rudzicz 27 31 0 16 May 2021
FNet: Mixing Tokens with Fourier Transforms James Lee-Thorp Joshua Ainslie Ilya Eckstein Santiago Ontanon 47 517 0 09 May 2021
Understanding by Understanding Not: Modeling Negation in Language Models Arian Hosseini Siva Reddy Dzmitry Bahdanau R. Devon Hjelm Alessandro Sordoni Rameswar Panda 22 87 0 07 May 2021
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality Adithya V Ganesan Matthew Matero Aravind Reddy Ravula Huy-Hien Vu H. A. Schwartz 27 35 0 07 May 2021
Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses Aina Garí Soler Marianna Apidianaki MILM 211 68 0 29 Apr 2021
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? William Merrill Yoav Goldberg Roy Schwartz Noah A. Smith 25 67 0 22 Apr 2021
Knowledge Neurons in Pretrained Transformers Damai Dai Li Dong Y. Hao Zhifang Sui Baobao Chang Furu Wei KELM MU 19 417 0 18 Apr 2021
A multilabel approach to morphosyntactic probing Naomi Tachikawa Shapiro Amandalynne Paullada Shane Steinert-Threlkeld 37 10 0 17 Apr 2021
Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models Matteo Alleman J. Mamou Miguel Rio Hanlin Tang Yoon Kim SueYeon Chung NAI 35 17 0 15 Apr 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 45 243 0 14 Apr 2021
What's in your Head? Emergent Behaviour in Multi-Task Transformer Models Mor Geva Uri Katz Aviv Ben-Arie Jonathan Berant LRM 43 11 0 13 Apr 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 32 27 0 13 Apr 2021
CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation J. Clark Dan Garrette Iulia Turc John Wieting 36 210 0 11 Mar 2021
The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models Go Inoue Bashar Alhafni Nurpeiis Baimukan Houda Bouamor Nizar Habash 35 224 0 11 Mar 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 34 8 0 02 Mar 2021
Investigating the Limitations of Transformers with Simple Arithmetic Tasks Rodrigo Nogueira Zhiying Jiang Jimmy J. Li LRM 24 122 0 25 Feb 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 269 346 0 01 Feb 2021
Language Modelling as a Multi-Task Problem Leon Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 26 13 0 27 Jan 2021
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT Benjamin Muller Yanai Elazar Benoît Sagot Djamé Seddah LRM 29 71 0 26 Jan 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 43 10 0 20 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 52 132 0 09 Jan 2021
Reservoir Transformers Sheng Shen Alexei Baevski Ari S. Morcos Kurt Keutzer Michael Auli Douwe Kiela 35 17 0 30 Dec 2020
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 39 745 0 29 Dec 2020
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade Lei Li Yankai Lin Deli Chen Shuhuai Ren Peng Li Jie Zhou Xu Sun 29 51 0 29 Dec 2020
Parameter-Efficient Transfer Learning with Diff Pruning Demi Guo Alexander M. Rush Yoon Kim 13 385 0 14 Dec 2020
Self-Explaining Structures Improve NLP Models Zijun Sun Chun Fan Qinghong Han Xiaofei Sun Yuxian Meng Fei Wu Jiwei Li MILM XAI LRM FAtt 43 38 0 03 Dec 2020
Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Framework S. Yadav Jainish Chauhan Joy Prakash Sain K. Thirunarayan A. Sheth Jeremiah A. Schumm 21 42 0 12 Nov 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 29 136 0 10 Nov 2020
CxGBERT: BERT meets Construction Grammar Harish Tayyar Madabushi Laurence Romain Dagmar Divjak P. Milin 19 39 0 09 Nov 2020
Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation Diego Kozlowski Jennifer Dusdal Jun Pang A. Zilian 18 13 0 05 Nov 2020
Introducing various Semantic Models for Amharic: Experimentation and Evaluation with multiple Tasks and Datasets Seid Muhie Yimam A. Ayele Gopalakrishnan Venkatesh Ibrahim Gashaw Christian Biemann 30 27 0 02 Nov 2020
ABNIRML: Analyzing the Behavior of Neural IR Models Sean MacAvaney Sergey Feldman Nazli Goharian Doug Downey Arman Cohan 17 49 0 02 Nov 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 95 142 0 24 Oct 2020
Long Document Ranking with Query-Directed Sparse Transformer Jyun-Yu Jiang Chenyan Xiong Chia-Jung Lee Wei Wang 30 25 0 23 Oct 2020
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling Wenxuan Zhou Kevin Huang Tengyu Ma Jing Huang 24 273 0 21 Oct 2020
Towards Interpreting BERT for Reading Comprehension Based QA Sahana Ramnath Preksha Nema Deep Sahni Mitesh M. Khapra 42 30 0 18 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 242 611 0 13 Oct 2020
Probing Pretrained Language Models for Lexical Semantics Ivan Vulić E. Ponti Robert Litschko Goran Glavas Anna Korhonen KELM 33 233 0 12 Oct 2020
On the Sub-Layer Functionalities of Transformer Decoder Yilin Yang Longyue Wang Shuming Shi Prasad Tadepalli Stefan Lee Zhaopeng Tu 26 27 0 06 Oct 2020
Linguistic Profiling of a Neural Language Model Alessio Miaschi D. Brunato F. Dell’Orletta Giulia Venturi 36 46 0 05 Oct 2020
Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling Yan Shvartzshnaider Ananth Balashankar Vikas Patidar Thomas Wies L. Subramanian 19 4 0 01 Oct 2020