Tracking linguistic information in transformer-based sentence embeddings through targeted sparsification

25 July 2024

Papers citing "Tracking linguistic information in transformer-based sentence embeddings through targeted sparsification"

17 / 17 papers shown

Title
Towards Automated Circuit Discovery for Mechanistic Interpretability Arthur Conmy Augustine N. Mavor-Parker Aengus Lynch Stefan Heimersheim Adrià Garriga-Alonso 64 318 0 28 Apr 2023
Semantic Role Labeling Meets Definition Modeling: Using Natural Language to Describe Predicate-Argument Structures Simone Conia Edoardo Barba Alessandro Sciré Roberto Navigli 69 7 0 02 Dec 2022
SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features Juri Opitz Anette Frank 71 37 0 14 Jun 2022
Low-Complexity Probing via Finding Subnetworks Steven Cao Victor Sanh Alexander M. Rush 43 54 0 08 Apr 2021
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks Róbert Csordás Sjoerd van Steenkiste Jürgen Schmidhuber 91 95 0 05 Oct 2020
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Pengcheng He Xiaodong Liu Jianfeng Gao Weizhu Chen AAML 163 2,750 0 05 Jun 2020
Toward Better Storylines with Sentence-Level Language Models Daphne Ippolito David Grangier Douglas Eck Chris Callison-Burch LRM 51 24 0 11 May 2020
A Primer in BERTology: What we know about how BERT works Anna Rogers Olga Kovaleva Anna Rumshisky OffRL 92 1,498 0 27 Feb 2020
Winning the Lottery with Continuous Sparsification Pedro H. P. Savarese Hugo Silva Michael Maire 76 137 0 10 Dec 2019
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 373 6,467 0 26 Sep 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,301 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
What Does BERT Look At? An Analysis of BERT's Attention Kevin Clark Urvashi Khandelwal Omer Levy Christopher D. Manning MILM 223 1,602 0 11 Jun 2019
What do you learn from context? Probing for sentence structure in contextualized word representations Ian Tenney Patrick Xia Berlin Chen Alex Jinpeng Wang Adam Poliak ... Najoung Kim Benjamin Van Durme Samuel R. Bowman Dipanjan Das Ellie Pavlick 183 865 0 15 May 2019
BERT Rediscovers the Classical NLP Pipeline Ian Tenney Dipanjan Das Ellie Pavlick MILM SSeg 138 1,478 0 15 May 2019
Assessing BERT's Syntactic Abilities Yoav Goldberg 73 496 0 16 Jan 2019
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 349 896 0 03 May 2018