The Architectural Bottleneck Principle

11 November 2022

Papers citing "The Architectural Bottleneck Principle"

7 / 7 papers shown

Title
Linguistic Interpretability of Transformer-based Language Models: a systematic review Miguel López-Otal Jorge Gracia Jordi Bernad Carlos Bobed Lucía Pitarch-Ballesteros Emma Anglés-Herrero VLM 81 0 0 09 Apr 2025
Low-Complexity Probing via Finding Subnetworks Steven Cao Victor Sanh Alexander M. Rush 37 51 0 08 Apr 2021
Pareto Probing: Trading Off Accuracy for Complexity Tiago Pimentel Naomi Saphra Adina Williams Ryan Cotterell 46 60 0 05 Oct 2020
A Theory of Usable Information Under Computational Constraints Yilun Xu Shengjia Zhao Jiaming Song Russell Stewart Stefano Ermon 62 171 0 25 Feb 2020
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 274 6,420 0 26 Sep 2019
BERT Rediscovers the Classical NLP Pipeline Ian Tenney Dipanjan Das Ellie Pavlick MILM SSeg 105 1,458 0 15 May 2019
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 285 888 0 03 May 2018