On the Expressivity Role of LayerNorm in Transformers' Attention

4 May 2023

Papers citing "On the Expressivity Role of LayerNorm in Transformers' Attention"

4 / 4 papers shown

Title
Lambda-Skip Connections: the architectural component that prevents Rank Collapse Federico Arangath Joseph Jerome Sieber M. Zeilinger Carmen Amo Alonso 33 0 0 14 Oct 2024
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference Anton Xue Avishree Khare Rajeev Alur Surbhi Goel Eric Wong 58 2 0 21 Jun 2024
Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers Andy Yang David Chiang 38 8 0 05 Apr 2024
Grappa -- A Machine Learned Molecular Mechanics Force Field Leif Seute Eric Hartmann Jan Stühmer Frauke Gräter 29 3 0 25 Mar 2024