When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective

14 March 2025

Papers citing "When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective"

4 / 4 papers shown

Title
Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks Luca Arnaboldi Bruno Loureiro Ludovic Stephan Florent Krzakala Lenka Zdeborová 67 0 0 03 Jun 2025
Mixture of Parrots: Experts improve memorization more than reasoning Samy Jelassi Clara Mohri David Brandfonbrener Alex Gu Nikhil Vyas Nikhil Anand David Alvarez-Melis Yuanzhi Li Sham Kakade Eran Malach MoE 120 5 0 24 Oct 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 101 8 0 14 Aug 2024
Implicit Bias and Fast Convergence Rates for Self-attention Bhavya Vasudeva Puneesh Deora Christos Thrampoulidis 117 21 0 08 Feb 2024