Mixture of Parrots: Experts improve memorization more than reasoning

24 October 2024

Papers citing "Mixture of Parrots: Experts improve memorization more than reasoning"

4 / 4 papers shown

Title
The power of fine-grained experts: Granularity boosts expressivity in Mixture of Experts Enric Boix Adserà Philippe Rigollet MoE 28 0 0 11 May 2025
When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective Alireza Mousavi-Hosseini Clayton Sanford Denny Wu Murat A. Erdogdu 48 0 0 14 Mar 2025
Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models Samira Abnar Harshay Shah Dan Busbridge Alaaeldin Mohamed Elnouby Ali J. Susskind Vimal Thilak MoE LRM 39 5 0 28 Jan 2025
Theoretical limitations of multi-layer Transformer Lijie Chen Binghui Peng Hongxun Wu AI4CE 72 6 0 04 Dec 2024