Unbiased Gradient Estimation with Balanced Assignments for Mixtures of Experts

24 September 2021

Papers citing "Unbiased Gradient Estimation with Balanced Assignments for Mixtures of Experts"

2 / 2 papers shown

Title
Stochastic gradient descent with gradient estimator for categorical features Paul Peseux Maxime Bérar Thierry Paquet Victor Nicollet 25 0 0 08 Sep 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 27 177 0 02 Feb 2022