Routing Networks and the Challenges of Modular and Compositional Computation

29 April 2019

Papers citing "Routing Networks and the Challenges of Modular and Compositional Computation"

26 / 26 papers shown

Title
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts R. Teo Tan M. Nguyen MoE 33 3 0 18 Oct 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 46 0 0 10 Oct 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 41 2 0 09 Sep 2024
Revisiting Single-gated Mixtures of Experts Amelie Royer I. Karmanov Andrii Skliar B. Bejnordi Tijmen Blankevoort MoE MoMe 36 6 0 11 Apr 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić E. Ponti MoMe OOD 32 73 0 22 Feb 2023
Reusable Slotwise Mechanisms Trang Nguyen Amin Mansouri Kanika Madan Khuong N. Nguyen Kartik Ahuja Dianbo Liu Yoshua Bengio OCL 28 4 0 21 Feb 2023
Learning Label Modular Prompts for Text Classification in the Wild Hailin Chen Amrita Saha Shafiq R. Joty Steven C. H. Hoi OOD VLM 18 5 0 30 Nov 2022
A Short Survey of Systematic Generalization Yuanpeng Li AI4CE 38 1 0 22 Nov 2022
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks Andrew M. Saxe Shagun Sodhani Sam Lewallen AI4CE 28 34 0 21 Jul 2022
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition Jorge Armando Mendez Mendez Eric Eaton KELM CLL 32 27 0 15 Jul 2022
Spot-adaptive Knowledge Distillation Jie Song Ying Chen Jingwen Ye Mingli Song 20 72 0 05 May 2022
Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions Vibhor Gupta Jyoti Narwariya Pankaj Malhotra L. Vig Gautam M. Shroff AI4TS 26 20 0 14 Mar 2022
Combining Modular Skills in Multitask Learning E. Ponti Alessandro Sordoni Yoshua Bengio Siva Reddy MoE 12 37 0 28 Feb 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 27 177 0 02 Feb 2022
All-In-One: Artificial Association Neural Networks Seokjun Kim Jaeeun Jang Hyeoncheol Kim 32 0 0 31 Oct 2021
Dynamic Inference with Neural Interpreters Nasim Rahaman Muhammad Waleed Gondal S. Joshi Peter V. Gehler Yoshua Bengio Francesco Locatello Bernhard Schölkopf 34 31 0 12 Oct 2021
Discrete-Valued Neural Communication Dianbo Liu DianboLiu Alex Lamb Kenji Kawaguchi Anirudh Goyal Chen Sun Michael C. Mozer Yoshua Bengio 23 50 0 06 Jul 2021
Scaling Vision with Sparse Mixture of Experts C. Riquelme J. Puigcerver Basil Mustafa Maxim Neumann Rodolphe Jenatton André Susano Pinto Daniel Keysers N. Houlsby MoE 17 575 0 10 Jun 2021
Neural Production Systems: Learning Rule-Governed Visual Dynamics Anirudh Goyal Aniket Didolkar Nan Rosemary Ke Charles Blundell Philippe Beaudoin N. Heess Michael C. Mozer Yoshua Bengio OCL 50 82 0 02 Mar 2021
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks Róbert Csordás Sjoerd van Steenkiste Jürgen Schmidhuber 44 87 0 05 Oct 2020
Multi-Task Learning with Deep Neural Networks: A Survey M. Crawshaw CVBM 39 609 0 10 Sep 2020
A Study of Compositional Generalization in Neural Models Tim Klinger D. Adjodah Vincent Marois Joshua Joseph Matthew D Riemer Alex Pentland Murray Campbell CoGe NAI 28 12 0 16 Jun 2020
Multi-Task Reinforcement Learning with Soft Modularization Ruihan Yang Huazhe Xu Yi Wu Xiaolong Wang 27 175 0 30 Mar 2020
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 21 1,165 0 19 Jan 2020
Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives Anirudh Goyal Shagun Sodhani Jonathan Binas Xue Bin Peng Sergey Levine Yoshua Bengio 19 47 0 25 Jun 2019
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,329 0 05 Nov 2016