Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning

3 November 2017

Papers citing "Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning"

50 / 62 papers shown

Title
R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts Zhongyang Li Ziyue Li Dinesh Manocha MoE 53 0 0 27 Feb 2025
E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detection Jiaqing Zhang Mingxiang Cao Weiying Xie Jie Lei Daixun Li Wenbo Huang Yunsong Li Xue Yang 62 5 0 28 Jan 2025
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 55 1 0 18 Oct 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 46 0 0 10 Oct 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 48 2 0 09 Sep 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 52 18 0 08 Jul 2024
Attention as a Hypernetwork Simon Schug Seijin Kobayashi Yassir Akram João Sacramento Razvan Pascanu GNN 37 3 0 09 Jun 2024
Differentiable Weight Masks for Domain Transfer Samarth Khanna Skanda Vaidyanath Akash Velu 34 0 0 26 Aug 2023
Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives Chuntao Ding Zhichao Lu Shangguang Wang Ran Cheng Vishnu Naresh Boddeti MoMe 21 16 0 03 Aug 2023
Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms Peiyao Xiao Hao Ban Kaiyi Ji 35 19 0 28 May 2023
Learning to Extrapolate: A Transductive Approach Aviv Netanyahu Abhishek Gupta Max Simchowitz Kaipeng Zhang Pulkit Agrawal 49 15 0 27 Apr 2023
Out-of-distribution Few-shot Learning For Edge Devices without Model Fine-tuning Xinyun Zhang Lanqing Hong OODD 43 0 0 13 Apr 2023
HiNet: Novel Multi-Scenario & Multi-Task Learning with Hierarchical Information Extraction Jie Zhou Xia Cao Wenhao Li Lin Bo Kun Zhang Chuan Luo Qian Yu 29 24 0 10 Mar 2023
Provable Pathways: Learning Multiple Tasks over Multiple Paths Yingcong Li Samet Oymak MoE 29 4 0 08 Mar 2023
Computing with Categories in Machine Learning Eli Sennesh T. Xu Yoshihiro Maruyama 28 2 0 07 Mar 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić E. Ponti MoMe OOD 32 73 0 22 Feb 2023
GDOD: Effective Gradient Descent using Orthogonal Decomposition for Multi-Task Learning Xin Dong Ruize Wu Chao Xiong Hai Li Lei Cheng Yong He Shiyou Qian Jian Cao Linjian Mo 11 4 0 31 Jan 2023
Selector-Enhancer: Learning Dynamic Selection of Local and Non-local Attention Operation for Speech Enhancement Xinmeng Xu Weiping Tu Yuhong Yang 32 8 0 07 Dec 2022
Spatial Mixture-of-Experts Nikoli Dryden Torsten Hoefler MoE 34 9 0 24 Nov 2022
Highly Scalable Task Grouping for Deep Multi-Task Learning in Prediction of Epigenetic Events Mohammad Shiri Jiangwen Sun 13 1 0 24 Sep 2022
On the Convergence Theory of Meta Reinforcement Learning with Personalized Policies Haozhi Wang Qing Wang Yunfeng Shao Dong Li Jianye Hao Yinchuan Li 20 0 0 21 Sep 2022
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition Jorge Armando Mendez Mendez Eric Eaton KELM CLL 32 27 0 15 Jul 2022
Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts Qinyuan Ye Juan Zha Xiang Ren MoE 18 12 0 25 May 2022
Spot-adaptive Knowledge Distillation Mingli Song Ying Chen Jingwen Ye Mingli Song 25 72 0 05 May 2022
Auto-Lambda: Disentangling Dynamic Task Relationships Shikun Liu Stephen James Andrew J. Davison Edward Johns 37 75 0 07 Feb 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 27 177 0 02 Feb 2022
Auto-Transfer: Learning to Route Transferrable Representations K. Murugesan Vijay Sadashivaiah Ronny Luss Karthikeyan Shanmugam Pin-Yu Chen Amit Dhurandhar AAML 43 5 0 02 Feb 2022
Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments A. Iyer Karan Grewal Akash Velu Lucas O. Souza Jérémy Forest Subutai Ahmad AI4CE 35 41 0 31 Dec 2021
Compositional Learning-based Planning for Vision POMDPs Sampada Deglurkar M. H. Lim Johnathan Tucker Zachary Sunberg Aleksandra Faust Claire Tomlin 48 4 0 17 Dec 2021
Conflict-Averse Gradient Descent for Multi-task Learning Bo Liu Xingchao Liu Xiaojie Jin Peter Stone Qiang Liu 47 298 0 26 Oct 2021
Dynamic Inference with Neural Interpreters Nasim Rahaman Muhammad Waleed Gondal S. Joshi Peter V. Gehler Yoshua Bengio Francesco Locatello Bernhard Schölkopf 48 31 0 12 Oct 2021
Auxiliary Task Update Decomposition: The Good, The Bad and The Neutral Lucio Dery Yann N. Dauphin David Grangier MoMe 18 29 0 25 Aug 2021
Exploring Data Aggregation and Transformations to Generalize across Visual Domains Antono DÍnnocente OOD 33 0 0 20 Aug 2021
Discrete-Valued Neural Communication Dianbo Liu DianboLiu Alex Lamb Kenji Kawaguchi Anirudh Goyal Chen Sun Michael C. Mozer Yoshua Bengio 26 50 0 06 Jul 2021
Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference Lu Gan Youngji Kim J. Grizzle Jeffrey M. Walls Ayoung Kim Ryan Eustice Maani Ghaffari 32 15 0 28 Jun 2021
Scaling Vision with Sparse Mixture of Experts C. Riquelme J. Puigcerver Basil Mustafa Maxim Neumann Rodolphe Jenatton André Susano Pinto Daniel Keysers N. Houlsby MoE 17 575 0 10 Jun 2021
Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer M. Elmahdy Laurens Beljaards Sahar Yousefi Hessam Sokooti F. Verbeek U. A. van der Heide Marius Staring 27 20 0 05 May 2021
Neural Production Systems: Learning Rule-Governed Visual Dynamics Anirudh Goyal Aniket Didolkar Nan Rosemary Ke Charles Blundell Philippe Beaudoin N. Heess Michael C. Mozer Yoshua Bengio OCL 50 82 0 02 Mar 2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity W. Fedus Barret Zoph Noam M. Shazeer MoE 11 2,075 0 11 Jan 2021
Meta Learning Backpropagation And Improving It Louis Kirsch Jürgen Schmidhuber 53 56 0 29 Dec 2020
Continual Learning in Low-rank Orthogonal Subspaces Arslan Chaudhry Naeemullah Khan P. Dokania Philip Torr CLL 33 114 0 22 Oct 2020
Controllable Pareto Multi-Task Learning Xi Lin Zhiyuan Yang Qingfu Zhang Sam Kwong MoE 77 73 0 13 Oct 2020
Multi-Task Learning with Deep Neural Networks: A Survey M. Crawshaw CVBM 48 609 0 10 Sep 2020
A Study of Compositional Generalization in Neural Models Tim Klinger D. Adjodah Vincent Marois Joshua Joseph Matthew D Riemer Alex Pentland Murray Campbell CoGe NAI 30 12 0 16 Jun 2020
Learning to Branch for Multi-Task Learning Pengsheng Guo Chen-Yu Lee Daniel Ulbricht 18 174 0 02 Jun 2020
Multi-Task Learning for Dense Prediction Tasks: A Survey Simon Vandenhende Stamatios Georgoulis Wouter Van Gansbeke Marc Proesmans Dengxin Dai Luc Van Gool CVBM 29 72 0 28 Apr 2020
Multi-Task Reinforcement Learning with Soft Modularization Ruihan Yang Huazhe Xu Yi Wu Xiaolong Wang 27 176 0 30 Mar 2020
Using Hindsight to Anchor Past Knowledge in Continual Learning Arslan Chaudhry Albert Gordo P. Dokania Philip Torr David Lopez-Paz KELM CLL 21 233 0 19 Feb 2020
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 41 1,172 0 19 Jan 2020
Attention over Parameters for Dialogue Systems Andrea Madotto Zhaojiang Lin Chien-Sheng Wu Jamin Shin Pascale Fung 30 20 0 07 Jan 2020