Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models

12 October 2020

Papers citing "Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models"

28 / 128 papers shown

Title
Combining Modular Skills in Multitask Learning E. Ponti Alessandro Sordoni Yoshua Bengio Siva Reddy MoE 12 37 0 28 Feb 2022
Structured Multi-task Learning for Molecular Property Prediction Shengchao Liu Meng Qu Zuobai Zhang Huiyu Cai Jian Tang 20 24 0 22 Feb 2022
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 30 111 0 03 Feb 2022
Multi-Task Learning as a Bargaining Game Aviv Navon Aviv Shamsian Idan Achituve Haggai Maron Kenji Kawaguchi Gal Chechik Ethan Fetaya 25 140 0 02 Feb 2022
In Defense of the Unitary Scalarization for Deep Multi-Task Learning Vitaly Kurin Alessandro De Palma Ilya Kostrikov Shimon Whiteson M. P. Kumar 39 74 0 11 Jan 2022
Parameter Differentiation based Multilingual Neural Machine Translation Qian Wang Jiajun Zhang 27 17 0 27 Dec 2021
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning V. Aribandi Yi Tay Tal Schuster J. Rao H. Zheng ... Jianmo Ni Jai Gupta Kai Hui Sebastian Ruder Donald Metzler MoE 23 213 0 22 Nov 2021
Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning Baijiong Lin Feiyang Ye Yu Zhang Ivor W. Tsang 42 93 0 20 Nov 2021
An Empirical Study of Training End-to-End Vision-and-Language Transformers Zi-Yi Dou Yichong Xu Zhe Gan Jianfeng Wang Shuohang Wang ... Pengchuan Zhang Lu Yuan Nanyun Peng Zicheng Liu Michael Zeng VLM 32 369 0 03 Nov 2021
Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning Yi-Chen Chen Shu-Wen Yang Cheng-Kuang Lee Simon See Hung-yi Lee SSL 19 12 0 18 Oct 2021
Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research Ross Gruetzemacher D. Paradice 25 30 0 18 Oct 2021
Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits Julia Kreutzer David Vilar Artem Sokolov 49 15 0 13 Oct 2021
Balancing Average and Worst-case Accuracy in Multitask Learning Paul Michel Sebastian Ruder Dani Yogatama 21 11 0 12 Oct 2021
Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning Seanie Lee Haebeom Lee Juho Lee Sung Ju Hwang MoMe CLL 45 16 0 06 Oct 2021
Multi-Task Learning in Natural Language Processing: An Overview Shijie Chen Yu Zhang Qiang Yang AIMat 41 99 0 19 Sep 2021
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion Budhaditya Deb Guoqing Zheng Milad Shokouhi Ahmed Hassan Awadallah 31 1 0 15 Sep 2021
Improving Multilingual Translation by Representation and Gradient Regularization Yilin Yang Akiko Eriguchi Alexandre Muzio Prasad Tadepalli Stefan Lee Hany Hassan 47 40 0 10 Sep 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 213 238 1 10 Sep 2021
Distributionally Robust Multilingual Machine Translation Chunting Zhou Daniel Levy Xian Li Marjan Ghazvininejad Graham Neubig 73 24 0 09 Sep 2021
Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training Minghao Wu Yitong Li Meng Zhang Liangyou Li Gholamreza Haffari Qun Liu 34 22 0 06 Sep 2021
Domain Generalization via Gradient Surgery Lucas Mansilla Rodrigo Echeveste Diego H. Milone Enzo Ferrante OOD 24 78 0 03 Aug 2021
Scaling End-to-End Models for Large-Scale Multilingual ASR Bo-wen Li Ruoming Pang Tara N. Sainath Anmol Gulati Yu Zhang James Qin Parisa Haghani Yifan Jiang Min Ma Junwen Bai CLL 34 76 0 30 Apr 2021
Adaptive Sparse Transformer for Multilingual Translation Hongyu Gong Xian Li Dmitriy Genzel 20 14 0 15 Apr 2021
RotoGrad: Gradient Homogenization in Multitask Learning Adrián Javaloy Isabel Valera 21 86 0 03 Mar 2021
Measuring and Harnessing Transference in Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 28 15 0 29 Oct 2020
A Survey on Negative Transfer Wen Zhang Lingfei Deng Lei Zhang Dongrui Wu AAML 27 205 0 02 Sep 2020
Investigating Multilingual NMT Representations at Scale Sneha Kudugunta Ankur Bapna Isaac Caswell N. Arivazhagan Orhan Firat LRM 144 120 0 05 Sep 2019
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism Orhan Firat Kyunghyun Cho Yoshua Bengio LRM AIMat 231 623 0 06 Jan 2016