Do Current Multi-Task Optimization Methods in Deep Learning Even Help?

23 September 2022

Justin Gilmer

Papers citing "Do Current Multi-Task Optimization Methods in Deep Learning Even Help?"

26 / 26 papers shown

Title
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Zhiqi Bu Xiaomeng Jin Bhanukiran Vinzamuri Anil Ramakrishna Kai-Wei Chang Volkan Cevher Mingyi Hong MU 146 13 0 29 Oct 2024
Federated Communication-Efficient Multi-Objective Optimization Baris Askin Pranay Sharma Gauri Joshi Carlee Joe-Wong FedML 255 1 0 21 Oct 2024
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 102 1 0 06 Oct 2024
Can Optimization Trajectories Explain Multi-Task Transfer? David Mueller Mark Dredze Nicholas Andrews 122 1 0 26 Aug 2024
Enhancing Domain Adaptation through Prompt Gradient Alignment Hoang Phan Lam C. Tran Quyen Tran Trung Le 129 1 0 13 Jun 2024
mSLAM: Massively multilingual joint pre-training for speech and text Ankur Bapna Colin Cherry Yu Zhang Ye Jia Melvin Johnson Yong Cheng Simran Khanuja Jason Riesa Alexis Conneau VLM 55 114 0 03 Feb 2022
In Defense of the Unitary Scalarization for Deep Multi-Task Learning Vitaly Kurin Alessandro De Palma Ilya Kostrikov Shimon Whiteson M. P. Kumar 84 75 0 11 Jan 2022
Unsupervised Domain Adaptation: A Reality Check Kevin Musgrave Serge Belongie Ser-Nam Lim 76 22 0 30 Nov 2021
Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits Julia Kreutzer David Vilar Artem Sokolov 80 15 0 13 Oct 2021
Efficiently Identifying Task Groupings for Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 290 255 1 10 Sep 2021
Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout Zhao Chen Jiquan Ngiam Yanping Huang Thang Luong Henrik Kretzschmar Yuning Chai Dragomir Anguelov 75 221 0 14 Oct 2020
Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models Zirui Wang Yulia Tsvetkov Orhan Firat Yuan Cao 72 202 0 12 Oct 2020
In Search of Lost Domain Generalization Ishaan Gulrajani David Lopez-Paz OOD 91 1,156 0 02 Jul 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Zhiwen Chen MoE 124 1,191 0 30 Jun 2020
Gradient Surgery for Multi-Task Learning Tianhe Yu Saurabh Kumar Abhishek Gupta Sergey Levine Karol Hausman Chelsea Finn 180 1,228 0 19 Jan 2020
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges N. Arivazhagan Ankur Bapna Orhan Firat Dmitry Lepikhin Melvin Johnson ... George F. Foster Colin Cherry Wolfgang Macherey Zhiwen Chen Yonghui Wu 90 428 0 11 Jul 2019
Multi-Task Learning as Multi-Objective Optimization Ozan Sener V. Koltun 169 1,288 0 10 Oct 2018
A Call for Clarity in Reporting BLEU Scores Matt Post 179 2,996 0 23 Apr 2018
GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks Zhao Chen Vijay Badrinarayanan Chen-Yu Lee Andrew Rabinovich ODL 165 1,290 0 07 Nov 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics Alex Kendall Y. Gal R. Cipolla 3DH 272 3,135 0 19 May 2017
Automated Curriculum Learning for Neural Networks Alex Graves Marc G. Bellemare Jacob Menick Rémi Munos Koray Kavukcuoglu 88 531 0 10 Apr 2017
The Cityscapes Dataset for Semantic Urban Scene Understanding Marius Cordts Mohamed Omran Sebastian Ramos Timo Rehfeld Markus Enzweiler Rodrigo Benenson Uwe Franke Stefan Roth Bernt Schiele 1.1K 11,644 0 06 Apr 2016
LSTM: A Search Space Odyssey Klaus Greff R. Srivastava Jan Koutník Bas R. Steunebrink Jürgen Schmidhuber AI4TS VLM 130 5,309 0 13 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,312 0 22 Dec 2014
Deep Learning Face Attributes in the Wild Ziwei Liu Ping Luo Xiaogang Wang Xiaoou Tang CVBM 247 8,426 0 28 Nov 2014