Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models

12 October 2020

Papers citing "Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models"

50 / 128 papers shown

Title
Implicit biases in multitask and continual learning from a backward error analysis perspective Benoit Dherin 36 3 0 01 Nov 2023
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training Mingyang Wang Heike Adel Lukas Lange Jannik Strötgen Hinrich Schütze 33 3 0 23 Oct 2023
Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systems Yunli Wang Zhiqiang Wang Jian Yang Shiyang Wen Dongying Kong Han Li Kun Gai 32 10 0 16 Oct 2023
Transformer-based Multimodal Change Detection with Multitask Consistency Constraints Biyuan Liu Huaixin Chen Kun Li Michael Ying Yang 41 14 0 13 Oct 2023
Scalarization for Multi-Task and Multi-Domain Learning at Scale Amelie Royer Tijmen Blankevoort B. Bejnordi 38 17 0 13 Oct 2023
Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Models Tianjian Li Haoran Xu Philipp Koehn Daniel Khashabi Kenton W. Murray 38 4 0 02 Oct 2023
HarmonyDream: Task Harmonization Inside World Models Haoyu Ma Jialong Wu Ningya Feng Chenjun Xiao Dong Li Jianye Hao Jianmin Wang Mingsheng Long 41 7 0 30 Sep 2023
GCL: Gradient-Guided Contrastive Learning for Medical Image Segmentation with Multi-Perspective Meta Labels YiXuan Wu Jintai Chen Jiahuan Yan Yiheng Zhu Danny Chen Jian Wu VLM 32 6 0 16 Sep 2023
Projected Task-Specific Layers for Multi-Task Reinforcement Learning Josselin Somerville Roberts Julia Di 12 1 0 15 Sep 2023
Dual-Balancing for Multi-Task Learning Baijiong Lin Weisen Jiang Feiyang Ye Yu Zhang Pengguang Chen Yingke Chen Shu Liu James T. Kwok CVBM 36 12 0 23 Aug 2023
Deep Task-specific Bottom Representation Network for Multi-Task Recommendation Qi Liu Zhilong Zhou Gangwei Jiang T. Ge Defu Lian 23 12 0 11 Aug 2023
TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Experts Hanrong Ye Dan Xu MoE 42 26 0 28 Jul 2023
When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review Maxime Fontana Michael W. Spratling Miaojing Shi 50 6 0 25 Jul 2023
Improving Multitask Retrieval by Promoting Task Specialization Wenzheng Zhang Chenyan Xiong K. Stratos Arnold Overwijk LRM 26 2 0 01 Jul 2023
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models A. Jaiswal Shiwei Liu Tianlong Chen Ying Ding Zhangyang Wang VLM 41 22 0 18 Jun 2023
Fairness in Multi-Task Learning via Wasserstein Barycenters Franccois Hu Philipp Ratz Arthur Charpentier 37 10 0 16 Jun 2023
Addressing Negative Transfer in Diffusion Models Hyojun Go Jinyoung Kim Yunsung Lee Seunghyun Lee Shinhyeok Oh Hyeongdon Moon Seungtaek Choi DiffM VLM 32 24 0 01 Jun 2023
Independent Component Alignment for Multi-Task Learning Dmitry Senushkin Nikolay Patakin Arseny Kuznetsov Anton Konushin CVBM 40 41 0 30 May 2023
Exploring Representational Disparities Between Multilingual and Bilingual Translation Models Neha Verma Kenton W. Murray Kevin Duh 21 0 0 23 May 2023
When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLP Jingwei Ni Zhijing Jin Qian Wang Mrinmaya Sachan Markus Leippold AIFin 26 6 0 23 May 2023
FedAds: A Benchmark for Privacy-Preserving CVR Estimation with Vertical Federated Learning Penghui Wei Hongjian Dou Shaoguo Liu Rong Tang Li Liu Liangji Wang Bo Zheng FedML 24 12 0 15 May 2023
KINLP at SemEval-2023 Task 12: Kinyarwanda Tweet Sentiment Analysis Antoine Nzeyimana 20 3 0 25 Apr 2023
UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining Hyung Won Chung Noah Constant Xavier Garcia Adam Roberts Yi Tay Sharan Narang Orhan Firat 29 50 0 18 Apr 2023
On the Pareto Front of Multilingual Neural Machine Translation Liang Chen Shuming Ma Dongdong Zhang Furu Wei Baobao Chang MoE 23 5 0 06 Apr 2023
Efficient Diffusion Training via Min-SNR Weighting Strategy Tiankai Hang Shuyang Gu Chen Li Jianmin Bao Dong Chen Han Hu Xin Geng B. Guo 30 150 0 16 Mar 2023
Gradient Coordination for Quantifying and Maximizing Knowledge Transference in Multi-Task Learning Xuanhua Yang Jianxin R. Zhao Shaoguo Liu Liangji Wang Bo Zheng 20 1 0 10 Mar 2023
Modular Deep Learning Jonas Pfeiffer Sebastian Ruder Ivan Vulić E. Ponti MoMe OOD 32 73 0 22 Feb 2023
Scaling Laws for Multilingual Neural Machine Translation Patrick Fernandes Behrooz Ghorbani Xavier Garcia Markus Freitag Orhan Firat 38 29 0 19 Feb 2023
GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks Salah Ghamizi Jingfeng Zhang Maxime Cordy Mike Papadakis Masashi Sugiyama Yves Le Traon AAML 28 2 0 06 Feb 2023
Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Data Alon Albalak Colin Raffel William Yang Wang 21 12 0 01 Feb 2023
Auxiliary Learning as an Asymmetric Bargaining Game Aviv Shamsian Aviv Navon Neta Glazer Kenji Kawaguchi Gal Chechik Ethan Fetaya 35 8 0 31 Jan 2023
ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning Junguang Jiang Baixu Chen Junwei Pan Ximei Wang Liu Dapeng Jie Jiang Mingsheng Long MoMe 29 20 0 30 Jan 2023
Causes and Cures for Interference in Multilingual Translation Uri Shaham Maha Elbayad Vedanuj Goswami Omer Levy Shruti Bhosale 23 26 0 14 Dec 2022
FairRoad: Achieving Fairness for Recommender Systems with Optimized Antidote Data Minghong Fang Jia-Wei Liu Michinari Momma Yi Sun 30 4 0 13 Dec 2022
Do Text-to-Text Multi-Task Learners Suffer from Task Conflict? David Mueller Nicholas Andrews Mark Dredze 31 6 0 13 Dec 2022
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Dinh Q. Phung Trung Le 27 11 0 24 Nov 2022
Multi-Head Adapter Routing for Cross-Task Generalization Lucas Caccia E. Ponti Zhan Su Matheus Pereira Nicolas Le Roux Alessandro Sordoni 21 20 0 07 Nov 2022
Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonization Junru Wu Yi Liang Feng Han Hassan Akbari Zhangyang Wang Cong Yu 39 9 0 03 Nov 2022
M $^3$ ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design Hanxue Liang Zhiwen Fan Rishov Sarkar Ziyu Jiang Tianlong Chen Kai Zou Yu Cheng Cong Hao Zhangyang Wang MoE 42 81 0 26 Oct 2022
PaCo: Parameter-Compositional Multi-Task Reinforcement Learning Lingfeng Sun Haichao Zhang Wei-ping Xu Masayoshi Tomizuka MoE 30 37 0 21 Oct 2022
Forging Multiple Training Objectives for Pre-trained Language Models via Meta-Learning Hongqiu Wu Ruixue Ding Haizhen Zhao Boli Chen Pengjun Xie Fei Huang Min Zhang MoMe 30 8 0 19 Oct 2022
Do Current Multi-Task Optimization Methods in Deep Learning Even Help? Derrick Xin Behrooz Ghorbani Ankush Garg Orhan Firat Justin Gilmer MoMe 73 63 0 23 Sep 2022
Informative Language Representation Learning for Massively Multilingual Neural Machine Translation Renren Jin Deyi Xiong 30 4 0 04 Sep 2022
Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling Zheqi Lv Feng Wang Shengyu Zhang Kun Kuang Hongxia Yang Fei Wu 37 8 0 19 Aug 2022
Dynamic Restrained Uncertainty Weighting Loss for Multitask Learning of Vocal Expression Meishu Song Zijiang Yang Andreas Triantafyllopoulos Xin Jing Vincent Karas Xie Jiangjian Zixing Zhang Yamamoto Yoshiharu Bjoern W. Schuller 22 6 0 22 Jun 2022
All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass Jiaxin Huang Tianqi Liu Jialu Liu Á. Lelkes Cong Yu Jiawei Han 37 1 0 22 May 2022
Improving Multi-Task Generalization via Regularizing Spurious Correlation Ziniu Hu Zhe Zhao Xinyang Yi Tiansheng Yao Lichan Hong Yizhou Sun Ed H. Chi OOD LRM 93 29 0 19 May 2022
Por Qué Não Utiliser Alla Språk? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfer Haoran Xu Kenton W. Murray 29 12 0 29 Apr 2022
LibMTL: A Python Library for Multi-Task Learning Baijiong Lin Yu Zhang OffRL AI4CE 27 37 0 27 Mar 2022
X-Learner: Learning Cross Sources and Tasks for Universal Visual Representation Yinan He Gengshi Huang Siyu Chen Jianing Teng Wang Kun Zhen-fei Yin Lu Sheng Ziwei Liu Yu Qiao Jing Shao VLM SSL ViT 43 7 0 16 Mar 2022