Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models

12 October 2020

Papers citing "Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models"

50 / 128 papers shown

Title
HBO: Hierarchical Balancing Optimization for Fine-Tuning Large Language Models Weixuan Wang Minghao Wu Barry Haddow Alexandra Birch 2 0 0 18 May 2025
BoundarySeg:An Embarrassingly Simple Method To Boost Medical Image Segmentation Performance for Low Data Regimes Tushar Kataria Shireen Y. Elhabian 29 0 0 14 May 2025
Learning Compatible Multi-Prize Subnetworks for Asymmetric Retrieval Yushuai Sun Zikun Zhou D. Jiang Yaowei Wang Jun Yu Guangming Lu Wenjie Pei 34 0 0 16 Apr 2025
CONGRAD:Conflicting Gradient Filtering for Multilingual Preference Alignment Jiangnan Li Thuy-Trang Vu Christian Herold Amirhossein Tebbifakhr Shahram Khadivi Gholamreza Haffari 33 0 0 31 Mar 2025
Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning Shijie Zhu Hui Zhao Tianshu Wu Pengjie Wang Hongbo Deng Jian Xu Bo Zheng 70 1 0 05 Mar 2025
Go Beyond Your Means: Unlearning with Per-Sample Gradient Orthogonalization Aviv Shamsian E. Shaar Aviv Navon Gal Chechik Ethan Fetaya MU 64 0 0 04 Mar 2025
Generative Modeling of Individual Behavior at Scale Nabil Omi Lucas Caccia Anurag Sarkar Jordan T. Ash S. Sen 76 0 0 24 Feb 2025
PiKE: Adaptive Data Mixing for Multi-Task Learning Under Low Gradient Conflicts Zeman Li Yuan Deng Peilin Zhong Meisam Razaviyayn Vahab Mirrokni MoMe 75 1 0 10 Feb 2025
Ensembles of Low-Rank Expert Adapters Yinghao Li Vianne Gao Chao Zhang MohamadAli Torkamani 72 0 0 31 Jan 2025
Commute Your Domains: Trajectory Optimality Criterion for Multi-Domain Learning Alexey Rukhovich Alexander Podolskiy Irina Piontkovskaya 48 0 0 28 Jan 2025
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond Weiyu Chen Xiaoyuan Zhang Baijiong Lin Xi Lin Han Zhao Qingfu Zhang James T. Kwok 75 3 0 19 Jan 2025
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning Hanwen Zhong Jiaxin Chen Yutong Zhang Di Huang Yunhong Wang MoE 44 0 0 12 Jan 2025
Preventing Conflicting Gradients in Neural Marked Temporal Point Processes T. Bosser Souhaib Ben Taieb AI4TS 69 0 0 11 Dec 2024
Optimizing Dense Visual Predictions Through Multi-Task Coherence and Prioritization Maxime Fontana Michael W. Spratling Miaojing Shi MoE VLM 69 0 0 04 Dec 2024
AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers Jake Grigsby Justin Sasek Samyak Parajuli Daniel Adebi Amy Zhang Yuke Zhu OffRL 26 3 0 17 Nov 2024
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Zhiqi Bu Xiaomeng Jin Bhanukiran Vinzamuri Anil Ramakrishna Kai-Wei Chang V. Cevher Mingyi Hong MU 88 7 0 29 Oct 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 37 4 0 18 Oct 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 55 1 0 18 Oct 2024
Decouple-Then-Merge: Finetune Diffusion Models as Multi-Task Learning Qianli Ma Xuefei Ning Dongrui Liu Li Niu Linfeng Zhang MoMe 57 0 0 09 Oct 2024
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 35 1 0 06 Oct 2024
X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale Haoran Xu Kenton W. Murray Philipp Koehn Hieu T. Hoang Akiko Eriguchi Huda Khayrallah 37 8 0 04 Oct 2024
Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling David Grangier Simin Fan Skyler Seto Pierre Ablin 44 3 0 30 Sep 2024
Task Weighting through Gradient Projection for Multitask Learning Christian Bohn Ido Freeman Hasan Tercan Tobias Meisen 29 1 0 03 Sep 2024
Can Optimization Trajectories Explain Multi-Task Transfer? David Mueller Mark Dredze Nicholas Andrews 61 1 0 26 Aug 2024
Customizing Language Models with Instance-wise LoRA for Sequential Recommendation Xiaoyu Kong Jiancan Wu An Zhang Leheng Sheng Hui Lin Xiang Wang Xiangnan He AI4TS 61 7 0 19 Aug 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard F. Fleuret MoE 67 6 0 10 Jul 2024
DMTG: One-Shot Differentiable Multi-Task Grouping Yuan Gao Shuguo Jiang Moran Li Jin-Gang Yu Gui-Song Xia 54 2 0 06 Jul 2024
Exploring Training on Heterogeneous Data with Mixture of Low-rank Adapters Yuhang Zhou Zihua Zhao Haolin Li Siyuan Du Jiangchao Yao Ya Zhang Yanfeng Wang MoMe MoE 40 3 0 14 Jun 2024
Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Models Minghao Wu Thuy-Trang Vu Lizhen Qu Gholamreza Haffari 31 5 0 13 Jun 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Caccia Alessandro Sordoni MoMe 44 31 0 18 May 2024
Neuron Specialization: Leveraging intrinsic task modularity for multilingual machine translation Shaomu Tan Di Wu Christof Monz MoMe 36 8 0 17 Apr 2024
Multi-Task Learning as enabler for General-Purpose AI-native RAN Hasan Farooq Julien Forgeat Shruti Bothe K. Čyras Md Moin 36 0 0 05 Apr 2024
Low-resource neural machine translation with morphological modeling Antoine Nzeyimana 39 4 0 03 Apr 2024
Make Continual Learning Stronger via C-Flat Ang Bian Wei Li Hangjie Yuan Chengrong Yu Zixiang Zhao Mang Wang Aojun Lu Tao Feng 26 9 0 01 Apr 2024
FastCAR: Fast Classification And Regression Multi-Task Learning via Task Consolidation for Modelling a Continuous Property Variable of Object Classes Anoop Kini A. Jansche T. Bernthaler Gerhard Schneider 25 0 0 26 Mar 2024
DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Data Hanrong Ye Dan Xu DiffM 60 4 0 22 Mar 2024
Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts Shengzhuang Chen Jihoon Tack Yunqiao Yang Yee Whye Teh Jonathan Richard Schwarz Ying Wei MoE 43 1 0 13 Mar 2024
Smooth Tchebycheff Scalarization for Multi-Objective Optimization Xi Lin Xiao-Yan Zhang Zhiyuan Yang Fei Liu Zhenkun Wang Qingfu Zhang 35 16 0 29 Feb 2024
Fair Resource Allocation in Multi-Task Learning Hao Ban Kaiyi Ji 35 11 0 23 Feb 2024
Robust Training of Federated Models with Extremely Label Deficiency Yonggang Zhang Zhiqin Yang Xinmei Tian Nannan Wang Tongliang Liu Bo Han FedML 41 6 0 22 Feb 2024
Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning Idan Achituve I. Diamant Arnon Netzer Gal Chechik Ethan Fetaya UQCV 34 4 0 06 Feb 2024
Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglement Dayou Mao Yuhao Chen Yifan Wu Maximilian Gilles Alexander Wong AAML 41 0 0 05 Feb 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 56 38 0 05 Feb 2024
Careful with that Scalpel: Improving Gradient Surgery with an EMA Yu-Guan Hsieh James Thornton Eugène Ndiaye Michal Klein Marco Cuturi Pierre Ablin MedIm 39 0 0 05 Feb 2024
A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization Feiyang Ye Baijiong Lin Xiao-Qun Cao Yu Zhang Ivor Tsang 50 6 0 17 Jan 2024
Multitask Learning in Minimally Invasive Surgical Vision: A Review Oluwatosin O. Alabi Tom Kamiel Magda Vercauteren Miaojing Shi 31 3 0 16 Jan 2024
Multi-dimensional Fair Federated Learning Cong Su Guoxian Yu Jun Wang Hui Li Qingzhong Li Han Yu FedML 27 3 0 09 Dec 2023
Customizable Combination of Parameter-Efficient Modules for Multi-Task Learning Haowen Wang Tao Sun Cong Fan Jinjie Gu MoE 16 7 0 06 Dec 2023
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective Can Jin Tianjin Huang Yihua Zhang Mykola Pechenizkiy Sijia Liu Shiwei Liu Tianlong Chen VLM 36 26 0 03 Dec 2023
FedHCA $^2$ : Towards Hetero-Client Federated Multi-Task Learning Yuxiang Lu Suizhi Huang Yuwen Yang Shalayiding Sirejiding Yue Ding Hongtao Lu FedML 50 3 0 22 Nov 2023