GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks

7 November 2017

Chen-Yu Lee

Papers citing "GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks"

45 / 45 papers shown

Title
An Uncertainty-Aware ED-LSTM for Probabilistic Suffix Prediction Henryk Mustroph Michel Kunkler Stefanie Rinderle-Ma 75 0 0 27 May 2025
Flexible-weighted Chamfer Distance: Enhanced Objective Function for Point Cloud Completion Jie Li Shengwei Tian Long Yu Xin Ning 157 0 0 20 May 2025
Gradient-Guided Annealing for Domain Generalization Aristotelis Ballas Christos Diou OOD 481 1 0 27 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 180 0 0 17 Feb 2025
DeepGate4: Efficient and Effective Representation Learning for Circuit Design at Scale Ziyang Zheng Shan Huang Jianyuan Zhong Zhengyuan Shi Guohao Dai Ningyi Xu Qiang Xu GNN 123 4 0 02 Feb 2025
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts Wenju Sun Qingyong Li Wen Wang Yangli-ao Geng Boyang Li 140 5 0 28 Jan 2025
E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detection Jiaqing Zhang Mingxiang Cao Weiying Xie Jie Lei Daixun Li Wenbo Huang Yunsong Li Xue Yang 90 5 0 28 Jan 2025
Pareto Set Learning for Multi-Objective Reinforcement Learning Erlong Liu Yu-Chang Wu Xiaobin Huang Chengrui Gao Ren-Jian Wang Ke Xue Chao Qian OffRL 184 2 0 12 Jan 2025
SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection Yuchen Li Xianrui Li Yunheng Li Yanzhe Zhang Yimian Dai Qibin Hou Ming-Ming Cheng Jian Yang 158 7 0 31 Dec 2024
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate Zhiqi Bu Xiaomeng Jin Bhanukiran Vinzamuri Anil Ramakrishna Kai-Wei Chang Volkan Cevher Mingyi Hong MU 109 7 0 29 Oct 2024
Relaxed Equivariance via Multitask Learning Ahmed A. A. Elhag T. Konstantin Rusch Francesco Di Giovanni Michael Bronstein 82 3 0 23 Oct 2024
radarODE-MTL: A Multi-Task Learning Framework with Eccentric Gradient Alignment for Robust Radar-Based ECG Reconstruction Yanmei Zhang Rui Yang Yutao Yue Eng Gee Lim 59 1 0 11 Oct 2024
Metamizer: a versatile neural optimizer for fast and accurate physics simulations Nils Wandel Stefan Schulz Reinhard Klein PINN AI4CE 95 1 0 10 Oct 2024
A Parameter Update Balancing Algorithm for Multi-task Ranking Models in Recommendation Systems Jun Yuan Guohao Cai Zhenhua Dong 142 0 0 08 Oct 2024
Restructuring Vector Quantization with the Rotation Trick Christopher Fifty Ronald G. Junkins Dennis Duan Aniketh Iger Jerry W. Liu Ehsan Amid Sebastian Thrun Christopher Ré LLMSV 92 12 0 08 Oct 2024
Using dynamic loss weighting to boost improvements in forecast stability Daan Caljon Jeff Vercauteren Simon De Vos Wouter Verbeke Jente Van Belle 67 0 0 26 Sep 2024
ID-Guard: A Universal Framework for Combating Facial Manipulation via Breaking Identification Zuomin Qu Wei Lu Xiangyang Luo Qian Wang Xiaochun Cao AAML PICV 101 1 0 20 Sep 2024
Can Optimization Trajectories Explain Multi-Task Transfer? David Mueller Mark Dredze Nicholas Andrews 108 1 0 26 Aug 2024
BihoT: A Large-Scale Dataset and Benchmark for Hyperspectral Camouflaged Object Tracking Hanzheng Wang Wei Li X. Xia Qian Du 105 2 0 22 Aug 2024
Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences Nikolaos Dimitriadis Pascal Frossard François Fleuret MoE 187 8 0 10 Jul 2024
Collaborative Pareto Set Learning in Multiple Multi-Objective Optimization Problems Chikai Shang Rongguang Ye Jiaqi Jiang Fangqing Gu 58 1 0 01 Apr 2024
Diverse Target and Contribution Scheduling for Domain Generalization Shaocong Long Qianyu Zhou Soham Dan Lizhuang Ma Yuan Luo 108 8 0 28 Sep 2023
Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs Yi Sun Xin Xu Jiaqiang Li Xiaochang Hu Yifei Shi L. Zeng 147 2 0 31 May 2023
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Tuan Truong Qi Lei Nhat Ho Dinh Q. Phung Trung Le 164 11 0 24 Nov 2022
Maximum Roaming Multi-Task Learning Lucas Pascal Pietro Michiardi Xavier Bost B. Huet Maria A. Zuluaga 143 31 0 17 Jun 2020
Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics Alex Kendall Y. Gal R. Cipolla 3DH 244 3,101 0 19 May 2017
Automated Curriculum Learning for Neural Networks Alex Graves Marc G. Bellemare Jacob Menick Rémi Munos Koray Kavukcuoglu 64 523 0 10 Apr 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 320 27,018 0 20 Mar 2017
RoomNet: End-to-End Room Layout Estimation Chen-Yu Lee Vijay Badrinarayanan Tomasz Malisiewicz Andrew Rabinovich 3DV 46 161 0 18 Mar 2017
Universal representations:The missing link between faces, text, planktons, and cat breeds Hakan Bilen Andrea Vedaldi 63 147 0 25 Jan 2017
YOLO9000: Better, Faster, Stronger Joseph Redmon Ali Farhadi VLM ObjD 173 15,573 0 25 Dec 2016
MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving Marvin Teichmann Michael Weber Johann Marius Zöllner R. Cipolla R. Urtasun SSeg 56 697 0 22 Dec 2016
Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification Y. Lu Abhishek Kumar Shuangfei Zhai Yu Cheng T. Javidi Rogerio Feris 3DH 44 384 0 16 Nov 2016
A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks Kazuma Hashimoto Caiming Xiong Yoshimasa Tsuruoka R. Socher KELM 78 575 0 05 Nov 2016
UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory Iasonas Kokkinos SSeg SSL 109 672 0 07 Sep 2016
Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer Jonathan Long Trevor Darrell VOS SSeg 462 37,704 0 20 May 2016
Cross-stitch Networks for Multi-task Learning Ishan Misra Abhinav Shrivastava Abhinav Gupta M. Hebert 81 1,339 0 12 Apr 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.7K 192,638 0 10 Dec 2015
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation Vijay Badrinarayanan Alex Kendall R. Cipolla SSeg 903 15,718 0 02 Nov 2015
Learning Multiple Tasks with Multilinear Relationship Networks Mingsheng Long Zhangjie Cao Jianmin Wang Philip S. Yu 57 98 0 06 Jun 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 398 43,154 0 11 Feb 2015
Self-informed neural network structure learning David Warde-Farley Andrew Rabinovich Dragomir Anguelov SSL 48 18 0 20 Dec 2014
Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture David Eigen Rob Fergus VLM MDE 181 2,677 0 18 Nov 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.2K 99,991 0 04 Sep 2014
Clustered Multi-Task Learning: A Convex Formulation Laurent Jacob Francis R. Bach Jean-Philippe Vert 114 509 0 11 Sep 2008