v1v2 (latest)

Improving Training Stability for Multitask Ranking Models in Recommender Systems

17 February 2023

Justin Gilmer

Papers citing "Improving Training Stability for Multitask Ranking Models in Recommender Systems"

21 / 21 papers shown

Title
A Parameter Update Balancing Algorithm for Multi-task Ranking Models in Recommendation Systems Jun Yuan Guohao Cai Zhenhua Dong 184 0 0 08 Oct 2024
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models Rohan Anil S. Gadanho Danya Huang Nijith Jacob Zhuoshu Li ... Cristina Pop Kevin Regan G. Shamir Rakesh Shivanna Qiqi Yan 3DV 72 42 0 12 Sep 2022
Understanding Scaling Laws for Recommendation Models Newsha Ardalani Carole-Jean Wu Zeliang Chen Bhargav Bhushanam Adnan Aziz 82 31 0 17 Aug 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 515 6,279 0 05 Apr 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 140 1,600 0 20 Jan 2022
A Loss Curvature Perspective on Training Instability in Deep Learning Justin Gilmer Behrooz Ghorbani Ankush Garg Sneha Kudugunta Behnam Neyshabur David E. Cardoze George E. Dahl Zachary Nado Orhan Firat ODL 63 36 0 08 Oct 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 251 156 0 17 Sep 2021
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability Jeremy M. Cohen Simran Kaur Yuanzhi Li J. Zico Kolter Ameet Talwalkar ODL 84 277 0 26 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 285 522 0 11 Feb 2021
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 841 42,332 0 28 May 2020
Why gradient clipping accelerates training: A theoretical justification for adaptivity J.N. Zhang Tianxing He S. Sra Ali Jadbabaie 76 467 0 28 May 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 257 999 0 01 Apr 2019
Towards Neural Mixture Recommender for Long Range Dependent User Sequences Jiaxi Tang Francois Belletti Sagar Jain Minmin Chen Alex Beutel Can Xu Ed H. Chi 45 92 0 22 Feb 2019
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost Noam M. Shazeer Mitchell Stern ODL 84 1,051 0 11 Apr 2018
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 420 10,526 0 21 Jul 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 433 18,361 0 27 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,328 0 11 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 2.0K 150,260 0 22 Dec 2014
On the difficulty of training Recurrent Neural Networks Razvan Pascanu Tomas Mikolov Yoshua Bengio ODL 204 5,360 0 21 Nov 2012