Variance Reduction in SGD by Distributed Importance Sampling

20 November 2015

Aaron Courville

Papers citing "Variance Reduction in SGD by Distributed Importance Sampling"

34 / 34 papers shown

Title
Multiple Importance Sampling for Stochastic Gradient Estimation Corentin Salaün Xingchang Huang Iliyan Georgiev Niloy J. Mitra Gurprit Singh 32 1 0 22 Jul 2024
Diversified Batch Selection for Training Acceleration Feng Hong Yueming Lyu Jiangchao Yao Ya Zhang Ivor W. Tsang Yanfeng Wang 42 4 0 07 Jun 2024
A Negative Result on Gradient Matching for Selective Backprop Lukas Balles Cédric Archambeau Giovanni Zappella 37 0 0 08 Dec 2023
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models Jean Kaddour Oscar Key Piotr Nawrot Pasquale Minervini Matt J. Kusner 22 41 0 12 Jul 2023
Personalized Privacy Amplification via Importance Sampling Dominik Fay Sebastian Mair Jens Sjölund 60 0 0 05 Jul 2023
AdaSelection: Accelerating Deep Learning Training through Data Subsampling Minghe Zhang Chaosheng Dong Jinmiao Fu Tianchen Zhou Jia Liang ... Bo Liu Michinari Momma Bryan Wang Yan Gao Yi Sun 35 3 0 19 Jun 2023
GaitMPL: Gait Recognition with Memory-Augmented Progressive Learning Huanzhang Dou Pengyi Zhang Yuhan Zhao Lin Dong Zequn Qin Xi Li CVBM VLM 33 24 0 06 Jun 2023
NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Tasks Jean-Michel Attendu Jean-Philippe Corbeil 38 15 0 05 Jun 2023
Informative Sample-Aware Proxy for Deep Metric Learning Aoyu Li Ikuro Sato Kohta Ishikawa Rei Kawakami Rio Yokota 24 1 0 18 Nov 2022
Information FOMO: The unhealthy fear of missing out on information. A method for removing misleading data for healthier models Ethan Pickering T. Sapsis 24 6 0 27 Aug 2022
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt Sören Mindermann J. Brauner Muhammed Razzak Mrinank Sharma Andreas Kirsch ... Benedikt Höltgen Aidan Gomez Adrien Morisot Sebastian Farquhar Y. Gal 62 149 0 14 Jun 2022
DELTA: Diverse Client Sampling for Fasting Federated Learning Lung-Chuang Wang Yongxin Guo Tao R. Lin Xiaoying Tang FedML 23 22 0 27 May 2022
Tricks and Plugins to GBM on Images and Sequences Biyi Fang J. Utke Diego Klabjan 25 0 0 01 Mar 2022
Characterizing & Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods Amirkeivan Mohtashami Sebastian U. Stich Martin Jaggi 26 13 0 03 Feb 2022
FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients Jaemin Shin Yuanchun Li Yunxin Liu Sung-Ju Lee FedML 17 74 0 05 Jan 2022
Curriculum generation using Autoencoder based continuous optimization Dipankar Sarkar Mukur Gupta 22 0 0 16 Jun 2021
Dynamic Gradient Aggregation for Federated Domain Adaptation Dimitrios Dimitriadis K. Kumatani R. Gmyr Yashesh Gaur Sefik Emre Eskimez FedML 26 5 0 14 Jun 2021
One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning Chaosheng Dong Xiaojie Jin Weihao Gao Yijia Wang Hongyi Zhang Xiang Wu Jianchao Yang Xiaobing Liu 28 5 0 27 Apr 2021
Efficient training of physics-informed neural networks via importance sampling M. A. Nabian R. J. Gladstone Hadi Meidani DiffM PINN 71 223 0 26 Apr 2021
Variance Reduced Training with Stratified Sampling for Forecasting Models Yucheng Lu Youngsuk Park Lifan Chen Bernie Wang Christopher De Sa Dean Phillips Foster AI4TS 38 17 0 02 Mar 2021
Federated Learning under Importance Sampling Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 18 52 0 14 Dec 2020
Optimal Importance Sampling for Federated Learning Elsa Rizk Stefan Vlaski Ali H. Sayed FedML 40 46 0 26 Oct 2020
A Survey on Curriculum Learning Xin Wang Yudong Chen Wenwu Zhu SyDa 32 22 0 25 Oct 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 22 109 0 10 Aug 2020
Federated Transfer Learning with Dynamic Gradient Aggregation Dimitrios Dimitriadis K. Kumatani R. Gmyr Yashesh Gaur Sefik Emre Eskimez FedML 24 15 0 06 Aug 2020
Adaptive Task Sampling for Meta-Learning Chenghao Liu Zhihao Wang Doyen Sahoo Yuan Fang Kun Zhang Guosheng Lin 30 54 0 17 Jul 2020
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul N. Bennett Junaid Ahmed Arnold Overwijk 11 1,180 0 01 Jul 2020
Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization Kenji Kawaguchi Haihao Lu ODL 22 62 0 09 Jul 2019
Submodular Batch Selection for Training Deep Neural Networks K. J. Joseph R. VamshiTeja Krishnakant Singh V. Balasubramanian 11 23 0 20 Jun 2019
Auto-Vectorizing TensorFlow Graphs: Jacobians, Auto-Batching And Beyond Ashish Agarwal Igor Ganichev 27 8 0 08 Mar 2019
Distributed Prioritized Experience Replay Dan Horgan John Quan David Budden Gabriel Barth-Maron Matteo Hessel H. V. Hasselt David Silver 86 731 0 02 Mar 2018
Safe Adaptive Importance Sampling Sebastian U. Stich Anant Raj Martin Jaggi 29 54 0 07 Nov 2017
Online Batch Selection for Faster Training of Neural Networks I. Loshchilov Frank Hutter ODL 37 296 0 19 Nov 2015
Efficient Per-Example Gradient Computations Ian Goodfellow 186 74 0 07 Oct 2015