Accelerating Deep Learning by Focusing on the Biggest Losers

2 October 2019

Papers citing "Accelerating Deep Learning by Focusing on the Biggest Losers"

26 / 26 papers shown

Title
Teaching Models to Understand (but not Generate) High-risk Data Ryan Yixiang Wang Matthew Finlayson Luca Soldaini Swabha Swayamdipta Robin Jia 198 0 0 05 May 2025
DONOD: Robust and Generalizable Instruction Fine-Tuning for LLMs via Model-Intrinsic Dataset Pruning Jucheng Hu Steve Yang Dongzhan Zhou Lijun Wu 36 0 0 21 Apr 2025
ToReMi: Topic-Aware Data Reweighting for Dynamic Pre-Training Data Selection Xiaoxuan Zhu Zhouhong Gu Baiqian Wu Suhang Zheng Tao Wang Tianyu Li Hongwei Feng Yanghua Xiao 46 0 0 01 Apr 2025
Diversified Batch Selection for Training Acceleration Feng Hong Yueming Lyu Jiangchao Yao Ya Zhang Ivor W. Tsang Yanfeng Wang 49 4 0 07 Jun 2024
SAVA: Scalable Learning-Agnostic Data Valuation Samuel Kessler Tam Le Vu Nguyen TDI 71 0 0 03 Jun 2024
Rho-1: Not All Tokens Are What You Need Zheng-Wen Lin Zhibin Gou Yeyun Gong Xiao Liu Yelong Shen ... Chen Lin Yujiu Yang Jian Jiao Nan Duan Weizhu Chen CLL 50 58 0 11 Apr 2024
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding Talfan Evans Shreya Pathak Hamza Merzic Jonathan Schwarz Ryutaro Tanno Olivier J. Hénaff 31 16 0 08 Dec 2023
A Negative Result on Gradient Matching for Selective Backprop Lukas Balles Cédric Archambeau Giovanni Zappella 47 0 0 08 Dec 2023
REDUCR: Robust Data Downsampling Using Class Priority Reweighting William Bankes George Hughes Ilija Bogunovic Zi Wang 38 3 0 01 Dec 2023
Farzi Data: Autoregressive Data Distillation Noveen Sachdeva Zexue He Wang-Cheng Kang Jianmo Ni D. Cheng Julian McAuley DD 35 3 0 15 Oct 2023
Towards a statistical theory of data selection under weak supervision Germain Kolossov Andrea Montanari Pulkit Tandon 24 14 0 25 Sep 2023
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models Jean Kaddour Oscar Key Piotr Nawrot Pasquale Minervini Matt J. Kusner 32 41 0 12 Jul 2023
AdaSelection: Accelerating Deep Learning Training through Data Subsampling Minghe Zhang Chaosheng Dong Jinmiao Fu Tianchen Zhou Jia Liang ... Bo Liu Michinari Momma Bryan Wang Yan Gao Yi Sun 40 3 0 19 Jun 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 35 41 0 07 Apr 2023
Enhancing Efficiency in Multidevice Federated Learning through Data Selection Fan Mo Mohammad Malekzadeh S. Chatterjee F. Kawsar Akhil Mathur FedML 40 2 0 08 Nov 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 57 27 0 20 Sep 2022
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt Sören Mindermann J. Brauner Muhammed Razzak Mrinank Sharma Andreas Kirsch ... Benedikt Höltgen Aidan Gomez Adrien Morisot Sebastian Farquhar Y. Gal 67 152 0 14 Jun 2022
On Continual Model Refinement in Out-of-Distribution Data Streams Bill Yuchen Lin Sida I. Wang Xi Lin Robin Jia Lin Xiao Xiang Ren Wen-tau Yih CLL 33 29 0 04 May 2022
ACPL: Anti-curriculum Pseudo-labelling for Semi-supervised Medical Image Classification Fengbei Liu Yu Tian Yuanhong Chen Yuyuan Liu Vasileios Belagiannis G. Carneiro 43 78 0 25 Nov 2021
Chaos as an interpretable benchmark for forecasting and data-driven modelling W. Gilpin AI4TS 27 77 0 11 Oct 2021
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference Chaojian Li Wuyang Chen Yuchen Gu Tianlong Chen Yonggan Fu Zhangyang Wang Yingyan Lin 35 0 0 16 Jul 2021
Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast Yawen Wu Zhepeng Wang Dewen Zeng Yiyu Shi Jingtong Hu SSL 41 28 0 07 Jun 2021
One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning Chaosheng Dong Xiaojie Jin Weihao Gao Yijia Wang Hongyi Zhang Xiang Wu Jianchao Yang Xiaobing Liu 28 5 0 27 Apr 2021
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training Y. Fu Haoran You Yang Katie Zhao Yue Wang Chaojian Li K. Gopalakrishnan Zhangyang Wang Yingyan Lin MQ 38 32 0 24 Dec 2020
Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies Yae Jee Cho Jianyu Wang Gauri Joshi FedML 47 402 0 03 Oct 2020
L $^2$ -GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks Yuning You Tianlong Chen Zhangyang Wang Yang Shen GNN 101 82 0 30 Mar 2020