Critical Learning Periods in Deep Neural Networks

24 November 2017

Papers citing "Critical Learning Periods in Deep Neural Networks"

25 / 25 papers shown

Title
Generalization Guarantees for Representation Learning via Data-Dependent Gaussian Mixture Priors Romain Chor Milad Sefidgaran Piotr Krasnowski 91 1 0 21 Feb 2025
Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning E. Chimoto Jay Gala Orevaoghene Ahia Julia Kreutzer Bruce A. Bassett Sara Hooker VLM 42 4 0 29 May 2024
The Garden of Forking Paths: Observing Dynamic Parameters Distribution in Large Language Models Carlo Nicolini Jacopo Staiano Bruno Lepri Raffaele Marino MoE 34 1 0 13 Mar 2024
Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning Michal Nauman Michal Bortkiewicz Piotr Milo's Tomasz Trzciñski M. Ostaszewski Marek Cygan OffRL 30 17 0 01 Mar 2024
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models Anton Razzhigaev Matvey Mikhalchuk Elizaveta Goncharova Ivan Oseledets Denis Dimitrov Andrey Kuznetsov 32 7 0 10 Nov 2023
Maintaining Plasticity in Continual Learning via Regenerative Regularization Saurabh Kumar Henrik Marklund Benjamin Van Roy CLL KELM 34 16 0 23 Aug 2023
Accelerating Distributed ML Training via Selective Synchronization S. Tyagi Martin Swany FedML 32 3 0 16 Jul 2023
GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training S. Tyagi Martin Swany 25 4 0 20 May 2023
Accelerating Dataset Distillation via Model Augmentation Lei Zhang Jie M. Zhang Bowen Lei Subhabrata Mukherjee Xiang Pan Bo Zhao Caiwen Ding Heng Chang Dongkuan Xu DD 43 62 0 12 Dec 2022
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 23 3 0 28 Oct 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 44 27 0 20 Sep 2022
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models Rohan Anil S. Gadanho Danya Huang Nijith Jacob Zhuoshu Li ... Cristina Pop Kevin Regan G. Shamir Rakesh Shivanna Qiqi Yan 3DV 26 41 0 12 Sep 2022
On the Importance of Critical Period in Multi-stage Reinforcement Learning Junseok Park Inwoo Hwang Min Whoo Lee Hyunseok Oh Minsu Lee Youngki Lee Byoung-Tak Zhang OffRL 24 0 0 09 Aug 2022
A precortical module for robust CNNs to light variations R. Fioresi J. Petkovic 23 1 0 15 Feb 2022
Does Optimal Source Task Performance Imply Optimal Pre-training for a Target Task? Steven Gutstein Brent Lance Sanjay Shakkottai 27 1 0 21 Jun 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 30 30 0 01 May 2021
Intraclass clustering: an implicit learning ability that regularizes DNNs Simon Carbonnelle Christophe De Vleeschouwer 60 8 0 11 Mar 2021
Anti-Distillation: Improving reproducibility of deep networks G. Shamir Lorenzo Coviello 42 20 0 19 Oct 2020
The Early Phase of Neural Network Training Jonathan Frankle D. Schwab Ari S. Morcos 19 171 0 24 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 50 154 0 21 Feb 2020
DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks Simon Wiedemann H. Kirchhoffer Stefan Matlage Paul Haase Arturo Marbán ... Ahmed Osman D. Marpe H. Schwarz Thomas Wiegand Wojciech Samek 49 92 0 27 Jul 2019
Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade T. Lindeberg 27 19 0 29 May 2019
DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression Simon Wiedemann H. Kirchhoffer Stefan Matlage Paul Haase Arturo Marbán ... Ahmed Osman D. Marpe H. Schwarz Thomas Wiegand Wojciech Samek MQ 19 21 0 15 May 2019
Task2Vec: Task Embedding for Meta-Learning Alessandro Achille Michael Lam Rahul Tewari Avinash Ravichandran Subhransu Maji Charless C. Fowlkes Stefano Soatto Pietro Perona SSL 22 309 0 10 Feb 2019
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016