Revisit Micro-batch Clipping: Adaptive Data Pruning via Gradient
Manipulation

Revisit Micro-batch Clipping: Adaptive Data Pruning via Gradient Manipulation

29 August 2024

Lun Wang

ArXiv (abs)PDF HTML

Papers citing "Revisit Micro-batch Clipping: Adaptive Data Pruning via Gradient Manipulation"

15 / 15 papers shown

Title
Efficiently Train ASR Models that Memorize Less and Perform Better with Per-core Clipping Lun Wang Om Thakkar Zhong Meng Nicole Rafidi Rohit Prabhavalkar Arun Narayanan VLM 35 1 0 04 Jun 2024
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 150 180 0 01 Mar 2023
Beyond neural scaling laws: beating power law scaling via data pruning Ben Sorscher Robert Geirhos Shashank Shekhar Surya Ganguli Ari S. Morcos 97 444 0 29 Jun 2022
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger Zhiqi Bu Yu Wang Sheng Zha George Karypis 121 71 0 14 Jun 2022
Self-supervised Learning with Random-projection Quantizer for Speech Recognition Chung-Cheng Chiu James Qin Yu Zhang Jiahui Yu Yonghui Wu SSL 90 169 0 03 Feb 2022
Active Learning at the ImageNet Scale Z. Emam Hong-Min Chu Ping Yeh-Chiang W. Czaja R. Leapman Micah Goldblum Tom Goldstein 69 35 0 25 Nov 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 119 461 0 15 Jul 2021
Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering Siddharth Karamcheti Ranjay Krishna Li Fei-Fei Christopher D. Manning 85 92 0 06 Jul 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,802 0 23 Dec 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 229 3,155 0 16 May 2020
Libri-Light: A Benchmark for ASR with Limited or No Supervision Jacob Kahn M. Rivière Weiyi Zheng Evgeny Kharitonov Qiantong Xu ... Tatiana Likhomanenko Gabriel Synnaeve Armand Joulin Abdel-rahman Mohamed Emmanuel Dupoux AuLLM 75 673 0 17 Dec 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 128 741 0 12 Dec 2018
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,162 0 01 Jul 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.