Data-Efficient Double-Win Lottery Tickets from Robust Pre-training

9 June 2022

ArXiv (abs)PDF HTML Github (9★)

Papers citing "Data-Efficient Double-Win Lottery Tickets from Robust Pre-training"

38 / 38 papers shown

Title
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen Decebal Constantin Mocanu Elena Mocanu OOD 3DH 201 3 0 03 Oct 2024
Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks Yonggan Fu Qixuan Yu Yang Zhang Shan-Hung Wu Ouyang Xu David D. Cox Yingyan Lin AAML OOD 105 30 0 26 Oct 2021
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang Kartik Ahuja Yilun Xu Yisen Wang Aaron Courville OOD 86 96 0 05 Jun 2021
GANs Can Play Lottery Tickets Too Xuxi Chen Zhenyu Zhang Yongduo Sui Tianlong Chen GAN 71 58 0 31 May 2021
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets Xiaohan Chen Yu Cheng Shuohang Wang Zhe Gan Zhangyang Wang Jingjing Liu 105 100 0 31 Dec 2020
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Michael Carbin Zhangyang Wang 68 123 0 12 Dec 2020
Robust Pre-Training by Adversarial Contrastive Learning Ziyu Jiang Tianlong Chen Ting-Li Chen Zhangyang Wang 106 234 0 26 Oct 2020
Do Adversarially Robust ImageNet Models Transfer Better? Hadi Salman Andrew Ilyas Logan Engstrom Ashish Kapoor Aleksander Madry 96 426 0 16 Jul 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 298 388 0 05 Mar 2020
Overfitting in adversarially robust deep learning Leslie Rice Eric Wong Zico Kolter 121 809 0 26 Feb 2020
Attacks Which Do Not Kill Training Make Adversarial Learning Stronger Jingfeng Zhang Xilie Xu Bo Han Gang Niu Li-zhen Cui Masashi Sugiyama Mohan S. Kankanhalli AAML 58 404 0 26 Feb 2020
HYDRA: Pruning Adversarially Robust Neural Networks Vikash Sehwag Shiqi Wang Prateek Mittal Suman Jana AAML 49 25 0 24 Feb 2020
Fast is better than free: Revisiting adversarial training Eric Wong Leslie Rice J. Zico Kolter AAML OOD 142 1,181 0 12 Jan 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 163 630 0 11 Dec 2019
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers Ari S. Morcos Haonan Yu Michela Paganini Yuandong Tian 77 229 0 06 Jun 2019
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP Haonan Yu Sergey Edunov Yuandong Tian Ari S. Morcos 56 150 0 06 Jun 2019
Adversarial Training for Free! Ali Shafahi Mahyar Najibi Amin Ghiasi Zheng Xu John P. Dickerson Christoph Studer L. Davis Gavin Taylor Tom Goldstein AAML 136 1,253 0 29 Apr 2019
The State of Sparsity in Deep Neural Networks Trevor Gale Erich Elsen Sara Hooker 163 763 0 25 Feb 2019
Model Compression with Adversarial Robustness: A Unified Optimization Framework Shupeng Gui Haotao Wang Chen Yu Haichuan Yang Zhangyang Wang Ji Liu MQ 65 138 0 10 Feb 2019
Certified Adversarial Robustness via Randomized Smoothing Jeremy M. Cohen Elan Rosenfeld J. Zico Kolter AAML 169 2,052 0 08 Feb 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 69 150 0 02 Feb 2019
Rethinking ImageNet Pre-training Kaiming He Ross B. Girshick Piotr Dollár VLM SSeg 132 1,087 0 21 Nov 2018
Rethinking the Value of Network Pruning Zhuang Liu Mingjie Sun Tinghui Zhou Gao Huang Trevor Darrell 40 1,475 0 11 Oct 2018
Adversarially Robust Generalization Requires More Data Ludwig Schmidt Shibani Santurkar Dimitris Tsipras Kunal Talwar Aleksander Madry OOD AAML 161 797 0 30 Apr 2018
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 272 3,488 0 09 Mar 2018
Stochastic Activation Pruning for Robust Adversarial Defense Guneet Singh Dhillon Kamyar Azizzadenesheli Zachary Chase Lipton Jeremy Bernstein Jean Kossaifi Aran Khanna Anima Anandkumar AAML 84 547 0 05 Mar 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 264 1,901 0 28 Dec 2017
Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser Fangzhou Liao Ming Liang Yinpeng Dong Tianyu Pang Xiaolin Hu Jun Zhu 87 888 0 08 Dec 2017
Towards Robust Neural Networks via Random Self-ensemble Xuanqing Liu Minhao Cheng Huan Zhang Cho-Jui Hsieh FedML AAML 101 423 0 02 Dec 2017
Countering Adversarial Images using Input Transformations Chuan Guo Mayank Rana Moustapha Cissé Laurens van der Maaten AAML 128 1,406 0 31 Oct 2017
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 319 12,151 0 19 Jun 2017
Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks Weilin Xu David Evans Yanjun Qi AAML 97 1,273 0 04 Apr 2017
Adversarial Machine Learning at Scale Alexey Kurakin Ian Goodfellow Samy Bengio AAML 477 3,148 0 04 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 433 2,946 0 15 Sep 2016
A study of the effect of JPG compression on adversarial images Gintare Karolina Dziugaite Zoubin Ghahramani Daniel M. Roy AAML 94 535 0 02 Aug 2016
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,862 0 01 Oct 2015
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 282 19,129 0 20 Dec 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 289 14,968 1 21 Dec 2013