L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformers

12 May 2025

Papers citing "L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformers"

32 / 32 papers shown

Title
Graph is all you need? Lightweight data-agnostic neural architecture search without training Zhenhan Huang Tejaswini Pedapati Pin-Yu Chen Chunheng Jiang Jianxi Gao GNN 82 1 0 02 May 2024
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients Guihong Li Yuedong Yang Kartikeya Bhardwaj R. Marculescu 94 62 0 26 Jan 2023
Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training? J. Mok Byunggook Na Ji-Hoon Kim Dongyoon Han Sungroh Yoon AAML 91 25 0 28 Mar 2022
GradSign: Model Performance Inference with Theoretical Insights Zhihao Zhang Zhihao Jia 53 24 0 16 Oct 2021
TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search Yawen Duan Xin Chen Hang Xu Zewei Chen Li Xiaodan Tong Zhang Zhenguo Li ViT 75 73 0 25 May 2021
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective Wuyang Chen Xinyu Gong Zhangyang Wang OOD 111 238 0 23 Feb 2021
Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition Ming Lin Pichao Wang Zhenhong Sun Hesen Chen Xiuyu Sun Qi Qian Hao Li Rong Jin 70 108 0 01 Feb 2021
Zero-Cost Proxies for Lightweight NAS Mohamed S. Abdelfattah Abhinav Mehrotra Łukasz Dudziak Nicholas D. Lane 75 260 0 20 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,802 0 23 Dec 2020
Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks Arber Zela Julien N. Siems Lucas Zimmer Jovita Lukasik Margret Keuper Frank Hutter 73 81 0 22 Aug 2020
Evaluating Efficient Performance Estimators of Neural Architectures Xuefei Ning Changcheng Tang Wenshuo Li Zixuan Zhou Shuang Liang Huazhong Yang Yu Wang 79 78 0 07 Aug 2020
BRP-NAS: Prediction-based NAS using GCNs Łukasz Dudziak Thomas C. P. Chau Mohamed S. Abdelfattah Royson Lee Hyeji Kim Nicholas D. Lane GNN 64 205 0 16 Jul 2020
On the Number of Linear Regions of Convolutional Neural Networks Huan Xiong Lei Huang Mengyang Yu Li Liu Fan Zhu Ling Shao MLT 61 68 0 01 Jun 2020
Low-Rank Bottleneck in Multi-head Attention Models Srinadh Bhojanapalli Chulhee Yun A. S. Rawat Sashank J. Reddi Sanjiv Kumar 66 96 0 17 Feb 2020
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search Xuanyi Dong Yi Yang 141 714 0 02 Jan 2020
Neural Predictor for Neural Architecture Search W. Wen Hanxiao Liu H. Li Yiran Chen Gabriel Bender Pieter-Jan Kindermans 81 200 0 02 Dec 2019
Searching for A Robust Neural Architecture in Four GPU Hours Xuanyi Dong Yezhou Yang 126 654 0 10 Oct 2019
Understanding and Robustifying Differentiable Architecture Search Arber Zela T. Elsken Tonmoy Saikia Yassine Marrakchi Thomas Brox Frank Hutter OOD AAML 140 373 0 20 Sep 2019
Once-for-All: Train One Network and Specialize it for Efficient Deployment Han Cai Chuang Gan Tianzhe Wang Zhekai Zhang Song Han OOD 113 1,283 0 26 Aug 2019
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search Yuhui Xu Lingxi Xie Xiaopeng Zhang Xin Chen Guo-Jun Qi Qi Tian H. Xiong 88 609 0 12 Jul 2019
Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation Xin Chen Lingxi Xie Jun Wu Qi Tian AI4TS MQ 73 664 0 29 Apr 2019
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours Dimitrios Stamoulis Ruizhou Ding Di Wang Dimitrios Lymberopoulos B. Priyantha Jie Liu Diana Marculescu 60 285 0 05 Apr 2019
SNAS: Stochastic Neural Architecture Search Sirui Xie Hehui Zheng Chunxiao Liu Liang Lin 85 938 0 24 Dec 2018
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 839 0 19 Dec 2018
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Han Cai Ligeng Zhu Song Han 102 1,875 0 02 Dec 2018
SNIP: Single-shot Network Pruning based on Connection Sensitivity Namhoon Lee Thalaiyasingam Ajanthan Philip Torr VLM 266 1,207 0 04 Oct 2018
Neural Architecture Optimization Renqian Luo Fei Tian Tao Qin Enhong Chen Tie-Yan Liu 3DV 90 655 0 22 Aug 2018
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 204 4,366 0 24 Jun 2018
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 150 612 0 01 Jun 2018
Efficient Neural Architecture Search via Parameter Sharing Hieu H. Pham M. Guan Barret Zoph Quoc V. Le J. Dean 117 2,766 0 09 Feb 2018
Progressive Neural Architecture Search Chenxi Liu Barret Zoph Maxim Neumann Jonathon Shlens Wei Hua Li Li Li Fei-Fei Alan Yuille Jonathan Huang Kevin Patrick Murphy 114 1,994 0 02 Dec 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 478 5,381 0 05 Nov 2016