Zero-Cost Proxies for Lightweight NAS

20 January 2021

Mohamed S. Abdelfattah

Papers citing "Zero-Cost Proxies for Lightweight NAS"

50 / 53 papers shown

Title
GreenFactory: Ensembling Zero-Cost Proxies to Estimate Performance of Neural Networks Gabriel Cortes Nuno Lourenço Paolo Romano Penousal Machado UQCV FedML 37 0 0 14 May 2025
L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformers S. Casarin Sergio Escalera Oswald Lanz 34 0 0 12 May 2025
Multi-Task Neural Architecture Search Using Architecture Embedding and Transfer Rank TingJie Zhang HaiLin Liu 43 1 0 01 Apr 2025
RBFleX-NAS: Training-Free Neural Architecture Search Using Radial Basis Function Kernel and Hyperparameter Detection Tomomasa Yamasaki Zhehui Wang Tao Luo Niangjun Chen Bo Wang 32 0 0 26 Mar 2025
Variation Matters: from Mitigating to Embracing Zero-Shot NAS Ranking Function Variation P. Rumiantsev Mark Coates 55 0 0 27 Feb 2025
NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance Raphael T. Husistein Markus Reiher Marco Eckhoff 142 1 0 20 Feb 2025
NASH: Neural Architecture and Accelerator Search for Multiplication-Reduced Hybrid Models Yang Xu Huihong Shi Zhongfeng Wang 45 0 0 07 Sep 2024
Efficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search An Vo Ngoc Hoang Luong 33 0 0 30 Jul 2024
ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models Yash Akhauri Ahmed F. AbouElhamayed Jordan Dotzel Zhiru Zhang Alexander M Rush Safeen Huda Mohamed S. Abdelfattah 23 2 0 24 Jun 2024
TG-NAS: Leveraging Zero-Cost Proxies with Transformer and Graph Convolution Networks for Efficient Neural Architecture Search Ye Qiao Haocheng Xu Sitao Huang 29 0 0 30 Mar 2024
Anytime Neural Architecture Search on Tabular Data Naili Xing Shaofeng Cai Zhaojing Luo Bengchin Ooi Jian Pei 34 1 0 15 Mar 2024
Robustifying and Boosting Training-Free Neural Architecture Search Zhenfeng He Yao Shu Zhongxiang Dai K. H. Low 43 1 0 12 Mar 2024
Efficient Stitchable Task Adaptation Haoyu He Zizheng Pan Jing Liu Jianfei Cai Bohan Zhuang 34 3 0 29 Nov 2023
DONNAv2 -- Lightweight Neural Architecture Search for Vision tasks Sweta Priyadarshi Tianyu Jiang Hsin-Pai Cheng S. Rama Krishna Viswanath Ganapathy C. Patel 44 0 0 26 Sep 2023
Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Search A. Chebykin A. Dushatskiy Tanja Alderliesten Peter A. N. Bosman 39 0 0 28 Jul 2023
An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustness Jovita Lukasik Michael Moeller M. Keuper 30 1 0 18 Jul 2023
Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyML M. Deutel G. Kontes Christopher Mutschler Jürgen Teich 52 0 0 23 May 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 36 19 0 06 Apr 2023
Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models? Boris Knyazev Doha Hwang Simon Lacoste-Julien AI4CE 31 17 0 07 Mar 2023
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients Guihong Li Yuedong Yang Kartikeya Bhardwaj R. Marculescu 36 60 0 26 Jan 2023
Efficient Evaluation Methods for Neural Architecture Search: A Survey Xiangning Xie Xiaotian Song Zeqiong Lv Gary G. Yen Weiping Ding Yizhou Sun 32 12 0 14 Jan 2023
DAS: Neural Architecture Search via Distinguishing Activation Score Yuqiao Liu Haipeng Li Yanan Sun Shuaicheng Liu 36 1 0 23 Dec 2022
GENNAPE: Towards Generalized Neural Architecture Performance Estimators Keith G. Mills Fred X. Han Jialin Zhang Fabián A. Chudak A. Mamaghani Mohammad Salameh Wei Lu Shangling Jui Di Niu 21 4 0 30 Nov 2022
Deep Model Reassembly Xingyi Yang Zhou Daquan Songhua Liu Jingwen Ye Xinchao Wang MoMe 20 120 0 24 Oct 2022
BLOX: Macro Neural Architecture Search Benchmark and Algorithms Thomas C. P. Chau L. Dudziak Hongkai Wen Nicholas D. Lane Mohamed S. Abdelfattah 30 13 0 13 Oct 2022
Towards Theoretically Inspired Neural Initialization Optimization Yibo Yang Hong Wang Haobo Yuan Zhouchen Lin 21 9 0 12 Oct 2022
Siamese-NAS: Using Trained Samples Efficiently to Find Lightweight Neural Architecture by Prior Knowledge Yumeng Zhang J. Hsieh Chun-Chieh Lee Kuo-Chin Fan 36 0 0 02 Oct 2022
EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoring Yash Akhauri J. P. Muñoz Nilesh Jain Ravi Iyer 46 13 0 15 Sep 2022
On the Privacy Risks of Cell-Based NAS Architectures Haiping Huang Zhikun Zhang Yun Shen Michael Backes Qi Li Yang Zhang 27 7 0 04 Sep 2022
Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey Dalin Zhang Kaixuan Chen Yan Zhao B. Yang Li-Ping Yao Christian S. Jensen 43 3 0 22 Aug 2022
Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution Yushu Wu Yifan Gong Pu Zhao Yanyu Li Zheng Zhan Wei Niu Hao Tang Minghai Qin Bin Ren Yanzhi Wang SupR MQ 35 23 0 25 Jul 2022
Machine Learning for Microcontroller-Class Hardware: A Review Swapnil Sayan Saha S. Sandha Mani B. Srivastava 27 118 0 29 May 2022
Efficient Automated Deep Learning for Time Series Forecasting Difan Deng Florian Karl Frank Hutter Bernd Bischl Marius Lindauer AI4TS 32 16 0 11 May 2022
When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search Guocheng Qian Xuanyang Zhang Bernard Ghanem Chen Zhao Yukang Chen Xinming Zhang Guohao Li Jian Sun 39 4 0 11 Apr 2022
Generalizing Few-Shot NAS with Gradient Matching Shou-Yong Hu Ruochen Wang Lanqing Hong Zhenguo Li Cho-Jui Hsieh Jiashi Feng 25 23 0 29 Mar 2022
Training-free Transformer Architecture Search Qinqin Zhou Kekai Sheng Xiawu Zheng Ke Li Xing Sun Yonghong Tian Jie Chen Rongrong Ji ViT 32 46 0 23 Mar 2022
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning Haoxiang Wang Yite Wang Ruoyu Sun Bo-wen Li 29 27 0 17 Mar 2022
Structured Pruning is All You Need for Pruning CNNs at Initialization Yaohui Cai Weizhe Hua Hongzheng Chen G. E. Suh Christopher De Sa Zhiru Zhang CVBM 44 14 0 04 Mar 2022
Neural Architecture Search for Dense Prediction Tasks in Computer Vision T. Elsken Arber Zela J. H. Metzen B. Staffler Thomas Brox Abhinav Valada Frank Hutter OOD 33 17 0 15 Feb 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Neural Architecture Search for Spiking Neural Networks Youngeun Kim Yuhang Li Hyoungseob Park Yeshwanth Venkatesha Priyadarshini Panda 26 88 0 23 Jan 2022
Automated Deep Learning: Neural Architecture Search Is Not the End Xuanyi Dong D. Kedziora Katarzyna Musial Bogdan Gabrys 25 26 0 16 Dec 2021
One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search Bingqian Lu Jianyi Yang Weiwen Jiang Yiyu Shi Shaolei Ren 24 24 0 01 Nov 2021
In Search of Probeable Generalization Measures Jonathan Jaegerman Khalil Damouni M. M. Ankaralı Konstantinos N. Plataniotis 21 2 0 23 Oct 2021
ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies Yu Shen Yang Li Jian Zheng Wentao Zhang Peng Yao Jixiang Li Sen Yang Ji Liu Cui Bin AI4CE 42 30 0 20 Oct 2021
Accelerating Multi-Objective Neural Architecture Search by Random-Weight Evaluation Shengran Hu Ran Cheng Cheng He Zhichao Lu Jing Wang Miao Zhang 32 7 0 08 Oct 2021
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Yao Shu Shaofeng Cai Zhongxiang Dai Beng Chin Ooi K. H. Low 22 43 0 02 Sep 2021
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving Ruochen Wang Xiangning Chen Minhao Cheng Xiaocheng Tang Cho-Jui Hsieh 35 13 0 18 Aug 2021
FLASH: Fast Neural Architecture Search with Hardware Optimization Guihong Li Sumit K. Mandal Ümit Y. Ogras R. Marculescu 27 20 0 01 Aug 2021
FEAR: A Simple Lightweight Method to Rank Architectures Debadeepta Dey Shital C. Shah Sébastien Bubeck OOD 27 4 0 07 Jun 2021