Training-free Transformer Architecture Search

23 March 2022

Yonghong Tian

Jie Chen

Rongrong Ji

ViT

ArXiv PDF HTML

Papers citing "Training-free Transformer Architecture Search"

28 / 28 papers shown

Title
ZeroLM: Data-Free Transformer Architecture Search for Language Models Zhen-Song Chen Hong-Wei Ding Xian-Jia Wang Witold Pedrycz 55 0 0 24 Mar 2025
Merino: Entropy-driven Design for Generative Language Models on IoT Devices Youpeng Zhao Ming Lin Huadong Tang Qiang Wu Jun Wang 83 0 0 28 Jan 2025
Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insights Sy-Tuyen Ho Tuan Van Vo Somayeh Ebrahimkhani Ngai-man Cheung 42 0 0 08 Jan 2025
HyTAS: A Hyperspectral Image Transformer Architecture Search Benchmark and Analysis Fangqin Zhou Mert Kilickaya Joaquin Vanschoren Ran Piao 21 1 0 23 Jul 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 47 45 0 17 May 2024
Large Language Models Synergize with Automated Machine Learning Jinglue Xu Jialong Li Zhen Liu Nagar Anthel Venkatesh Suryanarayanan Guoyuan Zhou Jia Guo Hitoshi Iba Kenji Tei 43 4 0 06 May 2024
AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search Junghyup Lee Bumsub Ham 32 6 0 28 Mar 2024
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression Hancheng Ye Chong Yu Peng Ye Renqiu Xia Yansong Tang Jiwen Lu Tao Chen Bo-Wen Zhang 56 3 0 23 Mar 2024
When Training-Free NAS Meets Vision Transformer: A Neural Tangent Kernel Perspective Qiqi Zhou Yichen Zhu ViT 16 1 0 15 Mar 2024
ParZC: Parametric Zero-Cost Proxies for Efficient NAS Peijie Dong Lujun Li Xinglin Pan Zimian Wei Xiang Liu Qiang-qiang Wang Xiaowen Chu 66 3 0 03 Feb 2024
TVT: Training-Free Vision Transformer Search on Tiny Datasets Zimian Wei H. Pan Lujun Li Peijie Dong Zhiliang Tian Xin-Yi Niu Dongsheng Li ViT 47 7 0 24 Nov 2023
Entropic Score metric: Decoupling Topology and Size in Training-free NAS Niccolò Cavagnero Luc Robbiano Francesca Pistilli Barbara Caputo Giuseppe Averta 23 2 0 06 Oct 2023
A Survey of Techniques for Optimizing Transformer Inference Krishna Teja Chitty-Venkata Sparsh Mittal M. Emani V. Vishwanath Arun Somani 43 62 0 16 Jul 2023
Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities Guihong Li Duc-Tuong Hoang Kartikeya Bhardwaj Ming Lin Zhangyang Wang R. Marculescu 40 10 0 05 Jul 2023
AutoST: Training-free Neural Architecture Search for Spiking Transformers Ziqing Wang Qidong Zhao Jinku Cui Xu Liu Dongkuan Xu 21 5 0 01 Jul 2023
Training-free Neural Architecture Search for RNNs and Transformers Aaron Serianni Jugal Kalita 28 7 0 01 Jun 2023
GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samples T. Gao Chengzhong Xu Le Zhang Hui Kong 38 4 0 13 May 2023
Maximizing Spatio-Temporal Entropy of Deep 3D CNNs for Efficient Video Recognition Junyan Wang Zhenhong Sun Yichen Qian Dong Gong Xiuyu Sun Ming Lin M. Pagnucco Yang Song 3DPC 20 11 0 05 Mar 2023
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients Guihong Li Yuedong Yang Kartikeya Bhardwaj R. Marculescu 36 60 0 26 Jan 2023
Efficient Evaluation Methods for Neural Architecture Search: A Survey Xiangning Xie Xiaotian Song Zeqiong Lv Gary G. Yen Weiping Ding Yizhou Sun 32 12 0 14 Jan 2023
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 35 159 0 15 Dec 2022
NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxies Arjun Krishnakumar Colin White Arber Zela Renbo Tu Mahmoud Safari Frank Hutter 55 40 0 06 Oct 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 23 347 0 02 Jun 2022
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models Mojan Javaheripi Gustavo de Rosa Subhabrata Mukherjee S. Shah Tomasz Religa C. C. T. Mendes Sébastien Bubeck F. Koushanfar Debadeepta Dey 31 18 0 04 Mar 2022
Connection Sensitivity Matters for Training-free DARTS: From Architecture-Level Scoring to Operation-Level Sensitivity Analysis Miao Zhang Wei Huang Li Wang 26 1 0 22 Jun 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 289 1,524 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 304 3,623 0 24 Feb 2021
Bag of Tricks for Image Classification with Convolutional Neural Networks Tong He Zhi-Li Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li 221 1,399 0 04 Dec 2018