AutoFormer: Searching Transformers for Visual Recognition

1 July 2021

Papers citing "AutoFormer: Searching Transformers for Visual Recognition"

50 / 155 papers shown

Title
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Peiyan Dong Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng Zechao Li Xue Lin Zhenman Fang Yanzhi Wang ViT 36 63 0 15 Nov 2022
Data Level Lottery Ticket Hypothesis for Vision Transformers Xuan Shen Zhenglun Kong Minghai Qin Peiyan Dong Geng Yuan Xin Meng Hao Tang Xiaolong Ma Yanzhi Wang 30 6 0 02 Nov 2022
Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data Huy Hoang Nguyen Matthew B. Blaschko S. Saarakkala A. Tiulpin MedIm AI4CE 50 15 0 25 Oct 2022
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets Yong Guo Yaofo Chen Yin Zheng Qi Chen P. Zhao Jian Chen Junzhou Huang Mingkui Tan 30 5 0 14 Oct 2022
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning Weicong Liang Yuhui Yuan Henghui Ding Xiao Luo Weihong Lin Ding Jia Zheng-Wei Zhang Chao Zhang Hanhua Hu 42 26 0 03 Oct 2022
EAPruning: Evolutionary Pruning for Vision Transformers and CNNs Qingyuan Li Bo Zhang Xiangxiang Chu ViT VLM 27 3 0 01 Oct 2022
NasHD: Efficient ViT Architecture Performance Ranking using Hyperdimensional Computing Dongning Ma Pengfei Zhao Xun Jiao 33 0 0 23 Sep 2022
Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment Zhichao Lu Ran Cheng Yaochu Jin Kay Chen Tan Kalyanmoy Deb 27 53 0 08 Aug 2022
Neural Architecture Search on Efficient Transformers and Beyond Zexiang Liu Dong Li Kaiyue Lu Zhen Qin Weixuan Sun Jiacheng Xu Yiran Zhong 35 19 0 28 Jul 2022
TinyViT: Fast Pretraining Distillation for Small Vision Transformers Kan Wu Jinnian Zhang Houwen Peng Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 21 248 0 21 Jul 2022
UFO: Unified Feature Optimization Teng Xi Yifan Sun Deli Yu Bi Li Nan Peng ... Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang 34 10 0 21 Jul 2022
Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks Yongming Rao Zuyan Liu Wenliang Zhao Jie Zhou Jiwen Lu ViT 46 36 0 04 Jul 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 39 32 0 19 Jun 2022
Neural Prompt Search Yuanhan Zhang Kaiyang Zhou Ziwei Liu VPVLM VLM 55 145 0 09 Jun 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 26 348 0 02 Jun 2022
Fast Vision Transformers with HiLo Attention Zizheng Pan Jianfei Cai Bohan Zhuang 28 152 0 26 May 2022
ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks Haoran You Baopu Li Huihong Shi Y. Fu Yingyan Lin 52 17 0 17 May 2022
MiniViT: Compressing Vision Transformers with Weight Multiplexing Jinnian Zhang Houwen Peng Kan Wu Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 28 124 0 14 Apr 2022
A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search Stephen Cha Taehyeon Kim Hayeon Lee Se-Young Yun AI4CE 47 2 0 08 Apr 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 34 62 0 29 Mar 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 26 40 0 29 Mar 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 33 46 0 28 Mar 2022
Training-free Transformer Architecture Search Qinqin Zhou Kekai Sheng Xiawu Zheng Ke Li Xing Sun Yonghong Tian Jie Chen Rongrong Ji ViT 45 46 0 23 Mar 2022
DATA: Domain-Aware and Task-Aware Self-supervised Learning Qing Chang Junran Peng Lingxi Xie Jiajun Sun Hao Yin Qi Tian Zhaoxiang Zhang 42 8 0 17 Mar 2022
STDAN: Deformable Attention Network for Space-Time Video Super-Resolution Hai Wang Xiaoyu Xiang Yapeng Tian Wenming Yang Q. Liao AI4TS 34 14 0 14 Mar 2022
Multi-trial Neural Architecture Search with Lottery Tickets Zimian Wei H. Pan Lujun Li Menglong Lu Xin-Yi Niu Peijie Dong Dongsheng Li ViT 34 0 0 08 Mar 2022
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models Mojan Javaheripi Gustavo de Rosa Subhabrata Mukherjee S. Shah Tomasz Religa C. C. T. Mendes Sébastien Bubeck F. Koushanfar Debadeepta Dey 39 18 0 04 Mar 2022
Multi-Tailed Vision Transformer for Efficient Inference Yunke Wang Bo Du Wenyuan Wang Chang Xu ViT 220 6 0 03 Mar 2022
Auto-scaling Vision Transformers without Training Wuyang Chen Wei Huang Xianzhi Du Xiaodan Song Zhangyang Wang Denny Zhou ViT 32 23 0 24 Feb 2022
BViT: Broad Attention based Vision Transformer Nannan Li Yaran Chen Weifan Li Zixiang Ding Dong Zhao ViT 38 23 0 13 Feb 2022
Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search Bicheng Guo Tao Chen Shibo He Haoyu Liu Lili Xu Peng Ye Jiming Chen 44 13 0 30 Jan 2022
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models Dongkuan Xu Subhabrata Mukherjee Xiaodong Liu Debadeepta Dey Wenhui Wang Xiang Zhang Ahmed Hassan Awadallah Jianfeng Gao 33 4 0 29 Jan 2022
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer Mengshu Sun Haoyu Ma Guoliang Kang Yi Ding Tianlong Chen Xiaolong Ma Zhangyang Wang Yanzhi Wang ViT 33 46 0 17 Jan 2022
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space Arnav Chavan Zhiqiang Shen Zhuang Liu Zechun Liu Kwang-Ting Cheng Eric P. Xing ViT 33 70 0 03 Jan 2022
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Zhenglun Kong Peiyan Dong Xiaolong Ma Xin Meng Mengshu Sun ... Geng Yuan Bin Ren Minghai Qin Hao Tang Yanzhi Wang ViT 34 145 0 27 Dec 2021
Searching the Search Space of Vision Transformer Minghao Chen Kan Wu Bolin Ni Houwen Peng Bei Liu Jianlong Fu Hongyang Chao Haibin Ling ViT 42 52 0 29 Nov 2021
MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection Zhenhong Sun Ming Lin Xiuyu Sun Zhiyu Tan Hao Li Rong Jin 23 32 0 26 Nov 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 34 40 0 23 Nov 2021
Mesa: A Memory-saving Training Framework for Transformers Zizheng Pan Peng Chen Haoyu He Jing Liu Jianfei Cai Bohan Zhuang 31 20 0 22 Nov 2021
Sliced Recursive Transformer Zhiqiang Shen Zechun Liu Eric P. Xing ViT 25 27 0 09 Nov 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 30 39 0 10 Oct 2021
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers Changlin Li Guangrun Wang Bing Wang Xiaodan Liang Zhihui Li Xiaojun Chang 35 9 0 21 Sep 2021
Joint Graph Learning and Matching for Semantic Feature Correspondence He Liu Tao Wang Yidong Li Congyan Lang Yi Jin Haibin Ling 30 20 0 01 Sep 2021
Rethinking and Improving Relative Position Encoding for Vision Transformer Kan Wu Houwen Peng Minghao Chen Jianlong Fu Hongyang Chao ViT 53 330 0 29 Jul 2021
GLiT: Neural Architecture Search for Global and Local Image Transformer Boyu Chen Peixia Li Chuming Li Baopu Li Lei Bai Chen Lin Ming Sun Junjie Yan Wanli Ouyang ViT 35 85 0 07 Jul 2021
ViTAS: Vision Transformer Architecture Search Xiu Su Shan You Jiyang Xie Mingkai Zheng Fei Wang Chao Qian Changshui Zhang Xiaogang Wang Chang Xu ViT 34 54 0 25 Jun 2021
Less is More: Pay Less Attention in Vision Transformers Zizheng Pan Bohan Zhuang Haoyu He Jing Liu Jianfei Cai ViT 24 82 0 29 May 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 120 209 0 26 Apr 2021
SoT: Delving Deeper into Classification Head for Transformer Jiangtao Xie Rui Zeng Qilong Wang Ziqi Zhou P. Li ViT 34 12 0 22 Apr 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 48 105 0 23 Mar 2021