Blockwisely Supervised Neural Architecture Search with Knowledge Distillation

29 November 2019

Xiaodan Liang

Xiaojun Chang

Papers citing "Blockwisely Supervised Neural Architecture Search with Knowledge Distillation"

33 / 33 papers shown

Title
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 77 1 0 28 Nov 2024
DONNAv2 -- Lightweight Neural Architecture Search for Vision tasks Sweta Priyadarshi Tianyu Jiang Hsin-Pai Cheng S. Rama Krishna Viswanath Ganapathy C. Patel 41 0 0 26 Sep 2023
Knowledge Distillation from Single to Multi Labels: an Empirical Study Youcai Zhang Yuzhuo Qin Heng-Ye Liu Yanhao Zhang Yaqian Li X. Gu VLM 53 2 0 15 Mar 2023
Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels Zifu Wang Xuefei Ning Matthew B. Blaschko VLM 30 12 0 11 Feb 2023
$β$ -DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search Peng Ye Tong He Baopu Li Tao Chen Lei Bai Wanli Ouyang OOD 43 7 0 16 Jan 2023
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets Yong Guo Yaofo Chen Yin Zheng Qi Chen P. Zhao Jian Chen Junzhou Huang Mingkui Tan 28 5 0 14 Oct 2022
BLOX: Macro Neural Architecture Search Benchmark and Algorithms Thomas C. P. Chau L. Dudziak Hongkai Wen Nicholas D. Lane Mohamed S. Abdelfattah 27 13 0 13 Oct 2022
Searching a High-Performance Feature Extractor for Text Recognition Network Hui Zhang Quanming Yao James T. Kwok X. Bai 28 7 0 27 Sep 2022
Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey Dalin Zhang Kaixuan Chen Yan Zhao B. Yang Li-Ping Yao Christian S. Jensen 43 3 0 22 Aug 2022
Long-term Reproducibility for Neural Architecture Search David Towers M. Forshaw Amir Atapour-Abarghouei A. Mcgough 19 1 0 11 Jul 2022
Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing? Keshigeyan Chandrasegaran Ngoc-Trung Tran Yunqing Zhao Ngai-man Cheung 83 41 0 29 Jun 2022
Distributed Training for Deep Learning Models On An Edge Computing Network Using ShieldedReinforcement Learning Tanmoy Sen Haiying Shen OffRL 11 5 0 01 Jun 2022
Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search Minbin Huang Zhijian Huang Changlin Li Xin Chen Hangyang Xu Zhenguo Li Xiaodan Liang 30 18 0 12 Apr 2022
When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search Guocheng Qian Xuanyang Zhang Bernard Ghanem Chen Zhao Yukang Chen Xinming Zhang Guohao Li Jian Sun 39 4 0 11 Apr 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 28 46 0 28 Mar 2022
Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval Binjie Zhang Yixiao Ge Yantao Shen Yu Li Chun Yuan Xuyuan Xu Yexin Wang Ying Shan VLM 33 7 0 24 Jan 2022
PP-LCNet: A Lightweight CPU Convolutional Neural Network Cheng Cui Tingquan Gao Shengyun Wei Yuning Du Ruoyu Guo ... X. Lv Qiwen Liu Xiaoguang Hu Dianhai Yu Yanjun Ma ObjD 39 121 0 17 Sep 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation Chenhe Dong Guangrun Wang Hang Xu Jiefeng Peng Xiaozhe Ren Xiaodan Liang 16 28 0 15 Sep 2021
A Light-weight Deep Human Activity Recognition Algorithm Using Multi-knowledge Distillation Runze Chen Haiyong Luo Fang Zhao Xuechun Meng Zhiqing Xie Yida Zhu VLM HAI 24 2 0 06 Jul 2021
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 36 259 0 01 Jul 2021
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients Miao Zhang Steven W. Su Shirui Pan Xiaojun Chang Ehsan Abbasnejad Reza Haffari 24 68 0 21 Jun 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 46 105 0 23 Mar 2021
Towards Personalized Federated Learning A. Tan Han Yu Li-zhen Cui Qiang Yang FedML AI4CE 209 840 0 01 Mar 2021
AlphaNet: Improved Training of Supernets with Alpha-Divergence Dilin Wang Chengyue Gong Meng Li Qiang Liu Vikas Chandra 155 44 0 16 Feb 2021
Inter-layer Transition in Neural Architecture Search Benteng Ma Jing Zhang Yong-quan Xia Dacheng Tao 34 4 0 30 Nov 2020
DOTS: Decoupling Operation and Topology in Differentiable Architecture Search Yuchao Gu Li-Juan Wang Yun-Hai Liu Yi Yang Yu-Huan Wu Shao-Ping Lu Ming-Ming Cheng 32 48 0 02 Oct 2020
Differentiable Feature Aggregation Search for Knowledge Distillation Yushuo Guan Pengyu Zhao Bingxuan Wang Yuanxing Zhang Cong Yao Kaigui Bian Jian Tang FedML 17 44 0 02 Aug 2020
Cyclic Differentiable Architecture Search Hongyuan Yu Houwen Peng Yan Huang Jianlong Fu Hao Du Liang Wang Haibin Ling 3DPC 25 48 0 18 Jun 2020
Multi-fidelity Neural Architecture Search with Knowledge Distillation I. Trofimov Nikita Klyuchnikov Mikhail Salnikov Alexander N. Filippov Evgeny Burnaev 32 15 0 15 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,843 0 09 Jun 2020
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions Pengzhen Ren Yun Xiao Xiaojun Chang Po-Yao (Bernie) Huang Zhihui Li Xiaojiang Chen Xin Wang AI4CE 48 653 0 01 Jun 2020
MixPath: A Unified Approach for One-shot Neural Architecture Search Xiangxiang Chu Shun Lu Xudong Li Bo-Wen Zhang 24 21 0 16 Jan 2020
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,329 0 05 Nov 2016