Parameter Prediction for Unseen Deep Architectures

25 October 2021

Boris Knyazev

M. Drozdzal

Graham W. Taylor

Adriana Romero Soriano

OOD

ArXiv PDF HTML

Papers citing "Parameter Prediction for Unseen Deep Architectures"

50 / 100 papers shown

Title
A Model Zoo of Vision Transformers Damian Falk Léo Meynent Florence Pfammatter Konstantin Schurholt Damian Borth 105 0 0 14 Apr 2025
Monomial Matrix Group Equivariant Neural Functional Networks Hoang V. Tran Thieu N. Vo Tho H. Tran An T. Nguyen Tan M. Nguyen 77 5 0 18 Sep 2024
Principled Weight Initialization for Hypernetworks Oscar Chang Lampros Flokas Hod Lipson 49 75 0 13 Dec 2023
Data-driven Weight Initialization with Sylvester Solvers Debasmit Das Yash Bhalgat Fatih Porikli ODL 38 3 0 02 May 2021
Meta Learning Black-Box Population-Based Optimizers H. Gomes B. Léger Christian Gagné 130 13 0 05 Mar 2021
GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training Chen Zhu Renkun Ni Zheng Xu Kezhi Kong Wenjie Huang Tom Goldstein ODL 48 56 0 16 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 244 514 0 11 Feb 2021
Characterizing signal propagation to close the performance gap in unnormalized ResNets Andrew Brock Soham De Samuel L. Smith 93 123 0 21 Jan 2021
SPAGAN: Shortest Path Graph Attention Network Yiding Yang Xinchao Wang Xiuming Zhang Junsong Yuan Dacheng Tao GNN 175 97 0 10 Jan 2021
Meta Learning Backpropagation And Improving It Louis Kirsch Jürgen Schmidhuber 75 56 0 29 Dec 2020
Building LEGO Using Deep Generative Models of Graphs Rylee Thompson Elahe Ghalebi Terrance Devries Graham W. Taylor GAN AI4CE 28 20 0 21 Dec 2020
Design Space for Graph Neural Networks Jiaxuan You Rex Ying J. Leskovec GNN AI4CE 92 317 0 17 Nov 2020
Are wider nets better given the same number of parameters? A. Golubeva Behnam Neyshabur Guy Gur-Ari 39 44 0 27 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 164 40,217 0 22 Oct 2020
Neural Architecture Performance Prediction Using Graph Neural Networks Jovita Lukasik David Friede Heiner Stuckenschmidt Margret Keuper GNN AI4CE 83 10 0 19 Oct 2020
Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves Luke Metz Niru Maheswaranathan C. Freeman Ben Poole Jascha Narain Sohl-Dickstein 110 62 0 23 Sep 2020
The Hardware Lottery Sara Hooker 45 205 0 14 Sep 2020
What is being transferred in transfer learning? Behnam Neyshabur Hanie Sedghi Chiyuan Zhang 52 513 0 26 Aug 2020
CATCH: Context-based Meta Reinforcement Learning for Transferrable Architecture Search Xin Chen Yawen Duan Zewei Chen Hang Xu Zihao Chen Xiaodan Liang Tong Zhang Zhenguo Li OffRL 21 21 0 18 Jul 2020
A Systematic Survey on Deep Generative Models for Graph Generation Xiaojie Guo Liang Zhao MedIm 68 147 0 13 Jul 2020
Graph Structure of Neural Networks Jiaxuan You J. Leskovec Kaiming He Saining Xie GNN 47 139 0 13 Jul 2020
The Computational Limits of Deep Learning Neil C. Thompson Kristjan Greenewald Keeheon Lee Gabriel F. Manso VLM 38 518 0 10 Jul 2020
On the Bottleneck of Graph Neural Networks and its Practical Implications Uri Alon Eran Yahav GNN 67 675 0 09 Jun 2020
Non-Local Graph Neural Networks Meng Liu Zhengyang Wang Shuiwang Ji 53 163 0 29 May 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 369 41,106 0 28 May 2020
Meta-Learning in Neural Networks: A Survey Timothy M. Hospedales Antreas Antoniou P. Micaelli Amos Storkey OOD 266 1,950 0 11 Apr 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 68 1,672 0 30 Mar 2020
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation Chaoyang He Haishan Ye Li Shen Tong Zhang 41 129 0 27 Mar 2020
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models Jiahui Yu Pengchong Jin Hanxiao Liu Gabriel Bender Pieter-Jan Kindermans Mingxing Tan Thomas Huang Xiaodan Song Ruoming Pang Quoc V. Le 47 302 0 24 Mar 2020
Benchmarking Graph Neural Networks Vijay Prakash Dwivedi Chaitanya K. Joshi Anh Tuan Luu T. Laurent Yoshua Bengio Xavier Bresson 267 929 0 02 Mar 2020
Geom-GCN: Geometric Graph Convolutional Networks Hongbin Pei Bingzhen Wei Kevin Chen-Chuan Chang Yu Lei Bo Yang GNN 224 1,105 0 13 Feb 2020
Deep Graph Similarity Learning: A Survey Guixiang Ma Nesreen Ahmed Theodore L. Willke Philip S. Yu GNN 38 77 0 25 Dec 2019
Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild Xin Chen Lingxi Xie Jun Wu Qi Tian 65 94 0 23 Dec 2019
Why are Adaptive Methods Good for Attention Models? J.N. Zhang Sai Praneeth Karimireddy Andreas Veit Seungyeon Kim Sashank J. Reddi Surinder Kumar S. Sra 75 80 0 06 Dec 2019
Neural Predictor for Neural Architecture Search W. Wen Hanxiao Liu H. Li Yiran Chen Gabriel Bender Pieter-Jan Kindermans 63 196 0 02 Dec 2019
SGAS: Sequential Greedy Architecture Search Ge Li Guocheng Qian Itzel C. Delgadillo Matthias Muller Ali K. Thabet Guohao Li 3DPC 37 186 0 30 Nov 2019
Meta-Learning of Neural Architectures for Few-Shot Learning T. Elsken B. Staffler J. H. Metzen Frank Hutter 47 137 0 25 Nov 2019
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark Xiaohua Zhai J. Puigcerver Alexander Kolesnikov P. Ruyssen C. Riquelme ... Michael Tschannen Marcin Michalski Olivier Bousquet Sylvain Gelly N. Houlsby SSL 49 432 0 01 Oct 2019
CARS: Continuous Evolution for Efficient Neural Architecture Search Zhaohui Yang Yunhe Wang Xinghao Chen Boxin Shi Chao Xu Chunjing Xu Qi Tian Chang Xu 25 226 0 11 Sep 2019
Once-for-All: Train One Network and Specialize it for Efficient Deployment Han Cai Chuang Gan Tianzhe Wang Zhekai Zhang Song Han OOD 64 1,267 0 26 Aug 2019
AutoML: A Survey of the State-of-the-Art Xin He Kaiyong Zhao Xiaowen Chu 69 1,440 0 02 Aug 2019
Faster Neural Network Training with Data Echoing Dami Choi Alexandre Passos Christopher J. Shallue George E. Dahl 35 48 0 12 Jul 2019
Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes James Requeima Jonathan Gordon J. Bronskill Sebastian Nowozin Richard Turner 39 241 0 18 Jun 2019
Position-aware Graph Neural Networks Jiaxuan You Rex Ying J. Leskovec 71 492 0 11 Jun 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 35 2,633 0 05 Jun 2019
LGM-Net: Learning to Generate Matching Networks for Few-Shot Learning Huaiyu Li Weiming Dong Xing Mei Chongyang Ma Feiyue Huang Bao-Gang Hu OffRL 57 98 0 15 May 2019
Searching for MobileNetV3 Andrew G. Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen ... Yukun Zhu Ruoming Pang Vijay Vasudevan Quoc V. Le Hartwig Adam 248 6,685 0 06 May 2019
Batch Normalization is a Cause of Adversarial Vulnerability A. Galloway A. Golubeva T. Tanay M. Moussa Graham W. Taylor ODL AAML 29 80 0 06 May 2019
DAG-GNN: DAG Structure Learning with Graph Neural Networks Yue Yu Jie Chen Tian Gao Mo Yu BDL CML GNN 52 481 0 22 Apr 2019
Knowledge Flow: Improve Upon Your Teachers Iou-Jen Liu Jian-wei Peng Alex Schwing 79 62 0 11 Apr 2019