The Impact of Model Zoo Size and Composition on Weight Space Learning

14 April 2025

Papers citing "The Impact of Model Zoo Size and Composition on Weight Space Learning"

38 / 38 papers shown

Title
Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction Léo Meynent Ivan Melev Konstantin Schurholt Göran Kauermann Damian Borth 111 3 0 21 Mar 2025
Towards Scalable and Versatile Weight Space Learning Konstantin Schurholt Michael W. Mahoney Damian Borth 98 19 0 14 Jun 2024
Transferring Knowledge from Large Foundation Models to Small Downstream Models Shikai Qiu Boran Han Danielle C. Maddix Shuai Zhang Yuyang Wang Andrew Gordon Wilson 45 3 0 11 Jun 2024
Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling Cristian Rodriguez-Opazo Ehsan Abbasnejad Damien Teney Edison Marrese-Taylor Hamed Damirchi Anton Van Den Hengel VLM 96 1 0 27 May 2024
The Platonic Representation Hypothesis Minyoung Huh Brian Cheung Tongzhou Wang Phillip Isola 115 141 0 13 May 2024
Learning Useful Representations of Recurrent Neural Network Weight Matrices Vincent Herrmann Francesco Faccio Jürgen Schmidhuber 61 7 0 18 Mar 2024
Training-Free Pretrained Model Merging Zhenxing Xu Ke Yuan Huiqiong Wang Yong Wang Mingli Song Mingli Song MoMe 85 17 0 04 Mar 2024
Improved Generalization of Weight Space Networks via Augmentations Aviv Shamsian Aviv Navon David W. Zhang Yan Zhang Ethan Fetaya Gal Chechik Haggai Maron 95 14 0 06 Feb 2024
Graph Metanetworks for Processing Diverse Neural Architectures Derek Lim Haggai Maron Marc T. Law Jonathan Lorraine James Lucas AI4CE 77 39 0 07 Dec 2023
Equivariant Deep Weight Space Alignment Aviv Navon Aviv Shamsian Ethan Fetaya Gal Chechik Nadav Dym Haggai Maron 62 24 0 20 Oct 2023
Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models? Boris Knyazev Doha Hwang Simon Lacoste-Julien AI4CE 81 21 0 07 Mar 2023
Permutation Equivariant Neural Functionals Allan Zhou Kaien Yang Kaylee Burns Adriano Cardace Yiding Jiang Samuel Sokota J. Zico Kolter Chelsea Finn 90 56 0 27 Feb 2023
Equivariant Architectures for Learning in Deep Weight Spaces Aviv Navon Aviv Shamsian Idan Achituve Ethan Fetaya Gal Chechik Haggai Maron 95 69 0 30 Jan 2023
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization Alexandre Ramé Kartik Ahuja Jianyu Zhang Matthieu Cord Léon Bottou David Lopez-Paz MoMe OODD 99 86 0 20 Dec 2022
Deep Model Reassembly Xingyi Yang Zhou Daquan Songhua Liu Jingwen Ye Xinchao Wang MoMe 87 129 0 24 Oct 2022
Model Zoos: A Dataset of Diverse Populations of Neural Network Models Konstantin Schurholt Diyar Taskiran Boris Knyazev Xavier Giró-i-Nieto Damian Borth 126 30 0 29 Sep 2022
Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights Konstantin Schurholt Boris Knyazev Xavier Giró-i-Nieto Damian Borth 120 42 0 29 Sep 2022
Learning to Learn with Generative Models of Neural Network Checkpoints William S. Peebles Ilija Radosavovic Tim Brooks Alexei A. Efros Jitendra Malik UQCV 150 68 0 26 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 298 344 0 11 Sep 2022
Factorizing Knowledge in Neural Networks Xingyi Yang Jingwen Ye Xinchao Wang MoMe 102 125 0 04 Jul 2022
Learning the Space of Deep Models G. Berardi Luca de Luigi Samuele Salti Luigi Di Stefano SSL 51 3 0 10 Jun 2022
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time Mitchell Wortsman Gabriel Ilharco S. Gadre Rebecca Roelofs Raphael Gontijo-Lopes ... Hongseok Namkoong Ali Farhadi Y. Carmon Simon Kornblith Ludwig Schmidt MoMe 161 1,009 1 10 Mar 2022
Parameter Prediction for Unseen Deep Architectures Boris Knyazev M. Drozdzal Graham W. Taylor Adriana Romero Soriano OOD 97 83 0 25 Oct 2021
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs Kaichao You Yong Liu Ziyang Zhang Jianmin Wang Michael I. Jordan Mingsheng Long 187 33 0 20 Oct 2021
Robust fine-tuning of zero-shot models Mitchell Wortsman Gabriel Ilharco Jong Wook Kim Mike Li Simon Kornblith ... Raphael Gontijo-Lopes Hannaneh Hajishirzi Ali Farhadi Hongseok Namkoong Ludwig Schmidt VLM 158 739 0 04 Sep 2021
Zoo-Tuning: Adaptive Transfer from a Zoo of Models Yang Shu Zhi Kou Zhangjie Cao Jianmin Wang Mingsheng Long 62 44 0 29 Jun 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 125 635 0 18 Jun 2021
Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types Thomas Mensink J. Uijlings Alina Kuznetsova Michael Gygli V. Ferrari VLM 96 84 0 24 Mar 2021
Predicting Neural Network Accuracy from Weights Thomas Unterthiner Daniel Keysers Sylvain Gelly Olivier Bousquet Ilya O. Tolstikhin 69 107 0 26 Feb 2020
Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data Charles H. Martin Tongsu Peng Peng Michael W. Mahoney 90 110 0 17 Feb 2020
Classifying the classifier: dissecting the weight space of neural networks Gabriel Eilertsen Daniel Jonsson Timo Ropinski Jonas Unger Anders Ynnerman 59 54 0 13 Feb 2020
Traditional and Heavy-Tailed Self Regularization in Neural Network Models Charles H. Martin Michael W. Mahoney 79 126 0 24 Jan 2019
Graph HyperNetworks for Neural Architecture Search Chris Zhang Mengye Ren R. Urtasun GNN 76 280 0 12 Oct 2018
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification P. Helber B. Bischke Andreas Dengel Damian Borth 156 1,834 0 31 Aug 2017
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms Han Xiao Kashif Rasul Roland Vollgraf 285 8,926 0 25 Aug 2017
HyperNetworks David R Ha Andrew M. Dai Quoc V. Le 170 1,633 0 27 Sep 2016
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,745 0 09 Mar 2015
How transferable are features in deep neural networks? J. Yosinski Jeff Clune Yoshua Bengio Hod Lipson OOD 236 8,353 0 06 Nov 2014