Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data

17 February 2020

Papers citing "Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data"

50 / 64 papers shown

Title
A Model Zoo on Phase Transitions in Neural Networks Konstantin Schurholt Léo Meynent Yefan Zhou Haiquan Lu Yaoqing Yang Damian Borth 68 0 0 25 Apr 2025
The Impact of Model Zoo Size and Composition on Weight Space Learning Damian Falk Konstantin Schurholt Damian Borth 36 0 0 14 Apr 2025
A Model Zoo of Vision Transformers Damian Falk Léo Meynent Florence Pfammatter Konstantin Schurholt Damian Borth 34 0 0 14 Apr 2025
Regional Tiny Stories: Using Small Models to Compare Language Learning and Tokenizer Performance Nirvan Patil Malhar Abhay Inamdar Agnivo Gosai Guruprasad Pathak Anish Joshi Aryan Sagavekar Anish Joshirao Raj Abhijit Dandekar Rajat Dandekar Sreedath Panat 46 0 0 07 Apr 2025
ZeroLM: Data-Free Transformer Architecture Search for Language Models Zhen-Song Chen Hong-Wei Ding Xian-Jia Wang Witold Pedrycz 53 0 0 24 Mar 2025
Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction Léo Meynent Ivan Melev Konstantin Schurholt Göran Kauermann Damian Borth 47 2 0 21 Mar 2025
Can We Optimize Deep RL Policy Weights as Trajectory Modeling? Hongyao Tang OffRL 77 0 0 06 Mar 2025
Using Pre-trained LLMs for Multivariate Time Series Forecasting Malcolm Wolff Shenghao Yang Kari Torkkola Michael W. Mahoney AI4TS AIFin 46 1 0 10 Jan 2025
LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics Tiankai Xie Jiaqing Chen Yaoqing Yang Caleb Geniesse Ge Shi ... J. Cava Michael W. Mahoney Talita Perciano Gunther H. Weber Ross Maciejewski 77 0 0 17 Dec 2024
Evaluating Loss Landscapes from a Topology Perspective Tiankai Xie Caleb Geniesse Jiaqing Chen Yaoqing Yang Dmitriy Morozov Michael W. Mahoney Ross Maciejewski Gunther H. Weber 28 1 0 14 Nov 2024
A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities Yatin Dandi Luca Pesce Hugo Cui Florent Krzakala Yue M. Lu Bruno Loureiro MLT 37 1 0 24 Oct 2024
How Does Data Diversity Shape the Weight Landscape of Neural Networks? Yang Ba M. Mancenido Rong Pan 26 0 0 18 Oct 2024
Model Balancing Helps Low-data Training and Fine-tuning Zihang Liu Yihan Hu Tianyu Pang Yefan Zhou Pu Ren Yaoqing Yang 36 2 0 16 Oct 2024
AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models Haiquan Lu Yefan Zhou Shiwei Liu Zhangyang Wang Michael W. Mahoney Yaoqing Yang 29 0 0 14 Oct 2024
AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality Peijun Qing Chongyang Gao Yefan Zhou Xingjian Diao Yaoqing Yang Soroush Vosoughi MoMe MoE 24 3 0 14 Oct 2024
Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning Siyuan Li Juanxi Tian Zedong Wang Luyuan Zhang Zicheng Liu Weiyang Jin Yang Liu Baigui Sun Stan Z. Li 34 0 0 08 Oct 2024
Mitigating Memorization In Language Models Mansi Sakarvadia Aswathy Ajith Arham Khan Nathaniel Hudson Caleb Geniesse Kyle Chard Yaoqing Yang Ian Foster Michael W. Mahoney KELM MU 58 0 0 03 Oct 2024
MD tree: a model-diagnostic tree grown on loss landscape Yefan Zhou Jianlong Chen Qinxue Cao Konstantin Schürholt Yaoqing Yang 33 2 0 24 Jun 2024
Towards Scalable and Versatile Weight Space Learning Konstantin Schurholt Michael W. Mahoney Damian Borth 50 15 0 14 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 58 13 0 13 Jun 2024
Crafting Heavy-Tails in Weight Matrix Spectrum without Gradient Noise Vignesh Kothapalli Tianyu Pang Shenyang Deng Zongmin Liu Yaoqing Yang 37 3 0 07 Jun 2024
Transferable and Principled Efficiency for Open-Vocabulary Segmentation Jingxuan Xu Wuyang Chen Yao-Min Zhao Yunchao Wei VLM 36 2 0 11 Apr 2024
Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance Chiraag Kaushik Ran Liu Chi-Heng Lin Amrit Khera Matthew Y Jin Wenrui Ma Vidya Muthukumar Eva L. Dyer 43 3 0 18 Feb 2024
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training Yefan Zhou Tianyu Pang Keqin Liu Charles H. Martin Michael W. Mahoney Yaoqing Yang 36 7 0 01 Dec 2023
Can we infer the presence of Differential Privacy in Deep Learning models' weights? Towards more secure Deep Learning Daniel Jiménez-López Daniel Nuria Rodríguez Barroso Nuria M. V. Luzón M. Victoria Francisco Herrera Francisco AAML 19 0 0 20 Nov 2023
A PAC-Bayesian Perspective on the Interpolating Information Criterion Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 41 1 0 13 Nov 2023
Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities L. Akoglu Jaemin Yoo 33 1 0 28 Aug 2023
To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networks Rajan Sahu Shivam Chadha N. Nagaraj A. Mathur Snehanshu Saha 8 0 0 19 Aug 2023
Fast Unsupervised Deep Outlier Model Selection with Hypernetworks Xueying Ding Yue Zhao L. Akoglu OODD 22 4 0 20 Jul 2023
The Interpolating Information Criterion for Overparameterized Models Liam Hodgkinson Christopher van der Heide Roberto Salomone Fred Roosta Michael W. Mahoney 20 7 0 15 Jul 2023
Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices M. Ferrag Mthandazo Ndhlovu Norbert Tihanyi Lucas C. Cordeiro Merouane Debbah Thierry Lestable Narinderjit Singh Thandi 18 72 0 25 Jun 2023
Revealing Model Biases: Assessing Deep Neural Networks via Recovered Sample Analysis M. Mehmanchi Mahbod Nouri Mohammad Sabokrou AAML 30 1 0 10 Jun 2023
A Three-regime Model of Network Pruning Yefan Zhou Yaoqing Yang Arin Chang Michael W. Mahoney 29 10 0 28 May 2023
Heavy-Tailed Regularization of Weight Matrices in Deep Neural Networks Xuanzhe Xiao Zengyi Li Chuanlong Xie Fengwei Zhou 23 3 0 06 Apr 2023
Deep Learning Weight Pruning with RMT-SVD: Increasing Accuracy and Reducing Overfitting Yitzchak Shmalo Jonathan Jenkins Oleksii Krupchytskyi 24 3 0 15 Mar 2023
Model scale versus domain knowledge in statistical forecasting of chaotic systems W. Gilpin AI4TS 30 14 0 13 Mar 2023
Greedy Ordering of Layer Weight Matrices in Transformers Improves Translation Elicia Ye 21 1 0 04 Feb 2023
Spectral Evolution and Invariance in Linear-width Neural Networks Zhichao Wang A. Engel Anand D. Sarwate Ioana Dumitriu Tony Chiang 40 14 0 11 Nov 2022
Multi-Object Navigation with dynamically learned neural implicit representations Pierre Marza L. Matignon Olivier Simonin Christian Wolf 29 23 0 11 Oct 2022
Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights Konstantin Schurholt Boris Knyazev Xavier Giró-i-Nieto Damian Borth 52 38 0 29 Sep 2022
The SVD of Convolutional Weights: A CNN Interpretability Framework Brenda Praggastis Davis Brown Carlos Ortiz Marrero Emilie Purvine Madelyn Shapiro Bei Wang FAtt 35 9 0 14 Aug 2022
Latent Properties of Lifelong Learning Systems Corban G. Rivera C. Ashcraft Alexander New J. Schmidt Gautam K. Vallabha CLL 12 0 0 28 Jul 2022
Dimension of activity in random neural networks David G. Clark L. F. Abbott Ashok Litwin-Kumar 13 23 0 25 Jul 2022
Hyper-Representations for Pre-Training and Transfer Learning Konstantin Schurholt Boris Knyazev Xavier Giró-i-Nieto Damian Borth 17 10 0 22 Jul 2022
Boundary between noise and information applied to filtering neural network weight matrices Max Staats M. Thamm B. Rosenow 23 3 0 08 Jun 2022
TorchNTK: A Library for Calculation of Neural Tangent Kernels of PyTorch Models A. Engel Zhichao Wang Anand D. Sarwate Sutanay Choudhury Tony Chiang 22 3 0 24 May 2022
Random matrix analysis of deep neural network weight matrices M. Thamm Max Staats B. Rosenow 29 12 0 28 Mar 2022
Extended critical regimes of deep neural networks Chengqing Qu Asem Wardak P. Gong AI4CE 21 1 0 24 Mar 2022
Learning continuous models for continuous physics Aditi S. Krishnapriyan A. Queiruga N. Benjamin Erichson Michael W. Mahoney AI4CE 24 33 0 17 Feb 2022
Investigating Power laws in Deep Representation Learning Arna Ghosh Arnab Kumar Mondal Kumar Krishna Agrawal Blake A. Richards SSL OOD 11 19 0 11 Feb 2022