v1v2v3v4v5 (latest)

Essentially No Barriers in Neural Network Energy Landscape

2 March 2018

Papers citing "Essentially No Barriers in Neural Network Energy Landscape"

50 / 295 papers shown

Title
Towards Scalable and Versatile Weight Space Learning Konstantin Schurholt Michael W. Mahoney Damian Borth 107 19 0 14 Jun 2024
Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion Anke Tang Li Shen Yong Luo Shiwei Liu Han Hu Di Lin MoMe 73 7 0 14 Jun 2024
FusionBench: A Comprehensive Benchmark of Deep Model Fusion Anke Tang Li Shen Yong Luo Han Hu Di Lin Dacheng Tao ELM MoMe VLM 82 27 0 05 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang Haotian He Jinbo Wang Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li E. Weinan Lei Wu 100 8 0 31 May 2024
$C^2M^3$ : Cycle-Consistent Multi-Model Merging Donato Crisostomi Marco Fumero Daniele Baieri F. Bernard Emanuele Rodolà MoMe 101 10 0 28 May 2024
Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks Xin-Chun Li Jinli Tang Bo Zhang Lan Li De-Chuan Zhan 93 2 0 21 May 2024
Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networks Yichu Xu Xin-Chun Li Lan Li De-Chuan Zhan 93 2 0 21 May 2024
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability Lucius Bushnaq Jake Mendel Stefan Heimersheim Dan Braun Nicholas Goldowsky-Dill Kaarel Hänni Cindy Wu Marius Hobbhahn 88 7 0 17 May 2024
Localizing Task Information for Improved Model Merging and Compression Ke Wang Nikolaos Dimitriadis Guillermo Ortiz-Jimenez Franccois Fleuret Pascal Frossard MoMe 94 60 0 13 May 2024
Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity Zhanran Lin Puheng Li Lei Wu 257 9 0 09 Apr 2024
Do Deep Neural Network Solutions Form a Star Domain? Ankit Sonthalia Alexander Rubinstein Ehsan Abbasnejad Seong Joon Oh MoMe 596 4 5 12 Mar 2024
Federated Learning over Connected Modes Dennis Grinwald Philipp Wiesner Shinichi Nakajima FedML 195 0 0 05 Mar 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 119 8 0 01 Mar 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 106 12 0 26 Feb 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning Paradigm Zhanpeng Zhou Zijun Chen Yilan Chen Bo Zhang Junchi Yan MoMe 105 11 0 06 Feb 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 170 6 0 06 Feb 2024
How Good is a Single Basin? Kai Lion Lorenzo Noci Thomas Hofmann Gregor Bachmann UQCV 53 3 0 05 Feb 2024
Training-time Neuron Alignment through Permutation Subspace for Improving Linear Mode Connectivity and Model Fusion Zexi Li Zhiqi Li Jie Lin Tao Shen Tao Lin Chao Wu 131 5 0 02 Feb 2024
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts Anke Tang Li Shen Yong Luo Nan Yin Lefei Zhang Dacheng Tao MoMe 90 54 0 01 Feb 2024
Enhancing Neural Training via a Correlated Dynamics Model Jonathan Brokman Roy Betser Rotem Turjeman Tom Berkov I. Cohen Guy Gilboa 54 3 0 20 Dec 2023
Disentangling Linear Mode-Connectivity Gul Sena Altintas Gregor Bachmann Lorenzo Noci Thomas Hofmann 134 8 0 15 Dec 2023
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion Anke Tang Li Shen Yong Luo Liang Ding Han Hu Bo Du Dacheng Tao MoMe 96 22 0 11 Dec 2023
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 109 12 0 07 Dec 2023
On original and latent space connectivity in deep neural networks Boyang Gu Anastasia Borovykh GNN 3DPC 60 1 0 12 Nov 2023
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How Timm Hess Tinne Tuytelaars Gido M. van de Ven 77 7 0 08 Nov 2023
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport Damien Ferbach Baptiste Goujaud Gauthier Gidel Hadrien Hendrikx MoMe 129 16 0 29 Oct 2023
Linear Mode Connectivity in Sparse Neural Networks Luke McDermott Daniel Cummings 43 1 0 28 Oct 2023
A Quadratic Synchronization Rule for Distributed Deep Learning Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 92 1 0 22 Oct 2023
Revisiting Deep Ensemble for Out-of-Distribution Detection: A Loss Landscape Perspective Kun Fang Qinghua Tao Xiaolin Huang Jie Yang OODD 112 3 0 22 Oct 2023
Equivariant Deep Weight Space Alignment Aviv Navon Aviv Shamsian Ethan Fetaya Gal Chechik Nadav Dym Haggai Maron 88 24 0 20 Oct 2023
On permutation symmetries in Bayesian neural network posteriors: a variational perspective Simone Rossi Ankit Singh T. Hannagan 69 3 0 16 Oct 2023
A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors Olivier Laurent Emanuel Aldea Gianni Franchi BDL UQCV 79 8 0 12 Oct 2023
Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory Yiting Chen Zhanpeng Zhou Junchi Yan 61 9 0 10 Oct 2023
Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy Pingzhi Li Zhenyu Zhang Prateek Yadav Yi-Lin Sung Yu Cheng Mohit Bansal Tianlong Chen MoMe 85 39 0 02 Oct 2023
Towards guarantees for parameter isolation in continual learning Giulia Lanzillotta Sidak Pal Singh Benjamin Grewe Thomas Hofmann 65 0 0 02 Oct 2023
Mode Connectivity and Data Heterogeneity of Federated Learning Tailin Zhou Jun Zhang Danny H. K. Tsang FedML 80 3 0 29 Sep 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 117 62 0 27 Sep 2023
Policy Optimization in a Noisy Neighborhood: On Return Landscapes in Continuous Control Nate Rahn P. DÓro Harley Wiltzer Pierre-Luc Bacon Marc G. Bellemare 98 3 0 26 Sep 2023
Stochastic Gradient Descent outperforms Gradient Descent in recovering a high-dimensional signal in a glassy energy landscape Persia Jana Kamali Pierfrancesco Urbani 82 6 0 09 Sep 2023
Mode Combinability: Exploring Convex Combinations of Permutation Aligned Models Adrián Csiszárik M. Kiss Péter Korösi-Szabó Márton Muntag Gergely Papp D. Varga MoMe 57 1 0 22 Aug 2023
DOT: A Distillation-Oriented Trainer Borui Zhao Quan Cui Renjie Song Jiajun Liang 60 7 0 17 Jul 2023
Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature Connectivity Zhanpeng Zhou Yongyi Yang Xiaojiang Yang Junchi Yan Wei Hu 104 34 0 17 Jul 2023
Layer-wise Linear Mode Connectivity Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 119 17 0 13 Jul 2023
On The Impact of Machine Learning Randomness on Group Fairness Prakhar Ganesh Hong Chang Martin Strobel Reza Shokri FaML 65 31 0 09 Jul 2023
Large-scale global optimization of ultra-high dimensional non-convex landscapes based on generative neural networks Jiaqi Jiang Jonathan A. Fan 13 0 0 09 Jul 2023
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization Khashayar Gatmiry Zhiyuan Li Ching-Yao Chuang Sashank J. Reddi Tengyu Ma Stefanie Jegelka ODL 79 12 0 22 Jun 2023
Traversing Between Modes in Function Space for Fast Ensembling Eunggu Yun Hyungi Lee G. Nam Juho Lee UQCV 64 3 0 20 Jun 2023
Lookaround Optimizer: $k$ steps around, 1 step average Jiangtao Zhang Shunyu Liu Mingli Song Tongtian Zhu Zhenxing Xu Mingli Song MoMe 113 6 0 13 Jun 2023
Consistent Explanations in the Face of Model Indeterminacy via Ensembling Dan Ley Leonard Tang Matthew Nazari Hongjin Lin Suraj Srinivas Himabindu Lakkaraju 68 2 0 09 Jun 2023
TIES-Merging: Resolving Interference When Merging Models Prateek Yadav Derek Tam Leshem Choshen Colin Raffel Joey Tianyi Zhou MoMe 147 319 0 02 Jun 2023