Linear Mode Connectivity and the Lottery Ticket Hypothesis

11 December 2019

Jonathan Frankle

Gintare Karolina Dziugaite

Papers citing "Linear Mode Connectivity and the Lottery Ticket Hypothesis"

50 / 172 papers shown

Title
MergeBench: A Benchmark for Merging Domain-Specialized LLMs Yifei He Siqi Zeng Yuzheng Hu Rui Yang Tong Zhang Han Zhao MoMe ALM 34 0 0 16 May 2025
Are Spatial-Temporal Graph Convolution Networks for Human Action Recognition Over-Parameterized? Jianyang Xie Yitian Zhao Y. Meng He Zhao Anh Nguyen Yalin Zheng 22 0 0 15 May 2025
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry Mohammed Adnan Rohan Jain Ekansh Sharma Rahul Krishnan Yani Andrew Ioannou 61 0 0 08 May 2025
DMM: Building a Versatile Image Generation Model via Distillation-Based Model Merging Tianhui Song Weixin Feng Shuai Wang Xinfeng Li Tiezheng Ge Bo Zheng Limin Wang MoMe 64 0 0 16 Apr 2025
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang Ming Wang Sijia Liu Pin-Yu Chen MoMe 71 4 0 15 Apr 2025
On the Cone Effect in the Learning Dynamics Zhanpeng Zhou Yongyi Yang Jie Ren Mahito Sugiyama Junchi Yan 53 0 0 20 Mar 2025
Analyzing the Role of Permutation Invariance in Linear Mode Connectivity Keyao Zhan Puheng Li Lei Wu MoMe 87 0 0 13 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 89 0 0 07 Mar 2025
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation Qiuming Zhao Guangzhi Sun Chao Zhang Mingxing Xu Thomas Fang Zheng MoMe VLM 238 1 0 24 Feb 2025
Robust Concept Erasure Using Task Vectors Minh Pham Kelly O. Marshall Chinmay Hegde Niv Cohen 126 18 0 21 Feb 2025
Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models Shuqi Liu Han Wu Bowei He Xiongwei Han M. Yuan Linqi Song MoMe 66 2 0 20 Feb 2025
Unveiling Mode Connectivity in Graph Neural Networks Bingheng Li Z. Chen Haoyu Han Shenglai Zeng J. Liu Jiliang Tang 48 0 0 18 Feb 2025
Linear Mode Connectivity in Differentiable Tree Ensembles Ryuichi Kanoh M. Sugiyama 75 1 0 17 Feb 2025
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs Mohammad Mozaffari Amir Yazdanbakhsh Zhao Zhang M. Dehnavi 93 5 0 28 Jan 2025
Information Consistent Pruning: How to Efficiently Search for Sparse Networks? Soheil Gharatappeh Salimeh Yasaei Sekeh 59 0 0 28 Jan 2025
Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networks Giulia Fracastoro Sophie M. Fosson Andrea Migliorati G. Calafiore 45 1 0 19 Jan 2025
Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic Yifei He Yuzheng Hu Yong Lin Tong Zhang Han Zhao FedML MoMe 70 19 0 08 Jan 2025
Pushing the Limits of Sparsity: A Bag of Tricks for Extreme Pruning Andy Li A. Durrant Milan Markovic Lu Yin Georgios Leontidis Tianlong Chen Lu Yin Georgios Leontidis 80 0 0 20 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 52 5 0 05 Nov 2024
Navigating Extremes: Dynamic Sparsity in Large Output Spaces Nasib Ullah Erik Schultheis Mike Lasby Yani Andrew Ioannou Rohit Babbar 37 0 0 05 Nov 2024
Subgraph Aggregation for Out-of-Distribution Generalization on Graphs Bowen Liu Haoyang Li Shuning Wang Shuo Nie Shanghang Zhang OODD CML 76 0 0 29 Oct 2024
Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace Jinluan Yang Anke Tang Didi Zhu Zhengyu Chen Li Shen Fei Wu MoMe AAML 65 3 0 17 Oct 2024
Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamics Daniel Paulin P. Whalley Neil K. Chada B. Leimkuhler BDL 52 4 0 14 Oct 2024
Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense Rui Min Zeyu Qin Nevin L. Zhang Li Shen Minhao Cheng AAML 41 4 0 13 Oct 2024
Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion Bowen Tian Songning Lai Yutao Yue MoMe 35 0 0 08 Oct 2024
DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation Changdae Oh Yixuan Li Kyungwoo Song Sangdoo Yun Dongyoon Han OOD MoMe 50 4 0 03 Oct 2024
Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness Boqian Wu Q. Xiao Shunxin Wang N. Strisciuglio Mykola Pechenizkiy M. V. Keulen Decebal Constantin Mocanu Elena Mocanu OOD 3DH 60 0 0 03 Oct 2024
Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learning Da-Wei Zhou Zi-Wen Cai Han-Jia Ye Lijun Zhang De-Chuan Zhan CLL AI4CE 76 2 0 01 Oct 2024
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training Kun Song Zhiquan Tan Bochao Zou Jiansheng Chen Huimin Ma Weiran Huang 47 1 0 25 Sep 2024
S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training Yuezhou Hu Jun-Jie Zhu Jianfei Chen 53 0 0 13 Sep 2024
Input Space Mode Connectivity in Deep Neural Networks Jakub Vrabel Ori Shem-Ur Yaron Oz David Krueger 60 1 0 09 Sep 2024
Can Optimization Trajectories Explain Multi-Task Transfer? David Mueller Mark Dredze Matthew Wiesner 61 1 0 26 Aug 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 49 3 0 19 Aug 2024
Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality Hao Li Gopi Krishnan Rajbahadur C. Bezemer 41 5 0 07 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 37 9 0 07 Jul 2024
Neural Networks Trained by Weight Permutation are Universal Approximators Yongqiang Cai Gaohang Chen Zhonghua Qiao 72 1 0 01 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 62 14 0 24 Jun 2024
Towards Scalable and Versatile Weight Space Learning Konstantin Schurholt Michael W. Mahoney Damian Borth 52 16 0 14 Jun 2024
Geometric sparsification in recurrent neural networks Wyatt Mackey Ioannis Schizas Jared Deighton David L. Boothe, Jr. Vasileios Maroulas 38 0 0 10 Jun 2024
Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction Christoph Jürgen Hemmer Manuel Brenner Florian Hess Daniel Durstewitz 38 4 0 07 Jun 2024
Ensembling Diffusion Models via Adaptive Feature Aggregation Cong Wang Kuan Tian Yonghang Guan Jun Zhang Zhiwei Jiang Fei Shen Xiao Han 47 5 0 27 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su Edoardo Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Caccia Alessandro Sordoni MoMe 49 31 0 18 May 2024
Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Scheme Qianyu Long Qiyuan Wang Christos Anagnostopoulos Daning Bi FedML 28 0 0 24 Apr 2024
ONNXPruner: ONNX-Based General Model Pruning Adapter Dongdong Ren Wenbin Li Tianyu Ding Lei Wang Qi Fan Jing Huo Hongbing Pan Yang Gao 48 3 0 10 Apr 2024
FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning Rishub Tamirisa Chulin Xie Wenxuan Bao Andy Zhou Ron Arel Aviv Shamsian 33 6 0 03 Apr 2024
Federated Learning over Connected Modes Dennis Grinwald Philipp Wiesner Shinichi Nakajima FedML 45 0 0 05 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 103 19 0 04 Mar 2024
Fine-tuning with Very Large Dropout Jianyu Zhang Léon Bottou 46 1 0 01 Mar 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 67 5 0 06 Feb 2024
Reviving Undersampling for Long-Tailed Learning Hao Yu Yingxiao Du Jianxin Wu 38 1 0 30 Jan 2024