Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging

29 June 2023

Christoph Spiegel

Sebastian Pokutta

Papers citing "Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging"

14 / 14 papers shown

Title
PatientDx: Merging Large Language Models for Protecting Data-Privacy in Healthcare José G. Moreno Jesus Lovon M'Rick Robin-Charlet Christine Damase-Michel L. Tamine MoMe LM&MA 58 0 0 24 Apr 2025
On the Byzantine-Resilience of Distillation-Based Federated Learning Christophe Roux Max Zimmer Sebastian Pokutta AAML 57 1 0 19 Feb 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 55 10 0 23 Dec 2023
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models A. Jaiswal Shiwei Liu Tianlong Chen Ying Ding Zhangyang Wang VLM 44 22 0 18 Jun 2023
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Simon Lacoste-Julien MoMe 47 17 0 06 Apr 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 255 318 0 11 Sep 2022
Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training Lu Yin Vlado Menkovski Meng Fang Tianjin Huang Yulong Pei Mykola Pechenizkiy Decebal Constantin Mocanu Shiwei Liu 46 8 0 30 May 2022
Pruning has a disparate impact on model accuracy Cuong Tran Ferdinando Fioretto Jung-Eun Kim Rakshit Naidu 43 38 0 26 May 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 199 128 0 19 May 2022
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 153 685 0 31 Jan 2021
What is the State of Neural Network Pruning? Davis W. Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle John Guttag 191 1,032 0 06 Mar 2020
Comparing Rewinding and Fine-tuning in Neural Network Pruning Alex Renda Jonathan Frankle Michael Carbin 235 383 0 05 Mar 2020
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 276 5,683 0 05 Dec 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 307 39,238 0 01 Sep 2014