Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g. WANDA), while using other methods lead to fast performance degradation.
View on arXiv@article{ghattas2025_2502.18886,
title={ On Pruning State-Space LLMs },
author={ Tamer Ghattas and Michael Hassid and Roy Schwartz },
journal={arXiv preprint arXiv:2502.18886},
year={ 2025 }
}