412
4
v1v2v3 (latest)

Tracking the Feature Dynamics in LLM Training: A Mechanistic Study

Main:9 Pages
45 Figures
Bibliography:3 Pages
3 Tables
Appendix:22 Pages
Abstract

Understanding training dynamics and feature evolution is crucial for the mechanistic interpretability of large language models (LLMs). Although sparse autoencoders (SAEs) have been used to identify features within LLMs, a clear picture of how these features evolve during training remains elusive. In this study, we (1) introduce SAE-Track, a novel method for efficiently obtaining a continual series of SAEs, providing the foundation for a mechanistic study that covers (2) the semantic evolution of features, (3) the underlying processes of feature formation, and (4) the directional drift of feature vectors. Our work provides new insights into the dynamics of features in LLMs, enhancing our understanding of training mechanisms and feature evolution. For reproducibility, our code is available atthis https URL.

View on arXiv
@article{xu2025_2412.17626,
  title={ Tracking the Feature Dynamics in LLM Training: A Mechanistic Study },
  author={ Yang Xu and Yi Wang and Hengguan Huang and Hao Wang },
  journal={arXiv preprint arXiv:2412.17626},
  year={ 2025 }
}
Comments on this paper

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.