Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting

This paper investigates the sample dependence of critical points for neural networks. We introduce a sample-independent critical lifting operator that associates a parameter of one network with a set of parameters of another, thus defining sample-dependent and sample-independent lifted critical points. We then show by example that previously studied critical embeddings do not capture all sample-independent lifted critical points. Finally, we demonstrate the existence of sample-dependent lifted critical points for sufficiently large sample sizes and prove that saddles appear among them.

@article{zhang2025_2505.13582,
  title={ Uncovering Critical Sets of Deep Neural Networks via Sample-Independent Critical Lifting },
  author={ Leyang Zhang and Yaoyu Zhang and Tao Luo },
  journal={arXiv preprint arXiv:2505.13582},
  year={ 2025 }
}