Consistent order estimation and the local geometry of mixtures

Consider an i.i.d. sequence of random variables whose distribution lies in one of a nested family of models , . The smallest index such that contains is called the model order. We establish strong consistency of the penalized likelihood order estimator in a general setting with penalties of order , where is a dimensional quantity. Moreover, such penalties are shown to be minimal. In contrast to previous work, an a priori upper bound on the model order is not assumed. The local dimension of the model is defined in terms of the bracketing entropy of a class of weighted densities, whose computation is a nonstandard problem which is of independent interest. We perform the requisite computations for the case of one-dimensional location mixtures, thus demonstrating the consistency of the penalized likelihood mixture order estimator. The proof requires a delicate analysis of the local geometry of the mixture family in a neighborhood of , for . The extension to more general mixture models remains an open problem.
View on arXiv