We develop a worst-case analysis of aggregation of ensembles of binary classifiers in a semi-supervised setting, for a broad class of losses including but not limited to all convex surrogates. The result is a family of parameter-free ensemble aggregation algorithms which use labeled and unlabeled data; these are as efficient as linear learning and prediction for convex risk minimization but work without any relaxations on many nonconvex losses like the 0-1 loss. The prediction algorithms take a familiar form, applying "inverse link functions" to a generalized notion of ensemble margin, but without the assumptions typically made in margin-based learning. All this structure follows from interpreting loss minimization as a game played over unlabeled data.
View on arXiv