Optimal measures and transition kernels

We study positive measures that are solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, constraints on information distance of the Kullback-Leibler type lead to solutions from a one-parameter exponential family, and measures within such a family have the property of mutual absolute continuity. Here we show that this property is related to strict convexity of a functional that is dual to the functional representing information, and variational problems for measures with constraints on a generalized information resource with this property define other families of mutually absolutely continuous optimal measures. This result carries an interesting implication to optimization of transitions between two sets, which are considered in many different areas including statistical decisions, estimation, control, communication and computation. Mutual absolute continuity of optimal joint measures implies that optimal transitions cannot be defined by deterministic Markov kernels, unless they communicate unconstrained information. For illustration, we construct an example where, unlike non-deterministic, any deterministic kernel either has negatively infinite expected utility (unbounded expected error) or communicates infinite information.