Codivergences and information matrices

Abstract

We propose a new concept of codivergence, which quantifies the similarity between two probability measures P₁,P₂ relative to a reference probability measure P₀. In the neighborhood of the reference measure P₀, a codivergence behaves like an inner product between the measures P₁-P₀ and P₂-P₀. Codivergences of covariance-type and correlation-type are introduced and studied with a focus on two specific correlation-type codivergences, the χ²-codivergence and the Hellinger codivergence. We derive explicit expressions for several common parametric families of probability distributions. For a codivergence, we introduce moreover the divergence matrix as an analogue of the Gram matrix. It is shown that the χ²-divergence matrix satisfies a data-processing inequality.