KullbackLeibler

class odl.functionals.default_functionals.KullbackLeibler(*args, **kwargs)[source]

Bases: Functional

The Kullback-Leibler divergence functional.

Notes

The functional F with prior g>=0 is given by:

F(x) = \begin{cases} \int \left( x(t) - g(t) + g(t) \log \left( \frac{g(t)}{x(t)} \right) \right) dt & \text{if } x(t) > 0 \; \forall t \\ +\infty & \text{else.} \end{cases}

Note that we use the common definition 0 log(0) := 0. KL based objectives are common in MLEM optimization problems and are often used as data-matching term when data noise governed by a multivariate Poisson probability distribution is significant.

The functional is related to the Kullback-Leibler cross entropy functional KullbackLeiblerCrossEntropy. The KL cross entropy is the one diescribed in this Wikipedia article, and the functional F is obtained by switching place of the prior and the varialbe in the KL cross entropy functional.

For a theoretical exposition, see Csiszar1991.

See Also

KullbackLeiblerConvexConj : the convex conjugate functional KullbackLeiblerCrossEntropy : related functional

References

__init__(space, prior=None)[source]

Initialize a new instance.

Parameters

spaceDiscretizedSpace or TensorSpace

Domain of the functional.

priorspace element-like, optional

Depending on the context, the prior, target or data distribution. It is assumed to be nonnegative. Default: if None it is take as the one-element.

Examples

Test that KullbackLeibler(x,x) = 0

>>> space = odl.rn(3)
>>> prior = 3 * space.one()
>>> func = odl.functionals.KullbackLeibler(space, prior=prior)
>>> func(prior)
0.0

Test that zeros in the prior are handled correctly

>>> prior = space.zero()
>>> func = odl.functionals.KullbackLeibler(space, prior=prior)
>>> x = space.one()
>>> func(x)
3.0