KullbackLeiblerCrossEntropy

class odl.functionals.default_functionals.KullbackLeiblerCrossEntropy(*args, **kwargs)[source]

Bases: Functional

The Kullback-Leibler Cross Entropy divergence functional.

Notes

The functional F with prior g>0 is given by:

F(x) = \begin{cases} \sum_{i} \left( g_i - x_i + x_i \log \left( \frac{x_i}{g_i} \right) \right) & \text{if } g_i > 0 \forall i \\ +\infty & \text{else} \end{cases}

For further information about the functional, see the Wikipedia article on the Kullback Leibler divergence, or read for example this article.

The KL cross entropy functional F, described above, is related to another functional which is also know as KL divergence. This functional is often used as data discrepancy term in inverse problems, when data is corrupted with Poisson noise. It is obtained by changing place of the prior and the variable. See the See Also section.

For a theoretical exposition, see Csiszar1991.

See Also

KullbackLeibler : related functional KullbackLeiblerCrossEntropyConvexConj : the convex conjugate functional

References

__init__(space, prior=None)[source]

Initialize a new instance.

Parameters

spaceDiscretizedSpace or TensorSpace

Domain of the functional.

priorspace element-like, optional

Depending on the context, the prior, target or data distribution. It is assumed to be nonnegative. Default: if None it is take as the one-element.