Title: On divergences, surrogate loss functions and decentralized detection
Author: X. Nguyen, M. J. Wainwright and M. I. Jordan
Date: October 2005
Pub: PDF
Url: http://www.stat.berkeley.edu/~wainwrig/695.pdf


Abstract: We develop a general correspondence between a family of loss
functions that act as surrogates to 0-1 loss, and the class of
Ali-Silvey or $f$-divergence functionals.  This correspondence
provides the basis for choosing and evaluating various surrogate
losses frequently used in statistical learning (e.g., hinge loss,
exponential loss, logistic loss); conversely, it provides a
decision-theoretic framework for the choice of divergences in signal
processing and quantization theory.  We exploit this correspondence to
characterize the statistical behavior of a nonparametric decentralized
hypothesis testing algorithms that operate by minimizing convex
surrogate loss functions.  In particular, we specify the family of
loss functions that are equivalent to 0-1 loss in the sense of
producing the same quantization rules and discriminant functions.