Effects of different types of regularization on the complexity of boundaries

Some work stated that for classification, the proper complexity to regularize is the boundary complexity, rather than the functional complexity of F. Indeed the functional complexity . The goal of classification is to recover the Bayes optimal decision boundary, which divides the input space into non-overlapping regions with respect to labels. Therefore, classification is better to be thought of as estimation of sets in R^d, rather than estimation of functions on R^d. ot be closely connected. The set difference reflects the 0-1 loss much more directly than functional norms on F.
Indeed we can have f in the functional space F that approximates the optimal classifier, \eta, so well that ||f -\eta||_{\infty}< \epsilon, but there is still no guarantee of matching the sign of \eta(x)- 1/2 close to the decision boundary. In this work we aim to study the effect of different types of regularization (l_1, l_2 norm, dropout , batch norm) on the boundary complexity and study how this is have an impact on generalization and robustness of the architecture.

Advisor: Fabrizio Silvestri