Margin Distribution Analysis.

Margin is an important concept in machine learning; theoretical analyses further reveal that the distribution of margin plays a more critical role than the minimum margin in generalization power. Recently, several approaches have achieved performance breakthroughs by optimizing the margin distribution, but their computational cost, which is usually higher than before, still hinders them to be widely applied. In this article, we propose margin distribution analysis (MDA), which optimizes the margin distribution more simply by maximizing the margin mean and minimizing the margin variance simultaneously. MDA is efficient and resistive to class-imbalance naturally, since its objective distinguishes the margin means of different classes and can be broken up into two linear equations. In practice, it can also cooperate with other frameworks such as reweight-minimization when facing complex circumstances with noise and outliers. Empirical studies validate the superiority of MDA in real-world data sets, and demonstrate that simple approaches can also perform competitively by optimizing margin distribution.