In this paper we aim to provide machine learning practitioners with tools to answer the question: have the labels in a dataset been corrupted? In order to simplify the problem, we assume the practitioner already has preconceptions on possible distortions that may have affected the labels, which allow us to pose the task as the design of hypothesis tests.
As a first approach, we focus on scenarios where a given dataset of instance-label pairs has been corrupted with class-conditional label noise , as opposed to uniform label noise , with the former biasing learning, while the latter — under mild conditions — does not.
(mh=WIvL-TsQ0mM_tejs)15.jpg "SEX ESCORT Saint-Étienne-du-Rouvray")
While previous works explore the direct estimation of the noise rates, this is known to be hard in practice and does not offer a real understanding of how trustworthy the estimates are. These methods typically require anchor points — examples whose true posterior is either 0 or 1. The proposed hypothesis tests are built upon the asymptotic properties of Maximum Likelihood Estimators for Logistic Regression models.
We establish the main properties of the tests, including a theoretical and empirical analysis of the dependence of the power on the test on the training sample size, the number of anchor points, the difference of the noise rates and the use of relaxed anchors. This is a preview of subscription content, log in via an institution to check access. Institutional subscriptions.
Different notions -related to our definition- of anchor points have been used before in the literature under different names. We review their uses and assumptions in Sect. What we have so far presented is aligned with the Neyman-Pearson theory of hypothesis testing.
