文本分类中第一类错误对无意识数据失真的有意控制
Intentional Control of Type I Error over Unconscious Data Distortion: a Neyman-Pearson Approach to Text Classification
摘要：
本文讨论了从开放的在线平台获得的文本数据分类的挑战，这是容易失真。大多数现有的分类方法使总体分类误差最小化，并且可能产生不期望的大的第一类错误（相关的文本消息被分类为不相关的），特别是当可用数据表现出相关和不相关信息之间的不对称时。数据失真加剧了这种情况，往往导致错误的预测。为了处理不可估量的数据失真，我们建议使用的奈曼-皮尔逊（NP）分类范式，最大限度地减少用户指定的第一类错误的约束下的第二类错误。理论上，我们表明，NP预言是不受数据失真时，类条件分布保持不变。从经验上讲，我们研究了一个案例，对从中国领先的微博平台获得的关于工人罢工的帖子进行分类，这些帖子经常容易受到广泛的、不可预测的和不可估量的审查。我们证明，即使训练和测试数据容易受到不同的失真，因此可能遵循不同的分布，我们提出的NP方法控制测试数据的第一类错误在目标水平。在我们的案例研究中提出的方法和实现管道适用于许多其他问题，涉及数据失真。
英文摘要：
This paper addresses the challenges in classifying textual data obtained from open online platforms, which are vulnerable to distortion. Most existing classification methods minimize the overall classification error and may yield an undesirably large type I error (relevant textual messages are classified as irrelevant), particularly when available data exhibit an asymmetry between relevant and irrelevant information. Data distortion exacerbates this situation and often leads to fallacious prediction. To deal with inestimable data distortion, we propose the use of the Neyman-Pearson (NP) classification paradigm, which minimizes type II error under a user-specified type I error constraint. Theoretically, we show that the NP oracle is unaffected by data distortion when the class conditional distributions remain the same. Empirically, we study a case of classifying posts about worker strikes obtained from a leading Chinese microblogging platform, which are frequently prone to extensive, unpredictable and inestimable censorship. We demonstrate that, even though the training and test data are susceptible to different distortion and therefore potentially follow different distributions, our proposed NP methods control the type I error on test data at the targeted level. The methods and implementation pipeline proposed in our case study are applicable to many other problems involving data distortion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
