LOBO --Twitter Bot分类器中泛化缺陷的评估
LOBO -- Evaluation of Generalization Deficiencies in Twitter Bot Classifiers
摘要：
在线社交网络中的僵尸网络越来越经常地影响正常的讨论流，攻击普通用户及其帖子，向他们发送不相关或攻击性内容的垃圾邮件，甚至操纵消息和账户的受欢迎程度。研究人员和网络犯罪分子参与了军备竞赛，不断开发旨在击败当前检测系统的新的和更新的僵尸网络，使这些检测系统过时。在本文中，我们激发了Twitter机器人检测中的广义评估的需要，并提出了一种方法来评估机器人分类器，通过测试它们看不见的机器人类。我们证明了这种方法在经验上是稳健的，使用不同大小和特征的机器人类并达到类似的结果，并认为在单个机器人类或数据集上训练和测试的方法可能无法推广到新的机器人类。我们训练一个这样的分类超过20万个数据点，并表明它达到了97%以上的准确率。用于训练和测试此分类器的数据包括文献中使用的一些最大和最多样化的机器人集合。然后，我们使用我们的方法测试这个理论上合理的分类器，强调它不能很好地推广到看不见的机器人类。最后，我们讨论了我们的结果的影响，以及为什么一些机器人类比其他机器人类更容易和更快地检测的原因。
英文摘要：
Botnets in online social networks are increasingly often affecting the regular flow of discussion, attacking regular users and their posts, spamming them with irrelevant or offensive content, and even manipulating the popularity of messages and accounts. Researchers and cybercriminals are involved in an arms race, and new and updated botnets designed to defeat current detection systems are constantly developed, rendering such detection systems obsolete. In this paper, we motivate the need for a generalized evaluation in Twitter bot detection and propose a methodology to evaluate bot classifiers by testing them on unseen bot classes. We show that this methodology is empirically robust, using bot classes of varying sizes and characteristics and reaching similar results, and argue that methods trained and tested on single bot classes or datasets might not able to generalize to new bot classes. We train one such classifier on over 200,000 data points and show that it achieves over 97% accuracy. The data used to train and test this classifier includes some of the largest and most varied collections of bots used in literature. We then test this theoretically sound classifier using our methodology, highlighting that it does not generalize well to unseen bot classes. Finally, we discuss the implications of our results, and reasons why some bot classes are easier and faster to detect than others.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
