利用多语言深度学习实现开放网络上的自动注册识别
Automatic register identification for the open web using multilingual deep learning
AI总结：
本文提出多语言深度学习模型，用于识别开放网络中的文本注册，通过多标签分类在16种语言中达到79%的F1值，并发现性能上限源于注册本身的歧义性。
摘要：
本文提出了用于识别网页注册（如新闻报道和讨论论坛等文本变体）的多语言深度学习模型，覆盖16种语言。我们引入了Multilingual CORE语料库，其中包含超过72000篇标注有25种注册层次分类的文档，旨在覆盖整个开放网络。通过多标签分类，我们的最佳模型在跨语言平均达到79%的F1值，与先前使用更简单分类方案的研究结果相当或更优。这表明，即使在多语言规模上使用复杂的注册方案，模型也能表现良好。然而，我们发现所有模型和配置都存在一致的性能上限。当通过数据修剪移除标签不确定的文档时，性能提升至超过90%的F1值，表明这一上限源于网页注册本身的固有歧义，而非模型限制。对混合文本（包含多种注册的文本）的分析表明，主要挑战不在于分类混合文本本身，而在于区分混合文本与非混合文本。多语言模型在单语言模型上表现更优，特别是对于训练数据有限的语言。在未见过的语言上进行零样本测试时，性能平均下降7%，尽管不同语言的下降幅度不同（3-8%），这表明虽然注册在不同语言中共享特征，但它们也保留了语言特有的特性。
英文摘要：
This article presents multilingual deep learning models for identifying web registers -- text varieties such as news reports and discussion forums -- across 16 languages. We introduce the Multilingual CORE corpora, which contain over 72,000 documents annotated with a hierarchical taxonomy of 25 registers designed to cover the entire open web. Using multi-label classification, our best model achieves 79% F1 averaged across languages, matching or exceeding previous studies that used simpler classification schemes. This demonstrates that models can perform well even with a complex register scheme at multilingual scale. However, we observe a consistent performance ceiling across all models and configurations. When we remove documents with uncertain labels through data pruning, performance increases to over 90% F1, suggesting that this ceiling stems from inherent ambiguity in web registers rather than model limitations. Analysis of hybrid texts (those combining multiple registers) reveals that the main challenge lies not in classifying hybrids themselves, but in distinguishing hybrid from non-hybrid documents. Multilingual models consistently outperform monolingual ones, particularly for languages with limited training data. Zero-shot performance on unseen languages drops by an average of 7%, though this varies by language (3--8%), indicating that while registers share features across languages, they also retain language-specific characteristics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
