RooseBERT: 政治语言建模的新协议
RooseBERT: A New Deal For Political Language Modelling
AI总结：
针对政治语言特殊性，提出领域预训练模型RooseBERT，在大型政治辩论语料上训练，在多项政治分析任务中优于通用模型。
摘要：
政治辩论和与政治相关讨论的日益增多，要求定义新颖的计算方法来自动分析此类内容，最终目标是让公民更清晰地了解政治审议。然而，政治语言的特殊性和这些辩论的论证形式（采用隐藏的沟通策略并利用隐含论点）使得这项任务非常具有挑战性，即使是对于当前通用的预训练语言模型（LMs）也是如此。为了解决这个问题，我们引入了一种新颖的预训练语言模型，专门用于政治话语语言，称为RooseBERT。在专业领域上预训练语言模型面临着不同的技术和语言挑战，需要大量的计算资源和大规模数据。RooseBERT是在大型英语政治辩论和演讲语料库（11GB）上训练的。为了评估其性能，我们在多个与政治辩论分析相关的下游任务上对其进行了微调，即立场检测、情感分析、论证成分检测与分类、论证关系预测与分类、政策分类、命名实体识别（NER）。我们的结果显示，在大多数这些任务上，RooseBERT相比通用语言模型有所改进，突显了领域特定预训练如何增强政治辩论分析的性能。我们将RooseBERT发布给研究社区。
英文摘要：
The increasing amount of political debates and politics-related discussions calls for the definition of novel computational methods to automatically analyse such content with the final goal of lightening up political deliberation to citizens. However, the specificity of the political language and the argumentative form of these debates (employing hidden communication strategies and leveraging implicit arguments) make this task very challenging, even for current general-purpose pre-trained Language Models (PLMs). To address this, we introduce a novel PLM for political discourse language called RooseBERT. Pre-training a language model on a specialised domain presents different technical and linguistic challenges, requiring extensive computational resources and large-scale data. RooseBERT has been trained on large political debate and speech corpora (11GB) in English. To evaluate its performances, we fine-tuned it on multiple downstream tasks related to political debate analysis, i.e., stance detection, sentiment analysis, argument component detection and classification, argument relation prediction and classification, policy classification, named entity recognition (NER). Our results show improvements over general-purpose PLMs on the majority of these tasks, highlighting how domain-specific pre-training enhances performance in political debate analysis. We release RooseBERT for the research community: https://huggingface.co/collections/MARIANNE-INRIA/roosebert.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
