arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1911.04525cs.CL
收藏

了解BERT在宣传分析中的表现

Understanding BERT performance in propaganda analysis

Cornell Tech(康奈尔科技学院)

Yiqing Hua

摘要:

在本文中,我们描述了我们的系统中使用的共享任务细粒度的宣传分析在句子级别。尽管任务具有挑战性,但我们的预训练BERT模型(YMJA团队)在共享任务提供的训练数据集上进行了微调,在测试集上获得了0.62 F1,在参加比赛的25个团队中排名第三。我们提出了一组说明性的实验,以更好地了解我们的BERT模型在这个共享任务的性能。此外,我们还在给定的数据集之外探索可能由我们的系统产生的假阳性案例。我们发现,尽管给定的测试集上的高性能,我们的系统可能有分类的意见件作为宣传的倾向,并不能区分宣传讲话的报价从实际使用的宣传技术。

英文摘要:

In this paper, we describe our system used in the shared task for fine-grained propaganda analysis at sentence level. Despite the challenging nature of the task, our pretrained BERT model (team YMJA) fine tuned on the training dataset provided by the shared task scored 0.62 F1 on the test set and ranked third among 25 teams who participated in the contest. We present a set of illustrative experiments to better understand the performance of our BERT model on this shared task. Further, we explore beyond the given dataset for false-positive cases that likely to be produced by our system. We show that despite the high performance on the given testset, our system may have the tendency of classifying opinion pieces as propaganda and cannot distinguish quotations of propaganda speech from actual usage of propaganda techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑