arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.08127cs.LGcs.IR
收藏

领域预处理对主题聚类的影响

The Influence of Domain-Based Preprocessing on Subject-Specific Clustering

University of Bath(巴斯大学)

Alexandra Gkolia, Nikhil Fernandes, Nicolas Pizzo, James Davenport, Akshar Nair

摘要:

由于全球新冠肺炎疫情,大学突然改变将大部分教学转移到网上,这给学者带来了更大的工作量。其中一个促成因素是回答来自学生的大量查询。由于这些查询不限于讲座的同步时间范围,因此其中许多查询很有可能是相关的,甚至是等同的。处理这个问题的一种方法是根据主题将这些问题分组。在我们之前的工作中,我们的目标是使用循环LDA模型找到一种改进的聚类方法,从而提高效率。我们的数据集包含了巴斯大学计算机科学课程在线发布的问题。这些问题中有相当一部分包含代码摘录,我们发现这在聚类中造成了问题,因为某些术语被认为是英语中的常用词,而不是被认为是特定的代码术语。为了解决这个问题,我们使用Python实现了这些技术术语的标记,作为数据集预处理的一部分。在本文中,我们将探讨标记数据集的领域,重点是识别代码摘录和提供实证结果,以证明我们的推理。

英文摘要:

The sudden change of moving the majority of teaching online at Universities due to the global Covid-19 pandemic has caused an increased amount of workload for academics. One of the contributing factors is answering a high volume of queries coming from students. As these queries are not limited to the synchronous time frame of a lecture, there is a high chance of many of them being related or even equivalent. One way to deal with this problem is to cluster these questions depending on their topic. In our previous work, we aimed to find an improved method of clustering that would give us a high efficiency, using a recurring LDA model. Our data set contained questions posted online from a Computer Science course at the University of Bath. A significant number of these questions contained code excerpts, which we found caused a problem in clustering, as certain terms were being considered as common words in the English language and not being recognised as specific code terms. To address this, we implemented tagging of these technical terms using Python, as part of preprocessing the data set. In this paper, we explore the realms of tagging data sets, focusing on identifying code excerpts and providing empirical results in order to justify our reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑