arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2012.07521cs.CL
收藏

Procode:瑞士职业和经济活动自动编码和记录的多语言解决方案

Procode: the Swiss Multilingual Solution for Automatic Coding and Recoding of Occupations and Economic Activities

Unisanté(瑞士大学卫生中心(Unisanté));University of Lausanne(洛桑大学);Research Institute for Environmental and Occupational Health(环境与职业健康研究院)

Nenad Savic, Nicolas Bovio, Fabian Gilbert, Irina Guseva Canu

摘要:

Objective.流行病学研究需要符合职业或经济活动分类的数据。分类通常包括数百个代码和标题。原始数据的手动编码可能会导致错误分类,并且非常耗时。目标是开发和测试一个名为Procode的网络工具,用于对照分类对自由文本进行编码,并在不同分类之间重新编码。方法.采用k折交叉验证的方法,研究了三种文本分类器,即互补朴素贝叶斯(CNB)、支持向量机(SVM)和随机森林分类器(RFC)。有30000个自由文本,带有手动分配的法国职业分类(PCS)和法国活动分类(NAF)的分类代码。对于重新编码,Procode集成了一个工作流程,根据现有的人行横道将一种分类的代码转换为另一种。由于这是一个简单的操作,因此只测量了重新编码时间。结果在三个被调查的文本分类器中,CNB导致最好的性能,其中分类器分别准确地预测PCS和NAF的分类代码的57-81%和63-83%。SVM导致结果稍低(1-2%),而RFC准确编码高达30%的数据。编码操作需要每10000个记录一分钟,而重新编码则更快,即5-10秒。结论集成在Procode中的算法表现出令人满意的性能,因为该工具必须通过在500-700个不同的选择中进行选择来分配正确的代码。根据结果,作者决定在Procode中实现CNB。将来,如果另一个分类器表现出更好的性能,更新将包括所需的修改。

英文摘要:

Objective. Epidemiological studies require data that are in alignment with the classifications established for occupations or economic activities. The classifications usually include hundreds of codes and titles. Manual coding of raw data may result in misclassification and be time consuming. The goal was to develop and test a web-tool, named Procode, for coding of free-texts against classifications and recoding between different classifications. Methods. Three text classifiers, i.e. Complement Naive Bayes (CNB), Support Vector Machine (SVM) and Random Forest Classifier (RFC), were investigated using a k-fold cross-validation. 30 000 free-texts with manually assigned classification codes of French classification of occupations (PCS) and French classification of activities (NAF) were available. For recoding, Procode integrated a workflow that converts codes of one classification to another according to existing crosswalks. Since this is a straightforward operation, only the recoding time was measured. Results. Among the three investigated text classifiers, CNB resulted in the best performance, where the classifier predicted accurately 57-81% and 63-83% classification codes for PCS and NAF, respectively. SVM lead to somewhat lower results (by 1-2%), while RFC coded accurately up to 30% of the data. The coding operation required one minute per 10 000 records, while the recoding was faster, i.e. 5-10 seconds. Conclusion. The algorithm integrated in Procode showed satisfactory performance, since the tool had to assign the right code by choosing between 500-700 different choices. Based on the results, the authors decided to implement CNB in Procode. In future, if another classifier shows a superior performance, an update will include the required modifications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑