arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.04532cs.IRcs.LGstat.ML
收藏

同伴互评中问题陈述的发现

Detecting Problem Statements in Peer Assessments

North Carolina State University(北卡罗来纳州立大学);Northeastern University(东北大学);Shanghai Jiao Tong University(上海交通大学);University of North Carolina at Wilmington(北卡罗来纳大学威尔明顿分校);Southern University of Science and Technology(南方科技大学)
Yunkai Xiao, Gabriel Zingle, Qinjin Jia, Harsh R. Shah, Yi Zhang, Tianyi Li, Mohsin Karovaliya, Weixiang Zhao, Yang Song, Jie Ji, Ashwin Balasubramaniam, Harshi… 展开作者

Yunkai Xiao, Gabriel Zingle, Qinjin Jia, Harsh R. Shah, Yi Zhang, Tianyi Li, Mohsin Karovaliya, Weixiang Zhao, Yang Song, Jie Ji, Ashwin Balasubramaniam, Harshit Patel, Priyankha Bhalasubbramanian, Vikram Patel, Edward F. Gehringer

摘要:

有效的同伴评估要求学生注意他们所评估的工作中的不足之处。因此,对它们的审查应查明问题。但是有什么方法可以检查他们做的呢?我们试图自动化决定评论是否检测到问题的过程。我们使用了超过18,000条评论,这些评论被评论者标记为检测到或没有检测到工作中的问题。我们部署了几个传统的机器学习模型,以及使用GloVe和BERT嵌入的神经网络模型。我们发现表现最好的是分层注意力网络分类器,其次是双向门控递归单元(GRU)注意力和胶囊模型,得分分别为93.1%和90.5%。最好的非神经网络模型是支持向量机,得分为89.71%。其次是随机梯度下降模型和Logistic回归模型,分别为89.70%和88.98%。

英文摘要:

Effective peer assessment requires students to be attentive to the deficiencies in the work they rate. Thus, their reviews should identify problems. But what ways are there to check that they do? We attempt to automate the process of deciding whether a review comment detects a problem. We use over 18,000 review comments that were labeled by the reviewees as either detecting or not detecting a problem with the work. We deploy several traditional machine-learning models, as well as neural-network models using GloVe and BERT embeddings. We find that the best performer is the Hierarchical Attention Network classifier, followed by the Bidirectional Gated Recurrent Units (GRU) Attention and Capsule model with scores of 93.1% and 90.5% respectively. The best non-neural network model was the support vector machine with a score of 89.71%. This is followed by the Stochastic Gradient Descent model and the Logistic Regression model with 89.70% and 88.98%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑