用于自动生成移动应用用户反馈答案的预训练神经语言模型
Pre-Trained Neural Language Models for Automatic Mobile App User Feedback Answer Generation
摘要:
研究表明,开发人员对移动应用程序用户在应用程序商店的反馈的回答可以提高应用程序的星级。为了帮助应用程序开发人员生成与用户问题相关的答案,最近的研究开发了自动生成答案的模型。目的:应用响应生成模型使用深度神经网络,需要训练数据。用于自然语言处理(NLP)的预训练神经语言模型(PTM)以无监督的方式利用从大型语料库中学习到的信息,并且可以减少所需的训练数据的量。在本文中,我们评估PTM来生成对移动应用程序用户反馈的回复。方法:我们从头开始训练一个Transformer模型,并微调两个PTM来评估生成的响应,并将其与当前的应用响应模型RRGEN进行比较。我们还评估了模型的不同部分的训练数据。结果:通过自动度量评估的大型数据集上的结果显示,PTM获得的分数低于基线。然而,我们的人工评估证实,PTM可以对发布的反馈产生更相关和更有意义的响应。当训练数据减少到1/3时,PTM的性能下降较小。结论:PTM在生成对应用程序评论的响应时很有用,并且对于所提供的训练数据量来说是更鲁棒的模型。然而,预测时间是RRGEN的19倍。这项研究可以提供新的途径,研究在适应PTM分析移动应用程序用户反馈。Index Terms-移动应用用户反馈分析、神经预训练语言模型、自动答案生成
英文摘要:
Studies show that developers' answers to the mobile app users' feedbacks on app stores can increase the apps' star rating. To help app developers generate answers that are related to the users' issues, recent studies develop models to generate the answers automatically. Aims: The app response generation models use deep neural networks and require training data. Pre-Trained neural language Models (PTM) used in Natural Language Processing (NLP) take advantage of the information they learned from a large corpora in an unsupervised manner, and can reduce the amount of required training data. In this paper, we evaluate PTMs to generate replies to the mobile app user feedbacks. Method: We train a Transformer model from scratch and fine-tune two PTMs to evaluate the generated responses, which are compared to RRGEN, a current app response model. We also evaluate the models with different portions of the training data. Results: The results on a large dataset evaluated by automatic metrics show that PTMs obtain lower scores than the baselines. However, our human evaluation confirms that PTMs can generate more relevant and meaningful responses to the posted feedbacks. Moreover, the performance of PTMs has less drop compared to other models when the amount of training data is reduced to 1/3. Conclusion: PTMs are useful in generating responses to app reviews and are more robust models to the amount of training data provided. However, the prediction time is 19X than RRGEN. This study can provide new avenues for research in adapting the PTMs for analyzing mobile app user feedbacks. Index Terms-mobile app user feedback analysis, neural pre-trained language models, automatic answer generation
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。