可再现性、可复制性及其他：评估基于方面的情感分析的实际生产准备情况
Reproducibility, Replicability and Beyond: Assessing Production Readiness of Aspect Based Sentiment Analysis in the Wild
摘要：
随着在线市场和其中的用户生成内容的指数增长，基于方面的情感分析变得比以往任何时候都更加重要。在这项工作中，我们批判性地审查了在过去六年中发布的模型的代表性样本，通过从业者的镜头，着眼于生产部署。首先，我们严格的经验评估显示重现性差：整个样本的测试准确度平均下降4-5%。其次，为了进一步增强我们对实证评估的信心，我们报告了两个具有挑战性的数据切片的实验，并观察到准确性持续下降12-55%。第三，我们研究了跨域转移的可能性，并观察到，当与同一区域内其他域的数据集结合使用时，只有10-25%的特定于域的训练数据集，在很大程度上缩小了完全跨域和完全域内预测性能之间的差距。最后，我们从印度的一个大型电子商务门户网站开源了两个大型注释评论语料库，以帮助研究可复制性和转移，希望它将推动该领域的进一步发展。
英文摘要：
With the exponential growth of online marketplaces and user-generated content therein, aspect-based sentiment analysis has become more important than ever. In this work, we critically review a representative sample of the models published during the past six years through the lens of a practitioner, with an eye towards deployment in production. First, our rigorous empirical evaluation reveals poor reproducibility: an average 4-5% drop in test accuracy across the sample. Second, to further bolster our confidence in empirical evaluation, we report experiments on two challenging data slices, and observe a consistent 12-55% drop in accuracy. Third, we study the possibility of transfer across domains and observe that as little as 10-25% of the domain-specific training dataset, when used in conjunction with datasets from other domains within the same locale, largely closes the gap between complete cross-domain and complete in-domain predictive performance. Lastly, we open-source two large-scale annotated review corpora from a large e-commerce portal in India in order to aid the study of replicability and transfer, with the hope that it will fuel further growth of the field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
