arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.13074cs.CLcs.AI
收藏

走向可教推理系统:使用用户反馈的动态记忆来持续改进系统

Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvement

Allen Institute for AI(艾伦人工智能研究所)

Bhavana Dalvi Mishra, Oyvind Tafjord, Peter Clark

摘要:

我们的目标是一个可教学的问答推理系统(QA),用户可以与忠实的答案解释进行交互,并纠正其错误,以便系统随着时间的推移而改进。我们的方法是增加一个QA模型与用户反馈的动态记忆,包含用户提供的纠正错误的模型信念,用户在互动过程中识别。从记忆中提取的信息被用作QA的额外上下文,以帮助避免在类似的新情况下出现以前的错误-这是基于记忆的连续学习的一种新应用。通过模拟反馈,我们发现我们的系统(称为TeachMe)随着时间的推移不断改进,并且无需模型再训练,仅需要25%的训练样本的反馈即可达到上限(所有样本的反馈)的1%以内。同样,在对真实用户的实验中,我们观察到了类似的趋势,教学后隐藏测试集的性能提高了15%以上。这为在交互式环境中使用冻结的语言模型提供了新的机会,用户可以检查、调试和纠正模型的信念,从而随着时间的推移提高系统的性能。

英文摘要:

Our goal is a teachable reasoning system for question-answering (QA), where a user can interact with faithful answer explanations, and correct its errors so that the system improves over time. Our approach is to augment a QA model with a dynamic memory of user feedback, containing user-supplied corrections to erroneous model beliefs that users identify during interaction. Retrievals from memory are used as additional context for QA, to help avoid previous mistakes in similar new situations - a novel application of memory-based continuous learning. With simulated feedback, we find that our system (called TeachMe) continually improves with time, and without model retraining, requiring feedback on only 25% of training examples to reach within 1% of the upper-bound (feedback on all examples). Similarly, in experiments with real users, we observe a similar trend, with performance improving by over 15% on a hidden test set after teaching. This suggests new opportunities for using frozen language models in an interactive setting where users can inspect, debug, and correct the model's beliefs, leading to improved system's performance over time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑