POSSCORE：一种简单有效的词性标注会话搜索评估方法
POSSCORE: A Simple Yet Effective Evaluation of Conversational Search with Part of Speech Labelling
摘要：
诸如Google Assistant和Microsoft Cortana之类的对话搜索系统提供了一种新的搜索范例，其中允许用户经由自然语言对话与搜索系统进行通信。由于搜索结果是以自然语言句子的格式呈现的，因此评估这样的系统是非常具有挑战性的。鉴于可能的答复数量无限，收集所有可能答复的相关性评估是不可行的。在本文中，我们提出了POSSCORE，一个简单而有效的对话搜索自动评估方法。建议的嵌入为基础的度量考虑到响应中的术语的词性（POS）的影响。据最好的知识，我们的工作是第一次系统地展示了纳入句法信息，如POS标签，会话搜索评估的重要性。实验结果表明，我们的指标可以与人类的喜好，实现了显着的改进，国家的最先进的基线指标。
英文摘要：
Conversational search systems, such as Google Assistant and Microsoft Cortana, provide a new search paradigm where users are allowed, via natural language dialogues, to communicate with search systems. Evaluating such systems is very challenging since search results are presented in the format of natural language sentences. Given the unlimited number of possible responses, collecting relevance assessments for all the possible responses is infeasible. In this paper, we propose POSSCORE, a simple yet effective automatic evaluation method for conversational search. The proposed embedding-based metric takes the influence of part of speech (POS) of the terms in the response into account. To the best knowledge, our work is the first to systematically demonstrate the importance of incorporating syntactic information, such as POS labels, for conversational search evaluation. Experimental results demonstrate that our metrics can correlate with human preference, achieving significant improvements over state-of-the-art baseline metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
