arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1601.02543cs.CLcs.AIcs.HC
收藏

基于语音识别系统的性能评估

Evaluating the Performance of a Speech Recognition based System

TCS Innovation Labs - Mumbai, Tata Consultancy Services(塔塔咨询服务公司TCS创新实验室-孟买)

Vinod Kumar Pandey, Sunil Kumar Kopparapu

摘要:

随着服务业的发展,基于语音的解决方案已经占据了中心舞台,服务业需要满足来自社会各阶层的大量人群的需求。虽然自然语言语音接口是研究界的热门话题,但在实践中,基于菜单的语音解决方案蓬勃发展。通常,在基于菜单的语音解决方案中,当系统提示时,用户需要通过从封闭的单词集合说话来响应。对IVR提示的一系列人类语音响应导致交易完成。如果语音解决方案能够在系统提示时正确识别用户的所有口头话语,则认为交易成功。评估语音解决方案性能的通常机制是对系统进行广泛的测试,将其投入实际使用,然后通过分析成功事务的日志来评估性能。这种评估可能会导致测试用户不满意,特别是如果系统的性能导致较差的事务完成率。为了否定这一点,在语音系统的评估过程中采用了绿野仙踪方法。总的来说,这类评估在时间和成本方面都是一个昂贵的提议。在本文中,我们提出了一种方法来评估性能的语音解决方案,而实际上并没有把它的人使用。我们首先描述的方法,然后通过实验表明,这可以用来识别性能瓶颈的语音解决方案,甚至在系统实际使用,从而节省评估时间和费用。

英文摘要:

Speech based solutions have taken center stage with growth in the services industry where there is a need to cater to a very large number of people from all strata of the society. While natural language speech interfaces are the talk in the research community, yet in practice, menu based speech solutions thrive. Typically in a menu based speech solution the user is required to respond by speaking from a closed set of words when prompted by the system. A sequence of human speech response to the IVR prompts results in the completion of a transaction. A transaction is deemed successful if the speech solution can correctly recognize all the spoken utterances of the user whenever prompted by the system. The usual mechanism to evaluate the performance of a speech solution is to do an extensive test of the system by putting it to actual people use and then evaluating the performance by analyzing the logs for successful transactions. This kind of evaluation could lead to dissatisfied test users especially if the performance of the system were to result in a poor transaction completion rate. To negate this the Wizard of Oz approach is adopted during evaluation of a speech system. Overall this kind of evaluations is an expensive proposition both in terms of time and cost. In this paper, we propose a method to evaluate the performance of a speech solution without actually putting it to people use. We first describe the methodology and then show experimentally that this can be used to identify the performance bottlenecks of the speech solution even before the system is actually used thus saving evaluation time and expenses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑