多模态延续式人机交互体系结构
Multimodal Continuation-style Architectures for Human-Robot Interaction
摘要：
我们提出了一个架构，用于集成实时，多模态输入到一个计算代理的上下文模型。在虚拟世界中使用人与化身的交互，我们将对齐的手势和语音视为一个整体，其中内容可以通过任何一种方式进行通信。利用修改的非确定性下推自动机架构，计算机系统：（1）使用连续传递样式递增地消耗输入，直到其实现对用户目标的充分理解;（2）在必要时使用已建立的上下文信息构造并询问问题;以及（3）使用多模态线索保持对先前话语项的跟踪。这种类型的架构支持下推和有限状态自动机的特殊情况，以及集成来自机器学习模型的输出。我们提出的例子，这种架构的使用在多模态一次性学习互动的新的手势和现场动作组成。
英文摘要：
We present an architecture for integrating real-time, multimodal input into a computational agent's contextual model. Using a human-avatar interaction in a virtual world, we treat aligned gesture and speech as an ensemble where content may be communicated by either modality. With a modified nondeterministic pushdown automaton architecture, the computer system: (1) consumes input incrementally using continuation-passing style until it achieves sufficient understanding the user's aim; (2) constructs and asks questions where necessary using established contextual information; and (3) maintains track of prior discourse items using multimodal cues. This type of architecture supports special cases of pushdown and finite state automata as well as integrating outputs from machine learning models. We present examples of this architecture's use in multimodal one-shot learning interactions of novel gestures and live action composition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
