基于时间深度网络的人类社会交互建模
Human Social Interaction Modeling Using Temporal Deep Networks
摘要:
我们提出了一种新的方法来计算建模的基础上建模的基本社会互动谓词(ESIP),如联合注意和夹带的社会互动。基于良好的社会心理学理论和方法,我们收集了一个新的“塔游戏”数据集,包括视听捕获的二元互动与ESIP标记。我们希望这个数据集为计算社会交互建模的研究提供一个新的途径。我们提出了一种新的联合判别条件限制玻尔兹曼机(DCRBM)模型,该模型将判别组件与CRBM的生成能力相结合。这样的组合使我们能够通过两个步骤揭示ESIP的可操作成分。首先,我们在标记数据上训练DCRBM模型,并获得谓词的准确检测(在各种ESIP中为76%-49%)。其次,我们利用DCRBM的生成能力来激活训练模型,以便生成与实际训练数据紧密匹配的特定ESIP对应的较低级别数据(生成100帧的均方误差为0.01-0.1)。因此,我们能够将ESIP分解为其组成的可操作行为。如此纯粹的计算决定如何建立一个ESIP,如接触,是前所未有的。
英文摘要:
We present a novel approach to computational modeling of social interactions based on modeling of essential social interaction predicates (ESIPs) such as joint attention and entrainment. Based on sound social psychological theory and methodology, we collect a new "Tower Game" dataset consisting of audio-visual capture of dyadic interactions labeled with the ESIPs. We expect this dataset to provide a new avenue for research in computational social interaction modeling. We propose a novel joint Discriminative Conditional Restricted Boltzmann Machine (DCRBM) model that combines a discriminative component with the generative power of CRBMs. Such a combination enables us to uncover actionable constituents of the ESIPs in two steps. First, we train the DCRBM model on the labeled data and get accurate (76\%-49\% across various ESIPs) detection of the predicates. Second, we exploit the generative capability of DCRBMs to activate the trained model so as to generate the lower-level data corresponding to the specific ESIP that closely matches the actual training data (with mean square error 0.01-0.1 for generating 100 frames). We are thus able to decompose the ESIPs into their constituent actionable behaviors. Such a purely computational determination of how to establish an ESIP such as engagement is unprecedented.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。