arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.09402cs.HC
收藏

利用多模态数据检测日常生活事件

Detecting Events of Daily Living Using Multimodal Data

Hyungik Oh, Ramesh Jain

摘要:

事件是理解人们如何体验生活的基础。然而,自动记录日常生活中的所有事件具有挑战性。对多媒体信号的理解允许识别日常生活事件并尽可能自动地获取其属性。在本文中,我们考虑的问题,通过采用常用的多媒体数据从智能手机和可穿戴设备获得的日常事件的识别。我们开发了一种不引人注目的方法来获得潜在的语义信息的数据,因此,日常事件识别的语义上下文丰富的基础上的方法。我们通过一个事件知识图谱来表示丰富过程,该知识图谱从语义上丰富了日常事件。为了展示这种丰富的具体例子,我们使用三个用户14个月的数据进行了一个关于进食活动的实验,这可能是最复杂的事件之一。在这个过程中,为了不引人注目地补充语义信息的缺乏,我们提出了一种新的食物识别/分类方法,只关注对食物消费的身体反应。实验结果表明,我们的方法是能够自动抽象的生活经验。然后,这些日常事件可以用于创建个人模型,该模型可以捕捉一个人在特定条件下对不同刺激的反应。

英文摘要:

Events are fundamental for understanding how people experience their lives. It is challenging, however, to automatically record all events in daily life. An understanding of multimedia signals allows recognizing events of daily living and getting their attributes as automatically as possible. In this paper, we consider the problem of recognizing a daily event by employing the commonly used multimedia data obtained from a smartphone and wearable device. We develop an unobtrusive approach to obtain latent semantic information from the data, and therefore an approach for daily event recognition based on semantic context enrichment. We represent the enrichment process through an event knowledge graph that semantically enriches a daily event from a low-level daily activity. To show a concrete example of this enrichment, we perform an experiment with eating activity, which may be one of the most complex events, by using 14 months of data for three users. In this process, to unobtrusively complement the lack of semantic information, we suggest a new food recognition/classification method that focuses only on a physical response to food consumption. Experimental results indicate that our approach is able to show automatic abstraction of life experience. These daily events can then be used to create a personal model that can capture how a person reacts to different stimuli under specific conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑