arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2011.11400cs.AIcs.CLq-bio.NC
收藏

语言引导的机器动作

Language guided machine action

Alibaba-Group(阿里巴巴集团);University of Oxford(牛津大学)

Feng Qi

摘要:

本文构建了一个层次化的模块化网络,称为语言引导机器动作(LGMA),其模块模仿人类大脑皮层网络处理信息流,可以完成语言引导动作、意图分解和动作执行前的心理模拟等多个一般任务。LGMA包含3个主要系统:(1)初级感觉系统,包含视觉、语言和感觉运动的多模态感觉信息。(2)联想系统涉及到认知语言的理解和合成、感觉运动和语言之间的转换、语言和视觉之间的转换、以及将注意的视觉对象和手臂状态整合到未来空间动作的认知地图中的上顶叶。前辅助运动区(pre-SMA)可以将高级意图转化为顺序的原子动作,而SMA可以将这些原子动作、当前手臂和被关注对象状态整合为感觉运动向量,通过手臂的前运动和主运动对手臂施加相应的力矩,实现意图。高级执行系统包括前额叶皮层(PFC),前额叶皮层负责外显推理,引导基于语言的自主行为;

英文摘要:

Here we build a hierarchical modular network called Language guided machine action (LGMA), whose modules process information stream mimicking human cortical network that allows to achieve multiple general tasks such as language guided action, intention decomposition and mental simulation before action execution etc. LGMA contains 3 main systems: (1) primary sensory system that multimodal sensory information of vision, language and sensorimotor. (2) association system involves and Broca modules to comprehend and synthesize language, BA14/40 module to translate between sensorimotor and language, midTemporal module to convert between language and vision, and superior parietal lobe to integrate attended visual object and arm state into cognitive map for future spatial actions. Pre-supplementary motor area (pre-SMA) can converts high level intention into sequential atomic actions, while SMA can integrate these atomic actions, current arm and attended object state into sensorimotor vector to apply corresponding torques on arm via pre-motor and primary motor of arm to achieve the intention. The high-level executive system contains PFC that does explicit inference and guide voluntary action based on language, while BG is the habitual action control center.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑