We present a neural-symbolic framework for observing the environment and continuously learning visual semantics and intuitive physics to reproduce them in an interactive simulation. The framework consists of five parts, a neural-symbolic hybrid network based on capsules for inverse graphics, an episodic memory to store observations, an interaction network for intuitive physics, a meta-learning agent that continuously improves the framework and a querying language that acts as the framework's interface for simulation. By means of lifelong meta-learning, the capsule network is expanded and trained continuously, in order to better adapt to its environment with each iteration. This enables it to learn new semantics using a few-shot approach and with minimal input from an oracle over its lifetime. From what it learned through observation, the part for intuitive physics infers all the required physical properties of the objects in a scene, enabling predictions. Finally, a custom query language ties all parts together, which allows to perform various mental simulation tasks, such as navigation, sorting and simulation of a game environment, with which we illustrate the potential of our novel approach.
Efficient Rotation Invariance in Deep Neural Networks through Artificial Mental Rotation
ZHAW Centre for Artificial Intelligence(苏黎世应用科学大学人工智能中心);University of Lugano(卢加诺大学);European Centre for Living Technology(欧洲生命技术中心);The Swiss AI Lab IDSIA(瑞士人工智能实验室 IDSIA);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学)
Explainable and Interpretable Diabetic Retinopathy Classification Based on Neural-Symbolic Learning
Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Singapore Eye Research Institute(新加坡眼科研究所);Duke-NUS Medical School(杜克-新加坡国立大学医学院);Institute for Molecular and Clinical Ophthalmology(分子与临床眼科研究所);National University of Singapore(新加坡国立大学)