Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalize well and are robust to changes in the input distribution. While methods that harness spatial and temporal structures find broad application, recent work has demonstrated the potential of models that leverage sparse and modular structure using an ensemble of sparingly interacting modules. In this work, we take a step towards dynamic models that are capable of simultaneously exploiting both modular and spatiotemporal structures. We accomplish this by abstracting the modeled dynamical system as a collection of autonomous but sparsely interacting sub-systems. The sub-systems interact according to a topology that is learned, but also informed by the spatial structure of the underlying real-world system. This results in a class of models that are well suited for modeling the dynamics of systems that only offer local views into their state, along with corresponding spatial locations of those views. On the tasks of video prediction from cropped frames and multi-agent world modeling from partial observations in the challenging Starcraft2 domain, we find our models to be more robust to the number of available views and better capable of generalization to novel tasks without additional training, even when compared against strong baselines that perform equally well or better on the training distribution.
Efficient Recurrent Neural Networks using Structured Matrices in FPGAs
Department of Electrical Engineering and Computer Science, Syracuse University, USA（Syracuse大学电气工程与计算机科学系）；Center for Energy-efficient Computing and Applications (CECA), Peking University, China（北京大学能源高效计算与应用中心）
Reservoir Computing meets Recurrent Kernels and Structured Transforms
Ecole Normale Supérieure（巴黎高等师范学院）；Université PSL（巴黎文理研究大学）；CNRS（法国国家科学研究中心）；Sorbonne Université（索邦大学）；Université de Paris（巴黎大学）；Collège de France（法兰西公学院）；LightOn（LightOn公司）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation
University of Edinburgh（爱丁堡大学）；Intuitive Robots Lab, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院直觉机器人实验室）；NVIDIA（英伟达）；Robotics Institute Germany (RIG)（德国机器人研究所）
Learning to generalize to new compositions in image understanding
Bar Ilan University（巴伊兰大学）；Google Research（谷歌研究院）；Tel Aviv University（特拉维夫大学）
最近，递归神经网络已经被用于学习使用自然语言来描述图像。然而，已经观察到，这些模型对在训练期间未观察到的场景的概括性较差，可能过于强烈地依赖于训练数据中的文本的统计。在这里，我们建议使用简短的结构化表示来描述图像，旨在捕捉描述的关键。这些结构化表示允许我们分别梳理和评估两种类型的泛化：对具有相似场景的新图像的标准泛化，以及对已知实体的新组合的泛化。我们在MS-COCO数据集上比较了两种学习方法：基于LSTM（Show，Attend and Tell）的最先进的递归网络，以及基于深度网络的简单结构化预测模型。我们发现，结构化模型对新组合的泛化能力远远好于LSTM，是预测结构化表示的精度的7倍。通过提供一个具体的方法来量化概括看不见的组合，我们认为，结构化的表示和成分分裂是一个有用的基准图像描述，并主张组成模型，捕捉语言和视觉结构。
