Systematic Generalization refers to a learning algorithm's ability to extrapolate learned behavior to unseen situations that are distinct but semantically similar to its training data. As shown in recent work, state-of-the-art deep learning models fail dramatically even on tasks for which they are designed when the test set is systematically different from the training data. We hypothesize that explicitly modeling the relations between objects in their contexts while learning their representations will help achieve systematic generalization. Therefore, we propose a novel method that learns objects' contextualized embeddings with dynamic message passing conditioned on the input natural language and end-to-end trainable with other downstream deep learning modules. To our knowledge, this model is the first one that significantly outperforms the provided baseline and reaches state-of-the-art performance on grounded-SCAN (gSCAN), a grounded natural language navigation dataset designed to require systematic generalization in its test splits.
University of Amsterdam（阿姆斯特丹大学）；Massachusetts Institute of Technology（麻省理工学院）；ICREA（加泰罗尼亚高等研究院）；Facebook Artificial Intelligence Research（脸书人工智能研究院）；Facebook AI Research（脸书人工智能研究院）；New York University（纽约大学）
Humans easily interpret expressions that describe unfamiliar situations composed from familiar parts ("greet the pink brontosaurus by the ferris wheel"). Modern neural networks, by contrast, struggle to interpret novel compositions. In this paper, we introduce a new benchmark, gSCAN, for evaluating compositional generalization in situated language understanding. Going beyond a related benchmark that focused on syntactic aspects of generalization, gSCAN defines a language grounded in the states of a grid world, facilitating novel evaluations of acquiring linguistically motivated rules. For exam…
University College London（伦敦大学学院）；New York University（纽约大学）；Facebook AI Research（脸书人工智能研究院）
Systematic generalization is the ability to combine known parts into novel meaning; an important aspect of efficient human learning, but a weakness of neural network learning. In this work, we investigate how two well-known modeling principles -- modularity and data augmentation -- affect systematic generalization of neural networks in grounded language learning. We analyze how large the vocabulary needs to be to achieve systematic generalization and how similar the augmented data needs to be to the problem at hand. Our findings show that even in the controlled setting of a synthetic benchmark…
Humans are remarkably flexible when understanding new sentences that include combinations of concepts they have never encountered before. Recent work has shown that while deep networks can mimic some human language abilities when presented with novel sentences, systematic variation uncovers the limitations in the language-understanding abilities of networks. We demonstrate that these limitations can be overcome by addressing the generalization challenges in the gSCAN dataset, which explicitly measures how well an agent is able to interpret novel linguistic commands grounded in vision, e.g., no…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Alibaba Group（阿里巴巴集团）；Shanghai Jiao Tong University（上海交通大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；University of Alberta（阿尔伯塔大学）
Humans can systematically generalize to novel compositions of existing concepts. Recent studies argue that neural networks appear inherently ineffective in such cognitive capacity, leading to a pessimistic view and a lack of attention to optimistic results. We revisit this controversial topic from the perspective of meaningful learning, an exceptional capability of humans to learn novel concepts by connecting them with known ones. We reassess the compositional skills of sequence-to-sequence models conditioned on the semantic links between new and old concepts. Our observations suggest that mod…
Polypersona: Persona-Grounded LLM for Synthetic Survey Responses
George Mason University, Virginia, USA（乔治·马歇尔大学）；New Jersey Institute of Technology (NJIT), New Jersey, USA（新泽西理工学院）；University of Southern Queensland, Queensland, Australia（南方昆士兰大学）；Edinburgh Napier University, Edinburgh, Scotland（爱丁堡纳皮尔大学）
Semantically Conditioned Diffusion Models for Cerebral DSA Synthesis
Department of Statistics, LMU Munich（统计学系）；Department of Biomedical Informatics (DBMI), Medical Faculty Mannheim, Heidelberg University（生物医学信息学系）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Clinic for Diagnostic and Interventional Neuroradiology, Medical Faculty Mannheim, Heidelberg University（诊断和介入神经放射科）；Department of Anesthesiology and Intensive Care Medicine, BG Trauma Center Tuebingen（麻醉和重症医学科）
Scaffold-Conditioned Preference Triplets for Controllable Molecular Optimization with Large Language Models
National Key Laboratory of Parallel and Distributed Processing（平行与分布式处理国家重点实验室）；College of Computer Science and Technology（计算机科学与技术学院）；National University of DefenseTechnology（国防科技大学）；DP Technology（DP科技）
University of Washington（华盛顿大学）；TCS Research（塔塔咨询服务公司研究院）
Systematic generalization aims to evaluate reasoning about novel combinations from known components, an intrinsic property of human cognition. In this work, we study systematic generalization of NNs in forecasting future time series of dependent variables in a dynamical system, conditioned on past time series of dependent variables, and past and future control variables. We focus on systematic generalization wherein the NN-based forecasting model should perform well on previously unseen combinations or regimes of control variables after being trained on a limited set of the possible regimes. F…
