Compositional generalization is a troubling blind spot for neural language models. Recent efforts have presented techniques for improving a model's ability to encode novel combinations of known inputs, but less work has focused on generating novel combinations of known outputs. Here we focus on this latter "decode-side" form of generalization in the context of gSCAN, a synthetic benchmark for compositional generalization in grounded language understanding. We present Recursive Decoding (RD), a novel procedure for training and using seq2seq models, targeted towards decode-side generalization. Rather than generating an entire output sequence in one pass, models are trained to predict one token at a time. Inputs (i.e., the external gSCAN environment) are then incrementally updated based on predicted tokens, and re-encoded for the next decoder time step. RD thus decomposes a complex, out-of-distribution sequence generation task into a series of incremental predictions that each resemble what the model has already seen during training. RD yields dramatic improvement on two previously neglected generalization tasks in gSCAN. We provide analyses to elucidate these gains over failure of a baseline, and then discuss implications for generalization in naturalistic grounded language understanding, and seq2seq more generally.
MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
AI Singapore, National University of Singapore（新加坡国立大学新加坡人工智能）；School of Computing, National University of Singapore（新加坡国立大学计算学院）；Institute of Advanced Intelligence and Computing, A*STAR（新加坡科技研究局先进智能与计算研究所）
A Benchmark for Systematic Generalization in Grounded Language Understanding
University of Amsterdam（阿姆斯特丹大学）；Massachusetts Institute of Technology（麻省理工学院）；ICREA（加泰罗尼亚高等研究院）；Facebook Artificial Intelligence Research（脸书人工智能研究院）；Facebook AI Research（脸书人工智能研究院）；New York University（纽约大学）
Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Shanghai Key Laboratory of Intelligent Information Processing, Fudan University（复旦大学上海智能信息处理重点实验室）
The Compositional Architecture of Regret in Large Language Models
Provable Responsible AI and Data Analytics (PRADA) Lab（可证明负责任的人工智能与数据分析实验室）；King Abdullah University of Science and Technology（卡瓦尔大学科学与技术大学）；State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University（认知神经科学与学习国家重点实验室，北京师范大学）；University of Exeter（埃克塞特大学）；The Hong Kong Polytechnic University（香港理工大学）
