Refactoring Policy for Compositional Generalizability using Self-Supervised Object Proposals

University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Shanghai Jiao Tong University（上海交通大学）

Tongzhou Mu, Jiayuan Gu, Zhiwei Jia, Hao Tang, Hao Su

备注：: 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We study how to learn a policy with compositional generalizability. We propose a two-stage framework, which refactorizes a high-reward teacher policy into a generalizable student policy with strong inductive bias. Particularly, we implement an object-centric GNN-based student policy, whose input objects are learned from images through self-supervised learning. Empirically, we evaluate our approach on four difficult tasks that require compositional generalizability, and achieve superior performance compared to baselines.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning

Zhejiang University（浙江大学）；University of Technology Sydney（悉尼科技大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）

Temporal grounding in videos aims to localize one target video segment that semantically corresponds to a given query sentence. Thanks to the semantic diversity of natural language descriptions, temporal grounding allows activity grounding beyond pre-defined classes and has received increasing attention in recent years. The semantic diversity is rooted in the principle of compositionality in linguistics, where novel semantics can be systematically described by combining known words in novel ways (compositional generalization). However, current temporal grounding datasets do not specifically te…

2022-03-28cs.CV

Meta Module Network for Compositional Visual Reasoning

University of California, Santa Barbara（加州大学圣巴巴拉分校）；Microsoft Dynamics 365 AI Research（微软 Dynamics 365 AI 研究院）

Neural Module Network (NMN) exhibits strong interpretability and compositionality thanks to its handcrafted neural modules with explicit multi-hop reasoning capability. However, most NMNs suffer from two critical drawbacks: 1) scalability: customized module for specific function renders it impractical when scaling up to a larger set of functions in complex tasks; 2) generalizability: rigid pre-defined module inventory makes it difficult to generalize to unseen functions in new tasks/domains. To design a more powerful NMN architecture for practical use, we propose Meta Module Network (MMN) cent…

2020-11-08cs.CVcs.LG

Separate-and-Enhance：面向文本生成图像扩散模型的组合式微调

Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models

CMU（卡内基梅隆大学）；Adobe Research（奥多比研究院）；UIUC（伊利诺伊大学厄巴纳-香槟分校）

针对文本生成图像扩散模型多目标组合生成与提示对齐不佳的问题，提出结合Separate与Enhance损失的微调方法，提升了图像真实感、对齐度与适应性。

2024-01-31cs.AIcs.CV

CompoDistill：多模态大语言模型中基于注意力的知识蒸馏用于组合推理

CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs

KAIST（韩国科学技术院）

本文提出CompoDistill，通过显式对齐学生与教师模型的视觉注意力，提升多模态大语言模型的组合推理能力，同时保持视觉问答任务的性能。

2026-04-09cs.AIcs.CV

CCSD：跨模态组合自蒸馏用于缺失模态的鲁棒脑肿瘤分割

CCSD: Cross-Modal Compositional Self-Distillation for Robust Brain Tumor Segmentation with Missing Modalities

Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（上海智能自主系统研究院，同济大学）；College of Biomedical Engineering, Fudan University（生物医学工程学院，复旦大学）；School of Medicine, Tongji University（医学院，同济大学）；College of Information Engineering, China Jiliang University（信息工程学院，中国嘉兴大学）

CCSD通过跨模态组合自蒸馏方法，提升在缺失模态下的脑肿瘤分割鲁棒性与泛化能力。

2026-03-05cs.AIcs.CV

Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoning

University of Minnesota（明尼苏达大学）

Humans have the innate capability to answer diverse questions, which is rooted in the natural ability to correlate different concepts based on their semantic relationships and decompose difficult problems into sub-tasks. On the contrary, existing visual reasoning methods assume training samples that capture every possible object and reasoning problem, and rely on black-boxed models that commonly exploit statistical priors. They have yet to develop the capability to address novel objects or spurious biases in real-world scenarios, and also fall short of interpreting the rationales behind their …

2023-03-18cs.CV