arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

Think-Verify-Revise:结合视觉语言模型与动态逻辑张量网络的神经符号视觉推理

Think-Verify-Revise: Neuro-Symbolic Visual Reasoning with Vision-Language Models and Dynamic Logic Tensor Networks

Homayoun Afshari, Pietro Basci, Alessandro Russo, Lia Morra

arXiv 2609.05388首次发表:更新:

发表机构

Politecnico di Torino(都灵理工大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本文提出Think-Verify-Revise神经符号框架,结合VLM与D-LTN形成闭环迭代反馈,在ViSudo-PC基准的四个视觉领域上仅用3个示例归纳数独规则,AUC分数优于或匹配现有方法,实现自动规则发现。

AI 中文摘要

视觉推理任务要求系统同时感知视觉内容并应用形式化关系约束,这一组合是纯神经方法或纯符号方法单独处理时均难以胜任的。本文提出一种神经符号(NeSy)框架,通过将用于自动一阶逻辑(FOL)规则归纳的视觉语言模型(VLM)与用于可微分规则验证的动态逻辑张量网络(D-LTN)紧密耦合,形成闭环迭代反馈回路,填补上述空白。VLM接收少量带标签的视觉示例,提出符合严格语法的候选FOL规则(思考阶段);D-LTN在运行时由这些规则自动组装,并基于CNN生成的视觉嵌入对规则进行评估(验证阶段);验证失败信息被反馈回VLM,以指导其提出下一个假设(修正阶段)。在ViSudo-PC基准的四个视觉领域(MNIST、EMNIST、KMNIST、FMNIST)上进行评估,该系统仅使用3个训练示例作为视觉上下文,就能归纳出有效的数独约束规则。所提方法的AUC分数与现有方法(NeuPSL、LTN)相当或更优,展现出通过VLM自动发现规则的潜力。代码可在this https URL获取。

英文摘要

Visual reasoning tasks require a system to jointly perceive visual content and apply formal relational constraints---a combination that neither pure neural nor purely symbolic approaches handle well in isolation. This paper proposes a Neuro-Symbolic (NeSy) framework that closes this gap by tightly coupling a Vision-Language Model (VLM) for automatic First-Order Logic (FOL) rule induction with a Dynamic Logic Tensor Network (D-LTN) for differentiable rule verification, in a closed iterative feedback loop. The VLM receives a small set of labelled visual examples and proposes candidate FOL rules conforming to a strict grammar (Think); the D-LTN is automatically assembled from these rules at runtime and evaluates them grounding on CNN-produced visual embeddings (Verify); and verification failures are fed back to guide the VLM's next hypothesis (Revise). Evaluated on the ViSudo-PC benchmark across four visual domains (MNIST, EMNIST, KMNIST, FMNIST), the system induces valid Sudoku constraint rules using only three training examples as visual context. The proposed method achieves AUC scores matching or outperforming previous methods (NeuPSL, LTN), showing the potential for automatic rule discovery through VLM. Code is available at https://github.com/homayoun-afshari/nesy.

CommentsAccepted at the MARS2 Workshop @ ECCV 2026

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑