重新审视 KRISP：一种轻量级的知识增强视觉-语言模型的再现与分析
Revisiting KRISP: A Lightweight Reproduction and Analysis of Knowledge-Enhanced Vision-Language Models
AI总结：
本研究重新审视 KRISP，通过轻量级模型揭示知识增强 VQA 的设计缺陷与资源受限下的有效性。
摘要：
Facebook AI Research 引入了 KRISP [4]，该模型将结构化的外部知识整合到视觉-语言推理的流程中。尽管其效果显著，但原始模型是为工业级训练开发的，计算成本高，并且紧密连接到一个大型骨干网络。在本工作中，我们从不同的角度重新审视 KRISP，并提供了一个参数显著更少的轻量级再现。尽管我们的复现模型性能约为原始模型的 75%，但复现过程揭示了若干设计缺陷、现实中的陷阱以及未在原始论文中完全覆盖的隐含问题。我们通过系统性的消融研究，提供了关于在资源受限条件下知识增强的 VQA 架构的可扩展性和有效性洞察，包括在合成 VQA 数据上的概念验证以及在 DAQUAR 数据集上的评估。我们的模型配置为低参数设置，并受外部知识图谱领域的限制，防止了人工智能幻觉，并生成仅限于该领域的输出。少量参数使我们能够在智能手机和 AR-VR 等边缘设备上运行，进一步提高了离线视觉推理。
英文摘要：
Facebook AI Research introduced KRISP [4], which integrates structured external knowledge into pipelines for vision-language reasoning. Despite its effectiveness, the original model has been developed for industrial-scale training, is computationally demanding, and is tightly connected to a large backbone. In this work, we reexamine KRISP from a different angle and offer a lightweight reproduction with significantly fewer parameters. Even though our replicated model performs about 75 % of the original, the replication process uncovers a number of design flaws, real-world pitfalls, and implicit problems that were not fully covered in the original paper. We offer insights into the scalability and efficacy of knowledge-enhanced VQA architectures under resource constraints through systematic ablation studies, which include a proof-of-concept on synthetic VQA data and evaluation on the DAQUAR dataset. Our model, configured with a low parameter setup and constrained by the external Knowledge graph domain, prevents AI hallucinations and generates outputs solely within that domain. Minimal parameters allow us to function on edge devices like smartphones and AR-VR, further improving offline visual reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
