AI 中文总结
针对多视图VQA任务，提出ODSC本体驱动语义通信框架，通过共享本体知识库优化场景图，可减少SI数据量、提升问答准确率及多视图VQA准确率。
AI 中文摘要
本文针对多视图视觉问答（VQA）任务，提出了一种本体驱动的语义通信（ODSC）框架。在该框架中，多个发射端观测同一场景，通过视觉语言模型（VLMs）提取语义信息（SI），并将场景图传输至接收端。由于VLMs的完整性、异质性和不可解释性，提取的场景图存在冗余、模糊和不一致的问题。为解决这些问题，发射端与接收端共享一个基于本体的知识库，该库预先定义了同义词、推理规则和一致性约束。对于每个发射端，所提ODSC框架会移除可基于推理规则推断出的部分场景图；对于接收端，该框架会基于同义词对齐不同视图的SI，并基于约束检测视图间的不一致性。本文定义了多视图VQA准确率（MVA）指标来评估所提框架。仿真结果表明，与传输完整场景图相比，所提框架可将SI的数据量最多减少87.1%，同时将问答准确率提升4.5%；此外，与SI过滤方法相比，所提框架在MVA指标上最多提升16.0%。
英文摘要
In this paper, an ontology-driven semantic communication (ODSC) framework is proposed for multi-view visual question answering (VQA) tasks. In the considered framework, multiple transmitters observe a scene, extract the semantic information (SI) with vision-language models (VLMs), and transmit the scene graphs to a receiver. Due to the completeness, heterogeneity, and uninterpretability of the VLMs, the extracted scene graphs are redundant, ambiguous, and inconsistent. To solve these problems, the transmitters and the receiver share an ontology-based knowledge base that predefines synonyms, inference rules, and consistency constraints. For each transmitter, the proposed ODSC framework removes the partial scene graph that can be inferred based on the inference rules. For the receiver, the proposed framework aligns the SI of different views based on the synonyms and detects the inconsistency among the views based on the constraints. A metric of multi-view VQA accuracy (MVA) is defined to evaluate the proposed framework. Simulation results show that, compared with transmitting the complete scene graphs, the proposed framework reduces the data size of the SI by up to 87.1% while improving the answering accuracy by 4.5%. Moreover, the proposed framework yields up to a 16.0% improvement in terms of the MVA compared with the SI filtering approaches.
CommentsThe manuscript was submitted to arXiv without obtaining prior authorization from all listed co-authors. In particular, at least one listed co-author had not consented to the public posting of the manuscript on arXiv before submission. We therefore request withdrawal of the manuscript due to this authorship and submission authorization issue
