arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.03215cs.AI
收藏

COSINT-Agent: 一种面向中文开源情报的知识驱动多模态智能体

COSINT-Agent: A Knowledge-Driven Multimodal Agent for Chinese Open Source Intelligence

Wentao Li, Congcong Wang, Xiaoxiao Cui, Zhi Liu, Wei Guo, Lizhen Cui

AI总结:

COSINT-Agent通过整合多模态大语言模型与实体-事件-场景知识图谱,提升中文开源情报的多模态推理能力与情报生成效率。

摘要:

开源情报(OSINT)需要整合和推理多种多模态数据,这在从非结构化数据源中推导出可操作见解方面带来了显著挑战。传统方法,包括多模态大语言模型(MLLMs),往往难以推断复杂的上下文关系或从非结构化数据源中提供全面的情报。在本文中,我们介绍了COSINT-Agent,这是一种针对中文领域OSINT挑战而设计的知识驱动多模态智能体。COSINT-Agent无缝整合了微调后的MLLMs的感知能力与实体-事件-场景知识图谱(EES-KG)的结构推理能力。COSINT-Agent的核心是创新的EES-Match框架,该框架连接COSINT-MLLM和EES-KG,使多模态洞察的系统提取、推理和上下文化成为可能。这种整合促进了精确实体识别、事件解释和上下文检索,有效地将原始多模态数据转化为可操作的情报。广泛的实验验证了COSINT-Agent在核心OSINT任务上的优越性能,包括实体识别、EES生成和上下文匹配。这些结果突显了其作为稳健且可扩展解决方案的潜力,以推动自动化多模态推理,并增强OSINT方法的效率。

英文摘要:

Open Source Intelligence (OSINT) requires the integration and reasoning of diverse multimodal data, presenting significant challenges in deriving actionable insights. Traditional approaches, including multimodal large language models (MLLMs), often struggle to infer complex contextual relationships or deliver comprehensive intelligence from unstructured data sources. In this paper, we introduce COSINT-Agent, a knowledge-driven multimodal agent tailored to address the challenges of OSINT in the Chinese domain. COSINT-Agent seamlessly integrates the perceptual capabilities of fine-tuned MLLMs with the structured reasoning power of the Entity-Event-Scene Knowledge Graph (EES-KG). Central to COSINT-Agent is the innovative EES-Match framework, which bridges COSINT-MLLM and EES-KG, enabling systematic extraction, reasoning, and contextualization of multimodal insights. This integration facilitates precise entity recognition, event interpretation, and context retrieval, effectively transforming raw multimodal data into actionable intelligence. Extensive experiments validate the superior performance of COSINT-Agent across core OSINT tasks, including entity recognition, EES generation, and context matching. These results underscore its potential as a robust and scalable solution for advancing automated multimodal reasoning and enhancing the effectiveness of OSINT methodologies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑