面向科技情报（STI）的高效多模态多语言观点抽取：一种基于QLoRA的微调方法

中文摘要 AI 辅助

大型语言模型（LLMs）的最新进展重塑了语义分析。科技情报（STI）的观点抽取（OE）需要从海量信息流中提炼出简洁的核心观点。现有模型难以过滤这些信息流中的噪声，且在零样本多语言和多模态场景下结构化输出的可靠性有限。为解决信息过载和抽取焦点分散问题，本研究提出一种多模态核心观点抽取框架，其中视觉证据作为文本判断的上下文锚点。以VideoLLaMA2（VL2）和VideoLLaMA2.1（VL2.1）为基础模型，我们在包含2194个多语言多模态样本的精选数据集上应用量化低秩适配（QLoRA）微调。在选定的图像增强设置下，微调后的VL2.1生成结构化JSON格式的核心观点输出，达到64.98%的精确率、42.15%的召回率、51.14%的F1值和74.00%的样本级准确率。相较于零样本VL2.1设置，它将西班牙语和俄语的F1值分别从4.83%和0.45%提升至46.05%和51.93%。该框架还整合了基于模糊累积前景理论的抽取后分类模块，用于案例级价值评估，为下游STI筛选提供案例级价值信号。

英文摘要

Recent advances in large language models (LLMs) have reshaped semantic analysis. Opinion Extraction (OE) for Science and Technology Intelligence (STI) requires concise core opinions from large information streams. Off-the-shelf models struggle to filter noise from these streams and show limited structured-output reliability in zero-shot multilingual and multi-modal settings. To address information overload and extraction defocus, this study proposes a multimodal core-opinion extraction framework in which visual evidence serves as a contextual anchor for textual judgment. Using VideoLLaMA2 (VL2) and VideoLLaMA2.1 (VL2.1) as the base models, we apply Quantized Low-Rank Adaptation (QLoRA) fine-tuning on a curated dataset of 2,194 multilingual and multimodal samples. Under the selected Image-Augmented setting, fine-tuned VL2.1 generates structured JSON core-opinion outputs, achieving 64.98% Precision, 42.15% Recall, 51.14% F1-score, and 74.00% sample-level accuracy. Relative to the zero-shot VL2.1 setting, it raises the F1-scores of Spanish and Russian from 4.83% and 0.45% to 46.05% and 51.93%, respectively. The framework further incorporates a Fuzzy Cumulative Prospect Theory-based post-extraction triage module for case-level value assessment, providing a case-level value signal for downstream STI screening.

发表机构

• Beihang University（北京航空航天大学）
• Nanchang University（南昌大学）
• Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）

机构由 AI 辅助整理，请以论文原文为准。