在工业生成式AI平台上扩展视觉语言模型用于制药长格式视频推理
Scaling Vision Language Models for Pharmaceutical Long Form Video Reasoning on Industrial GenAI Platform
AI总结：
本文提出工业GenAI框架，解决制药领域长格式视频推理问题，通过多模态架构和实证分析提升效率并揭示现有VLMs的限制。
摘要：
视觉语言模型（VLMs）在多模态推理任务中表现出强劲的性能，但大多数评估集中在短视频上，并假设有不受限制的计算资源。在制药内容理解等工业环境中，从业者必须在严格的GPU、延迟和成本约束下处理长格式视频，其中许多现有方法无法扩展。在本工作中，我们提出了一种工业GenAI框架，能够处理超过200,000份PDF文件、25,326个跨八种格式（如MP4、M4V等）的视频，以及888个多语言音频文件，涵盖超过20种语言。我们的研究做出了三个贡献：（i）一种用于制药领域多模态推理的工业级大规模架构；（ii）在两个领先的基准（Video-MME和MMBench）和专有数据集（包含25,326个视频，覆盖14个疾病领域）上对超过40个VLMs的实证分析；（iii）四个与长格式视频推理相关的发现：多模态性的作用、注意力机制的权衡、时间推理的限制以及在GPU约束下的视频分割挑战。结果表明，使用SDPA注意力机制在商用GPU上可实现3-8倍的效率提升，多模态性在多达8/12个任务领域（特别是长度依赖性任务）中提升了性能，但开放源和闭源VLMs在时间对齐和关键帧检测上存在明显瓶颈。本文并非提出新的“A+B”模型，而是刻画了在现实部署约束下当前VLMs的实际限制、权衡和失败模式，并为研究人员和从业者设计可扩展的多模态系统提供了可行的指导，用于工业领域长格式视频理解。
英文摘要：
Vision Language Models (VLMs) have shown strong performance on multimodal reasoning tasks, yet most evaluations focus on short videos and assume unconstrained computational resources. In industrial settings such as pharmaceutical content understanding, practitioners must process long-form videos under strict GPU, latency, and cost constraints, where many existing approaches fail to scale. In this work, we present an industrial GenAI framework that processes over 200,000 PDFs, 25,326 videos across eight formats (e.g., MP4, M4V, etc.), and 888 multilingual audio files in more than 20 languages. Our study makes three contributions: (i) an industrial large-scale architecture for multimodal reasoning in pharmaceutical domains; (ii) empirical analysis of over 40 VLMs on two leading benchmarks (Video-MME and MMBench) and proprietary dataset of 25,326 videos across 14 disease areas; and (iii) four findings relevant to long-form video reasoning: the role of multimodality, attention mechanism trade-offs, temporal reasoning limits, and challenges of video splitting under GPU constraints. Results show 3-8 times efficiency gains with SDPA attention on commodity GPUs, multimodality improving up to 8/12 task domains (especially length-dependent tasks), and clear bottlenecks in temporal alignment and keyframe detection across open- and closed-source VLMs. Rather than proposing a new "A+B" model, this paper characterizes practical limits, trade-offs, and failure patterns of current VLMs under realistic deployment constraints, and provide actionable guidance for both researchers and practitioners designing scalable multimodal systems for long-form video understanding in industrial domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
