具有语言基础的稀疏编码的生成AI内容分析
Human-like Content Analysis for Generative AI with Language-Grounded Sparse Encoders
AI总结：
本文提出LanSE工具，通过自然语言描述将图像分解为可解释的视觉模式，实现了对生成AI内容的细粒度分析，提升了物理合理性评估并扩展至医学影像领域。
摘要：
生成AI的快速发展改变了内容创作、交流和人类发展，但在高风险领域引发了深刻担忧，需要严格方法分析和评估AI生成内容。现有分析方法常将图像视为不可分割的整体，而现实中的AI故障通常表现为特定的视觉模式，可逃避整体检测，更适合细粒度和分解分析。本文引入了语言基础稀疏编码（LanSE）工具，通过自然语言描述将图像分解为可解释的视觉模式。利用可解释模块和大规模多模态模型，LanSE能够自动识别数据模态中的视觉模式。我们的方法发现了超过5,000个视觉模式，人类同意率达93%，提供了优于现有方法的分解评估，建立了物理合理性的首个系统评估，并扩展到医学影像领域。我们的方法提取语言基础模式的能力可自然适应生物学、地理学及其他数据模态，如蛋白质结构和时间序列，从而推动生成AI的内容分析。
英文摘要：
The rapid development of generative AI has transformed content creation, communication, and human development. However, this technology raises profound concerns in high-stakes domains, demanding rigorous methods to analyze and evaluate AI-generated content. While existing analytic methods often treat images as indivisible wholes, real-world AI failures generally manifest as specific visual patterns that can evade holistic detection and suit more granular and decomposed analysis. Here we introduce a content analysis tool, Language-Grounded Sparse Encoders (LanSE), which decompose images into interpretable visual patterns with natural language descriptions. Utilizing interpretability modules and large multimodal models, LanSE can automatically identify visual patterns within data modalities. Our method discovers more than 5,000 visual patterns with 93\% human agreement, provides decomposed evaluation outperforming existing methods, establishes the first systematic evaluation of physical plausibility, and extends to medical imaging settings. Our method's capability to extract language-grounded patterns can be naturally adapted to numerous fields, including biology and geography, as well as other data modalities such as protein structures and time series, thereby advancing content analysis for generative AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
