SeetaPsych v1.0：用于基于行为的心理测量的开源计算机视觉工具包

中文摘要 AI 辅助

自动视觉分析为基于行为的心理测量开辟了新途径。然而，现有的技术模块通常分散在特定任务的系统中，具有异构接口和不同的部署要求。在这项工作中，我们提出了SeetaPsych v1.0，一个开源、统一且可扩展的计算机视觉工具包，旨在从面部图像和/或基于面部的视频中提取心理相关信号。当前版本包含四个主要核心模块，旨在实现基于行为的生理感知：统一的面部情感分析（同时进行面部表情识别、面部动作单元检测和效价-唤醒度估计）、基于摄像头的心率估计、屏幕注视点估计和场景注视跟随。还包括一套用于以人为中心的视觉分析的辅助预处理模块，包括人脸检测、面部关键点检测和头部检测。这些功能封装在模块化的Pipeline/Runner架构中，该架构自动解析属性依赖关系、构建计算图，并支持模块间的中间结果共享。SeetaPsych提供标准化的Python API以促进可重复的大规模分析，同时提供交互式WebUI以支持快速、无需代码的方法评估。总体而言，SeetaPsych为心理学、行为科学、人机交互及相关领域的研究提供了一个集成且可访问的视觉测量平台。

英文摘要

Automated visual analysis opens new avenues for behavior--based psychological measurement. Nevertheless, existing technological modules are typically scattered across task specific systems with heterogeneous interfaces and disparate deployment requirements. In this work, we present SeetaPsych v1.0, an open source, unified and extensible computer vision toolkit designed to extract psychologically relevant signals from facial images and/or face based videos. The current release encompasses four major core modules aiming at behavior--based physiological perception: unified face based emotion analysis (simultaneous facial expression recognition, facial action unit detection, and valence--arousal estimation), camera based heart rate estimation, screen point--of--gaze estimation, and scene gaze following. A suite of auxiliary preprocessing modules for human centric visual analysis is also included, comprising face detection, facial landmark detection, and head detection. These functionalities are encapsulated within a modular Pipeline/Runner architecture that automatically resolves attribute dependencies, constructs computation graphs, and support intermediate result sharing among modules. SeetaPsych provides standardized Python APIs to facilitate reproducible, large scale analyses, alongside an interactive WebUI for rapid, code--free method evaluation. Overall, SeetaPsych offers an integrated and accessible visual measurement platform for research in psychology, behavioral science, human computer interaction, and related fields.

发表机构

• Southeast University（东南大学）

机构由 AI 辅助整理，请以论文原文为准。