波形知道什么：使用字幕工作室实现透明优先的语音和音频智能

中文摘要 AI 辅助

字幕工作室是一个透明优先的语音和音频智能平台，通过自动转录、说话人识别、语音分析、信号级音频分析和字幕生成，将语音音频和视频转换为结构化、可搜索的内容。该系统基于具有实时仪表盘的FastAPI后端构建，采用三层架构，包括基于Whisper类自动语音识别和pyannote说话人识别的转录和识别核心、直接从音频信号中提取声学和语言特征的音频智能层、支持数据导出和下游工作流程集成的集成层。这项工作的主要贡献是透明优先框架，其中每个报告的指标都明确标识为测量、推导或不可用，从而提高了语音分析的可追溯性、可解释性和可靠性。本文介绍了系统架构、基准测试方法、可解释性和不确定性框架以及企业级部署的关键考虑因素。

英文摘要

Caption Studio is a transparency-first speech and audio intelligence platform that transforms spoken audio and video into structured, searchable content through automated transcription, speaker diarization, speech analytics, signal-level audio analysis, and subtitle generation. The system is built on a FastAPI backend with a real-time dashboard and adopts a three-layer architecture comprising (i) a transcription and diarization core based on Whisper-class automatic speech recognition and pyannote speaker diarization, (ii) an audio intelligence layer that extracts acoustic and linguistic features, including waveforms, spectrograms, pitch, speaking rate, silence, filler-word frequency, and sentiment, directly from the audio signal, and (iii) an integration layer that supports data export and downstream workflow integration. A principal contribution of this work is the transparency-first framework, in which every reported metric is explicitly identified as measured, derived, or unavailable, thereby improving the traceability, interpretability, and reliability of speech analytics. The paper presents the system architecture, benchmarking methodology, explainability and uncertainty framework, and key considerations for enterprise-scale deployment.

发表机构

• Faculty of Science, Agriculture, and Engineering, Newcastle University Singapore（新加坡纽卡斯尔大学科学、农业与工程学院）
• School of Information and Control Engineering, Qingdao University of Technology（青岛理工大学信息与控制工程学院）
• Department of EEE, Amrita School of Engineering, Amrita Vishwa Vidyapeetham（阿姆瑞塔工程学院电气与电子工程系，阿姆瑞塔大学）

机构由 AI 辅助整理，请以论文原文为准。