SpeakerLLM：一种面向说话人理解与验证推理的说话人专用音频大语言模型
SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning
AI总结：
SpeakerLLM通过统一说话人特征建模、录音条件理解及验证推理，提升音频大语言模型在说话人识别与验证中的性能与准确性。
摘要：
随着音频优先代理在物理AI、对话机器人和无屏幕可穿戴设备中的普及，音频大语言模型（audio-LLMs）必须整合说话人特定的理解以支持用户授权、个性化和上下文感知交互。这要求建模谁在说话、声音如何听起来以及录音条件如何影响说话人线索。传统说话人验证系统提供强标量分数但缺乏语言证据，而当前音频-LLMs和说话人意识语言模型在组织说话人信息方面能力有限。我们提出SpeakerLLM，一种面向说话人专用的音频大语言模型框架，通过自然语言接口统一单句说话人特征建模、录音条件理解、句子对说话人比较和证据组织的验证推理。我们构建了验证推理目标和决策组合策略，将特征层面证据与最终同/异决定分离，并将录音条件、特征证据和决定组织成结构化轨迹。核心在于使用分层说话人分词器，以捕捉说话人证据的多个粒度。句子级说话人嵌入总结身份和特征层面线索，而帧级说话人特征保留细粒度的声学描述。实验表明，SpeakerLLM-Base在说话人特征和录音条件理解上优于通用音频-LLMs，而SpeakerLLM-VR保持了强生成判断准确性并产生基于监督验证推理模式的决策轨迹。我们将发布元数据丰富的监督数据集和目标构建代码以确保可重复性。
英文摘要：
As audio-first agents become increasingly common in physical AI, conversational robots, and screenless wearables, audio large language models (audio-LLMs) must integrate speaker-specific understanding to support user authorization, personalization, and context-aware interaction. This requires modeling who is speaking, how the voice sounds, and how recording conditions affect speaker cues. Conventional speaker verification systems provide strong scalar scores but little linguistic evidence, while current audio-LLMs and speaker-aware language models have limited ability to organize speaker information beyond binary labels or descriptive profiles. We present SpeakerLLM, a speaker-specialized audio-LLM framework that unifies single-utterance speaker profiling, recording-condition understanding, utterance-pair speaker comparison, and evidence-organized verification reasoning within a natural-language interface. We construct verification-reasoning targets and a decision-composition policy that separate profile-level evidence from the final same-or-different decision and organize recording condition, profile evidence, and the decision into a structured trace. At its core, SpeakerLLM uses a hierarchical speaker tokenizer designed to capture multiple granularities of speaker evidence. Utterance-level speaker embeddings summarize identity and profile-level cues, whereas frame-level speaker features preserve fine-grained acoustic descriptors. Experiments show that SpeakerLLM-Base improves speaker-profile and recording-condition understanding over general audio-LLMs, while SpeakerLLM-VR preserves strong generated-verdict accuracy and produces decision traces grounded in the supervised verification reasoning schema. We will release the metadata-enriched supervision dataset and target-construction code for reproducibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
