评估大型语言模型在会议中的收话人、话轮转换和下一说话人预测能力
Evaluating Large Language Models Abilities for Addressee, Turn-change, and Next Speaker Prediction in Meetings
AI总结：
利用大型语言模型（LLMs）研究多模态多人对话中的话轮转换，构建了收话人检测、话轮转换预测和下一说话人预测三个任务的评估框架，实验表明LLMs在下一说话人预测上优于监督模型和人类，但多模态LLM在收话人检测和话轮转换预测上仍低于人类水平。
摘要：
我们利用大型语言模型（LLMs）研究多模态多人对话中的话轮转换。我们构建了一个评估框架，涵盖三个任务：收话人检测、话轮转换预测和下一说话人预测。我们比较了针对这些任务训练的监督模型、基于文本的LLMs、多模态LLMs（MM-LLMs）以及人类受试者。在AMI语料库上的实验表明，尽管LLMs未在目标领域进行训练，也无法访问音频或视觉信息，但它们在下一说话人预测方面优于监督模型和人类。多模态LLM在收话人检测和话轮转换预测上表现优于基于文本的LLMs，但仍低于人类表现，表明其难以利用原始音视频信号。消融分析显示，对话上下文至关重要，尤其是对于下一说话人预测。我们观察到人类和LLM的预测模式相似，且话轮转换频繁的区间对两者都具有挑战性。
英文摘要：
We investigate turn-taking in multimodal multi-party conversations using large language models (LLMs). We construct an evaluation framework for three tasks: addressee detection, turn-change prediction, and next speaker prediction. We compare supervised models trained for these tasks, text-based LLMs, multimodal LLMs (MM-LLMs), and human subjects. Experiments on the AMI corpus showed that LLMs outperformed supervised models and humans in next speaker prediction, despite not being trained on the target domain and without access to audio or visual information. An MM-LLM performed better than text-based LLMs on addressee detection and turn-change prediction but remained below human performance, indicating difficulty leveraging raw audio-visual signals. Ablation analyses revealed that conversational context was critical, particularly for next speaker prediction. We observed that human and LLM prediction patterns were similar, and intervals with frequent turn changes were difficult for both.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
