语音信号补充大语言模型用于预测速配中的人际吸引力
Speech Signals Complement LLMs for Predicting Interpersonal Attraction in Speed Dating
浏览论文内容
中文总结 AI 辅助
研究利用日语速配对话，结合仅基于转录本的LLM预测与语音预测器预测来估计参与者对约会对象的喜好，发现语音能补充LLM预测，二者结合可提高两两排序准确率，且互补性有条件，还明确了语音预测价值及互补性出现的情况。
中文摘要 AI 辅助
大语言模型（LLMs）能够根据对话转录本预测人际吸引力，但语音预测器在仅基于转录本的LLM预测之外还能增加什么尚不清楚。利用日语速配对话，我们将仅基于转录本的LLM预测和有监督的语音预测器的预测相结合，以估计参与者对其约会对象的喜好程度报告。我们表明语音可以补充仅基于转录本的LLM预测，但这种互补性是有条件的而非普遍的。在所有评估条件下，将两种预测相结合显著提高了两两排序准确率。相比之下，参与者的皮尔逊r值在不同对话轮次和评分方向上有所不同，校正后均无显著差异。回顾来看，这些r值的增加集中在语音预测器更准确的参与者中。因此，即使LLM根据转录本预测吸引力，语音仍可保留预测价值。相关问题不仅在于语音是否有帮助，还在于其互补性出现在何处。
英文摘要
Large language models (LLMs) can predict interpersonal attraction from conversation transcripts, but it remains unclear what a speech predictor can add beyond transcript-only LLM prediction. Using Japanese speed-dating conversations, we combine predictions from a transcript-only LLM and a supervised speech predictor to estimate participants' reported liking of their partners. We show that speech can complement transcript-only LLM prediction, but that this complementarity is conditional rather than universal. Combining the two predictions significantly improves pairwise ranking accuracy over the transcript-only LLM alone in all evaluated conditions. By contrast, gains in per-participant Pearson $r$ vary across conversation rounds and rating directions, with none significant after correction. Retrospectively, these $r$ gains are concentrated among participants for whom the speech predictor is more accurate. Speech can therefore retain predictive value even when an LLM predicts attraction from transcripts. The relevant question is not simply whether speech helps, but where its complementarity emerges.
发表机构
- Japan Advanced Institute of Science and Technology（日本先进科学技术学院）
- NTT, Inc.（日本电报电话公司）
机构由 AI 辅助整理，请以论文原文为准。
