音调下的偏见:LLM驱动的UX系统中音调偏见的实证刻画
Bias Beneath the Tone: Empirical Characterisation of Tone Bias in LLM-Driven UX Systems
AI总结:
本文研究了LLM驱动UX系统中的音调偏见问题,通过合成数据集和弱监督方法,揭示了模型在对话中存在系统性的音调偏差,为设计公平可信的对话AI提供了依据。
摘要:
大型语言模型正越来越多地用于对话系统,如数字个人助手,通过语言塑造人们与技术互动的方式。尽管它们的回应通常听起来流畅自然,但它们也可能携带细微的音调偏见,例如在中性预期的情况下显得过于礼貌、愉快或谨慎。这些倾向可能影响用户对对话中信任、同理心和公平性的感知。在本研究中,我们探索音调偏见作为大型语言模型隐藏的行为特征。本研究的创新之处在于将可控的大语言模型基于对话合成与音调分类模型相结合,使在个人助手互动中实现稳健和道德的情感识别成为可能。我们创建了两个合成对话数据集,一个是从中性提示生成的,另一个则明确指导生成积极或消极的音调。令人惊讶的是,即使中性数据集也显示出一致的音调偏差,这表明偏见可能源于模型本身的对话风格。通过使用预训练的DistilBERT模型进行弱监督,我们对音调进行了标注,并训练了多个分类器来检测这些模式。集成模型在宏F1分数上达到了高达0.92,表明音调偏见是系统性、可测量的,并且与设计公平和可信的对话AI相关。
英文摘要:
Large Language Models are increasingly used in conversational systems such as digital personal assistants, shaping how people interact with technology through language. While their responses often sound fluent and natural, they can also carry subtle tone biases such as sounding overly polite, cheerful, or cautious even when neutrality is expected. These tendencies can influence how users perceive trust, empathy, and fairness in dialogue. In this study, we explore tone bias as a hidden behavioral trait of large language models. The novelty of this research lies in the integration of controllable large language model based dialogue synthesis with tone classification models, enabling robust and ethical emotion recognition in personal assistant interactions. We created two synthetic dialogue datasets, one generated from neutral prompts and another explicitly guided to produce positive or negative tones. Surprisingly, even the neutral set showed consistent tonal skew, suggesting that bias may stem from the model's underlying conversational style. Using weak supervision through a pretrained DistilBERT model, we labeled tones and trained several classifiers to detect these patterns. Ensemble models achieved macro F1 scores up to 0.92, showing that tone bias is systematic, measurable, and relevant to designing fair and trustworthy conversational AI.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。