我的大语言模型可能会模仿非洲裔美国英语(AAE)——但何时应该这样做?
My LLM might Mimic AAE -- But When Should it?
AI总结:
本研究通过调查与标注,探究美国黑人对LLM生成真实AAE的看法及期望语境,发现其重视选择自主权,倾向LLM在正式场合默认用美国主流英语、非正式场合用AAE,且适当提示下LLM生成的AAE真实性与黑人语音转录相当。
摘要:
我们研究了大型语言模型(LLMs)中非洲裔美国英语(AAE)的表现,探索(a)美国黑人对这些技术生成真实AAE的有效性的看法,以及(b)美国黑人在哪些语境下认为这是可取的。通过对美国黑人的调查(n=104)和美国黑人对LLM生成的AAE的标注(n=228),我们发现美国黑人倾向于在决定LLM输出中何时使用AAE合适时拥有选择和自主权。他们往往更希望LLM在正式场合默认使用美国主流英语进行交流,而在不太正式的场合对生成AAE更感兴趣。当LLM得到适当提示并提供上下文示例时,参与者发现其输出的AAE真实性与美国黑人的语音转录水平相当。我们项目的部分代码和数据可在此处获取:https://github.com/smelliecat/AAEMime.git
英文摘要:
We examine the representation of African American English (AAE) in large language models (LLMs), exploring (a) the perceptions Black Americans have of how effective these technologies are at producing authentic AAE, and (b) in what contexts Black Americans find this desirable. Through both a survey of Black Americans ($n=$ 104) and annotation of LLM-produced AAE by Black Americans ($n=$ 228), we find that Black Americans favor choice and autonomy in determining when AAE is appropriate in LLM output. They tend to prefer that LLMs default to communicating in Mainstream U.S. English in formal settings, with greater interest in AAE production in less formal settings. When LLMs were appropriately prompted and provided in context examples, our participants found their outputs to have a level of AAE authenticity on par with transcripts of Black American speech. Select code and data for our project can be found here: https://github.com/smelliecat/AAEMime.git
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。