arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.12294cs.CL
收藏

评估Transformer模型与人类行为在汉字命名中的表现

Evaluating Transformer Models and Human Behaviors on Chinese Character Naming

The Graduate Center, CUNY(纽约市立大学研究生中心);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)

Xiaomeng Ma, Lingyu Gao

AI总结:

本研究通过对比Transformer模型与人类在未知汉字命名任务上的表现,发现两者行为高度相似,表明模型能有效捕捉人类的汉字命名行为。

摘要:

神经网络模型已被提出用于解释人类在许多字母语言中的字形-音素映射过程。这些模型不仅成功学习了字母字符串与其发音之间的对应关系,还在非词命名任务中捕捉到了人类行为。对于非字母语言(如中文)的未知汉字任务,神经网络模型的表现如何?模型能在多大程度上捕捉人类行为?在本研究中,我们首先收集了人类说话者在未知汉字命名任务中的回答,然后通过比较一组Transformer模型的表现与人类在未知汉字命名任务中的行为来评估这些模型。我们发现,模型与人类的行为非常相似,它们对每个汉字的准确率分布相似,并且在答案上有大量重叠。此外,模型的答案与人类的答案高度相关。这些结果表明,Transformer模型能够很好地捕捉人类的汉字命名行为。

英文摘要:

Neural network models have been proposed to explain the grapheme-phoneme mapping process in humans for many alphabet languages. These models not only successfully learned the correspondence of the letter strings and their pronunciation, but also captured human behavior in nonce word naming tasks. How would the neural models perform for a non-alphabet language (e.g., Chinese) unknown character task? How well would the model capture human behavior? In this study, we first collect human speakers' answers on unknown character naming tasks and then evaluate a set of transformer models by comparing their performances with human behaviors on an unknown Chinese character naming task. We found that the models and humans behaved very similarly, that they had similar accuracy distribution for each character, and had a substantial overlap in answers. In addition, the models' answers are highly correlated with humans' answers. These results suggested that the transformer models can well capture human's character naming behavior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑