arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.08692cs.CL
收藏

从名字预测国籍:神经模型和大语言模型的比较研究

Nationality and Region Prediction from Names: A Comparative Study of Neural Models and Large Language Models

Keito Inoshita

AI总结:

本研究比较神经模型与大语言模型在国籍预测中的表现,发现LLMs在所有粒度级别均优于神经模型,且在低频国籍上表现下降,错误类型差异显著。

摘要:

从个人名字预测国籍在市场营销、人口研究和族谱研究中具有实际价值。传统神经模型从任务特定的训练数据中学习名字与国籍之间的统计对应关系,在泛化到低频国籍和区分同一地区内相似国籍时面临挑战。大语言模型(LLMs)有潜力通过利用预训练过程中获得的世界知识来解决这些挑战。在本研究中,我们全面比较了神经模型和LLMs在国籍预测中的表现,评估了六个神经模型和六个LLM提示策略,在三个粒度级别(国籍、地区和大洲)上进行比较,包括基于频率的分层分析和错误分析。结果表明,LLMs在所有粒度级别上均优于神经模型,随着粒度变粗,差距逐渐缩小。简单的机器学习方法在频率鲁棒性方面表现最高,而预训练模型和LLMs在低频国籍上表现下降。错误分析表明,LLMs倾向于做出“近似错误”,即使国籍预测错误,仍能正确预测地区,而神经模型则更容易出现跨地区错误,并偏向高频类别。这些发现表明,LLMs的优越性源于世界知识,模型选择应考虑所需的粒度,评估应考虑错误质量而不仅仅是准确率。

英文摘要:

Predicting nationality from personal names has practical value in marketing, demographic research, and genealogical studies. Conventional neural models learn statistical correspondences between names and nationalities from task-specific training data, posing challenges in generalizing to low-frequency nationalities and distinguishing similar nationalities within the same region. Large language models (LLMs) have the potential to address these challenges by leveraging world knowledge acquired during pre-training. In this study, we comprehensively compare neural models and LLMs on nationality prediction, evaluating six neural models and six LLM prompting strategies across three granularity levels (nationality, region, and continent), with frequency-based stratified analysis and error analysis. Results show that LLMs outperform neural models at all granularity levels, with the gap narrowing as granularity becomes coarser. Simple machine learning methods exhibit the highest frequency robustness, while pre-trained models and LLMs show degradation for low-frequency nationalities. Error analysis reveals that LLMs tend to make ``near-miss'' errors, predicting the correct region even when nationality is incorrect, whereas neural models exhibit more cross-regional errors and bias toward high-frequency classes. These findings indicate that LLM superiority stems from world knowledge, model selection should consider required granularity, and evaluation should account for error quality beyond accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑