Predicting nationality from personal names has practical value in marketing, demographic research, and genealogical studies. Conventional neural models learn statistical correspondences between names and nationalities from task-specific training data, posing challenges in generalizing to low-frequency nationalities and distinguishing similar nationalities within the same region. Large language models (LLMs) have the potential to address these challenges by leveraging world knowledge acquired during pre-training. In this study, we comprehensively compare neural models and LLMs on nationality prediction, evaluating six neural models and six LLM prompting strategies across three granularity levels (nationality, region, and continent), with frequency-based stratified analysis and error analysis. Results show that LLMs outperform neural models at all granularity levels, with the gap narrowing as granularity becomes coarser. Simple machine learning methods exhibit the highest frequency robustness, while pre-trained models and LLMs show degradation for low-frequency nationalities. Error analysis reveals that LLMs tend to make ``near-miss'' errors, predicting the correct region even when nationality is incorrect, whereas neural models exhibit more cross-regional errors and bias toward high-frequency classes. These findings indicate that LLM superiority stems from world knowledge, model selection should consider required granularity, and evaluation should account for error quality beyond accuracy.
Comparative Study of Pre-Trained BERT and Large Language Models for Code-Mixed Named Entity Recognition
Dept. of Computer Engineering, Pune Institute of Computer Technology, Pune, India(计算机工程系,普那计算机技术研究所,普那,印度);Indian Institute of Technology Madras, Chennai, India(印度理工学院马德拉斯学院,钦奈,印度);L3Cube Labs, Pune, Maharashtra, India(L3Cube实验室,普那,马哈拉施特拉邦,印度)
A Comparative Study of Task Adaptation Techniques of Large Language Models for Identifying Sustainable Development Goals
Linkalab s.r.l.(Linkalab公司);Department of Mathematics and Computer Science(数学与计算机科学系);University of Cagliari(卡利亚里大学);Knowledge Media Institute(知识媒体研究所);The Open University(开放大学)
Utilizing Large Language Models for Named Entity Recognition in Traditional Chinese Medicine against COVID-19 Literature: Comparative Study
China Academy of Chinese Medical Sciences(中国中医科学院);GESIS—Leibniz Institute for the Social Sciences(GESIS——莱布尼茨社会科学研究所);University of Bonn(波恩大学);Capital Medical University(首都医科大学)
Systematic Exploration of Dialogue Summarization Approaches for Reproducibility, Comparative Assessment, and Methodological Innovations for Advancing Natural Language Processing in Abstractive Summarization
SRM Institute of Science and Technology(SRM科学技术学院);SRM University, AP(SRM大学(安得拉邦))