arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.18567cs.CL
收藏

对谁困难?日语词汇复杂性研究

Difficult for Whom? A Study of Japanese Lexical Complexity

Nara Institute of Science and Technology(奈良科学技术研究所);Universitat Pompeu Fabra(庞培法布拉大学)

Adam Nohejl, Akio Hayakawa, Yusuke Ide, Taro Watanabe

AI总结:

本研究验证日语词汇复杂性数据集的代表性,发现母语为汉语者对复杂性感知不同,并比较群体与个体模型性能,表明个体LCP预测困难,微调BERT仅带来边际改进。

摘要:

词汇复杂性预测(LCP)和复杂词识别(CWI)任务通常预设难以理解的词汇对目标人群是共通的。同时,个性化方法也被提出以使模型适应个体需求。我们通过部分复制标注,验证了一个近期日语LCP数据集对其目标人群具有代表性。通过另一次重新标注,我们表明由于日汉同形词(Sino-Japanese vocabulary)的存在,以汉语为母语的说话者对复杂性的感知有所不同。为探索个性化的可能性,我们比较了基于群体平均评分和个体评分训练的竞争性基线在个体上的表现。我们表明,在CWI任务中,基于群体平均训练的模型表现与个体模型相似,而实现个体良好的LCP性能则较为困难。我们还尝试对微调后的BERT模型进行适配,但在所有设置中仅带来边际改进。

英文摘要:

The tasks of lexical complexity prediction (LCP) and complex word identification (CWI) commonly presuppose that difficult to understand words are shared by the target population. Meanwhile, personalization methods have also been proposed to adapt models to individual needs. We verify that a recent Japanese LCP dataset is representative of its target population by partially replicating the annotation. By another reannotation we show that native Chinese speakers perceive the complexity differently due to Sino-Japanese vocabulary. To explore the possibilities of personalization, we compare competitive baselines trained on the group mean ratings and individual ratings in terms of performance for an individual. We show that the model trained on a group mean performs similarly to an individual model in the CWI task, while achieving good LCP performance for an individual is difficult. We also experiment with adapting a finetuned BERT model, which results only in marginal improvements across all settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑