arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.09996cs.CLcs.AIcs.CRcs.LG
收藏

没有愚蠢的问题:从土耳其视角评估离线LLM能力

There Are No Silly Questions: Evaluation of Offline LLM Capabilities from a Turkish Perspective

Edibe Yilmaz, Kahraman Kostas

AI总结:

本研究从土耳其视角评估了离线LLM在遗产语言教育中的鲁棒性和教学安全性,发现参数范围在8B-14B的模型在成本与安全性之间达到最佳平衡。

摘要:

将大型语言模型(LLMs)整合到教育过程中引入了关于数据隐私和可靠性的重大限制,特别是在土耳其遗产语言教育等教育脆弱背景下。本研究旨在系统评估在土耳其遗产语言教育背景下可本地部署的离线LLMs的鲁棒性和教学安全性。为此,开发了一个包含10个原始边缘案例场景的土耳其异常套件(TAS),以评估模型在知识抵抗、逻辑一致性和教学安全性方面的能力。在14个不同模型上进行的实验,从270M到32B参数不等,表明异常抵抗不仅仅取决于模型规模,而且奉承偏差甚至在大规模模型中也可能带来教学风险。研究结果表明,在8B-14B参数范围内,以推理为导向的模型在语言学习者成本-安全性的权衡上表现最为平衡。

英文摘要:

The integration of large language models (LLMs) into educational processes introduces significant constraints regarding data privacy and reliability, particularly in pedagogically vulnerable contexts such as Turkish heritage language education. This study aims to systematically evaluate the robustness and pedagogical safety of locally deployable offline LLMs within the context of Turkish heritage language education. To this end, a Turkish Anomaly Suite (TAS) consisting of 10 original edge-case scenarios was developed to assess the models' capacities for epistemic resistance, logical consistency, and pedagogical safety. Experiments conducted on 14 different models ranging from 270M to 32B parameters reveal that anomaly resistance is not solely dependent on model scale and that sycophancy bias can pose pedagogical risks even in large-scale models. The findings indicate that reasoning-oriented models in the 8B--14B parameter range represent the most balanced segment in terms of cost-safety trade-off for language learners.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑