发表机构
SKOLKOVO School of Management; University of Tyumen; Institute of Business Studies, Russian Presidential Academy of National Economy and Public Administration（斯科尔科沃管理学院; 秋明大学; 俄罗斯总统国民经济与公共管理学院商业研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
GYROval是一个稳健基准，用于在英格尔哈特-韦尔策尔轴上测量大语言模型的文化价值取向，通过二元对比场景测试二十个模型，并评估跨语言和温度设置的稳定性。
AI 中文摘要
我们提出了一个稳健的基准，用于在两个英格尔哈特-韦尔策尔轴上，跨多个领域和角色，测量大语言模型的文化价值取向（因此命名为GYROval——网格化稳健价值取向产出），并给出了对二十个模型进行测试的结果。题目是CDEval所引入意义上的二元对比场景：两个选项都是合法的行动方案，没有一个是正确的，没有答案键，模型在一个轴上的得分是其响应落在被计数极上的比例。二十个模型中的十一个额外接受了相同题目的配对俄语翻译版本以及第二次采样温度的测试。该工具以两种语言公开发布。稳定性通过将小场景作为分析单位，在每种扰动因素的层级内对模型进行排名，并使用经平局校正的Kendall W系数与经验排列零分布比较层级间的一致性来评估。
英文摘要
We present a robust benchmark for measuring cultural value orientation in large language models on the two Inglehart-Welzel axes over several domains and roles (hence GYROval - Gridded Yielding of Robust value Orientation), together with the results of administering it to twenty models. Items are binary contrastive scenarios in the sense introduced by CDEval: both options are legitimate courses of action, neither is correct, there is no answer key, and a model's score on an axis is the proportion of its responses falling on the counted pole. Eleven of the twenty models were additionally administered a paired Russian translation of the identical items and a second sampling temperature. The instrument is publicly released in both languages. Stability was assessed by treating the vignette as the unit of analysis, ranking the models within the levels of each perturbation factor, and summarising the agreement between levels by tie-corrected Kendall's \emph{W} against an empirical permutation null.
