它必须是主观的：通过零样本密度估计实现人类标注者模拟（HAS）
It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimation
AI总结：
本研究针对人类标注者模拟需体现人类主观变异性的问题，提出将HAS视作零样本密度估计的元学习框架及两类对应不同标注类型的流模型，在真实任务上表现更优。
摘要：
人类标注者模拟（Human Annotator Simulation，HAS）是数据标注、系统评估等人类评估的一种高性价比替代方案。人类评估过程中的感知与行为因认知过程差异和主观解读存在固有变异性，建模时需纳入该特性以更好地模拟人类感知与交互世界的方式。本文提出一种新型元学习框架，将HAS视为零样本密度估计问题，该框架纳入人类变异性，可高效为无标注测试输入生成类人标注。在此框架下，我们提出两类新模型：条件整数流与条件softmax流，分别处理有序标注和分类标注。该方法在3项真实人类评估任务上进行了评估，展现出更优的能力与效率，可预测人类标注者的聚合行为、匹配人类标注分布并模拟标注者间的分歧。
英文摘要：
Human annotator simulation (HAS) serves as a cost-effective substitute for human evaluation such as data annotation and system assessment. Human perception and behaviour during human evaluation exhibit inherent variability due to diverse cognitive processes and subjective interpretations, which should be taken into account in modelling to better mimic the way people perceive and interact with the world. This paper introduces a novel meta-learning framework that treats HAS as a zero-shot density estimation problem, which incorporates human variability and allows for the efficient generation of human-like annotations for unlabelled test inputs. Under this framework, we propose two new model classes, conditional integer flows and conditional softmax flows, to account for ordinal and categorical annotations, respectively. The proposed method is evaluated on three real-world human evaluation tasks and shows superior capability and efficiency to predict the aggregated behaviours of human annotators, match the distribution of human annotations, and simulate the inter-annotator disagreements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
