arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.05328cs.CLcs.AIcs.ETcs.HCecon.GNq-fin.EC
收藏

从文本到洞察:利用大型语言模型进行管理中的绩效评估

From Text to Insight: Leveraging Large Language Models for Performance Evaluation in Management

Tsinghua University(清华大学)

Ning Li, Huaikang Zhou, Mingze Xu

AI总结:

该研究通过两项比较分析验证了GPT-4在知识型绩效评估中可替代甚至优于人类评分者,具有更高一致性和可靠性,但也存在光环效应等情境偏差,为AI在管理评估中的应用提供了实证基础与边界界定。

摘要:

本研究探讨了大型语言模型(LLMs),特别是GPT-4,在提升组织任务绩效评估客观性方面的潜力。通过在两项研究中对多种任务绩效输出进行比较分析,我们证明LLMs可以作为可靠甚至优于人类评分者的替代方案,用于评估基于知识的绩效输出,而这类输出是知识工作者的关键贡献。我们的结果表明,GPT评分与人类评分具有可比性,但表现出更高的一致性和可靠性。此外,对同一绩效输出进行多次GPT评分合并后,与聚合的人类绩效评分显示出强相关性,这与绩效评估文献中观察到的共识原则类似。然而,我们也发现LLMs容易受到情境偏差的影响,例如光环效应,这反映了人类评估偏差的特征。我们的研究表明,虽然LLMs能够从基于文本的数据中提取有意义的构念,但其适用范围目前仅限于特定形式的绩效评估。通过强调LLMs的潜力和局限性,我们的研究为关于人工智能在管理研究中作用的讨论做出了贡献,并为未来研究完善人工智能在管理中的理论和实际应用奠定了基础。

英文摘要:

This study explores the potential of Large Language Models (LLMs), specifically GPT-4, to enhance objectivity in organizational task performance evaluations. Through comparative analyses across two studies, including various task performance outputs, we demonstrate that LLMs can serve as a reliable and even superior alternative to human raters in evaluating knowledge-based performance outputs, which are a key contribution of knowledge workers. Our results suggest that GPT ratings are comparable to human ratings but exhibit higher consistency and reliability. Additionally, combined multiple GPT ratings on the same performance output show strong correlations with aggregated human performance ratings, akin to the consensus principle observed in performance evaluation literature. However, we also find that LLMs are prone to contextual biases, such as the halo effect, mirroring human evaluative biases. Our research suggests that while LLMs are capable of extracting meaningful constructs from text-based data, their scope is currently limited to specific forms of performance evaluation. By highlighting both the potential and limitations of LLMs, our study contributes to the discourse on AI role in management studies and sets a foundation for future research to refine AI theoretical and practical applications in management.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑