arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.19098cs.HCcs.AI
收藏

评估人机协作:综述与方法论框架

Evaluating Human-AI Collaboration: A Review and Methodological Framework

Harokopio University of Athens(雅典哈罗科皮奥大学)

George Fragiadakis, Christos Diou, George Kousiouris, Mara Nikolaidou

AI总结:

本文综述了人机协作(HAIC)的评估方法,并提出一个包含结构化决策树的方法论框架,以根据AI为中心、人为中心和共生型三种模式选择定量与定性指标,从而更有效地评估HAIC系统。

摘要:

人工智能(AI)在个人工作环境中的使用,即人机协作(HAIC),已在众多领域中变得不可或缺,提升了决策、效率和创新能力。尽管HAIC具有广泛的潜力,但由于所涉及组件之间复杂的交互,评估其有效性仍然具有挑战性。本文对现有的HAIC评估方法进行了详细分析,并开发了一种新的范式,以更有效地评估这些系统。我们的框架包括一个结构化的决策树,该决策树有助于根据不同的HAIC模式(以AI为中心、以人为中心和共生型)选择相关指标。通过纳入定量和定性指标,该框架力求体现HAIC的动态性和互惠性,从而能够评估其影响和成功程度。该框架的实用性可以通过其在制造业、医疗保健、金融和教育等一系列领域中的应用来检验,每个领域都有独特的挑战和要求。我们希望这项研究将有助于推动在现实应用中系统评估HAIC的进一步研究。

英文摘要:

The use of artificial intelligence (AI) in working environments with individuals, known as Human-AI Collaboration (HAIC), has become essential in a variety of domains, boosting decision-making, efficiency, and innovation. Despite HAIC's wide potential, evaluating its effectiveness remains challenging due to the complex interaction of components involved. This paper provides a detailed analysis of existing HAIC evaluation approaches and develops a fresh paradigm for more effectively evaluating these systems. Our framework includes a structured decision tree which assists to select relevant metrics based on distinct HAIC modes (AI-Centric, Human-Centric, and Symbiotic). By including both quantitative and qualitative metrics, the framework seeks to represent HAIC's dynamic and reciprocal nature, enabling the assessment of its impact and success. This framework's practicality can be examined by its application in an array of domains, including manufacturing, healthcare, finance, and education, each of which has unique challenges and requirements. Our hope is that this study will facilitate further research on the systematic evaluation of HAIC in real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑