arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.25974cs.HCcs.LG
收藏

谁在领导?比较以人类为中心和以模型为中心的策略以定义机器学习目标变量

Who Leads? Comparing Human-Centric and Model-Centric Strategies for Defining ML Target Variables

University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)

Mengtian Guo, David Gotz, Yue Wang

AI总结:

本文探讨了人类-机器协作在定义机器学习目标变量中的影响,发现以性能优先的策略能加快迭代速度,但可能导致与应用目标不一致的偏差。

摘要:

预测建模有潜力提升人类决策,但许多预测模型因问题定义不当而失效,特别是在预测目标是抽象概念或构造时。选择合适的代理目标变量需要领域知识和迭代数据建模。此过程本质上是协作的,涉及领域专家和数据科学家。本文研究了人类-机器协作如何通过加速迭代并保留人类判断来支持此过程。我们探讨了两种人类-机器协作策略对代理构建的影响:1)相关性优先:人类主导选择相关代理;2)性能优先:机器主导推荐基于预测性能的代理。基于对代理构建任务的受控用户研究(N=20),我们发现性能优先策略加快了迭代和决策,但使用户偏向于与应用目标不一致的高性能代理。本研究揭示了人类-机器协作在定义机器学习目标变量中的机会与风险,为未来研究提供了洞察。

英文摘要:

Predictive modeling has the potential to enhance human decision-making. However, many predictive models fail in practice due to problematic problem formulation in cases where the prediction target is an abstract concept or construct and practitioners need to define an appropriate target variable as a proxy to operationalize the construct of interest. The choice of an appropriate proxy target variable is rarely self-evident in practice, requiring both domain knowledge and iterative data modeling. This process is inherently collaborative, involving both domain experts and data scientists. In this work, we explore how human-machine teaming can support this process by accelerating iterations while preserving human judgment. We study the impact of two human-machine teaming strategies on proxy construction: 1) relevance-first: humans leading the process by selecting relevant proxies, and 2) performance-first: machines leading the process by recommending proxies based on predictive performance. Based on a controlled user study of a proxy construction task (N = 20), we show that the performance-first strategy facilitated faster iterations and decision-making, but also biased users towards well-performing proxies that are misaligned with the application goal. Our study highlights the opportunities and risks of human-machine teaming in operationalizing machine learning target variables, yielding insights for future research to explore the opportunities and mitigate the risks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑