发表机构
The Hong Kong University of Science and Technology (Guangzhou); College of Future Technology, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）; 香港科技大学（广州）未来技术学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出一种基于视觉的注意力-动作框架，利用RGB-D和机器人状态实现人机协作装配中可解释的认知工作负荷连续评估，实验验证其有效且无需额外传感器。
AI 中文摘要
人机协作（HRC）在工业装配操作中的引入正在彻底改变制造业格局。在这种不断演变的环境中，操作员需要将其手动任务与实时任务信息和机器人行为无缝协调。这些需求在操作过程中会波动，然而传统的工作负荷评估依赖于穿戴式生理传感器，这使实际部署变得复杂。在此，我们提出了一种基于视觉的注意力-动作框架，用于HRC装配中连续且可解释的工作负荷相关评估。该框架将RGB-D观测与机器人状态和经过校准的任务相关区域相结合，构建操作员行为的时间确认表示。该表示识别任务需求集中的位置，并解释当注意力与动作背离、任务情境变化或操作员犹豫时需求如何发展。我们在一个包含十名参与者的三级协作齿轮箱装配实验中评估了该框架，使用主观评分和同步生理信号作为独立参考。原始NASA-TLX评分确认了各条件下感知工作负荷的增加，对整体工作负荷及其心理和时间维度有显著影响。视觉派生的HRC-CWL输出与九名具有完整相关数据的参与者中七名的心电图（ECG）衍生特征显著相关。同步交互片段进一步显示检测到的犹豫与生理活动之间的时间对应关系。实时部署表明，该框架可以在无需操作员佩戴额外传感器的情况下运行。这些发现支持HRC-CWL作为认知工效学分析和自适应机器人辅助的可解释行为代理，而非工作负荷的直接心理生理测量。
英文摘要
The introduction of human-robot collaboration (HRC) in industrial assembly operations is revolutionizing the manufacturing landscape. In this evolving environment, operators are required to seamlessly coordinate their manual tasks with real-time task information and robotic behaviors. These demands fluctuate during operation, yet conventional workload assessments depend on body-worn physiological sensors that complicate practical deployment. Here, we present a vision-based attention--action framework for continuous and interpretable workload-related assessment in HRC assembly. The framework combines RGB-D observations with robot states and calibrated task-related areas to construct a temporally confirmed representation of operator behavior. This representation identifies where task demand is concentrated and explains how it develops when attention and action diverge, the task context changes, or the operator hesitates. We evaluated the framework in a three-level collaborative gearbox assembly experiment with ten participants, using subjective ratings and synchronized physiological signals as independent references. Raw NASA-TLX ratings confirmed increasing perceived workload across conditions, with significant effects on overall workload and its mental and temporal dimensions. The vision-derived HRC-CWL output was significantly associated with ECG-derived features in seven of nine participants with complete correlation data. Synchronized interaction episodes further showed temporal correspondence between detected hesitation and physiological activity. Real-time deployment demonstrated that the framework can operate without requiring operators to wear additional sensors. These findings support HRC-CWL as an interpretable behavioral proxy for cognitive ergonomics analysis and adaptive robot assistance, rather than a direct psychophysiological measure of workload.
Comments42 pages, 11 figures
