信用分配：开发类人 AI 智能体的挑战与机遇
Credit Assignment: Challenges and Opportunities in Developing Human-like AI Agents
AI总结：
本文基于 IBLT 构建含 IBL-TD 在内的序列决策模型，在不同复杂度导航任务中比较信用分配机制，发现等信用 IBL 最贴合人类表现。
摘要：
时间信用分配对于自然智能和人工智能中的学习与技能发展都至关重要。尽管强化学习中的 TD 方法等计算方法已经被提出，但它们是否准确表征了人类处理反馈延迟的方式仍不清楚。认知模型旨在表征人类解决问题并执行多项任务时所经历的心理步骤，但认知科学中针对人类及认知模型的信用分配问题开展的研究仍然有限。我们的研究使用一种基于经验决策理论——即基于实例的学习理论（Instance-Based Learning Theory，IBLT）——的认知模型，在具有不同决策复杂度水平的目标寻求导航任务中检验不同的信用分配机制。基于实例的学习（IBL）模型使用不同信用分配机制模拟做出序列选择的过程，其中包括一种新的 IBL-TD 模型，该模型将 IBL 决策机制与 TD 方法相结合。我们发现：（1）对所有决策给予同等信用分配的 IBL 模型能够比包括 IBL-TD 和 Q-learning 在内的其他模型更好地匹配人类表现；（2）IBL-TD 和 Q-learning 模型最初的表现不及人类，但最终会超过人类；（3）人类会受到决策复杂度的影响，而模型则不会。我们的研究为理解捕捉人类行为的挑战，以及在未来 AI 系统中使用这些模型支持人类活动的潜在机遇提供了洞见。
英文摘要：
Temporal credit assignment is crucial for learning and skill development in natural and artificial intelligence. While computational methods like the TD approach in reinforcement learning have been proposed, it's unclear if they accurately represent how humans handle feedback delays. Cognitive models intend to represent the mental steps by which humans solve problems and perform a number of tasks, but limited research in cognitive science has addressed the credit assignment problem in humans and cognitive models. Our research uses a cognitive model based on a theory of decisions from experience, Instance-Based Learning Theory (IBLT), to test different credit assignment mechanisms in a goal-seeking navigation task with varying levels of decision complexity. Instance-Based Learning (IBL) models simulate the process of making sequential choices with different credit assignment mechanisms, including a new IBL-TD model that combines the IBL decision mechanism with the TD approach. We found that (1) An IBL model that gives equal credit assignment to all decisions is able to match human performance better than other models, including IBL-TD and Q-learning; (2) IBL-TD and Q-learning models underperform compared to humans initially, but eventually, they outperform humans; (3) humans are influenced by decision complexity, while models are not. Our study provides insights into the challenges of capturing human behavior and the potential opportunities to use these models in future AI systems to support human activities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
