如何用刀剥皮：将细粒度操作与人类偏好对齐
How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference
AI总结：
本文提出了一种通过剥皮任务学习细粒度操作与人类偏好的对齐方法，通过两阶段策略优化实现高成功率和泛化能力。
摘要：
许多关键的操作任务 - 如食品准备、手术和手工艺 - 仍然难以让自主机器人完成。这些任务不仅具有接触丰富、力敏感的动力学特性，还具有
英文摘要：
Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain intractable for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward engineering difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human notions of task quality. Using only 50-200 peeling trajectories, our system achieves over 90% average success rates on challenging produce including cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on a single produce category exhibit strong zero-shot generalization to unseen in-category instances and to out-of-distribution produce from different categories while maintaining over 90% success rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
