arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.08049cs.HCcs.AIcs.LG
收藏

TutoAI:面向物理任务的人工智能辅助混合媒体教程创建的跨领域框架

TutoAI: A Cross-domain Framework for AI-assisted Mixed-media Tutorial Creation on Physical Tasks

University of Maryland(马里兰大学);Adobe Research(奥多比研究院)

Yuexi Chen, Vlad I. Morariu, Anh Truong, Zhicheng Liu

AI总结:

TutoAI提出跨领域框架,通过提炼教程组件、组装评估AI模型并设计UI指南,实现物理任务混合媒体教程的AI辅助创建,初步用户研究中质量优于或持平基线。

摘要:

混合媒体教程整合了视频、图像、文本和图表来教授程序性技能,与基于时间轴的视频相比,提供了更具可浏览性的替代方案。然而,手动创建此类教程十分繁琐,现有的自动化解决方案往往局限于特定领域。尽管人工智能模型前景广阔,但鉴于所涉及的多模态数据和庞大的模型格局,如何有效利用其能力尚不明确。我们提出了TutoAI,一个面向物理任务的人工智能辅助混合媒体教程创建的跨领域框架。首先,我们通过调研现有工作来提炼常见的教程组件;然后,我们提出了一种识别、组装和评估用于组件提取的人工智能模型的方法;最后,我们提出了设计支持基于人工智能生成组件的教程创建的用户界面(UI)的指南。我们表明,在初步用户研究中,TutoAI与基线模型相比,实现了更高或相当的质量。

英文摘要:

Mixed-media tutorials, which integrate videos, images, text, and diagrams to teach procedural skills, offer more browsable alternatives than timeline-based videos. However, manually creating such tutorials is tedious, and existing automated solutions are often restricted to a particular domain. While AI models hold promise, it is unclear how to effectively harness their powers, given the multi-modal data involved and the vast landscape of models. We present TutoAI, a cross-domain framework for AI-assisted mixed-media tutorial creation on physical tasks. First, we distill common tutorial components by surveying existing work; then, we present an approach to identify, assemble, and evaluate AI models for component extraction; finally, we propose guidelines for designing user interfaces (UI) that support tutorial creation based on AI-generated components. We show that TutoAI has achieved higher or similar quality compared to a baseline model in preliminary user studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑