嗨，Sheldon！从电视剧中创建深度个性化角色
Hi Sheldon! Creating Deep Personalized Characters from TV Shows
AI总结：
本文提出深度个性化角色创建任务DPCC，构建规模约10倍于既有数据集的DPCD多模态对话数据集，并给出基线以生成符合角色性格的文本、音频和视频回复。
摘要：
想象一个有趣的多模态交互场景：你可以看见、听见一个由AI生成的数字角色并与其聊天，该角色能够表现得像《生活大爆炸》中的Sheldon一样，成为从外貌到性格的深度复制。为实现这一奇妙的多模态聊天场景，我们提出了一项名为深度个性化角色创建（DPCC）的新任务：从电视剧等多模态数据中创建多模态聊天个性化角色。具体而言，给定单模态或多模态输入（文本、音频、视频），DPCC的目标是生成多模态（文本、音频、视频）回复，该回复应与Sheldon等特定角色的性格高度匹配，并且具有高质量。为支持这一新任务，我们进一步从电视剧中收集了一个以角色为中心的多模态对话数据集，名为深度个性化角色数据集（DPCD）。DPCD包含特定角色的多模态对话数据，每个角色约有1万条话语和约6小时的音频/视频，与现有相关数据集相比规模约大10倍。在DPCD上，我们提出了一种用于DPCC任务的基线方法，并从《生活大爆炸》电视剧中创建了5个深度个性化数字角色（DeepCharacters）。我们开展了主观和客观实验，从角色塑造和质量方面评估DeepCharacters的多模态回复。结果表明，在我们收集的DPCD数据集上，所提出的基线能够创建可生成多模态回复的个性化数字角色。我们收集的DPCD数据集、数据收集代码和基线将很快发布。
英文摘要：
Imagine an interesting multimodal interactive scenario that you can see, hear, and chat with an AI-generated digital character, who is capable of behaving like Sheldon from The Big Bang Theory, as a DEEP copy from appearance to personality. Towards this fantastic multimodal chatting scenario, we propose a novel task, named Deep Personalized Character Creation (DPCC): creating multimodal chat personalized characters from multimodal data such as TV shows. Specifically, given a single- or multi-modality input (text, audio, video), the goal of DPCC is to generate a multi-modality (text, audio, video) response, which should be well-matched the personality of a specific character such as Sheldon, and of high quality as well. To support this novel task, we further collect a character centric multimodal dialogue dataset, named Deep Personalized Character Dataset (DPCD), from TV shows. DPCD contains character-specific multimodal dialogue data of ~10k utterances and ~6 hours of audio/video per character, which is around 10 times larger compared to existing related datasets.On DPCD, we present a baseline method for the DPCC task and create 5 Deep personalized digital Characters (DeepCharacters) from Big Bang TV Shows. We conduct both subjective and objective experiments to evaluate the multimodal response from DeepCharacters in terms of characterization and quality. The results demonstrates that, on our collected DPCD dataset, the proposed baseline can create personalized digital characters for generating multimodal response.Our collected DPCD dataset, the code of data collection and our baseline will be published soon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
