PhotoChat:一个具有照片共享行为的人-人对话数据集及其图文联合建模
PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling
摘要:
我们提出了一个新的人与人之间的对话数据集- PhotoChat,第一个数据集,投光的照片共享行为在线电子邮件。PhotoChat包含12 k个对话,每个对话都与对话期间共享的用户照片配对。基于此数据集,我们提出了两个任务,以促进图像-文本建模的研究:照片共享意图预测任务,预测是否打算在下一个会话回合中共享照片,以及照片检索任务,根据对话上下文检索最相关的照片。此外,对于这两个任务,我们使用最先进的模型提供基线模型,并报告其基准性能。最好的图像检索模型达到了10.4%的recall@1(在1000个候选者中),最好的照片意图预测模型达到了58.1%的F1分数,这表明数据集提出了有趣但具有挑战性的现实问题。我们正在发布PhotoChat,以促进社区未来的研究工作。
英文摘要:
We present a new human-human dialogue dataset - PhotoChat, the first dataset that casts light on the photo sharing behavior in onlin emessaging. PhotoChat contains 12k dialogues, each of which is paired with a user photo that is shared during the conversation. Based on this dataset, we propose two tasks to facilitate research on image-text modeling: a photo-sharing intent prediction task that predicts whether one intends to share a photo in the next conversation turn, and a photo retrieval task that retrieves the most relevant photo according to the dialogue context. In addition, for both tasks, we provide baseline models using the state-of-the-art models and report their benchmark performances. The best image retrieval model achieves 10.4% recall@1 (out of 1000 candidates) and the best photo intent prediction model achieves 58.1% F1 score, indicating that the dataset presents interesting yet challenging real-world problems. We are releasing PhotoChat to facilitate future research work among the community.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。