LogicalDefender:发现、提取和利用常识知识
LogicalDefender: Discovering, Extracting, and Utilizing Common-Sense Knowledge
AI总结:
针对文生图模型生成图像逻辑不合理的问题,提出LogicalDefender,结合图像与人类文本逻辑知识,提升模型逻辑学习与提取能力,实验验证其有效性和可迁移性。
摘要:
大型文生图模型在根据文本合成多样且高质量的图像方面已取得了惊人的性能。通过注重细节的条件控制,甚至可以实现更细粒度的空间控制。然而,即使生成的图像具有丰富的物体特征和和谐的风格,一些图像仍然显得不合理。在本文中,我们深入探讨了其根本原因,发现作为常识知识的深层逻辑信息在理解和处理图像中起着重要作用。尽管如此,几乎所有模型都忽视了图像中逻辑关系的重要性,导致在这方面的表现不佳。基于这一观察,我们提出了LogicalDefender,它将图像与人类已经总结在文本中的逻辑知识相结合。这鼓励模型更快、更好地学习逻辑知识,同时从图像和人类知识中提取广泛适用的逻辑知识。实验表明,我们的模型在逻辑性能上取得了更好的表现,并且提取的逻辑知识可以有效地应用于其他场景。
英文摘要:
Large text-to-image models have achieved astonishing performance in synthesizing diverse and high-quality images guided by texts. With detail-oriented conditioning control, even finer-grained spatial control can be achieved. However, some generated images still appear unreasonable, even with plentiful object features and a harmonious style. In this paper, we delve into the underlying causes and find that deep-level logical information, serving as common-sense knowledge, plays a significant role in understanding and processing images. Nonetheless, almost all models have neglected the importance of logical relations in images, resulting in poor performance in this aspect. Following this observation, we propose LogicalDefender, which combines images with the logical knowledge already summarized by humans in text. This encourages models to learn logical knowledge faster and better, and concurrently, extracts the widely applicable logical knowledge from both images and human knowledge. Experiments show that our model has achieved better logical performance, and the extracted logical knowledge can be effectively applied to other scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。