FaceGemma：利用面部属性增强人像图像的图像描述生成能力
FaceGemma: Enhancing Image Captioning with Facial Attributes for Portrait Images
AI总结：
本研究提出FaceGemma模型，通过结合面部标注属性、人工标注描述与提示工程微调PaliGemma，构建匹配属性的人像描述数据集，显著提升了人像图像描述的质量与准确性。
摘要：
自动化图像描述生成对于提升视觉内容的可访问性与理解度至关重要。本研究提出了FaceGemma模型，该模型可准确描述情绪、表情、面部特征等面部属性。我们基于FaceAttdb数据，使用Llama 3-70B模型为2000张人脸生成描述文本，再用这些描述对PaliGemma模型进行微调。基于FaceAttDB提供的属性与描述，我们构建了一个全新的描述数据集，其中每条描述都能完美匹配人工标注的属性，涵盖吸引力、厚嘴唇、大鼻子、金发、棕发、浓眉、眼镜、男性、微笑、年轻等关键特征。这种精细化方法确保生成的描述与图像中细微的视觉细节高度契合。FaceGemma模型采用创新的图像描述思路，结合标注属性、人工标注描述与提示工程技术，生成高质量的面部描述。我们的方法显著提升了描述质量，平均BLEU-1得分达0.364，METEOR得分达0.355。这些指标证明了将面部属性融入图像描述的有效性，可为人像图像生成更准确、更具描述性的字幕。
英文摘要：
Automated image caption generation is essential for improving the accessibility and understanding of visual content. In this study, we introduce FaceGemma, a model that accurately describes facial attributes such as emotions, expressions, and features. Using FaceAttdb data, we generated descriptions for 2000 faces with the Llama 3 - 70B model and fine-tuned the PaliGemma model with these descriptions. Based on the attributes and captions supplied in FaceAttDB, we created a new description dataset where each description perfectly depicts the human-annotated attributes, including key features like attractiveness, full lips, big nose, blond hair, brown hair, bushy eyebrows, eyeglasses, male, smile, and youth. This detailed approach ensures that the generated descriptions are closely aligned with the nuanced visual details present in the images. Our FaceGemma model leverages an innovative approach to image captioning by using annotated attributes, human-annotated captions, and prompt engineering to produce high-quality facial descriptions. Our method significantly improved caption quality, achieving an average BLEU-1 score of 0.364 and a METEOR score of 0.355. These metrics demonstrate the effectiveness of incorporating facial attributes into image captioning, providing more accurate and descriptive captions for portrait images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
