大型语言模型驱动的特征发现提升视觉内容可信度感知的预测与解释
Large Language Model-Informed Feature Discovery Improves Prediction and Interpretation of Credibility Perceptions of Visual Content
AI总结：
针对视觉内容可信度感知预测与解释的挑战，提出LLM驱动的特征发现框架，利用GPT-4o等多模态LLM提取可解释特征并整合到模型中，在4191条样本上R2提升13%，揭示关键影响特征。
摘要：
在当今视觉主导的社交媒体环境中，预测视觉内容的感知可信度并理解驱动人类判断的因素对于对抗错误信息至关重要。然而，由于视觉特征的多样性和丰富性，这些任务具有挑战性。我们提出一种大型语言模型（LLM）驱动的特征发现框架，利用GPT-4o等多模态LLM评估内容可信度并解释其推理过程。我们通过目标提示提取和量化可解释特征，并将其整合到机器学习模型中以提升可信度预测。我们在科学、健康和政治领域8个主题的4191条视觉社交媒体帖子上测试了该方法，使用5355名众包工作者的可信度评分。我们的方法在R2指标上比零样本GPT预测高出13%，并揭示了信息具体性和图像格式等关键特征。我们讨论了其在错误信息缓解、视觉可信度以及LLM在社会科学中的作用方面的意义。
英文摘要：
In today's visually dominated social media landscape, predicting the perceived credibility of visual content and understanding what drives human judgment are crucial for countering misinformation. However, these tasks are challenging due to the diversity and richness of visual features. We introduce a Large Language Model (LLM)-informed feature discovery framework that leverages multimodal LLMs, such as GPT-4o, to evaluate content credibility and explain its reasoning. We extract and quantify interpretable features using targeted prompts and integrate them into machine learning models to improve credibility predictions. We tested this approach on 4,191 visual social media posts across eight topics in science, health, and politics, using credibility ratings from 5,355 crowdsourced workers. Our method outperformed zero-shot GPT-based predictions by 13 percent in R2, and revealed key features like information concreteness and image format. We discuss the implications for misinformation mitigation, visual credibility, and the role of LLMs in social science.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
