arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.22752cs.CLcs.AI
收藏

向LLMs模拟社交媒体用户迈进:评估条件评论预测的运作有效性

Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Prediction

Trier University(特里尔大学);University of Luxembourg(卢森堡大学)

Nils Schwager, Simon Münker, Alistair Plum, Achim Rettinger

AI总结:

本文通过条件评论预测任务评估LLMs在模拟社交媒体用户行为方面的运作有效性,发现监督微调在低资源环境下会导致表层结构与语义脱节,强调真实行为轨迹优于描述性人设。

摘要:

大型语言模型(LLMs)从探索工具转变为社会科学中的活跃

英文摘要:

The transition of Large Language Models (LLMs) from exploratory tools to active "silicon subjects" in social science lacks extensive validation of operational validity. This study introduces Conditioned Comment Prediction (CCP), a task in which a model predicts how a user would comment on a given stimulus by comparing generated outputs with authentic digital traces. This framework enables a rigorous evaluation of current LLM capabilities with respect to the simulation of social media user behavior. We evaluated open-weight 8B models (Llama3.1, Qwen3, Ministral) in English, German, and Luxembourgish language scenarios. By systematically comparing prompting strategies (explicit vs. implicit) and the impact of Supervised Fine-Tuning (SFT), we identify a critical form vs. content decoupling in low-resource settings: while SFT aligns the surface structure of the text output (length and syntax), it degrades semantic grounding. Furthermore, we demonstrate that explicit conditioning (generated biographies) becomes redundant under fine-tuning, as models successfully perform latent inference directly from behavioral histories. Our findings challenge current "naive prompting" paradigms and offer operational guidelines prioritizing authentic behavioral traces over descriptive personas for high-fidelity simulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑