arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.15273cs.CYcs.CL
收藏

FrameRef: 一个用于建模有限理性信息健康的框架数据集和仿真测试平台

Stability Regimes for Framing-Sensitive Fine-Tuning in Language Models

University of Oklahoma(俄克拉荷马大学)

Victor De Lima, Jiqun Liu, Grace Hui Yang

AI总结:

FrameRef通过仿真框架研究有限理性信息健康的动态,提供系统性数据集和方法论,影响人类判断和信息健康轨迹。

摘要:

信息生态系统日益影响人们如何内化对不良数字体验的暴露,引发了对信息健康长期后果的担忧。在现代搜索和推荐系统中,排名和个性化政策在塑造这种暴露及其对用户长期影响方面起着核心作用。为了在受控环境中研究这些影响,我们提出了FrameRef,一个包含1,073,740个系统性重新表述声明的大规模数据集,涵盖五个框架维度:权威性、共识性、情绪性、 prestige性和煽动性,并提出了一种基于仿真的框架,用于建模序列信息暴露和强化动态,这些动态特征于排名和推荐系统。在此框架中,我们通过微调语言模型,使用框架条件化的损失衰减来构建对框架敏感的代理人物,诱导目标偏见的同时保持整体任务能力。使用蒙特卡洛轨迹采样,我们展示了小的、系统性的接受和信心变化会随时间累积,产生显著的累积信息健康轨迹的差异。人类评估进一步确认,FrameRef生成的框架对人类判断有可测量的影响。共同,我们的数据集和框架通过仿真为系统信息健康研究提供了基础,补充并 informing 负责任的人本研究。我们发布了FrameRef、代码、文档、人类评估数据和人物适配器模型,网址为https://github.com/infosenselab/frameref。

英文摘要:

AI systems increasingly shape how individuals access and evaluate information, intensifying concerns about misinformation. Computational simulation offers a controlled complement to direct empirical study, with fine-tuned LLMs enabling powerful agent-based simulations. However, fine-tuning often induces broad behavioral shifts including bias drift, degenerate heuristics, and changes in baseline task behavior, which undermine experimental validity. The central challenge is inducing localized decision shifts while preserving the model's aggregate behavior outside the targeted intervention. We study this through a stability analysis of supervised fine-tuning, attenuating loss on selected training slices to shift false-claim acceptance. We introduce FrameRef, a large-scale dataset of semantically equivalent claims with controlled surface framings across five dimensions (Authoritative, Consensus, Emotional, Sensationalist, Prestige), with human validation confirming systematic framing effects on claim acceptance. We present a methodology for identifying model-specific stability regimes in which targeted error shifts can be induced while preserving aggregate accuracy and calibration, and demonstrate its application to selected language models. A sequential exposure task over FrameRef shows that small framing-conditioned shifts compound into substantially different cumulative outcomes under feedback, but largely disappear when feedback is removed, confirming that interventions modify specific decision tendencies rather than global performance. We release code, LoRA adapters, and data at https://github.com/infosenselab/framing-sensitive-ft.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑