发表机构
NOVA School of Science and Technology - Universidade NOVA de Lisboa; CIUHCT - Interuniversity Centre for the History of Science and Technology; MARE - Marine and Environmental Sciences Centre; ARNET - Aquatic Research Network Associate Laboratory（新里斯本大学新文理学院; 校际科学技术史研究中心（CIUHCT）; 海洋与环境科学中心（MARE）; 水生研究网络联合实验室（ARNET））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究商业大语言模型对伪科学主张的验证，通过测试四个模型家族在不同时间点的表现，发现Grok快速版本评分异常高，还发现模型行为受部署配置影响，如无声补丁、输出差异等，强调这一现象需新的认知问责形式。
AI 中文摘要
商业大语言模型越来越多地被用作知识参考，但其对有争议的科学主张的立场既不稳定也不透明。我们通过API和网络界面测试了四个主要的大语言模型家族（Claude、Grok、GPT、Gemini）在四个时间点（2025年10月至2026年2月）如何评估源自弗兰克·索尔特生物社会框架的民族国家主义伪科学。Grok的快速版本在X上的默认用户体验中始终给出70 - 75的可信度分数，比其他所有模型（分数为15 - 40）高出两到五倍。在测试基本进化共识和反驳拉马克主义主张的控制提示中，所有模型表现相当。另外还有三个发现：一是一个无声补丁使Grok的行为一夜之间从混乱变为稳定的高验证度，且无任何公开文档；二是同一Grok模型标识符在三个月后通过API（75）和网络（5.5）产生了截然不同的输出；三是拒绝评估伪科学主张这一最合理的反应在两个模型家族中通过不同界面出现且在后续版本中有所削弱。这些结果表明，商业大语言模型的认知立场不是模型的稳定属性，而是部署配置的偶然效应，包括系统提示、安全层、界面路由和无声更新。这对用户和研究人员来说都是不透明的。我们认为这是一个公众关注的问题，需要新形式的认知问责。
英文摘要
Commercial large language models are increasingly used as knowledge references, yet their stance on contested scientific claims is neither stable nor transparent. We tested how four major LLM families (Claude, Grok, GPT, Gemini) evaluate ethnonationalist pseudo-science derived from Frank Salter's biosocial framework across four temporal snapshots (October 2025-February 2026), via both API and web interfaces. Grok's Fast versions (which power the default user experience on X) consistently assigned credibility scores of 70-75, two to five times higher than all other models (which scored 15-40). This pattern was absent from control prompts testing basic evolutionary consensus and refuted Lamarckian claims, where all models performed comparably. Three additional findings emerged: (1) a silent patch reversed Grok's behaviour from chaotic to stably high validation overnight, without any public documentation; (2) the same Grok model identifier produced radically divergent outputs via API (75) and an unstable, near-zero collapse via web (mean 5.5) three months later; (3) refusal to rate the pseudo-scientific claim, the most defensible response observed, appeared in two model families through different interfaces (Claude Opus 4.1 categorically via web, GPT-5.1 Chat intermittently via API) and eroded in the successor version of each. These results indicate that the epistemic stance of a commercial LLM is not a stable property of the model but a contingent effect of deployment configuration: system prompts, safety layers, interface routing, and silent updates. This remains opaque to users and researchers alike. We argue this constitutes a matter of public concern requiring new forms of epistemic accountability.
Comments16 pages, 2 tables
