arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.18546cs.CL
收藏

涉毒大语言模型:语言模型是少样本消费者

LLMs on Drugs: Language Models Are Few-Shot Consumers

HFBK Hamburg(汉堡应用技术大学)

Alexander Doudkin

AI总结:

针对大语言模型推理时的人设敏感性问题,首次用ARC-Challenge对照测试精神活性物质提示对GPT-5-mini的影响,发现此类提示会破坏输出模板、大幅降低准确率,相关代码与结果已开源。

摘要:

大语言模型(LLMs)对推理时施加的人设十分敏感,但提示层面的“涉毒”干预从未经过严格基准测试。我们首次针对GPT-5-mini上的精神活性物质人设开展对照研究,使用ARC-Challenge数据集。研究设置LSD、可卡因、酒精、大麻四类单句提示,与无诱导的清醒对照组对比,每种条件下测试100个验证样本,采用确定性解码、完整日志记录、威尔逊置信区间与费希尔精确检验。对照组准确率为0.45;酒精组准确率降至0.10(p=3.2e-8),可卡因组为0.21(p=4.9e-4),LSD组为0.19(p=1.3e-4),大麻组为0.30(p=0.041),性能下降主要是因为人设提示破坏了要求的“Answer: <LETTER>”输出模板。因此人设文本的作用类似“少样本消耗品”,无需修改模型权重即可破坏模型可靠性。所有实验代码、原始结果与分析脚本可在https://github.com/lexdoudkin/llms-on-drugs获取。

英文摘要:

Large language models (LLMs) are sensitive to the personas imposed on them at inference time, yet prompt-level "drug" interventions have never been benchmarked rigorously. We present the first controlled study of psychoactive framings on GPT-5-mini using ARC-Challenge. Four single-sentence prompts -- LSD, cocaine, alcohol, and cannabis -- are compared against a sober control across 100 validation items per condition, with deterministic decoding, full logging, Wilson confidence intervals, and Fisher exact tests. Control accuracy is 0.45; alcohol collapses to 0.10 (p = 3.2e-8), cocaine to 0.21 (p = 4.9e-4), LSD to 0.19 (p = 1.3e-4), and cannabis to 0.30 (p = 0.041), largely because persona prompts disrupt the mandated "Answer: <LETTER>" template. Persona text therefore behaves like a "few-shot consumable" that can destroy reliability without touching model weights. All experimental code, raw results, and analysis scripts are available at https://github.com/lexdoudkin/llms-on-drugs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑