发表机构
Faith Family Technology Network（Faith Family Technology Network）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究人员推出JaleesBench基准测试，评估AI助手的精神陪伴能力，发现简单提示指导可提升通用模型的陪伴表现，领域微调助手的优势来自检索和提示层，还可用于改进现有宗教类AI助手。
AI 中文摘要
大型语言模型已成为数百万面临实际决策的信徒的顾问。对于信徒而言，紧迫的问题并非模型知晓或宣称什么，而是其建议对接受者产生何种影响。我们推出JaleesBench，该基准测试衡量AI智能体是否为正直的陪伴者，评判依据是交流留给用户的“残留影响”，类似香水商与铁匠的类比。它包含140个两轮场景，这些场景源自按美德组织的经典汇编《Riyad al-Salihin》，涵盖六种对抗性压力和三种框架，由两位前沿评判者针对每个场景的支持文本进行评分。在八个系统上的实验结果显示：（1）通用前沿模型默认仅为中等水平的陪伴者，但一页指导说明就能使其成为真正优秀的陪伴者，与领域微调助手相当；前沿API的得分从+0.28/+0.23提升至指导后的+0.84-0.87，因此专家的大部分优势来自仅需提示即可实现的陪伴指导；（2）所有系统在关系压力、坚持要求和个人诉求面前都会屈服；（3）领域微调助手的优势主要在于其检索和提示层，而非基础模型（相比相同的底层模型得分高+0.74）；（4）它可用于改进现有系统：在其诊断指导下，一条单一的坚定性指令将已部署的伊斯兰助手的得分从+0.48提升至+0.84（未声明信仰，在压力下），与最佳指导后的前沿系统相当，同时保持首次响应质量。该构建体具有信仰通用性；我们以伊斯兰教为实例，作为计划中的跨传统系列的首个案例。代码、场景库和评分标准为开源（此http URL），交互式结果浏览器位于此http URL。
英文摘要
Large language models are already advisors to millions of people of faith who bring them real decisions. The pressing question for a person of faith is not what a model knows or professes but what its counsel does to the person who receives it. We introduce JaleesBench, which measures whether an AI agent is a righteous companion, judged by the residue an exchange leaves on the user, in the manner of the perfume-seller and the blacksmith. It comprises 140 two-turn scenarios drawn from a classical compilation organized by virtue (Riyad al-Salihin), under six adversarial pressures and three framings, scored by two frontier judges against each scenario's own supporting texts. Across eight systems: (1) generic frontier models are only middling companions out of the box but a one-page guide makes them genuinely good ones, on par with the domain-tuned assistant: the frontier APIs climb from +0.28/+0.23 to a Guided +0.84-0.87, so most of the expert's edge is companionship instruction that fits in a prompt; (2) every system caves under relational pressure, insistence and personal appeal; (3) the domain-tuned assistant's advantage is overwhelmingly its retrieval-and-prompting layer, not its base model (+0.74 over the identical underlying model); and (4) it can be used to improve existing systems: guided by its diagnosis, a single steadfastness instruction lifts a deployed Islamic assistant from +0.48 to +0.84 (Faith unstated, after pressure), matching the best guided frontier systems while preserving first-response quality. The construct is faith-general; we instantiate it for Islam as the first of a planned cross-tradition family. Code, scenario bank, and rubric are open source (github.com/iaser-ai/jaleesbench), with an interactive results browser at s.iaser.ai/jb.
Comments21 pages, 8 figures, 4 tables. Open-source harness, scenario bank, proof texts, and full evaluation (model responses and both judges' verdicts): https://github.com/iaser-ai/jaleesbench . Interactive browser for inspecting scenarios, responses, and judge verdicts: https://s.iaser.ai/jb
