发表机构
LoveMind AI（LoveMind AI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
CHOIR 框架利用自由列表启发法将 LLM 集成的开放式同质性转化为诊断测量，通过排序列表聚类和显著性分析，识别模型身份等可恢复特征并区分提示深度。
AI 中文摘要
开放式 LLM 的同质性可能造成虚假的多样性，即多个系统看似提供独立视角，实则返回相同的常见默认答案。单次回答掩盖了由高度受限的答案空间、提示词词汇回响以及表面之下存在稳定替代方案的更广泛答案空间所产生的共识之间的区别。我们引入了 CHOIR（集体分层有序查询响应），这是一个将认知人类学中的自由列表启发法适配到 LLM 集成中的框架。CHOIR 反复启发排序列表，将条目聚类为提示级概念，并衡量概念在不同模型、提示变体和角色条件下的显著性。我们在 Infinity-Chat 100（一个来自近期关于开放式模型同质性研究的、外部提示库）以及一个旨在隔离机制级对比的、包含 27 个问题的定向诊断库上评估了 CHOIR。在 Infinity-Chat 100 上，CHOIR 重现了高表面一致性（100 个提示中有 93 个高于随机水平），同时将狭窄提示与具有可恢复深度的广泛提示区分开来。在定向探针和外部提示库中，基础模型身份仍然是最强的可恢复特征，而角色提示会在基础模型特征内转移表面化的概念。一个来源盲排序模块优先考虑罕见但稳定的候选以供后续检查。CHOIR 通过询问模型在哪里趋同、为何趋同以及在结构化深度探测下仍可触及什么，将开放式同质性转化为一个诊断性测量问题。
英文摘要
Open-ended LLM homogeneity can create false plurality when several systems appear to offer independent perspectives while returning the same familiar default. Single-pass answers obscure the distinction between agreement produced by a tightly constrained answer space, prompt-vocabulary echo, and broader answer spaces with stable alternatives beneath the surface. We introduce CHOIR (Collective Hierarchically-Ordered Inquiry Responses), a framework that adapts free-list elicitation from cognitive anthropology to LLM ensembles. CHOIR repeatedly elicits ranked lists, clusters items into prompt-level concepts, and measures concept salience across models, prompt variants, and persona conditions. We evaluate CHOIR on Infinity-Chat 100, an external prompt bank from recent work on open-ended model homogeneity, and on a 27-question targeted diagnostic bank designed to isolate mechanism-level contrasts. On Infinity-Chat 100, CHOIR reproduces high surface agreement (93/100 prompts above chance) while separating narrow prompts from broad prompts with recoverable depth. Across targeted probes and the external prompt bank, base-model identity remains the strongest recoverable signature, and persona prompts shift surfaced concepts within base-model signatures. A source-blind ranking module prioritises rare-but-stable candidates for later inspection. CHOIR turns open-ended homogeneity into a diagnostic measurement problem by asking where models converge, why they converge, and what remains reachable under structured depth probing.
Comments23 pages, 6 figures. Published in the Proceedings of the Third Conference on Language Modeling (COLM 2026)
Journal refProceedings of the Third Conference on Language Modeling (COLM 2026), 2026
