大型语言模型类比生成的多样性研究
On the Diversity of Analogy Making in Large Language Models
浏览论文内容
中文总结 AI 辅助
本研究评估10种开源及闭源LLM的类比生成多样性，发现其存在领域同质化问题，多样性增强方法存在多样性与质量的权衡，且不同LLM控制类比多样性的敏感区域有显著差异。
中文摘要 AI 辅助
大型语言模型（LLMs）已展现出类比生成的卓越潜力，类比生成是驱动新颖性与创造力的核心认知能力。尽管已有大量研究探讨了基于LLM的类比生成的应用及潜在机制，但其输出多样性仍未得到充分探索，而多样性对于拓宽跨领域关联、促进科学创新至关重要。本研究对10种最先进的开源及闭源LLM的类比多样性展开全面评估，研究发现了一个值得关注的领域同质化问题：LLMs倾向于从狭窄的目标领域生成类比，这既限制了查询间多样性，也限制了模型内部多样性。此外，分析还揭示了现有LLM多样性增强方法中存在的根本权衡：提高输出多样性往往以牺牲输出质量为代价。最后，对LLM信息流的因果分析显示，不同LLM中控制类比多样性的模型敏感区域存在显著差异，这为观察到的多样性-质量权衡提供了潜在机制。据作者所知，本研究是首批系统探究基于LLM的类比生成输出多样性的研究之一。
英文摘要
Large Language Models (LLMs) have demonstrated remarkable potential for analogy making, a core cognitive capability that drives novelty and creativity. While prior research has extensively investigated the applications and underlying mechanisms of LLM-based analogy making, its output diversity remains largely unexplored, despite being essential for broadening cross-domain connections and fostering scientific innovation. In this work, we present a comprehensive evaluation of analogy diversity across ten state-of-the-art open- and closed-source LLMs. Our findings highlight a concerning issue of domain homogeneity, a prevalent tendency for LLMs to generate analogies from a narrow set of target domains, limiting both inter-query and intra-model diversity. Furthermore, our analysis reveals a fundamental trade-off in existing LLM diversity-enhancement methods: increasing output diversity often comes at the expense of output quality. Finally, our causal analysis of LLM information flow reveals substantial differences in the model-sensitive regions governing analogy diversity across LLMs, suggesting a potential mechanism for the observed diversity-quality trade-off. To our knowledge, this is among the first studies to systematically investigate output diversity in LLM-based analogy making.
