Large Language Models (LLMs) are highly capable but require significant computational resources for both training and inference. Within the LLM family, smaller models (those with fewer than 10 billion parameters) also perform well across various tasks. However, these smaller models share similar limitations to their larger counterparts, including the tendency to hallucinate. Despite the existence of many benchmarks to evaluate hallucination in LLMs, few have specifically focused on small LLMs (SLLMs). Additionally, SLLMs show widely varying performance across different benchmarks. In this paper, we introduce OnionEval, a multi-layer structured framework with a specific metric called the context-influence score (CI), designed to effectively assess the fact-conflicting hallucination tendencies of small LLMs across different contextual levels. Our experimental results reveal a key feature of SLLMs: they excel in factual analysis but face challenges with context reasoning. Further investigation shows that a simple Chain-of-Thought strategy can significantly reduce these limitations, improving the practical usefulness of SLLMs in real-world applications.
Detecting LLM Fact-conflicting Hallucinations Enhanced by Temporal-logic-based Reasoning
Huazhong University of Science and Technology（华中科技大学）；National University of Singapore（新加坡国立大学）；University of New South Wales（新南威尔士大学）；Nanyang Technological University（南洋理工大学）
MALM: A Multi-Information Adapter for Large Language Models to Mitigate Hallucination
Beijing Institute of Technology（北京理工大学）；The Open University（开放大学）；Beijing Information Science and Technology University（北京信息科技大学）；The Hong Kong Polytechnic University（香港理工大学）
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；ZJU-Ant Group Joint Research Center for Knowledge Graphs, Zhejiang University（浙江大学-蚂蚁集团知识图谱联合研究中心）；School of Software Technology, Zhejiang University（浙江大学软件学院）；Alibaba Group（阿里巴巴集团）
