Evaluating Reliability Asymmetries in Chinese Factual Search and AI Answers
Department of Electronics, Information and Bioengineering, Politecnico di Milano(电子、信息与生物工程系,米兰理工学院);University of Science and Technology of China(中国科学技术大学)
Search engines and AI-powered systems increasingly mediate access to factual information, yet their reliability remains difficult to evaluate in realistic information-seeking settings. We study this problem in the Chinese web ecosystem by constructing a query-based fact-checking dataset from real Chinese search logs and comparing nine systems across traditional search engines, standalone large language models, and search-integrated AI Overviews. Focusing on factual Chinese-language factual Yes/No questions, we evaluate whether systems provide correct, incorrect, or uncertain decisions against evidence-derived ground truth. We find that systems are similarly accurate when they provide definitive answers, but differ sharply in how often they do so. Conditional accuracy ranges from 73.2% to 78.9%, yet search engines answer definitively on over 83% of queries, while Qwen-Max does so on fewer than half. We also find a consistent polarity gap: all systems perform better on yes-labeled queries than on no-labeled queries. We also use Baidu Index data to identify Chinese provinces with higher health-related search attention, which may indicate greater potential exposure to misinformation. Overall, our results show that reliability depends not only on whether systems are correct when they answer, but also on how often they answer, how they handle negative claims, and where information demand may increase exposure risks.
A Large-Scale Chinese Knowledge Graph-Text Alignment Dataset for Benchmarking Knowledge-Grounded LLMs
Nanjing University of Science and Techonolgy(南京理工大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);University of Science and Technology Beijing(北京科技大学);Hefei University of Technology(合肥工业大学);Beijing University of Chemical Technology(北京化工大学);Renmin University of China(中国人民大学);Beihang University(北航);Beijing University of Posts and Telecommunications(北京邮电大学);Griffith University, Australia(澳大利亚格里菲斯大学)
Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups
Department of Electronics, Information and Bioengineering, Politecnico di Milano(电子、信息与生物工程系,米兰理工大学);University of Science and Technology of China(中国科学技术大学)
AsdKB: A Chinese Knowledge Base for the Early Screening and Diagnosis of Autism Spectrum Disorder
Southeast University(东南大学);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education(教育部新一代人工智能技术及其交叉应用重点实验室(东南大学));Hangzhou Dianzi University(杭州电子科技大学);The First Affiliated Hospital of Nanjing Medical University(南京医科大学第一附属医院);Nanjing Medical University(南京医科大学)
Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilization
Dept. of Comp. Sci. and Tech.(计算机科学与技术系);Institute for AI(人工智能研究院);Tsinghua-Bosch Joint ML Center(清华大学-博世联合机器学习中心);THBI Lab(THBI实验室);BNRist Center(BNRist中心);Tsinghua University(清华大学)
CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checking
School of Cyber Science and Engineering, Sichuan University, China(四川大学网络科学与工程学院);College of Computer Science, Sichuan University, China(四川大学计算机学院);Institute of Data Science, National University of Singapore, Singapore(新加坡国立大学数据科学研究所)
Quantifying Self-diagnostic Atomic Knowledge in Chinese Medical Foundation Model: A Computational Analysis
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Shenzhen Research Institute of Big Data(深圳大数据研究院);University of Science and Technology of China(中国科学技术大学)
Leveraging Group Relative Policy Optimization to Advance Large Language Models in Traditional Chinese Medicine
University of Missouri(密苏里大学);Christopher S. Bond Life Sciences Center, University of Missouri(密苏里大学克里斯托弗·S·邦德生命科学中心);Community Health Service Center Shanghai Pudong New Area(上海市浦东新区社区卫生服务中心);Shanghai University of Traditional Chinese Medicine(上海中医药大学)
MTCMB: A Multi-Task Benchmark Framework for Evaluating LLMs on Knowledge, Reasoning, and Safety in Traditional Chinese Medicine
School of Software Engineering, Sun Yat-sen University(中山大学软件学院);Institute of TCM Diagnostics, Hunan University of Chinese Medicine(湖南中医药大学中医诊断研究所);School of Artificial Intelligence, Guilin University of Electronic Technology(桂林电子科技大学人工智能学院);Department of Computer Science, Cornell University(康奈尔大学计算机科学系)
Universitat Politècnica de València(瓦伦西亚理工大学);University of Trento(特伦托大学);University of Sheffield(谢菲尔德大学);Bonn-Aachen International Center for Information Technology, University of Bonn(波恩-亚琛信息科技国际中心,波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);University of Pisa(比萨大学);University of Milano-Bicocca(米兰-比可卡大学);ValgrAI Valencian Graduate School and Research Network of Artificial Intelligence(瓦伦西亚人工智能研究生学校和研究网络)
One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Guanghua Law School, Zhejiang University(浙江大学光华法学院);School of Software Technology, Zhejiang University(浙江大学软件学院);Law & AI Lab, Zhejiang University(浙江大学法律与人工智能实验室);Shanghai AI Laboratory(上海人工智能实验室)