Large language models (LLMs) are now used for academic expert recommendation. Existing audits typically evaluate such recommendations in isolation, ignoring end-user inference-time interventions. Thus, it remains unclear whether failures (e.g., refusals, hallucinations, uneven coverage) stem from model choice or deployment decisions. We introduce LLMScholarBench, a benchmark for auditing LLM-based scholar recommendation that jointly evaluates model infrastructure and end-user interventions across multiple tasks. LLMScholarBench measures technical quality and social representation using nine metrics. We instantiate the benchmark in physics expert recommendation and audit 22 LLMs under temperature variation, representation-constrained prompting, and retrieval-augmented generation (RAG) via web search. Our results show that each intervention entails distinct tradeoffs. Higher temperature degrades validity, consistency, and factuality. Representation-constrained prompting improves diversity at the expense of factuality, while RAG primarily improves technical quality while reducing diversity and parity. Overall, end-user interventions reshape trade-offs rather than providing uniform gains. LLMScholarBench makes all these dynamics auditable across models and interventions in LLM-based scholar recommendations.
Auditing a KB Elicitation of Frontier LLM Knowledge: A Multi-dimensional Analysis of GPTKB v1.5
University of Tübingen, Germany（图宾根大学）；ScaDS.AI Dresden/Leipzig & TU Dresden, Germany（ScaDS.AI 德累斯顿/莱比希 & 德累斯顿技术大学）；VNU University of Engineering and Technology, Hanoi, Vietnam（越南河内工程与技术大学）
Auditing the Synthetic Memoir: Measuring Scene-Level Confabulation in LLM-Generated Autobiography Against the Documented Record of the Life It Describes
From Verification to Amplification: Auditing Reverse Image Search as Algorithmic Gatekeeping in Visual Misinformation Fact-checking
Tsinghua University（清华大学）；Georgia State University（佐治亚州立大学）；The Chinese University of Hong Kong（香港中文大学）；Northwestern University（西北大学）；University of Georgia（佐治亚大学）；University of California, Davis（加州大学戴维斯分校）
Named Entity Recognition in Historical Italian: The Case of Giacomo Leopardi's Zibaldone
Department of Humanities, University of Macerata（人文学院，马切拉塔大学）；Department of Political Sciences, Communication and International Relations, University of Macerata（政治科学、传播与国际关系学院，马切拉塔大学）
