Medical large language models (LLMs), including custom medical GPTs (MedGPTs) and open-source models, are increasingly deployed on web platforms to provide clinical guidance. However, they pose risks of hallucination, policy noncompliance, and unsafe design. We conduct a large-scale assessment of 6,233 MedGPTs, evaluating a stratified sample of 1,500, together with 10 open-source LLMs. We introduce two frameworks: MedGPT-HEval for hallucination detection and an LLM-based pipeline for assessing policy violations and developer intent. Our results show that 25-30% of MedGPTs exhibit low factual accuracy, with bottom- and middle-tier models at highest risk; 33.6-54.3% violate operational thresholds, and 57.06% of Action-enabled models lack adequate privacy disclosures. Compared with open-source models, MedGPTs achieve higher factual accuracy and semantic alignment, though open-source models are more stable. These results reveal systemic gaps in hallucination and compliance, highlighting the need for multi-metric evaluation and stronger safeguards. We release HAA-MedGPT, a structured dataset that supports future research on the safety of web-facing medical LLMs.
They Said Memes Were Harmless-We Found the Ones That Hurt: Decoding Jokes, Symbols, and Cultural References
Macquarie University(麦考瑞大学);The University of Western Australia(西澳大学);Stanford University(斯坦福大学);Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(巴黎-罗quentcourt研究所);Rajiv Gandhi University(拉贾·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒尔研究实验室)
Representation Noising: A Defence Mechanism Against Harmful Finetuning
Dalhousie University(达尔豪斯大学);Vector Institute for Artificial Intelligence(矢量人工智能研究所);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Swarthmore College(斯沃斯莫尔学院);University of Warwick(华威大学);University of Toronto(多伦多大学);Vijil(维吉尔公司)
Assessing Language Model Deployment with Risk Cards
University of Washington(华盛顿大学);ITU Copenhagen(哥本哈根信息技术大学);University of Oxford(牛津大学);Carnegie Mellon University(卡内基梅隆大学);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);National Research Council Canada(加拿大国家研究委员会)