Learned, Lagged, LLM-splained：大语言模型对终端用户安全问题的应答
Learned, Lagged, LLM-splained: LLM Responses to End User Security Questions
AI总结：
该研究通过定性评估3款主流大语言模型在900个终端用户安全问题上的表现，发现其存在答案过时失准、应答不当等局限，据此提出模型改进方向与用户使用策略。
摘要：
回答终端用户的安全问题颇具挑战性。尽管GPT、LLAMA、Gemini等大语言模型（LLM）远非零错误，但它们在安全领域之外的各类问题应答中已展现出应用潜力。我们通过在900个系统收集的终端用户安全问题上定性评估3款主流LLM，研究了其在终端用户安全领域的表现。虽然LLM具备终端用户安全信息的广泛通用“知识”，但各模型均存在模式化的错误与局限，包括答案过时失准、沟通风格间接或答非所问，这些都会影响用户获取信息的质量。基于这些模式，我们提出了模型改进方向，并为用户在寻求安全帮助时与LLM交互提供了策略建议。
英文摘要：
Answering end user security questions is challenging. While large language models (LLMs) like GPT, LLAMA, and Gemini are far from error-free, they have shown promise in answering a variety of questions outside of security. We studied LLM performance in the area of end user security by qualitatively evaluating 3 popular LLMs on 900 systematically collected end user security questions. While LLMs demonstrate broad generalist ``knowledge'' of end user security information, there are patterns of errors and limitations across LLMs consisting of stale and inaccurate answers, and indirect or unresponsive communication styles, all of which impacts the quality of information received. Based on these patterns, we suggest directions for model improvement and recommend user strategies for interacting with LLMs when seeking assistance with security.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
