AI 中文总结
本研究通过混合方法调研13个 venues 的征稿通知、2016-2025年的24篇复现论文及作者调查,得出可用安全与隐私领域复现研究的四项关键发现,并提出强化复现实践的建议。
AI 中文摘要
可用安全与隐私研究是一个新兴且不断扩张的领域,目前仍在完善其研究标准,例如关于复现研究的相关规范。本研究采用混合方法,以更好地了解该领域复现研究的现状:(1)我们调研了涵盖安全、隐私及人机交互领域的13个 venues 的论文征稿通知;(2)我们系统检索了2016至2025年间在这些 venues 发表的、报告复现用户研究的论文,共得到24篇相关出版物;(3)我们采用Olszewski等人(2025)提出的复现分类法对这24篇论文进行分类;(4)我们向这些论文的作者发放了调查,以了解他们开展复现研究的动机。我们的分析得出四项关键见解:(A)论文征稿通知若能为作者和审稿人提供关于复现工作的更明确指南,将更具价值;(B)阅读复现论文时,难以确定其相对于原始研究做出了哪些修改;(C)我们的样本中不存在严格的精确复现,24项研究中约三分之二对原始研究的多个方面进行了修改;(D)影响结果的时间与情境变化是被最频繁提及的复现研究动机之一。基于这些发现,我们为 venues、研究人员及同行审稿人提供了实用建议,以强化可用安全与隐私研究中的复现实践。
英文摘要
The field of usable security and privacy research is a young and expanding field, which is still developing standards for its research, e.g. regarding replications. We used a mixed-method approach, in order to get a better understanding of the current state of replications in the field of usable security and privacy: (1) we examine the Call for Papers of 13 venues spanning security, privacy, and human-computer interaction; (2) we conduct a systematic search for papers reporting replicated user studies published across these venues between 2016 and 2025, yielding 24 relevant publications; (3) we categorized these 24 papers employing the replication taxonomy proposed by Olszewski et al. (2025); (4) we distributed a survey to the authors of these papers to understand their motivations for conducting replications. Our analysis reveals four key insights: (A) Calls for Papers would benefit from clearer guidelines for authors and reviewers regarding replication work; (B) determining what modifications were made relative to the original study proves difficult when reading replication papers; (C) strict exact replications do not exist in our sample. Approximately two-thirds of the 24 studies altered multiple aspects of the original work; (D) temporal and contextual changes affecting results emerged as one of the most frequently cited motivations for replication. Based on these findings, we offer practical recommendations for venues, researchers, and peer reviewers to strengthen replication practices in usable security and privacy research.