AI 中文总结
本文提出声明校准方法，用于明确复现研究所支持的原始声明范围，并通过推荐系统五对论文案例展示其应用，最终提出声明证据档案以系统化报告复现结果。
AI 中文摘要
复现研究可能产生混合的结果。报告的值可能不同，而比较方法的排序保持不变；一个结果可能仅在特定实验条件下成立；或者发布的实现可能无法复现模型仍能达到的结果。术语可重复性、可再现性和可复制性描述了后续研究相对于原始实验的关系，但并未指明新结果支持原始声明的哪些部分。我们引入“声明校准”作为一种方法，用于陈述后续研究所支持的最强声明，以及该声明成立的条件和尚未测试的部分。我们将这一视角应用于推荐系统研究中的五对原始-后续论文。这些案例表明，数值、方法排名、统计结果和总体结论的一致性并不总是重合，后续研究往往仅支持原始声明的一部分。基于这些观察，我们提出了一种声明证据档案，用于报告原始声明、其范围、复现目标、报告的结果、校准后的声明以及原始声明中仍未解决的部分。
英文摘要
Reproduction studies can produce mixed outcomes. Reported values may differ while the ordering of the compared methods remains the same, a result may hold only under some experimental conditions, or a released implementation may fail to reproduce a result that the model can still reach. The terms repeatability, reproducibility, and replicability describe how a follow-up study relates to the original experiment, but not which parts of the original claim are supported by the new results. We introduce \emph{claim calibration} as a way of stating the strongest claim supported by a follow-up study, together with the conditions under which it holds and the parts that remain untested. We apply this perspective to five original--follow-up paper pairs from recommender-systems research. The cases show that agreement in numerical values, method rankings, statistical results, and overall conclusions does not always coincide, and that follow-up studies often support only part of the original claim. Based on these observations, we propose a Claim Evidence Profile for reporting the original claim, its scope, the reproduction target, the reported results, the calibrated claim, and the parts of the original claim that remain unresolved.
CommentsAccepted to the Workshop Methodology First - Rethinking Research Assessment in RecSys (FRAME) September 28, 2026, Minneapolis, Minnesota, USA
