找到恰当的平衡:对不同贡献类型下评估方法分布的系统性评估
Striking the Right Balance: Systematic Assessment of Evaluation Method Distribution Across Contribution Types
AI总结:
本研究针对信息可视化领域单篇论文多重评估的趋势,通过调研214篇IEEE TVCG VIS专题论文,分析不同贡献类型下的评估方法分布,为评估策略选择提供参考并推动相关讨论。
摘要:
在快速发展的信息可视化领域,严格的评估对于验证新技术、理解用户交互以及证明可视化的有效性和可用性至关重要。可靠的评估能为用户如何与系统交互、如何感知系统提供宝贵洞见,让设计者能够发现潜在缺陷,并就设计选择与改进做出明智决策。然而,单篇研究中采用多重评估的新兴趋势,引发了关于该方法的可持续性、可行性与方法严谨性的关键问题。受这一趋势影响,新晋研究者和学生可能会认为——无论研究的贡献类型是什么,都必须开展多重评估。但一项研究中的评估数量应当取决于其贡献与价值,而非为了强化论文而加入多重评估的潮流。那么,多少项评估才算足够?这是一个依情境而定的问题,无法通过公式确定。我们的目标是总结当前的趋势与模式,评估不同论文贡献类型下的评估方法分布情况。本文通过对2023年和2024年IEEE TVCG期刊近两年VIS专题的214篇论文中的评估模式进行非穷尽式文献调研,识别出了这一趋势。随后我们讨论了信息可视化领域的多种评估策略模式,以指导实际选择,并阐述本文将如何为进一步讨论开辟方向。
英文摘要:
In the rapidly evolving field of information visualization, rigorous evaluation is essential for validating new techniques, understanding user interactions, and demonstrating the effectiveness and usability of visualizations. Faithful evaluations provide valuable insights into how users interact with and perceive the system, enabling designers to identify potential weaknesses and make informed decisions about design choices and improvements. However, an emerging trend of multiple evaluations within a single research raises critical questions about the sustainability, feasibility, and methodological rigor of such an approach. New researchers and students, influenced by this trend, may believe -- multiple evaluations are necessary for a study, regardless of the contribution types. However, the number of evaluations in a study should depend on its contributions and merits, not on the trend of including multiple evaluations to strengthen a paper. So, how many evaluations are enough? This is a situational question and cannot be formulaically determined. Our objective is to summarize current trends and patterns to assess the distribution of evaluation methods over different paper contribution types. In this paper, we identify this trend through a non-exhaustive literature survey of evaluation patterns in 214 papers in the two most recent years' VIS issues in IEEE TVCG from 2023 and 2024. We then discuss various evaluation strategy patterns in the information visualization field to guide practical choices and how this paper will open avenues for further discussion.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。