Angler：帮助机器翻译实践者确定模型改进优先级
Angler: Helping Machine Translation Practitioners Prioritize Model Improvements
AI总结：
本文提出交互式可视化分析工具Angler，通过定量统计与定性句子分析帮助机器翻译实践者在无限输入空间和昂贵质量信号条件下更有效地确定模型改进优先级。
摘要：
机器学习（ML）模型在现实世界中可能以意想不到的方式失败，但并非所有模型失败都同等重要。在时间和资源有限的情况下，ML实践者被迫对模型调试和改进工作进行优先级排序。通过对Apple公司13名ML实践者的访谈，我们发现实践者会构建小型针对性测试集来估计错误的性质、范围及其对用户的影响。我们在机器翻译模型的案例研究中基于这一见解，开发了Angler，一种交互式可视化分析工具，用于帮助实践者确定模型改进的优先级。在一项有7名机器翻译专家参与的用户研究中，我们使用Angler来理解当输入空间无限且获取可靠的模型质量信号成本高昂时的优先级排序实践。我们的研究表明，参与者能够通过分析定量汇总统计数据和通过阅读句子进行定性数据评估，形成更有趣且更以用户为中心的优先级排序假设。
英文摘要：
Machine learning (ML) models can fail in unexpected ways in the real world, but not all model failures are equal. With finite time and resources, ML practitioners are forced to prioritize their model debugging and improvement efforts. Through interviews with 13 ML practitioners at Apple, we found that practitioners construct small targeted test sets to estimate an error's nature, scope, and impact on users. We built on this insight in a case study with machine translation models, and developed Angler, an interactive visual analytics tool to help practitioners prioritize model improvements. In a user study with 7 machine translation experts, we used Angler to understand prioritization practices when the input space is infinite, and obtaining reliable signals of model quality is expensive. Our study revealed that participants could form more interesting and user-focused hypotheses for prioritization by analyzing quantitative summary statistics and qualitatively assessing data by reading sentences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
