SPOLRE：用于图像描述系统测试的语义保留对象布局重构
SPOLRE: Semantic Preserving Object Layout Reconstruction for Image Captioning System Testing
AI总结：
提出自动化工具 SPOLRE，通过四种语义保留的转换技术重构对象布局生成真实测试套件，有效检测图像描述系统错误，在七个系统中识别出 31,544 个错误描述，准确率超 91%。
摘要：
图像描述（IC）系统，如 Microsoft Azure Cognitive Service，将图像内容翻译为描述性语言，但可能会生成不准确的内容，从而导致误解。诸如 MetaIC 和 ROME 等高级测试技术旨在解决这些问题，但面临着重大挑战。这些方法需要大量的人工进行详细标注，并且经常生成不真实的图像，要么添加了不相关的对象，要么未能移除现有对象。此外，它们生成的测试套件有限，MetaIC 仅限于插入特定对象，而 ROME 仅限于狭窄的变化范围。我们引入了 SPOLRE，这是一种用于 IC 系统测试中语义保留对象布局重构的新型自动化工具。SPOLRE 利用四种转换技术来修改对象布局，而不改变图像的语义。这种自动化方法消除了手动标注的需要，并创建了真实、多样的测试套件。我们的测试表明，超过 75% 的调查受访者认为 SPOLRE 生成的图像比最先进方法生成的图像更真实。SPOLRE 在识别描述错误方面表现出色，在七个 IC 系统中检测到了 31,544 个错误描述，平均准确率为 91.62%，超过了平均准确率为 85.65% 并识别出 17,160 个错误描述的其他方法。值得注意的是，SPOLRE 在 Azure 中识别出了 6,236 个独特问题，证明了它对最先进的 IC 系统之一的有效性。
英文摘要：
Image captioning (IC) systems, such as Microsoft Azure Cognitive Service, translate image content into descriptive language but can generate inaccuracies leading to misinterpretations. Advanced testing techniques like MetaIC and ROME aim to address these issues but face significant challenges. These methods require intensive manual labor for detailed annotations and often produce unrealistic images, either by adding unrelated objects or failing to remove existing ones. Additionally, they generate limited test suites, with MetaIC restricted to inserting specific objects and ROME limited to a narrow range of variations. We introduce SPOLRE, a novel automated tool for semantic-preserving object layout reconstruction in IC system testing. SPOLRE leverages four transformation techniques to modify object layouts without altering the image's semantics. This automated approach eliminates the need for manual annotations and creates realistic, varied test suites. Our tests show that over 75% of survey respondents find SPOLRE-generated images more realistic than those from state-of-the-art methods. SPOLRE excels in identifying caption errors, detecting 31,544 incorrect captions across seven IC systems with an average precision of 91.62%, surpassing other methods which average 85.65% accuracy and identify 17,160 incorrect captions. Notably, SPOLRE identified 6,236 unique issues within Azure, demonstrating its effectiveness against one of the most advanced IC systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
