Klarna 产品页数据集：基于图神经网络与大型语言模型的网页元素提名
The Klarna Product Page Dataset: Web Element Nomination with Graph Neural Networks and Large Language Models
AI总结：
本文提出 Klarna 产品页数据集以缓解网页自动化中元素提名任务数据稀缺问题，并通过 GCN 与大型语言模型结合的训练精化流程及新型训练方法显著提升提名准确率。
摘要：
网页自动化有潜力彻底改变用户与数字世界的交互方式，通过复杂的计算方法提供无与伦比的辅助并简化任务。这一演进的核心是网页元素提名任务，即识别网页上的唯一元素。遗憾的是，网页自动化算法设计的发展受到缺乏全面且真实的数据集的阻碍，这些数据集应能反映现实世界 Web 应用所面临的复杂性。为解决这一问题，我们引入了 Klarna 产品页数据集，这是一个全面且多样化的网页集合，在丰富性和多样性上超越了现有数据集。该数据集包含来自 8,175 个电子商务网站、覆盖八个地理区域的 51,701 个经人工标注的产品页，并附带渲染页面截图数据集。为启动对 Klarna 产品页数据集的研究，我们在网页元素提名任务上对一系列图神经网络（GNN）进行了实证基准测试。我们做出了三项重要贡献。第一，我们发现简单的卷积 GNN（GCN）优于复杂的最先进提名方法。第二，我们引入了一种训练精化流程，即利用上述 GCN 从每个页面中识别少量相关元素，然后将这些元素传递给大型语言模型进行最终提名。该流程在我们具有挑战性的数据集上将提名准确率显著提高了 16.8 个百分点，且无需任何微调。最后，针对该领域另一个普遍挑战——适用于元素提名的训练方法过多——我们引入了 Challenge Nomination Training Procedure，这是一种新颖的训练方法，可进一步提升提名准确率。
英文摘要：
Web automation holds the potential to revolutionize how users interact with the digital world, offering unparalleled assistance and simplifying tasks via sophisticated computational methods. Central to this evolution is the web element nomination task, which entails identifying unique elements on webpages. Unfortunately, the development of algorithmic designs for web automation is hampered by the scarcity of comprehensive and realistic datasets that reflect the complexity faced by real-world applications on the Web. To address this, we introduce the Klarna Product Page Dataset, a comprehensive and diverse collection of webpages that surpasses existing datasets in richness and variety. The dataset features 51,701 manually labeled product pages from 8,175 e-commerce websites across eight geographic regions, accompanied by a dataset of rendered page screenshots. To initiate research on the Klarna Product Page Dataset, we empirically benchmark a range of Graph Neural Networks (GNNs) on the web element nomination task. We make three important contributions. First, we found that a simple Convolutional GNN (GCN) outperforms complex state-of-the-art nomination methods. Second, we introduce a training refinement procedure that involves identifying a small number of relevant elements from each page using the aforementioned GCN. These elements are then passed to a large language model for the final nomination. This procedure significantly improves the nomination accuracy by 16.8 percentage points on our challenging dataset, without any need for fine-tuning. Finally, in response to another prevalent challenge in this field - the abundance of training methodologies suitable for element nomination - we introduce the Challenge Nomination Training Procedure, a novel training approach that further boosts nomination accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
