Structure-BiEval: 一种自监督、双轨框架，用于解耦Web信息系统中LLM评估的结构与内容
Structure-BiEval: A Self-Supervised, Dual-Track Framework for Decoupling Structure and Content in LLM Evaluation for Web Information Systems
AI总结：
本文提出Structure-BiEval框架，通过确定性中间表示解耦结构与内容，利用内容语义准确性和归一化树编辑距离评估Web数据工程中的LLM结构 fidelity，发现不同模型在Web数据格式化中表现差异显著。
摘要：
随着大型语言模型（LLMs）成为基于网络的自主代理和复杂网络信息系统的核心，其将自然语言准确转换为严格结构化格式的能力变得至关重要，因为这种能力对于网络API调用和数据交换至关重要。然而，评估Web原生数据负载中的结构保真度仍是一个挑战：传统文本度量无法捕捉半结构化网络数据中的拓扑一致性，而手动评估成本过高。为此，我们提出了Structure-BiEval，一种新的自监督框架，用于定量、无标注评估，专为网络数据工程设计。通过利用确定性中间表示，我们的框架有效解耦了结构与内容，利用内容语义准确性和归一化树编辑距离作为精确度量。我们实证地在双网络结构拓扑上基准测试了15种最先进的LLMs，即层次数据（网络后端负载）和表格数据（网络前端展示）。结果揭示了结构性能的显著差异，包括中等大小模型在Web数据格式化中意外超越较大模型的情况。此外，我们的发现表明，深度递归嵌套对不同参数规模的网络代理构成持续挑战。
英文摘要：
As Large Language Models (LLMs) evolve into the core of Web-based autonomous agents and complex Web Information Systems, their ability to faithfully translate natural language into rigorous structured formats has become paramount, as this capability is critical for Web API invocation and data exchange. However, evaluating this structural fidelity in Web-native payloads remains a challenge: traditional text metrics fail to capture topological consistency in semi-structured Web data, while manual evaluation is prohibitively costly. To address this, we propose Structure-BiEval, a novel self-supervised framework for quantitative, annotation-free assessment tailored for Web data engineering. By leveraging deterministic Intermediate Representations, our framework effectively decouples structure from content, utilizing Content Semantic Accuracy and Normalized Tree Edit Distance as precise metrics. We empirically benchmark 15 state-of-the-art LLMs across dual Web structural topologies, namely Hierarchical Data (Web backend payloads) and Tabular Data (Web frontend presentation). The results reveal substantial variability in structural performance, including cases where mid-sized models unexpectedly outperform larger counterparts in Web data formatting. Furthermore, our findings show that deep recursive nesting poses a consistent challenge for Web agents across varying parameter scales.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
