arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.11904cs.SEcs.AI
收藏

利用LLMs支持文本DSL定义与实例的共演化：系统评估

Leveraging LLMs to support co-evolution between definitions and instances of textual DSLs: A Systematic Evaluation

MCSE Group（MCSE研究组）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Engineering and Information Technology（工程与信息技术）；Adelaide University（阿德莱德大学）；Institute of Computer Science（计算机科学研究所）；University of Rostock（罗斯托克大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Chalmers University of Technology and University of Gothenburg（楚科斯技术大学和哥德堡大学）；Department of Software Science（软件科学系）

Weixing Zhang, Bowen Jiang, Yuhong Fu, Anne Koziolek, Regina Hebig, Daniel Strüber

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文利用LLMs评估文本DSL定义与实例的共演化，发现其在小规模案例中表现良好，但随着规模增大性能下降，且语法复杂性和删除粒度影响更大。

摘要：

软件语言随着时间推移演变，原因包括功能增加。当语法演变时，原本符合它们的文本实例可能会变得过时。虽然模型驱动工程提供了许多共演化模型与元模型变化的技术，但这些方法并非为文本DSL设计，可能导致丢失人类相关的信息，如布局和注释。本研究系统评估了大语言模型（LLMs）在文本DSL语法和实例共演化中的潜力。使用Claude Sonnet 4.5和GPT-5.2在十个案例语言上进行十次运行，评估了正确性和人类相关信息的保留情况。结果表明，在小规模案例（≥94%的精度和召回率，对于需要少于20行修改的实例）表现强劲，但随着规模增大，性能下降：Claude在40行时保持85%的召回率，而GPT在最大实例上失败。响应时间随实例规模显著增加，语法演化复杂性和删除粒度比变更类型对性能影响更大。这些发现澄清了何时基于LLM的共演化有效，以及当前限制仍存在的地方。

英文摘要：

Software languages evolve over time for reasons such as feature additions. When grammars evolve, textual instances that originally conformed to them may become outdated. While model-driven engineering provides many techniques for co-evolving models with metamodel changes, these approaches are not designed for textual DSLs and may lose human-relevant information such as layout and comments. This study systematically evaluates the potential of large language models (LLMs) for co-evolving grammars and instances of textual DSLs. Using Claude Sonnet 4.5 and GPT-5.2 across ten case languages with ten runs each, we assess both correctness and preservation of human-oriented information. Results show strong performance on small-scale cases ($\geq$94% precision and recall for instances requiring fewer than 20 modified lines), but performance degraded with scale: Claude maintains 85% recall at 40 lines, while GPT fails on the largest instances. Response time increases substantially with instance size, and grammar evolution complexity and deletion granularity affect performance more than change type. These findings clarify when LLM-based co-evolution is effective and where current limitations remain.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑