你的模拟运行但解错了物理:基于PDE的意图验证用于LLM生成的多物理场模拟代码
Your Simulation Runs but Solves the Wrong Physics: PDE-Grounded Intent Verification for LLM-Generated Multiphysics Simulation Code
AI总结:
本文提出基于PDE的意图验证方法,用于检查LLM生成的多物理场模拟代码是否符合预期物理,通过确定性重构和比较PDE结构提升代码的意图保真度。
摘要:
基于执行的LLM生成代码评估隐式地将成功执行视为正确性的代理。在科学模拟中,这种代理不足:生成的输入文件可能运行、网格化并收敛,但编码的控制方程可能与用户的意图不同。我们称这种意图物理与生成代码之间的不匹配为理解-生成差距。我们在MOOSE中实例化了这一点,其中内核和BC对象通过组成方式映射到弱形式残差项,使可以确定性地重构编码的PDE并与其预期合同进行比较。我们将这种比较形式化为意图保真度分数(IFS),一个涵盖控制项、BCs、ICs、系数和时间方案的结构度量。基于IFS,我们开发了一种基于PDE的细化循环,利用确定性违反报告来迭代修正生成的代码。我们评估了MooseBench,一个包含220个案例的多物理场基准,其PDE级的地面真相与本文一同发布。在该基准上,我们的方法在直接生成上一致提高了平均IFS,增益集中在困难案例上。在直接生成低于IFS 0.7的子集上,细化增加了+0.22至+0.41的绝对IFS。在部署审计中,仅执行修复提高了执行成功率,但使所有220个案例中仍有39-40%的案例可运行但仍在解决错误的物理。静态证明概念实验表明,重构和比较的模式扩展到了四个PDE导向的DSL(UFL/FEniCS、FreeFEM、FiPy和Devito)。这些发现强化了可执行模拟代码应验证其数学结构,而不是仅接受执行本身。
英文摘要:
Execution-based evaluation of LLM-generated code implicitly treats successful execution as a proxy for correctness. In scientific simulation, this proxy is insufficient: a generated input file can run, mesh, and converge while encoding governing equations that differ from the user's intent. We call this mismatch between intended physics and generated code the comprehension-generation gap. We instantiate this in MOOSE, where Kernel and BC objects map compositionally to weak-form residual terms, enabling deterministic reconstruction of the encoded PDE and comparison against an intended contract. We formalize this comparison as the Intent Fidelity Score (IFS), a structural metric covering governing terms, BCs, ICs, coefficients, and time scheme. Building on IFS, we develop a PDE-grounded refinement loop that uses deterministic violation reports to correct generated code iteratively. We evaluate on MooseBench, a 220-case multiphysics benchmark with PDE-level ground truth released with this work. On this benchmark, our method consistently improves mean IFS over direct generation, with gains concentrated on hard cases. On the subset where direct generation falls below IFS 0.7, refinement adds +0.22 to +0.41 absolute IFS. In the deployment audit, execution-only repair improves execution success while leaving 39-40% of all 220 cases runnable but still solving the wrong physics across the three main deployment-audit models, exposing executability and intent fidelity as separable failure modes. Static proof-of-concept experiments on four PDE-oriented DSLs (UFL/FEniCS, FreeFEM, FiPy, and Devito) suggest that the reconstruction-and-comparison pattern extends beyond MOOSE. These findings reinforce that executable simulation code should be verified against the mathematical structure it is intended to encode, not accepted on execution alone.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。