arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.28345cs.SEcs.AI
收藏

当代码遇见自然语言:基于分类法的LLM集成应用信息流分析

Reachability Across the NL/PL Boundary: A Taxonomy-Driven Dataflow Model for LLM-Integrated Applications

University of New South Wales(新南威尔士大学);Macquarie University(麦考瑞大学);National University of Singapore(新加坡国立大学)

Zihao Xu, Xiao Cheng, Ruijie Meng, Yuekang Li

AI总结:

提出一种基于定量信息流理论的分类法,定义24个标签以跨越LLM调用的自然语言与编程语言边界,实现信息流分析,并在污点传播和程序切片中验证有效性。

摘要:

LLM API调用正成为一种普遍的程序构造,但它们创建了一个现有程序分析无法跨越的边界:运行时值进入自然语言提示,在LLM内部经过不透明处理,然后作为程序消费的代码、SQL、JSON或文本重新出现。每个跨函数边界跟踪数据的分析,包括污点分析、程序切片、依赖分析和变更影响分析,都依赖于被调用者的数据流摘要。LLM调用没有这样的摘要,在我们称为NL/PL边界处打破了所有这些分析。我们提出了第一个信息流方法来跨越这个边界。基于定量信息流理论,我们的分类法沿两个正交维度定义了24个标签:信息保留级别(从词汇保留到完全阻塞)和输出模态(自然语言、结构化格式、可执行工件)。我们从4,154个真实世界Python文件中标记了9,083个占位符-输出对,并通过Cohen's κ=0.82和近乎完全的覆盖率(0.01%无法分类)验证了可靠性。我们在两个下游应用中展示了分类法的实用性:(1)一个两阶段污点传播管道,结合基于分类法的过滤和LLM验证,在353个专家注释对上达到F1=0.923,并在六个真实世界OpenClaw提示注入案例上的跨语言验证进一步确认了有效性;(2)基于分类法的反向切片在包含非传播占位符的文件中将切片大小平均减少了15%。每个标签的分析显示,四个阻塞标签几乎涵盖了所有非传播情况,为工具构建者提供了可操作的过滤标准。

英文摘要:

LLM API calls have become a standard programming primitive, but they create a program boundary that disrupts traditional dataflow analysis. A runtime value may be inserted into a natural-language prompt through a template placeholder, transformed opaquely by the LLM, and returned as code, JSON, or text consumed by downstream logic. Existing analyses such as taint analysis and program slicing require a dataflow summary that describes how a callee maps inputs to outputs; an LLM call provides no such summary, breaking analysis at what we call the NL/PL boundary. We introduce PRISM, the first reachability model for this boundary. PRISM abstracts the missing dataflow summary of an LLM call as placeholder-to-output reachability. Because the LLM's internal transformation is opaque, the only observable signal is the input-output relationship, which spans an unbounded range of behaviors. PRISM therefore uses a finite taxonomy grounded in quantitative information flow theory. It classifies placeholder-output behavior into 25 labels along two dimensions: information preservation and output modality. Each label yields a reachability predicate for a placeholder. The model is sound with respect to its labeling, with residual error bounded empirically. PRISM is dependable and effective. Independent models and human annotators assign its labels consistently (Fleiss' kappa >= 0.72), and the labels cover 8,119 real-world pairs, leaving no pair unclassifiable; the Good-Turing discovery probability is 0.09%. For taint analysis, PRISM nearly doubles the conservative baseline and outperforms a direct LLM baseline, achieving F1 = 81.7%. Across six real OpenClaw CVEs, it detects every vulnerable flow and confirms every patch (F1 = 100%). In backward slicing, it removes about a quarter of irrelevant code without discarding any true dependency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑