arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.01280cs.LGcs.PL
收藏

编程示例中的固定集鲁棒性:示例损坏与语义分区恢复

Fixed-Set Robustness in Programming by Example: Example Corruption and Semantic Partition Recovery

Yuan Si, Jialu Zhang

AI总结:

研究编程示例系统中对抗性示例损坏的鲁棒性,提出版本空间分区聚合(VPA)防御方法,实验表明低裕度任务易受攻击,VPA仅在语义分区投票裕度存在时有效。

摘要:

编程示例系统从少量输入-输出示例中推断程序。鲁棒的PBE工作通常将错误示例建模为来自随机噪声过程的样本,然后最小化期望或经验损失。本文研究一种不同的失败模式:一个看到合成器并选择其损坏最严重地损害返回程序的示例的对手。我们形式化了有限PBE版本空间的固定集最坏情况损坏,为字符串转换DSL实现了精确有界池和启发式损坏搜索,并引入了版本空间分区聚合(VPA),一种在不相交示例组上合成并通过语义签名投票的防御。中心主张是故意有界且部分否定的:低裕度PBE任务具有随机拼写错误和噪声PBE评估遗漏的对抗鲁棒性维度,而语义分区聚合仅在干净语义保持分区投票裕度时才有帮助,这在现实任务中经常失败。来自策划/生成的DSL任务、已接受的公共SyGuS PBE_SLIA切片、SYNTRA Playgol v2和噪声PBE目标基线的证据支持这一界限。一次策划编辑翻转了所有8个尖峰任务,而200次试验的拼写错误、DSL池和距离匹配随机控制分别成功10.3%、11.0%和16.7%;生成的裕度1行在预算1下翻转,但VPA恢复了它们;在公共SyGuS上,投票裕度接近1,因此自适应攻击者将VPA准确率降至零;已接受的公共SyGuS切片在精确有界池预算边界上移动;Playgol在141个已接受行上显示与拼写错误和同池随机控制相比的正配对自助法差距。一个小的精确输出提示工具在20个受控裕度1任务上展示了相同的定性干净到攻击模式,跨越本地和API模型,同时它被视为范围检查,而非广泛的LLM基准。

英文摘要:

Programming-by-example systems infer programs from a small set of input-output examples. Robust PBE work usually models wrong examples as samples from a stochastic noise process and then minimizes an expected or empirical loss. This paper studies a different failure mode: an adversary who sees the synthesizer and chooses the examples whose corruption most damages the returned program. We formalize fixed-set worst-case corruption for finite PBE version spaces, implement exact-within-bounded-pool and heuristic corruption searches for a string-transformation DSL, and introduce version-space partition aggregation (VPA), a defense that synthesizes on disjoint example groups and votes by semantic signatures. The central claim is deliberately bounded and partly negative: low-margin PBE tasks have an adversarial robustness dimension that random-typo and noisy-PBE evaluations miss, while semantic partition aggregation helps only when the clean semantics keep a partition vote margin, which often fails on realistic tasks. Evidence from curated/generated DSL tasks, accepted public SyGuS PBE_SLIA slices, SYNTRA Playgol v2, and noisy-PBE objective baselines supports that boundary. One curated edit flips all 8 spike tasks while 200-trial typo, DSL-pool, and distance-matched random controls succeed on 10.3%, 11.0%, and 16.7%; generated margin-1 rows flip under budget 1 yet VPA recovers them; on public SyGuS the vote margin is near one, so an adaptive attacker drives VPA accuracy to zero; accepted public SyGuS slices move across exact-within-pool budget boundaries; and Playgol shows positive paired-bootstrap gaps against typo and same-pool random controls on the 141 accepted rows. A small exact-output prompt harness over 20 controlled margin-1 tasks shows the same qualitative clean-to-attacked pattern across local and API models, while it is treated as a scope check, not a broad LLM benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑