发表机构
Zhejiang University; University of Bristol（浙江大学; 布里斯托大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究人员提出首个真实PLC硬件在环框架PLCBench，评估自主LLM智能体将PLC访问转化为持续物理影响的能力，发现31.3%实验达成物理目标，过程观测丰富可提升目标达成率，还发布了相关复现资源。
AI 中文摘要
工业控制系统（ICS）依赖可编程逻辑控制器（PLC）将网络化计算与物理控制相连。使用工具的大语言模型（LLM）智能体是一种新兴攻击威胁：自主智能体能否将网络可达的PLC转化为持续的不良物理影响？然而，现有评估聚焦于数字任务或PLC测试的单个阶段。在ICS中，仅停留在软件利用、已接受的写入或工具访问的评估可能会错误表征物理风险。我们提出PLCBench，据我们所知，这是首个用于表征这种网络到物理能力及其边界的真实PLC硬件在环（HIL）框架。它结合了厂商原生交互、商用PLC执行、闭环降阶过程仿真以及独立结果验证。确定性评估器对运行器、通信、PLC对象和过程记录应用固定规则，以分配六个隐藏诊断标志，区分可用的PLC交互、过程关联的操纵以及持续的物理影响。我们在四个商用PLC上实例化PLCBench，搭配四个闭环工作负载。在五个LLM系列和240次真实PLC实验中，75次实验（31.3%）实现了各自的物理目标。阶段性结果显示，98次实验在有效原生读取前停止，而62次达到过程关联写入但未实现最终目标。值得注意的是，更丰富的过程观测与过程关联写入后条件目标达成率从44.2%提升至64.0%相关。这些测量结果将失败定位在已配置的PLC-过程部署中，并确定了未来防御评估的干预点。为支持可复现性，我们通过配套工件发布了可安全披露的PLCBench代码和纯软件复现流水线。
英文摘要
Industrial control systems (ICSs) rely on programmable logic controllers (PLCs) to connect networked computation with physical control. Tool-using large language model (LLM) agents represent an emerging attack threat: can an autonomous agent convert a network-reachable PLC into sustained adverse physical impact? However, existing evaluations focus on digital tasks or individual stages of PLC testing. In ICSs, evaluations that stop at software exploitation, an accepted write, or tool access may therefore mischaracterize physical risk. We present PLCBENCH, to our knowledge, the first real-PLC hardware-in-the-loop (HIL) framework for characterizing this cyber-to-physical capability and its boundaries. It combines vendor-native interaction, commercial PLC execution, closed-loop reduced-order process simulation, and independent outcome verification. A deterministic evaluator applies fixed rules to runner, communication, PLC-object, and process records to assign six hidden diagnostic flags, distinguishing usable PLC interaction, process-linked manipulation, and sustained physical impact. We instantiate PLCBENCH on four commercial PLCs crossed with four closed-loop workloads. Across five LLM families and 240 real-PLC episodes, 75 episodes (31.3%) sustain their respective physical objectives. Stagewise results show that 98 episodes stop before a valid native read, whereas 62 reach a process-linked write but do not sustain the final objective. Notably, richer process observation is associated with an increase in conditional objective attainment after a process-linked write from 44.2% to 64.0%. These measurements localize failure in configured PLC-process deployments and identify intervention points for future defense evaluation. To support reproducibility, we release the safely disclosable PLCBENCH code and a software-only reproduction pipeline through the accompanying artifact.
Comments36 pages, 13 figures
