发表机构
Khalifa University（哈利法大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对LLM生成的网络意图翻译配置，提出用采样预测不确定性排序风险、令牌级熵定位歧义，验证了两类信号在部署流程中的应用潜力。
AI 中文摘要
基于意图的网络实现始于将高层意图翻译为低层网络配置，近期方法已转向基于大语言模型（LLM）的翻译。尽管取得了良好结果，但多数研究聚焦于翻译准确率，忽略了部署生成配置相关的风险。本研究通过分析模型的不确定性，探究LLM生成配置的部署前翻译风险，提出使用两种不确定性信号：基于采样的预测不确定性用于翻译风险排序，令牌级熵用于歧义源定位。在歧义可控的测试集上，针对不同上下文类型和采样预算，使用针对厂商特定交换机平台（瞻博网络Juniper EX3300）微调的Llama-3.1-8B-Instruct模型评估这些信号。结果表明，预测不确定性可在不同上下文类型和采样预算下为翻译风险排序提供有用信号，尽管在信息较少的上下文存在严重校准误差；此外，参数令牌熵与参数源歧义相关，关键词令牌熵与描述源歧义相关。这些结果表明，不确定性信号可用于LLM生成配置的部署流程，预测不确定性可支持选择性部署，令牌级熵可识别歧义源。
英文摘要
Intent-based networking realization starts by translating high-level intents into low-level network configurations. Recent approaches have shifted toward LLM-based translation. Despite promising results, most studies focus on translation accuracy and overlook risks associated with deploying the resulting configurations. In this work, we investigate the pre-deployment translation risk of LLM-generated configurations by analyzing the model's uncertainty. We propose to use two uncertainty signals, namely sampling-based predictive uncertainty for translation-risk ranking and token-level entropy for ambiguity-source localization. We evaluate these signals on an ambiguity-controlled test set across different context types and sampling budgets, using a Llama-3.1-8B-Instruct model fine-tuned for intent translation on a vendor-specific switch platform (Juniper EX3300). The results demonstrate that predictive uncertainty provides a useful signal for ranking translations by risk across context types and sampling budgets, albeit with substantial miscalibration under less informative contexts. Moreover, we show that parameter-token entropy correlates with parameter-sourced ambiguity and keyword-token entropy correlates with description-sourced ambiguity. These results indicate the potential of using uncertainty signals in an LLM-generated configuration deployment pipeline, where predictive uncertainty can support selective deployment, while token-level entropy can identify sources of ambiguity.
