arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

IDRAAK:从多智能体自然语言处理到用于技术需求语义漂移检测的少样本提示

IDRAAK: From Multi-Agent NLP to Few-Shot Prompting for Semantic Drift Detection in Technical Requirements

Shiva Ahir

arXiv 2608.08801首次发表:更新:

发表机构

Stony Brook University(石溪大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

IDRAAK是基于语义需求表示的可解释框架,通过含6个少样本示例的LLM单次调用检测技术需求语义漂移,性能优于多种替代方案,证明少样本提示是高效替代方法。

AI 中文摘要

跨语言翻译技术需求会引入语义漂移,改变数值约束、极性、模态或其他对规范至关重要的含义。本文提出IDRAAK,这是一个可解释框架,采用与语言无关的语义需求表示(SRR)来检测此类漂移,评估了六种检测工作流,范围从确定性比较到多智能体验证和少样本提示。在来自10个工程领域的300个需求上的890个合成扰动中,使用6个少样本示例的单次大语言模型(LLM)调用达到马修斯相关系数(MCC)=0.888、F1值=0.983,优于所评估的结构化和多阶段替代方案。在PAWS-X(805对、5种语言)和XNLI(700对、7种语言)上的进一步评估揭示了结构化方法和基于LLM方法的互补优势与局限。确定性SRR比较在技术需求上表现强劲(F1=0.898),但在通用领域文本上表现很差(F1=0.012),而结构化证据可提升对抗性释义的性能。事后Platt缩放进一步改善了置信度校准。结果表明,增加智能体复杂度不一定能提升语义漂移检测性能,简单的少样本提示可提供强大且高效的替代方案。

英文摘要

Translating technical requirements across languages can introduce semantic drift, altering numerical constraints, polarities, modalities, or other specification-critical meaning. IDRAAK is presented as an interpretable framework for detecting such drift using a language-independent Semantic Requirement Representation (SRR), with six detection workflows evaluated, ranging from deterministic comparison to multi-agent verification and few-shot prompting. On 890 synthetic perturbations across 300 requirements from 10 engineering domains, a single LLM call with six few-shot examples achieves MCC=0.888 and F1=0.983, outperforming the evaluated structured and multi-stage alternatives. Further evaluation on PAWS-X (805 pairs, 5 languages) and XNLI (700 pairs, 7 languages) exposes complementary strengths and limitations of structured and LLM-based approaches. Deterministic SRR comparison performs strongly on technical requirements (F1=0.898) but poorly on general-domain text (F1=0.012), while structured evidence improves performance on adversarial paraphrases. Post-hoc Platt scaling further improves confidence calibration. The results demonstrate that increased agentic complexity does not necessarily improve semantic-drift detection and that simple few-shot prompting can provide a strong and efficient alternative.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑