以质量为导向的代理推理用于大语言模型辅助软件设计:问题-思考(QoT)作为时间序列自我QA链
Quality-Driven Agentic Reasoning for LLM-Assisted Software Design: Questions-of-Thoughts (QoT) as a Time-Series Self-QA Chain
AI总结:
本文提出QoT框架,通过时间序列自我QA链提升大语言模型辅助软件设计的质量,通过多领域实验验证其在不同模型规模和复杂度下的效果。
摘要:
近期大语言模型(LLM)的进步加速了AI辅助软件开发,但实际部署仍受不完整的实现、弱模块化和不一致的安全实践限制。我们引入了问题-思考(QoT),一种以质量为导向的推理时间支架,将用户目标转化为(i)有序的工程步骤序列和(ii)逐步自我提问以验证约束并减少遗漏错误,同时保持轻量级推理记录以稳定后续设计决策。我们评估QoT在三个代表性的后端工程领域:API设计、数据通信和文件系统。每个任务都需要多模块分解,并暴露了LLM生成系统中的标准故障模式。为了实现数据驱动的比较,我们使用一个受ISO/IEC启发的质量评估标准对生成的成果进行评分,该标准衡量可扩展性、完整性、模块化和安全性。我们报告了领域层面的改进作为总质量分数的变化,定义为QoT分数减去NoQoT分数。结果表明容量依赖性的改进:QoT在较大模型和更复杂领域中产生一致的质量改进,而较小的模型在紧约束和规划预算下可能会出现权衡。我们发布了包含提示、评分指南、原始生成和可重复报告表格和图表的开源成果,以支持应用AI和数据分析研究。
英文摘要:
Recent advances in large language models (LLMs) have accelerated AI-assisted software development, yet practical deployment remains constrained by incomplete implementations, weak modularization, and inconsistent security practices. We introduce Questions-of-Thoughts (QoT), a quality-driven inference-time scaffold that turns a user goal into (i) an ordered sequence of engineering steps and (ii) stepwise self-questioning to verify constraints and reduce omission errors, while maintaining a lightweight reasoning record that stabilizes subsequent design decisions. We evaluate QoT across three representative backend engineering domains: API Design, Data Communication, and File Systems. Each task requires multi-module decomposition and exposes standard failure modes in LLM-generated systems. To enable data-driven comparison, we score generated artifacts using an ISO/IEC-inspired quality rubric that measures Scalability, Completeness, Modularity, and Security. We report domain-wise gains as the change in total quality score, defined as the QoT score minus the NoQoT score. Results show capacity-dependent improvements: QoT yields consistent quality improvements for larger models and more complex domains, while smaller models may exhibit trade-offs under tight context and planning budgets. We release an open artifact with prompts, scoring guidelines, raw generations, and scripts that reproduce the reported tables and figures to support applied AI and data analytics research.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。