AI 中文总结
本研究针对将自然语言需求转化为GUI原型及GUI应用需求验证的两大挑战，提出基于预训练语言模型的自动化方法，可显著减少相关手动工作量。
AI 中文摘要
需求获取是开发交互式软件系统的关键环节，有助于确保最终产品满足利益相关者的需求。由于需求获取通常依赖自然语言（NL），其固有的模糊性可能导致误解。形式化规范可减少模糊性，但需要专业技术知识。因此，图形用户界面（GUI）原型设计将需求转化为有形的视觉产物，为沟通、获取和验证提供支持，是一种有价值的替代方案。然而，创建高保真原型仍然耗时且成本高昂。同样，确保实现符合指定需求的需求验证工作目前仍以手动为主，而现有的自动化方法往往局限于静态、基于规则的技术。本研究解决两大挑战：（C1）减少将自然语言需求转化为GUI原型所需的工作量；（C2）减少GUI应用程序和原型中需求验证所需的工作量。针对C1，我们引入了基于自然语言的新型GUI检索与重排序方法、新基准，以及高效调整大语言模型（LLM）以生成GUI的技术，包括专有GUI表示，其有效性在带有人工标注的大型基准上得到验证。针对C2，我们提出了基于LLM的方法，用于在静态GUI原型上验证语义复杂的自然语言需求，并引入了基于多模态LLM的智能体，通过自动生成和评估交互轨迹，在动态GUI应用程序中验证复杂的功能和非功能需求。总体而言，所提出的方法大幅减少了GUI原型设计和需求验证中的手动工作量。
英文摘要
Requirements elicitation is essential for developing interactive software systems, as it helps ensure that the resulting product meets stakeholder needs. Since elicitation typically relies on natural language (NL), misunderstandings can arise from its inherent ambiguity. Formal specifications can reduce ambiguity but require technical expertise. GUI prototyping therefore provides a valuable alternative by turning requirements into tangible visual artifacts that support communication, elicitation, and validation. However, creating high-fidelity prototypes remains time-consuming and costly. Similarly, requirements verification, which ensures that implementations conform to specified requirements, is still largely manual, while existing automated approaches are often limited to static, rule-based techniques. This work addresses two challenges: (C1) reducing the effort required to transform NL requirements into GUI prototypes, and (C2) reducing the effort required for requirements verification in GUI applications and prototypes. For C1, we introduce novel NL-based GUI retrieval and reranking methods, new benchmarks, and techniques for efficiently adapting LLMs to GUI generation, including proprietary GUI representations. Their effectiveness is demonstrated on a large benchmark with human annotations. For C2, we propose LLM-based methods for verifying semantically complex NL requirements on static GUI prototypes and introduce a multimodal LLM-based agent for verifying complex functional and non-functional requirements in dynamic GUI applications through automatically generated and evaluated interaction trajectories. Overall, the proposed methods substantially reduce manual effort in GUI prototyping and requirements verification.
