评估大型语言模型在材料选择中的应用
Evaluating Large Language Models for Material Selection
AI总结：
本研究通过构建专家材料偏好数据集，系统评估了大型语言模型在产品设计材料选择任务中的表现，揭示了LLM与专家建议的显著差异及两种失败模式，并指出并行提示是有效的提示工程方法，为LLMs融入设计流程提供了实证依据和改进方向。
摘要：
材料选择是概念设计中的关键步骤，因为它对最终产品的功能性、美观性、可制造性和可持续性影响具有显著作用。本研究探讨了在产品设计过程中使用大型语言模型（LLMs）进行材料选择的方法，并将LLMs的表现与专家在各种设计场景下的选择进行比较。通过收集专家材料偏好数据集，本研究为评估LLMs通过提示工程和超参数调优与专家建议对齐的程度提供了基础。在不同模型配置、提示策略和温度设置下，测量了LLM与专家建议之间的差异。该方法允许对影响LLMs推荐材料有效性的因素进行详细分析。本研究的结果揭示了两种失败模式，并确定并行提示（parallel prompting）是在使用LLMs进行材料选择时一种有用的提示工程方法。研究结果进一步表明，尽管LLMs可以提供有价值的辅助，但其推荐往往与人类专家的推荐存在显著差异。这种差异强调了进一步研究的必要性，即如何更好地定制LLMs以复现专家在材料选择中的决策过程。这项工作为关于LLMs如何融入设计过程的不断增长的知识体系做出了贡献，并提供了对其当前局限性和未来改进潜力的见解。
英文摘要：
Material selection is a crucial step in conceptual design due to its significant impact on the functionality, aesthetics, manufacturability, and sustainability impact of the final product. This study investigates the use of Large Language Models (LLMs) for material selection in the product design process and compares the performance of LLMs against expert choices for various design scenarios. By collecting a dataset of expert material preferences, the study provides a basis for evaluating how well LLMs can align with expert recommendations through prompt engineering and hyperparameter tuning. The divergence between LLM and expert recommendations is measured across different model configurations, prompt strategies, and temperature settings. This approach allows for a detailed analysis of factors influencing the LLMs' effectiveness in recommending materials. The results from this study highlight two failure modes, and identify parallel prompting as a useful prompt-engineering method when using LLMs for material selection. The findings further suggest that, while LLMs can provide valuable assistance, their recommendations often vary significantly from those of human experts. This discrepancy underscores the need for further research into how LLMs can be better tailored to replicate expert decision-making in material selection. This work contributes to the growing body of knowledge on how LLMs can be integrated into the design process, offering insights into their current limitations and potential for future improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
