AI 中文总结
研究针对方法名预测中现有评估不能反映语义质量、基于LLM的方法与人命名过程不同的问题，通过实证研究比较多种评估器及策略，提出结合摘要和细化的SMNP方法，实验证明该方法在方法名预测上有效且鲁棒。
AI 中文摘要
方法名对软件质量至关重要，影响代码的可理解性、可维护性和开发者协作。手动设计有意义的方法名具有挑战性，方法名预测（MNP）自动从代码片段生成方法名，虽大语言模型（LLMs）在MNP上有潜力，但存在挑战。现有评估依赖令牌相似性指标，不能反映语义质量；当前基于LLM的MNP方法直接代码到名称映射，与人命名前理解功能的过程不同。为此进行实证研究，比较多种评估器，结果显示基于LLM的评估器更符合人类判断。还比较了直接生成和摘要-细化策略，发现摘要和细化能提高生成名称的语义质量。提出SMNP，实验证明其有效性和鲁棒性。
英文摘要
Method names are critical to software quality, affecting code comprehensibility, maintainability, and developer collaboration. However, manually designing meaningful method names is challenging. Method Name Prediction (MNP), which automatically generates method names from code snippets, has recently attracted attention. Although large language models (LLMs) show promising performance for MNP, two challenges remain. First, existing evaluations mainly rely on token similarity metrics, which often fail to reflect human judgments of semantic quality. Second, current LLM-based MNP methods usually generate names through direct code-to-name mapping, which differs from the human process of understanding functionality before naming. To address these challenges, we conduct empirical studies on LLM-based evaluation and MNP strategies. We compare 6 metric-based evaluators, 5 LLM-based evaluators, and 6 human evaluators. Results show that LLM-based evaluators, especially DeepSeek-based evaluators, are more consistent with human judgments than traditional metrics. We further compare direct generation and summarization-and-refinement strategies. Results indicate that summarization and refinement generally improve the semantic quality of generated names. Case studies reveal three limitations: inaccurate summaries, semantic misalignment, and close semantic scores. Based on these findings, we propose SMNP, an MNP approach combining MNP-oriented summarization and chain-of-thought enhanced refinement. Experiments on 5 LLMs and 2 datasets demonstrate the effectiveness and robustness of SMNP.
