从众包知识自动化生成API文档
Automating API Documentation from Crowdsourced Knowledge
AI总结：
AutoDoc通过从Stack Overflow提取API知识，生成更准确、简洁且全面的API文档，有效解决官方文档过时和不完整的问题。
摘要：
API文档对于开发者学习和使用API至关重要。然而，已知许多官方API文档都是过时且不完整的。为了解决这一挑战，我们提出了一种新的方法，称为AutoDoc，该方法通过从Stack Overflow（SO）上的在线讨论中提取API知识来生成API文档。AutoDoc利用一个微调的密集检索模型来从SO帖子中识别七种类型的API知识。然后，它使用GPT-4o将这些帖子中的API知识总结成简洁的文本。同时，我们设计了两个特定组件来处理生成内容中的LLM幻觉和冗余问题。我们对5个比较基线在48个不同流行度的API上评估了AutoDoc。我们的结果表明，由AutoDoc生成的API文档的准确性比基线高77.7%，重复率低9.5%，并且包含官方文档未覆盖的34.4%的知识。我们还测量了AutoDoc对不同LLM选择的敏感性。我们发现，尽管较大的LLM生成的API文档质量更高，但AutoDoc使较小的开源模型（例如Mistral-7B-v0.3）能够实现可比的结果。最后，我们进行了一项用户研究来评估由AutoDoc生成的API文档的有用性。所有参与者都发现由AutoDoc生成的API文档比比较基线更全面、简洁和有用。这突显了利用LLM进行API文档生成的可行性，只要经过精心设计以应对LLM幻觉和信息冗余问题。
英文摘要：
API documentation is crucial for developers to learn and use APIs. However, it is known that many official API documents are obsolete and incomplete. To address this challenge, we propose a new approach called AutoDoc that generates API documents with API knowledge extracted from online discussions on Stack Overflow (SO). AutoDoc leverages a fine-tuned dense retrieval model to identify seven types of API knowledge from SO posts. Then, it uses GPT-4o to summarize the API knowledge in these posts into concise text. Meanwhile, we designed two specific components to handle LLM hallucination and redundancy in generated content. We evaluated AutoDoc against five comparison baselines on 48 APIs of different popularity levels. Our results indicate that the API documents generated by AutoDoc are up to 77.7% more accurate, 9.5% less duplicated, and contain 34.4% knowledge uncovered by the official documents. We also measured the sensitivity of AutoDoc to the choice of different LLMs. We found that while larger LLMs produce higher-quality API documents, AutoDoc enables smaller open-source models (e.g., Mistral-7B-v0.3) to achieve comparable results. Finally, we conducted a user study to evaluate the usefulness of the API documents generated by AutoDoc. All participants found API documents generated by AutoDoc to be more comprehensive, concise, and helpful than the comparison baselines. This highlights the feasibility of utilizing LLMs for API documentation with careful design to counter LLM hallucination and information redundancy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
