学术写作中的生成式AI:DeepSeek、Qwen、ChatGPT、Gemini、Llama、Mistral和Gemma的比较
Generative AI in Academic Writing: A Comparison of DeepSeek, Qwen, ChatGPT, Gemini, Llama, Mistral, and Gemma
AI总结:
本研究比较了DeepSeek、Qwen等开源模型与ChatGPT等在学术写作中的表现,发现虽能生成足量且语义高度重叠的内容,但存在抄袭率高、易被AI检测识别及可读性不足等问题。
摘要:
中国开发的DeepSeek v3于2024年12月发布,随后阿里巴巴的Qwen 2.5 Max于2025年1月发布,Qwen3 235B于2025年4月发布。这些免费的开源模型为学术写作和内容创作提供了巨大潜力。本研究通过将它们与ChatGPT、Gemini、Llama、Mistral和Gemma进行比较,评估了它们的学术写作表现。关于这些工具能在多大程度上被利用,以及它们在质量、可读性和有效性方面生成原创内容的潜力,现有文献存在关键空白。使用40篇关于Digital Twin和Healthcare的论文,通过AI工具基于提出的问题和改写的摘要生成文本。生成的内容通过查重检测、AI检测、字数比较、语义相似度和可读性评估进行分析。结果表明,改写的摘要显示出较高的抄袭率,而基于问题的响应也超出了可接受的水平。AI检测工具一致地将所有输出识别为AI生成。字数分析显示,所有聊天机器人都产生了足够数量的内容。语义相似度测试显示,生成的文本与原始文本之间存在高度重叠。然而,可读性评估表明,文本在清晰度和可访问性方面不足。本研究比较性地突出了流行和最新的大型语言模型在学术写作方面的潜力和局限性。虽然这些模型生成大量且语义准确的内容,但必须解决关于抄袭、AI检测和可读性的担忧,才能在学术工作中有效使用它们。
英文摘要:
DeepSeek v3, developed in China, was released in December 2024, followed by Alibaba's Qwen 2.5 Max in January 2025 and Qwen3 235B in April 2025. These free and open-source models offer significant potential for academic writing and content creation. This study evaluates their academic writing performance by comparing them with ChatGPT, Gemini, Llama, Mistral, and Gemma. There is a critical gap in the literature concerning how extensively these tools can be utilized and their potential to generate original content in terms of quality, readability, and effectiveness. Using 40 papers on Digital Twin and Healthcare, texts were generated through AI tools based on posed questions and paraphrased abstracts. The generated content was analyzed using plagiarism detection, AI detection, word count comparisons, semantic similarity, and readability assessments. Results indicate that paraphrased abstracts showed higher plagiarism rates, while question-based responses also exceeded acceptable levels. AI detection tools consistently identified all outputs as AI-generated. Word count analysis revealed that all chatbots produced a sufficient volume of content. Semantic similarity tests showed a strong overlap between generated and original texts. However, readability assessments indicated that the texts were insufficient in terms of clarity and accessibility. This study comparatively highlights the potential and limitations of popular and latest large language models for academic writing. While these models generate substantial and semantically accurate content, concerns regarding plagiarism, AI detection, and readability must be addressed for their effective use in scholarly work.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。