基于大语言模型的生成式AI为创新评估设计实践带来了机遇。随着近期技术发展，有必要了解生成式AI在模拟认知技能方面的局限与能力。评估学生的批判性思维技能长期以来都是评估工作的一项重要内容，但数字化评估的需求在公平性、学术诚信和评估命题方面带来了独特挑战。教育工作者需要一个框架来判定其评估内容对生成式AI的脆弱性，从而为评估设计实践提供参考。本文提出了一个框架，用于探究当前行业基准的大语言模型应用ChatGPT4的能力。本文提出了“问题映射、AI脆弱性测试、评分、评估（Mapping of questions, AI vulnerability testing, Grading, Evaluation，MAGE）”框架，可在各自学科语境下系统地评判其评估内容。该评判将针对问题在批判性思维技能方面的脆弱性提供具体、有针对性的指示，进而可作为其任务评估设计的基础。
英文摘要：
Generative AI such as those with large language models have created opportunities for innovative assessment design practices. Due to recent technological developments, there is a need to know the limits and capabilities of generative AI in terms of simulating cognitive skills. Assessing student critical thinking skills has been a feature of assessment for time immemorial, but the demands of digital assessment create unique challenges for equity, academic integrity and assessment authorship. Educators need a framework for determining their assessments vulnerability to generative AI to inform assessment design practices. This paper presents a framework that explores the capabilities of the LLM ChatGPT4 application, which is the current industry benchmark. This paper presents the Mapping of questions, AI vulnerability testing, Grading, Evaluation (MAGE) framework to methodically critique their assessments within their own disciplinary contexts. This critique will provide specific and targeted indications of their questions vulnerabilities in terms of the critical thinking skills. This can go on to form the basis of assessment design for their tasks.
Don't Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University（清华大学计算机科学与技术系、人工智能研究院）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Tencent（腾讯）
Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China（中国人民大学人工智能学院）；Beijing Key Laboratory of Research on Large Models（北京大型模型研究关键实验室）；Engineering Research Center of Next-Generation Intelligent Search（下一代智能搜索工程研究中心）；College of Artificial Intelligence, Tianjin University of Science（天津科技大学人工智能学院）
A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Education
AI4STEM Education Center（AI4STEM教育中心）；University of Georgia（佐治亚大学）；National GENIUS Center（国家GENIUS中心）；School of Computing, University of Georgia（佐治亚大学计算学院）；School of Teacher Education, Nanjing Normal University（南京师范大学教师教育学院）；National Institute of Education, Singapore（新加坡国家教育学院）
Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use Scale
School of Social Sciences, Nanyang Technological University（南洋理工大学社会科学学院）；Interdisciplinary Graduate Programme, Nanyang Technological University（南洋理工大学跨学科研究生项目）；College of Health and Human Sciences, Purdue University（普渡大学健康与人类科学学院）；School of Social Sciences, Singapore Management University（新加坡管理学院社会科学学院）；Faculty of Information Technology, Monash University（墨尔本大学信息技术学院）
