ChatGPT has gained a huge popularity since its introduction. Its positive aspects have been reported through many media platforms, and some analyses even showed that ChatGPT achieved a decent grade in professional exams, adding extra support to the claim that AI can now assist and even replace humans in industrial fields. Others, however, doubt its reliability and trustworthiness. This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour, focusing specifically on semantic consistency and the properties of negation, symmetric, and transitive consistency. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions. We also ascertain via experiments that prompt designing, few-shot learning and employing larger large language models (LLMs) are unlikely to be the ultimate solution to resolve the inconsistency issue of LLMs.
Can Large Language Models Logically Predict Myocardial Infarction? Evaluation based on UK Biobank Cohort
Beijing University of Chinese Medicine（北京中医药大学）；Tianjin University（天津大学）；National Yang-Ming University（国立阳明大学）；Taipei Veterans General Hospital（台北荣民总医院）；Beijing Normal University（北京师范大学）
本研究基于 UK Biobank 队列评估 ChatGPT 和 GPT-4 逻辑预测心肌梗死风险的能力，发现当前 LLMs 性能不及传统模型，尚未准备好应用于临床医学。
The Application of ChatGPT in Responding to Questions Related to the Boston Bowel Preparation Scale
First Hospital of Quanzhou Affiliated to Fujian Medical University（福建医科大学附属泉州第一医院）；McGill University（麦吉尔大学）；Montreal Neurological Institute（蒙特利尔神经学研究所）；Mila - Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）
An Empirical Exploration of ChatGPT's Ability to Support Problem Formulation Tasks for Mission Engineering and a Documentation of its Performance Variability
University of Southern California（南加州大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Virginia Tech（弗吉尼亚理工大学）
本文通过 NASA 太空任务设计挑战实证评估 ChatGPT-3.5 在任务工程问题表述中识别利益相关者的能力，发现其对人本利益相关者识别较好但对环境因素识别差、抽象层次保持不足且并行输出变异性大，表明其可靠性有限。
From Knowledge Generation to Knowledge Verification: Examining the BioMedical Generative Capabilities of ChatGPT
MGEN -- College of Engineering, Northeastern University Miami（东北大学迈阿密分校MGEN工程学院）；University of Vermont（佛蒙特大学）；AGH, University of Krakow（克拉科夫AGH大学）；Howard University（霍华德大学）
Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives
University of Macau（澳门大学）；Westlake University（西湖大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Cambridge（剑桥大学）；University of Aberdeen（阿伯丁大学）
Large language models for crowd decision making based on prompt design strategies using ChatGPT: models, analysis and challenges
Department of Computer Sciences and Artificial Intelligence, and the Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada（计算机科学与人工智能系，加泰罗尼亚数据科学与计算智能研究所（DaSCI）格拉纳达大学）；Department of Software Engineering, and the Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI) University of Granada（软件工程系，加泰罗尼亚数据科学与计算智能研究所（DaSCI）格拉纳达大学）
Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction
Nanjing University of Science and Technology（南京理工大学）；Northeastern University（东北大学）；Singapore Institute of Technology（新加坡理工大学）；Singapore University of Technology and Design（新加坡科技设计大学）
ChatGPT for automated grading of short answer questions in mechanical ventilation
Shri Atal Bihari Vajpayee Medical College and Research Institute（阿塔尔·比哈里·瓦杰帕伊医学院与研究所）；Bowring and Lady Curzon Hospitals（鲍林与莱迪·柯曾医院）；Rajiv Gandhi University of Health Sciences（拉吉夫·甘地健康科学大学）；Intensive Care Service, Westmead Hospital（韦斯特米德医院重症监护服务部）
Causal Consistency Regularization: Training Verifiably Sensitive Reasoning in Large Language Models
Department of Computer Science, Iowa State University（计算机科学系，爱荷华州立大学）；Department of Civil, Construction & Environmental Engineering, Iowa State University（土木、建筑与环境工程系，爱荷华州立大学）
Understanding Temporal Logic Consistency in Video-Language Models through Cross-Modal Attention Discriminability
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China（北京理工大学计算机科学与技术学院，北京，中国）；Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing Institute of Technology, Beijing, China（高性能语言信息处理与云计算应用北京工程研究中心，北京理工大学，北京，中国）
