Context: Demonstrating high reliability and safety for safety-critical systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in a rigorous way: in particular, results of operational testing with other evidence from design and verification. Growing use of machine learning in SCSs, by precluding most established methods for gaining assurance, makes operational testing even more important for supporting safety and reliability claims. Objective: We use Autonomous Vehicles (AVs) as a current example to revisit the problem of demonstrating high reliability. AVs are making their debut on public roads: methods for assessing whether an AV is safe enough are urgently needed. We demonstrate how to answer 5 questions that would arise in assessing an AV type, starting with those proposed by a highly-cited study. Method: We apply new theorems extending Conservative Bayesian Inference (CBI), which exploit the rigour of Bayesian methods while reducing the risk of involuntary misuse associated with now-common applications of Bayesian inference; we define additional conditions needed for applying these methods to AVs. Results: Prior knowledge can bring substantial advantages if the AV design allows strong expectations of safety before road testing. We also show how naive attempts at conservative assessment may lead to over-optimism instead; why extrapolating the trend of disengagements is not suitable for safety claims; use of knowledge that an AV has moved to a less stressful environment. Conclusion: While some reliability targets will remain too high to be practically verifiable, CBI removes a major source of doubt: it allows use of prior knowledge without inducing dangerously optimistic biases. For certain ranges of required reliability and prior beliefs, CBI thus supports feasible, sound arguments. Useful conservative claims can be derived from limited prior knowledge.
Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
The State Key Laboratory of Blockchain and Data Security, Zhejiang University（区块链与数据安全国家重点实验室，浙江大学）；Sun Yat-sen University（中山大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
Heriot-Watt University（赫瑞-瓦特大学）；City, University of London（伦敦大学城市学院）
There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents, road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing - knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), w…
Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios
Zhongguancun Laboratory（中关村实验室）；Beihang University（北京航空航天大学）；Institute of Data Space, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究院）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；A*STAR（新加坡科技研究局）
Manufacturers of safety-critical systems must make the case that their product is sufficiently safe for public deployment. Much of this case often relies upon critical event outcomes from real-world testing, requiring manufacturers to be strategic about how they allocate testing resources in order to maximize their chances of demonstrating system safety. This work frames the partially observable and belief-dependent problem of test scheduling as a Markov decision process, which can be solved efficiently to yield closed-loop manufacturer testing policies. By solving for policies over a wide ran…
OffTopicEval: When Large Language Models Enter the Wrong Chat, Almost Always!
Nanyang Technological University（南洋理工大学）；Walled AI Labs（Walled AI实验室）；Infocomm Media Development Authority, Singapore（新加坡信息通信媒体发展局）；Lambda Labs（Lambda实验室）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Driving in a dynamic environment that consists of other actors is inherently a risky task as each actor influences the driving decision and may significantly limit the number of choices in terms of navigation and safety plan. The risk encountered by the Ego actor depends on the driving scenario and the uncertainty associated with predicting the future trajectories of the other actors in the driving scenario. However, not all objects pose a similar risk. Depending on the object's type, trajectory, position, and the associated uncertainty with these quantities; some objects pose a much higher ri…
NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalism
School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；Institute for Advanced Algorithms Research, China（中国高等算法研究院）；Northeastern University, China（中国东北大学）；State Key Laboratory of Media Convergence Production Technology and Systems, China（中国媒体融合生产技术与系统国家重点实验室）
Assessing High-Risk AI Systems under the EU AI Act: From Legal Requirements to Technical Verification
Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院（LIST））；LIST, University of Luxembourg（卢森堡大学LIST）；Research Institutes of Sweden (RISE)（瑞典研究院（RISE））
EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety
Department of Electrical & Computer Engineering, Princeton University（普林斯顿大学电气与计算机工程系）；Department of Computer Science, Princeton University（普林斯顿大学计算机科学系）；Department of Computer Science & Engineering, University of Michigan（密歇根大学计算机科学与工程系）；Department of Data Science & Engineering, University of Michigan（密歇根大学数据科学与工程系）；Department of Philosophy, Columbia University（哥伦比亚大学哲学系）；AI Lab, Princeton University（普林斯顿大学人工智能实验室）；Chen Frontier Lab for Al and Mental Health, Tianqiao and Chrissy Chen Institute（天桥及克里斯西·陈研究所人工智能与心理健康前沿实验室）；Theta Health Inc.（Theta健康公司）
Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety
School of Computing and Communications（计算与通信学院）；University of Cambridge（剑桥大学）；School of Software（软件学院）；Nanjing University of Information Science and Technology（南京信息科技大學）；TU Wien（维也纳技术大学）；ICREA
