Although AI systems are increasingly being leveraged to provide value to organizations, individuals, and society, significant attendant risks have been identified and have manifested. These risks have led to proposed regulations, litigation, and general societal concerns.
As with any promising technology, organizations want to benefit from the positive capabilities of AI technology while reducing the risks. The best way to reduce risks is to implement comprehensive AI lifecycle governance where policies and procedures are described and enforced during the design, development, deployment, and monitoring of an AI system. Although support for comprehensive governance is beginning to emerge, organizations often need to identify the risks of deploying an already-built model without knowledge of how it was constructed or access to its original developers. Such an assessment will quantitatively assess the risks of an existing model in a manner analogous to how a home inspector might assess the risks of an already-built home or a physician might assess overall patient health based on a battery of tests.
Several AI risks can be quantified using metrics from the technical community. However, there are numerous issues in deciding how these metrics can be leveraged to create a quantitative AI risk assessment. This paper explores these issues, focusing on the opportunities, challenges, and potential impacts of such an approach, and discussing how it might influence AI regulations.
A Review of Generative AI in Computer Science Education: Challenges and Opportunities in Accuracy, Authenticity, and Assessment
School of Computer Science and Engineering, California State University, San Bernardino, USA(加州州立大学圣伯纳迪诺分校计算机科学与工程学院);School of Information Systems and Technology, San Jose State University, USA(旧金山州立大学信息系统与技术学院);University of California, San Diego, USA(加州大学圣地亚哥分校)
CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment
Institute for Artificial Intelligence, University of Central Florida(中佛罗里达大学人工智能研究所);Nemours Cardiac Center, Nemours Children’s Hospital, Florida(佛罗里达州内穆尔斯儿童医院内穆尔斯心脏中心);Children’s Heart Center, WVU Golisano Children’s, West Virginia(西弗吉尼亚大学戈利萨诺儿童医院儿童心脏中心);Department of Clinical Sciences, College of Medicine, University of Central Florida(中佛罗里达大学医学院临床科学系)
Generative AI for Health Technology Assessment: Opportunities, Challenges, and Policy Considerations
National Institute of Biomedical Imaging and Bioengineering, National Institutes of Health(美国国立卫生研究院国家生物医学成像与生物工程研究所);University of Florida(佛罗里达大学);University of Florida Health(佛罗里达大学健康科学中心);Tulane University School of Public Health and Tropical Medicine(杜兰大学公共卫生与热带医学学院);Intelligent Medical Objects(智能医疗对象公司);Yale University(耶鲁大学);National Institute for Health and Care Excellence(英国国家卫生与临床优化研究所);Cairo University(开罗大学);ISPOR - The Professional Society for Health Economics and Outcomes Research(国际药物经济学与结果研究学会);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Harvard University(哈佛大学)
Learning the structure of Bayesian Networks: A quantitative assessment of the effect of different algorithmic schemes
Universitá degli Studi di Milano-Bicocca(米兰比可卡大学);NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa(里斯本大学学院NOVA信息管理学院);INESC Coimbra(科英布拉系统与计算机工程研究所);University of Coimbra(科英布拉大学);Stanford University(斯坦福大学)
The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges
University of Alberta(阿尔伯塔大学);Federation of American Scientists(美国科学家联合会);Educational Testing Service(教育考试服务中心);University of Maryland(马里兰大学);University of Iowa(爱荷华大学);Cambium Assessment(坎比姆评估公司);Washington State University(华盛顿州立大学);Acuity Insights(敏锐洞察公司);PSI Services(PSI服务公司);University of Delaware(特拉华大学);Concordia University of Edmonton(埃德蒙顿康考迪亚大学)
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
Arizona State University(亚利桑那州立大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Northwestern University(西北大学);University of California, Berkeley(加州大学伯克利分校);Emory University(埃默里大学)
Fairboard: a quantitative framework for equity assessment of healthcare models
Queen Square Institute of Neurology, University College London(伦敦大学学院皇后广场神经科学研究所);Dept. of Physiology, Development and Neuroscience, University of Cambridge(剑桥大学生理学、发育与神经科学系);Laboratoire Bordelais de Recherche en Informatique (LABRI), Université de Bordeaux(波尔多大学波尔多计算机科学实验室)