AI 中文总结
OASIS是一个基于量规的多模态评估平台，利用大语言模型对视频、音频和文本进行评分，具备会话管理、量规版本控制、模态感知执行等功能，已在医学教育中处理超7000个会话，架构领域无关。
AI 中文摘要
OASIS（开放评估与评分基础设施栈）是一个用于使用大语言模型对视频、音频和文本进行基于量规评分的系统平台。使用大语言模型对单个工件进行评分是直接的；但大规模部署评估则需要处理会话管理、量规版本控制、模态感知执行、来源捕获和人工审查。OASIS将独立的命令行界面与规范的集成Elephant + MAPLES栈配对，用于会话管理和多模态评分编排。两种路径都可以通过Ollama和兼容OpenAI的端点（如vLLM）来定位托管API或自托管开放权重模型。SimRubrics量规编写和Wayfinder对话代理网关是可选的扩展，它们使用与人类操作员相同的认证接口。给定量规和记录的会话，OASIS会生成每个标准的分、证据和理由，并保留执行工件以供审计。显著特征包括量规即程序编译、渐进式执行计划、内容可寻址评分身份、转录增强的多模态评分、显式审查状态，以及为人类和自主代理共享的命令界面。尽管是在医学教育中开发的，该架构是领域无关的，适用于任何可以从记录或书面工件中评估结构化表现的地方。自2023年秋季在UT西南医学中心投入生产以来，该平台已处理超过7,000个会话。本出版物包含报告和项目信息，不包含应用程序源代码、二进制文件、安装材料、示例数据或标记的软件发布版本。
英文摘要
OASIS (Open Assessment and Scoring Infrastructure Stack) is a systems platform for rubric-based grading of video, audio, and text with large language models. Scoring one artifact with an LLM is straightforward; deploying assessment at scale requires encounter management, rubric versioning, modality-aware execution, provenance capture, and human review. OASIS pairs a standalone command-line interface with a canonical integrated Elephant + MAPLES stack for encounter management and multimodal grading orchestration. Both paths can target hosted APIs or self-hosted open-weight models through Ollama and OpenAI-compatible endpoints such as vLLM. SimRubrics rubric authoring and the Wayfinder conversational agent gateway are optional extensions that use the same authenticated interfaces as human operators. Given a rubric and recorded encounters, OASIS produces per-criterion scores, evidence, and rationales, preserving execution artifacts for audit. Distinctive features include rubric-as-program compilation, progressive execution plans, content-addressable grading identity, transcript-augmented multimodal grading, explicit review state, and a shared command surface for humans and autonomous agents. Though developed in medical education, the architecture is domain-agnostic, applying wherever structured performance can be evaluated from recorded or written artifacts. In production at UT Southwestern Medical Center since Fall 2023, the platform has processed more than 7,000 encounters. This publication includes the report and project information, not application source, binaries, installation materials, sample data, or a tagged software release.
Comments27 pages, 4 figures; technical report. Project page: https://jamiesonlabutsw.github.io/oasis/
