AI 中文总结
本研究提出TractorBeam浏览器扩展系统，通过协作机器标注隐喻解决语言模型文档问答系统的事实性与来源性问题，初步用户研究显示其可支持用户迭代优化模型输出并促进探索性意义建构。
AI 中文摘要
基于语言模型的文档问答系统已成为流行的意义建构工具，但仍存在事实性和来源性问题，且倾向于支持验证性研究而非探索性研究。我们提出TractorBeam，一种基于浏览器扩展的混合主动系统，将协作标注作为意义建构的界面隐喻，把语言模型（LM）输出重新定义为我们所称的“协作机器标注”过程中的建议高亮。该隐喻可在PDF文档的上下文环境中呈现LM结果，直接解决来源性和事实性问题，同时允许用户在文档上下文环境中迭代构建心理模型和向语言模型提出查询。在初步用户研究中，所有参与者均认为TractorBeam使他们能够评估并迭代改进模型对其预期高亮的反映，部分参与者发现相关建议促使他们重新考虑原有心理模型。该研究表明，支持对单个文档进行探索性研究的系统可为用户带来可验证的意义建构，并能补充适用于更广泛语料库的工具。
英文摘要
Language model-based systems which allow asking questions of documents have become popular tools for sensemaking. Despite their implied capability, these systems still suffer from issues of factuality and provenance, while encouraging confirmatory, rather than exploratory, research. We present TractorBeam, a browser extension-based mixed-initiative system that uses collaborative annotation as an interface metaphor for sensemaking, re-framing language model (LM) outputs as suggested highlights in a process that we call collaborative machine annotation. This metaphor allows us to present LM results in-context on PDF documents, directly addressing concerns of provenance and factuality, while allowing users to iteratively construct mental schemas and queries for language models directly in the context of a document. In a preliminary user study, all of our participants felt that TractorBeam enabled them evaluate and iteratively improve the model's reflection of their intended highlighting, and several found suggestions that made them reconsider their original schema. TractorBeam suggests that systems that facilitate exploratory research on individual documents may lead to verifiable sensemaking for users and complement tools that work across broader corpora.
CommentsUIST Poster Extended Abstract
