LLMography：将人机对话转化为可追溯性、监督和可审计性指标
LLMography: Transforming Human-AI Conversations into Traceability, Oversight, and Auditability Indicators
AI总结：
提出LLMography框架，通过分析人机对话历史生成可追溯性、人类贡献、AI依赖等KPI指标，促进AI透明度从输出检测转向交互过程记录。
摘要：
大型语言模型（LLM）在教育、软件工程、学术写作和技术文档中的日益普及引发了一个关键问题：我们如何不仅评估AI辅助的输出，还评估产生这些输出的交互过程？当前的争论通常集中在检测最终成果是否由AI生成，而忽略了揭示人类指导、AI贡献、修正、验证和可追溯性的对话历史。本文介绍了LLMography，一个将人机对话转化为可测量指标（包括来源、人类贡献、AI依赖、可重复性和可审计性）的框架。类比于参考文献和网络文献，LLMography将人类与大型语言模型之间的动态交互轨迹记录为结构化的人机协作痕迹。我们展示了一个原型，该原型分析人机对话痕迹并生成KPI报告，包括提示质量分数、人类指导分数、AI依赖水平、可审计性分数、最终输出可追溯性、隐私风险水平以及推荐的LLMography标签。对19份来自工程学生的匿名审计报告进行了初步探索性评估。大多数交互被分类为人机协作，平均得分为：人类指导86.8/100，提示质量81.9/100，可审计性72.8/100，最终输出可追溯性77.1/100。本文还将LLMography应用于自身的写作过程，分类为人类发起、人类指导、AI辅助的协作生产。研究结果表明，AI透明度应超越输出检测，转向记录交互历史。
英文摘要：
The growing use of Large Language Models (LLMs) in education, software engineering, academic writing, and technical documentation raises a key question: how can we evaluate not only AI-assisted outputs, but also the interaction process that produced them? Current debates often focus on detecting whether a final artifact was generated by AI, while overlooking the conversation history that reveals human direction, AI contribution, corrections, validation, and traceability. This paper introduces LLMography, a framework for transforming Human-AI conversations into measurable indicators of provenance, human contribution, AI dependency, reproducibility, and auditability. By analogy with bibliography and webography, LLMography documents the dynamic trajectory of interaction between a human and a Large Language Model as a structured trace of Human-AI co-production. We present a prototype that analyzes Human-AI conversation traces and generates KPI reports including Prompt Quality Score, Human Direction Score, AI Dependency Level, Auditability Score, Final Output Traceability, Privacy Risk Level, and a recommended LLMography label. A preliminary exploratory evaluation was conducted on 19 anonymized audit reports from engineering students. Most interactions were classified as Human-AI co-produced, with average scores of 86.8/100 for Human Direction, 81.9/100 for Prompt Quality, 72.8/100 for Auditability, and 77.1/100 for Final Output Traceability. The paper also applies LLMography to its own writing process, classified as human-originated, human-directed, AI-assisted co-production. The findings suggest that AI transparency should move beyond output detection toward documenting the history of interaction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
