随机程序的几何与同调性质的统计研究
Statistical investigations into the geometry and homology of random programs
AI总结：
本文提出利用 Tree-edit 距离、空间点统计和拓扑数据分析等不依赖嵌入近似的方法，研究随机程序在几何与拓扑上的构型关系，并通过比较 ChatGPT-4 与 TinyLlama 验证其有效性。
摘要：
AI 支持的编程随着 Meta 的 Llama 和 openAI 的 chatGPT 等工具的出现取得了巨大飞跃。这些工具是程序的随机来源，已经极大地影响了我们编写代码和教授编程的方式。如果我们将输入到此类模型的内容视为随机来源，一个自然的问题是：输入分布与输出分布之间、chatGPT 提示与最终生成的程序之间存在什么关系？在本文中，我们将展示如何利用程序语法树之间的 Tree-edit 距离，在不显式建模底层空间的情况下，从几何和拓扑角度描述由 chatGPT 生成的随机 Python 程序之间的关系。研究度量空间中高维样本的一种流行方法是使用低维嵌入，例如多维缩放。此类方法会产生取决于数据和嵌入空间维度的误差。在本文中，我们提议将此类投影方法限制为纯粹的可视化用途，转而使用几何汇总统计量、空间点统计方法以及拓扑数据分析来刻画随机程序的构型，这些方法不依赖嵌入近似。为证明其有效性，我们比较了两个公开可用的模型：ChatGPT-4 和 TinyLlama，在一个与图像处理相关的简单问题上进行实验。应用领域包括：理解应如何提问以获得有用的程序；衡量给定大语言模型回答的一致性；以及比较不同大语言模型作为编程助手时的表现。最后，我们推测我们的方法未来可能为编程语言的结构提供新的见解。
英文摘要：
AI-supported programming has taken giant leaps with tools such as Meta's Llama and openAI's chatGPT. These are examples of stochastic sources of programs and have already greatly influenced how we produce code and teach programming. If we consider input to such models as a stochastic source, a natural question is, what is the relation between the input and the output distributions, between the chatGPT prompt and the resulting program? In this paper, we will show how the relation between random Python programs generated from chatGPT can be described geometrically and topologically using Tree-edit distances between the program's syntax trees and without explicit modeling of the underlying space. A popular approach to studying high-dimensional samples in a metric space is to use low-dimensional embedding using, e.g., multidimensional scaling. Such methods imply errors depending on the data and dimension of the embedding space. In this article, we propose to restrict such projection methods to purely visualization purposes and instead use geometric summary statistics, methods from spatial point statistics, and topological data analysis to characterize the configurations of random programs that do not rely on embedding approximations. To demonstrate their usefulness, we compare two publicly available models: ChatGPT-4 and TinyLlama, on a simple problem related to image processing. Application areas include understanding how questions should be asked to obtain useful programs; measuring how consistently a given large language model answers; and comparing the different large language models as a programming assistant. Finally, we speculate that our approach may in the future give new insights into the structure of programming languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
