机器学习决策系统的结构极限：基于信息论、交互与随机动力学的视角
On the Structural Limits of Machine Learning Decision Systems: An Information-Theoretic, Interaction-Based, and Stochastic-Dynamical Perspective
- Universidad Nacional de Tres de Febrero（国立三 Febrero 大学）
- Universidad de Buenos Aires（布宜诺斯艾利斯大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究从信息论等视角分析机器学习决策系统的结构极限，通过Fano型边界等方法推导分类与参数估计的性能极限，指出合适的数据模型是拓展预测能力的前提。
AI中文摘要：
机器学习程序通常以预测精度和计算效率来评估，但它们可达到的性能根本上受底层数据生成过程的结构特性约束，这些特性以信息边界的形式被形式化。本研究从信息论和交互的视角考察数据驱动决策系统的内在极限，通过Fano型边界分析分类任务中可达到的最小误差，利用Cramér-Rao不等式分析参数估计中的精度极限，强调这类极限仅取决于底层模型而非算法的复杂程度。我们进一步讨论独立性、遍历性和分布稳定性等隐含假设如何影响推理过程的有效性，基于交互建模原则综述马尔可夫随机场（Markov Random Fields）等典型框架及编码依赖机制的势能表示，还将包括集成大语言模型（LLM）的智能体架构在内的决策系统描述为反馈驱动的随机过程，其中依赖状态的动力学可能诱导涌现的宏观行为。该视角凸显拥有合适的数据模型是拓展预测能力的前提，将算法学习置于模型施加的信息边界之内。
英文摘要：
Machine learning procedures are commonly evaluated in terms of predictive accuracy and computational efficiency. However, their achievable performance is fundamentally constrained by structural properties of the underlying data-generating process, which are formalized in terms of informational bounds. In this work we examine intrinsic limits of data-driven decision systems from an information-theoretic and interaction-based perspective. We analyze minimal achievable error in classification through Fano-type bounds and precision limits in parametric estimation via the Cramér-Rao inequality, emphasizing that such limits depend on the underlying model rather than on algorithmic sophistication alone. We further discuss how implicit assumptions, such as independence, ergodicity, and distributional stability, affect the validity of inferential procedures. Building on interaction-based modeling principles, we review typical frameworks such as Markov Random Fields and potential based representations for encoding dependence mechanisms. We also describe decision systems, including LLM-integrated agent architectures, as feedback-driven stochastic processes where state-dependent dynamics may induce emergent macroscopic behavior. This perspective highlights the importance of having adequate models for the data as a prerequi- site for expanding predictive capability, and situates algorithmic learning within the informational limits imposed by the models.
