arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.01382cs.CCcs.LGmath.LO
收藏

从非平凡性与有限性的不可判定性到可学习性的不可判定性

From Undecidability of Non-Triviality and Finiteness to Undecidability of Learnability

Freie Universität Berlin(柏林自由大学)

Matthias C. Caro

AI总结:

该研究证明,在PAC二分类、在线学习和精确学习等框架下,可学习性判定问题在一般意义上是不可判定的,从而否定了存在通用算法评估新学习模型的可能性。

摘要:

机器学习研究者和实践者不断扩展成功学习模型的多样性。他们通过深入的理论分析和经验启发式方法来实现这一目标。然而,目前尚无已知的通用程序来严格评估新提出的模型是否确实能从数据中成功学习。我们证明这样的程序不可能存在。对于PAC二分类、一致与通用在线学习,以及通过教师-学习者交互的精确学习,可学习性在一般情况下是不可判定的,既在形式系统公理独立性的意义上,也在不可计算性的意义上。我们的证明通过可计算构造进行,这些构造将形式系统的一致性问题和图灵机的停机问题编码为某些函数类是否为平凡/有限或高度复杂,然后我们通过已建立的可学习性与复杂度度量的特征化,将这些性质与这些类是否可学习联系起来。我们的工作表明,不可判定性出现在人工智能的理论基础中:不存在一个通用的算法来决定一个机器学习模型能否成功。我们无法在一般情况下自动化评估新学习模型的过程。

英文摘要:

Machine learning researchers and practitioners steadily enlarge the multitude of successful learning models. They achieve this through in-depth theoretical analyses and experiential heuristics. However, there is no known general-purpose procedure for rigorously evaluating whether newly proposed models indeed successfully learn from data. We show that such a procedure cannot exist. For PAC binary classification, uniform and universal online learning, and exact learning through teacher-learner interactions, learnability is in general undecidable, both in the sense of independence of the axioms in a formal system and in the sense of uncomputability. Our proofs proceed via computable constructions that encode the consistency problem for formal systems and the halting problem for Turing machines into whether certain function classes are trivial/finite or highly complex, which we then relate to whether these classes are learnable via established characterizations of learnability through complexity measures. Our work shows that undecidability appears in the theoretical foundations of artificial intelligence: There is no one-size-fits-all algorithm for deciding whether a machine learning model can be successful. We cannot in general automatize the process of assessing new learning models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑