Machine Learning Comes up Against Unsolvable Problem Researchers Run Into a Logical Paradox Discovered by Famed Mathematician Kurt Gödel

IN FOCUS NEWS

MATHEMATICS Machine learning comes up against unsolvable problem Researchers run into a logical paradox discovered by famed mathematician Kurt Gödel.

BY DAVIDE CASTELVECCHI considered to be the foundation for all of mathematics. team of researchers has stum- Gödel and Cohen’s work on the con- bled on a question that is tinuum hypothesis implies that there mathematically unanswerable can exist parallel mathematical uni- Abecause it is linked to logical paradoxes verses that are both compatible with discovered by Austrian mathematician standard mathematics — one in which Kurt Gödel in the 1930s. the continuum hypothesis is added to The mathematicians, who were the standard axioms and therefore working on a machine-learning prob- declared to be true, and another in lem, show that the question of ‘learn- which it is declared false.

ALFRED EISENSTAEDT/LIFE PICTURE COLL./GETTY PICTURE ALFRED EISENSTAEDT/LIFE ability’ — whether an algorithm can extract a pattern from limited data LEARNABILITY LIMBO — is linked to a paradox known as the In the latest paper, Yehudayoff and his continuum hypothesis. Gödel showed collaborators define learnability as the that this cannot be proved either true ability to make predictions about a large or false using standard mathematical data set by sampling a small number language. The latest result were pub- of data points. The link with Cantor’s lished on 7 January (S. Ben-David et al. problem is that there are infinitely many Nature Mach. Intel. 1, 44–48; 2019). ways of choosing the smaller set, but the “For us, it was a surprise,” says Amir size of that infinity is unknown. Yehudayoff at the Technion–Israel The authors go on to show that if Institute of Technology in Haifa, who the continuum hypothesis is true, a is a co-author on the paper. He says that small sample is sufficient to make the although there are a number of tech- extrapolation. But if it is false, no finite nical maths questions that are known sample can ever be enough. In this way, to be similarly ‘undecidable’, he did they show that the problem of learn- not expect this phenomenon to show ability is equivalent to the continuum up in a seemingly simple problem in hypothesis. Therefore, the learnability machine learning. Austrian mathematician Kurt Gödel. problem, too, is in a state of limbo that John Tucker, a computer scientist can be resolved only by choosing the at Swansea University, UK, says that the as the sets in Venn diagrams. In particular, it axiomatic universe. paper is “a heavyweight result on the limits relates to the different sizes of sets containing The result also helps to give us a broader of our knowledge”, with foundational impli- infinitely many objects. understanding of learnability, Yehudayoff cations for both mathematics and machine Georg Cantor, the founder of set theory, says. “This connection between compression learning. demonstrated in the 1870s that not all infinite and generalization is really fundamental if you sets are created equal: in particular, the set of want to understand learning.” NOT ALL SETS ARE EQUAL integer numbers is ‘smaller’ than the set of all Researchers have discovered a number of Researchers often define learnability in terms real numbers, also known as the continuum. similarly ‘undecidable’ problems, says Peter of whether an algorithm can generalize its (The real numbers include the irrational O’Hearn, a computer scientist at University knowledge. The algorithm is given the answer numbers, as well as rationals and integers.) College London. In particular, following work to a ‘yes or no’ question — such as “Does this Cantor also suggested that there cannot be by Gödel, Alan Turing — who co-founded image show a cat?” — for a limited number of sets of intermediate size — that is, larger than the theory of algorithms — found a class of objects, and then has to guess the answer for the integers but smaller than the continuum. questions that no computer program can be new objects. But he was not able to prove this continuum guaranteed to answer in any finite number Yehudayoff and his collaborators arrived at hypothesis, and nor were many mathemati- of steps. their result while investigating the connection cians and logicians who followed him. But the undecidability in the latest results between learnability and ‘compression’, which Their efforts were in vain. A 1940 result by is “of a rare kind”, and much more surprising, involves finding a way to summarize the salient Gödel (which was completed in the 1960s by O’Hearn adds: it points to what Gödel found features of a large set of data in a smaller set of US mathematician Paul Cohen) showed that to be an intrinsic incompleteness in any math- data. The authors discovered that the infor- the continuum hypothesis cannot be proved ematical language. The findings will probably mation’s ability to be compressed efficiently either true or false starting from the standard be important for the theory of machine learn- boils down to a question in the theory of sets axioms — the statements taken to be true — ing, he adds, although he is “not sure it will — mathematical collections of objects such of the theory of sets, which are commonly have much impact on the practice”. ■