Common Sense Comes Closer to Computers

Common Sense Comes Closer to Computers By John Pavlus April 30, 2020 The problem of common-sense reasoning has plagued the field of artificial intelligence for over 50 years. Now a new approach, borrowing from two disparate lines of thinking, has made important progress. Add a match to a pile of wood. What do you get? For a human, it’s easy. But machines have long lacked the common-sense reasoning abilities needed to figure it out. Guillem Casasús Xercavins for Quanta Magazine One evening last October, the artificial intelligence researcher Gary Marcus was amusing himself on his iPhone by making a state-of-the-art neural network look stupid. Marcus’ target, a deep learning network called GPT-2, had recently become famous for its uncanny ability to generate plausible-sounding English prose with just a sentence or two of prompting. When journalists at The Guardian fed it text from a report on Brexit, GPT-2 wrote entire newspaper-style paragraphs, complete with convincing political and geographic references. Marcus, a prominent critic of AI hype, gave the neural network a pop quiz. He typed the following into GPT-2: What happens when you stack kindling and logs in a fireplace and then drop some matches is that you typically start a … Surely a system smart enough to contribute to The New Yorker would have no trouble completing the sentence with the obvious word, “fire.” GPT-2 responded with “ick.” In another attempt, it suggested that dropping matches on logs in a fireplace would start an “irc channel full of people.” Marcus wasn’t surprised. Common-sense reasoning — the ability to make mundane inferences using basic knowledge about the world, like the fact that “matches” plus “logs” usually equals “fire” — has resisted AI researchers’ efforts for decades. Marcus posted the exchanges to his Twitter account with his own added commentary: “LMAO,” internet slang for a derisive chortle. Neural networks might be impressive linguistic mimics, but they clearly lacked basic common sense. Minutes later, Yejin Choi saw Marcus’ snarky tweet. The timing was awkward. Within the hour Choi was scheduled to give a talk at a prominent AI conference on her latest research project: a system, nicknamed COMET, that was designed to use an earlier version of GPT-2 to perform common-sense reasoning. Quickly, Choi — a computer scientist at the University of Washington and the Allen Institute for Artificial Intelligence — fed COMET the same prompt Marcus had used (with its wording slightly modified to match COMET’s input format): Gary stacks kindling and logs and drops some matches COMET generated 10 inferences about why Gary might be dropping the matches. Not all of the responses made sense, but the first two did: He “wanted to start a fire” or “to make a fire.” Choi tweeted the results in reply to Marcus and strode up to the podium to include them in her presentation. “It seemed only appropriate,” she said. Two Paths to Common Sense Common sense has been called the “dark matter of AI” — both essential and frustratingly elusive. That’s because common sense consists of implicit information — the broad (and broadly shared) set of unwritten assumptions and rules of thumb that humans automatically use to make sense of the world. For example, consider the following scenario: A man went to a restaurant. He ordered a steak. He left a big tip. If you were asked what he ate, the answer — steak — comes effortlessly. But nowhere in that little scene is it ever stated that the man actually ate anything. When Ray Mooney, director of the Artificial Intelligence Laboratory at the University of Texas, Austin, pointed this out after giving me the same pop quiz, I didn’t believe him at first. “People don’t even realize that they’re doing this,” he said. Common sense lets us read between the lines; we don’t need to be explicitly told that food is typically eaten in restaurants after people order and before they leave a tip. Computers do. It’s no wonder that common-sense reasoning emerged as a primary concern of AI research in 1958 (in a paper titled “Programs With Common Sense”), not long after the field of AI was born. “In general, you can’t do natural language understanding or vision or planning without it,” said Ernest Davis, a computer scientist at New York University who has studied common sense in AI since the 1980s. Still, progress has been infamously slow. At first, researchers tried to translate common sense into the language of computers: logic. They surmised that if all the unwritten rules of human common sense could be written down, computers should be able to use them to reason with in the same way that they do arithmetic. This symbolic approach, which came to be known as “good old- fashioned artificial intelligence” (or GOFAI), enabled some early successes, but its handcrafted approach didn’t scale. “The amount of knowledge which can be conveniently represented in the formalisms of logic is kind of limited in principle,” said Michael Witbrock, an AI researcher at the University of Auckland in New Zealand. “It turned out to be a truly overwhelming task.” Even modest attempts to map all possible logical relationships quickly run into trouble. Some of the relationships above always hold (for example, swallowing is always a part of eating). Some hold only occasionally (a person eats at a diner). Some are inconsistent (a person can not eat a cake while it is also in the oven). And nodes such as “cook” can mean both a person who cooks and the activity of cooking. doi: 10.1109/MIS.2009.72 Deep learning with neural networks seemed to offer an alternative. These AI systems, designed to mimic the interconnected layers of neurons in biological brains, learn patterns without requiring programmers to specify them in advance. Over the past decade, increasingly sophisticated neural networks, trained with copious amounts of data, have revolutionized computer vision and natural language processing. But for all their flexibility and apparent intellectual power — neural networks can now steer cars in highway traffic and beat world-class players at chess and Go — these systems remain notorious for their own silly (and occasionally fatal) lapses in ordinary common sense. “Acquiring it, representing it, reasoning with it — it’s all hard,” Davis said. Now, Choi and her collaborators have united these approaches. COMET (short for “commonsense transformers”) extends GOFAI-style symbolic reasoning with the latest advances in neural language modeling — a kind of deep learning that aims to imbue computers with a statistical “understanding” of written language. COMET works by reimagining common-sense reasoning as a process of generating plausible (if imperfect) responses to novel input, rather than making airtight deductions by consulting a vast encyclopedia-like database. “It tries to blend two very fundamentally different approaches to AI,” said Mooney, who is already using COMET in his own research. “It’s an interesting new direction that says, ‘Hey, there’s a middle road there.’” Leora Morgenstern, an expert in common-sense reasoning and AI at the Palo Alto Research Center who has spent decades researching symbolic approaches to the problem, thinks that the ideas behind COMET can help move the field forward. “One of the reasons I’m so excited about what Yejin is doing is I think it will inject new life into the common-sense reasoning community,” she said. “Deep learning is really, really powerful — let’s figure out how to harness it for common sense.” Endless Unwritten Rules Common sense is easier to detect than to define. According to Witbrock, the phrase “common sense” can mean both a kind of knowledge and an attitude toward that knowledge. “I would say [it’s] broadly reusable background knowledge that’s not specific to a particular subject area,” he said. “It’s knowledge that you ought to have.” Like, for example, the fact that people eat food in restaurants, rather than just ordering and paying for it; or that dropping matches on a pile of stacked logs implies that one is trying to light a fire. The implicit nature of most common-sense knowledge makes it difficult and tedious to represent explicitly. “What you learn when you’re two or four years old, you don’t really ever put down in a book,” said Morgenstern. Nevertheless, early AI researchers believed that bridging this gap was possible. “It was like, ‘Let’s write down all the facts about the world. Surely there’s only a couple million of them,’” said Ellie Pavlick, a computer scientist at Brown University. Constructing such a resource, known as a knowledge base, has traditionally been the first step in any approach to automating common-sense reasoning. Building up a sufficient number of obvious facts is harder than it sounds. A common-sense reasoning project called Cyc began in 1984 with the modest-sounding goal of encoding the implicit common-sense knowledge necessary to represent 400 encyclopedia articles. It never stopped. More than three decades later, Cyc’s knowledge base — encoded in a dense, custom-designed logical notation — contains “millions of collections and concepts, and more than 25 million assertions.” Yet a 2015 review article by Davis and Marcus stated that “Cyc has had comparatively little impact on AI research.” Subsequent attempts to write entries for a knowledge base — or to create one by mining documents using machine learning — have failed to crack the common-sense reasoning problem. Why? For one thing, “there’s always exceptions to every case,” Pavlick explained. “If I hear some statement like ‘it’s raining,’ I could infer that if I go outside I’ll get wet, but not if [I’m] underneath something.” Other exceptions are harder to anticipate.

Load more