(12) United States Patent (10) Patent No.: US 9,548,050 B2 Gruber Et Al
Total Page:16
File Type:pdf, Size:1020Kb
USO09548050B2 (12) United States Patent (10) Patent No.: US 9,548,050 B2 Gruber et al. (45) Date of Patent: Jan. 17, 2017 (54) INTELLIGENT AUTOMATED ASSISTANT (2013.01); G06F 17/3087 (2013.01); GIOL 15/22 (2013.01); G 10L 15/26 (2013.01); GIOL (75) Inventors: Thomas Robert Gruber, Emerald 21/06 (2013.01) Hills, CA (US); Adam John Cheyer, (58) Field of Classification Search Oakland, CA (US); Dag Kittlaus, San CPC ........ G10L 15/28: G 10L 15/285; G 10L 15/30; Jose, CA (US); Didier Rene Guzzoni, G10L 15/32: G 10L 15/34 Mont-sur-Rolle (CH); Christopher USPC .......................................................... 704/275 Dean Brigham, San Jose, CA (US); See application file for complete search history. Richard Donald Giuli, Arroyo Grande, CA (US); Marcello Bastea-Forte, New (56) References Cited York, NY (US); Harry Joseph U.S. PATENT DOCUMENTS Saddler, Berkeley, CA (US) 1,559,320 A 10, 1925 Hirsh (73) Assignee: Apple Inc., Cupertino, CA (US) 2,180,522 A 11/1939 Henne (Continued) (*) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 FOREIGN PATENT DOCUMENTS U.S.C. 154(b) by 0 days. CA 2666438 C 6, 2013 (21) Appl. No.: 13/492.809 CH 681573 A5 4f1993 (Continued) (22) Filed: Jun. 9, 2012 OTHER PUBLICATIONS (65) Prior Publication Data Alfred App, 2011. http://www.alfredapp.com/, 5 pages. US 2012/0245944 A1 Sep. 27, 2012 (Continued) Primary Examiner — Michael N Opsasnick Related U.S. Application Data (74) Attorney, Agent, or Firm — Morrison & Foerster (63) Continuation of application No. 12/987,982, filed on LLP Jan. 10, 2011. (57) ABSTRACT (Continued) The intelligent automated assistant system engages with the (51) Int. C. user in an integrated, conversational manner using natural GIOL 5/28 (2013.01) language dialog, and invokes external services when appro GIOL I5/18 (2013.01) priate to obtain information or perform various actions. The G06F 7/30 (2006.01) system can be implemented using any of a number of GIOL I5/22 different platforms, such as the web, email, Smartphone, and (2006.01) the like, or any combination thereof. In one embodiment, the G06F 7/28 (2006.01) system is based on sets of interrelated domains and tasks, (Continued) and employs additional functionally powered by external (52) U.S. Cl. services with which the system can interact. CPC ........... G10L 15/1815 (2013.01); G06F 3/167 (2013.01); G06F 9/54 (2013.01); G06F 17/28 24 Claims, 47 Drawing Sheets 050 102 -tose Active Ontology A. Domain Entily Catabases main Models Restaurant 31G ----. ---..... Restaurants Dining Out DE 1682 Colmar Model 1623 8. Business - Name cation 383 162 W AcidgessStreet 1814 1088 y 1088 A. %. Cisles A. Task Fow Models y — is is Wocabulary databases s Event Planning v Y Cusace Tasteriali830 ... Meal618 Event Title period 1087 Party 1817 is 18 A. Party / 088 Dialog Flow Models - E - Service Models get constraints - is restaurant racured for reservation transion service 1872 US 9,548,050 B2 Page 2 Related U.S. Application Data 4,776,016 10, 1988 Hansen 4,783,804 11, 1988 Juang et al. 4,783,807 11, 1988 Marley (60) Provisional application No. 61/295,774, filed on Jan. 4,785,413 11, 1988 Atsumi 18, 2010. 4,790,028 12, 1988 Ramage 4,797,930 1, 1989 Goudie (51) Int. Cl. 4,802,223 1, 1989 Lin et al. G06F 3/16 4,803,729 2, 1989 Baker (2006.01) 4,807,752 2, 1989 Chodorow G06F 9/54 (2006.01) 4,811,243 3, 1989 Racine GIOL I5/26 (2006.01) 4,813,074 3, 1989 Marcus GIOL 2/06 (2013.01) 4,819,271 4, 1989 Bahl et al. 4,827,518 5, 1989 Feustel et al. References Cited 4,827,520 5, 1989 Zeinstra (56) 4,829,576 5, 1989 Porter 4,829,583 5, 1989 Monroe et al. U.S. PATENT DOCUMENTS 4,831,551 5, 1989 Schalk et al. 4,833,712 5, 1989 Bahl et al. 2,495.222 1, 1950 Bierig 4,833,718 5, 1989 Sprague 3,704,345 11, 1972 Coker et al. 4,837,798 6, 1989 Cohen et al. 3,710,321 1, 1973 Rubenstein 4,837,831 6, 1989 Gillick et al. 3,787,542 1, 1974 Gallagher et al. 4,839,853 6, 1989 Deerwester et al. 3,828,132 8, 1974 Flanagan et al. 4,852,168 7, 1989 Sprague 3,979,557 9, 1976 Schulman et al. 4,862.504 8, 1989 Nomura 4,013,085 3, 1977 Wright 4,875,187 10, 1989 Smith 4,081,631 3, 1978 Feder 4,878,230 10, 1989 Murakami et al. 4,090,216 5, 1978 Constable 4,887.212 12, 1989 Zamora et al. 4,107,784 8, 1978 Van Bemmelen 4,896.359 1, 1990 Yamamoto et al. 4,108,211 8, 1978 Tanaka 4,903,305 2, 1990 Gillick et al. 4,159,536 6, 1979 Kehoe et al. 4.905,163 2, 1990 Garber et al. 4,181,821 1, 1980 Pirz et al. 4,908,867 3, 1990 Silverman 4,204,089 5, 1980 Key et al. 4.914,586 4, 1990 Swinehart et al. 4,241,286 12, 1980 Gordon 4.914,590 4, 1990 Loatman et al. 4,253,477 3, 1981 Eichman 4,918,723 4, 1990 Iggulden et al. 4.278,838 T. 1981 Antonov 4,926,491 5, 1990 Maeda et al. 4,282.405 8, 1981 Taguchi 4,928.307 5, 1990 Lynn 4,310,721 1, 1982 Manley et al. 4,931,783 6, 1990 Atkinson 4,332,464 6, 1982 Bartulis et al. 4,935,954 6, 1990 Thompson et al. 4,348,553 9, 1982 Baker et al. 4,939,639 7, 1990 Lee et al. 4,384,169 5, 1983 Mozer et al. 4.941,488 7, 1990 Marxer et al. 4,386,345 5, 1983 Narveson et al. 4,944,013 7, 1990 Gouvianakis et al. 4.433,377 2, 1984 Eustis et al. 4,945,504 7, 1990 Nakama et al. 4.451,849 5, 1984 Fuhrer 4,953, 106 8, 1990 Gansner et al. 4.485439 11, 1984 Rothstein 4,955,047 9, 1990 Morganstein et al. 4,495,644 1, 1985 Parks et al. 4,965,763 10, 1990 Zamora 4.513,379 4, 1985 Wilson et al. 4,972.462 11, 1990 Shibata 4,513.435 4, 1985 Sakoe et al. 4,974,191 11, 1990 Amirghodsi et al. 4,555,775 11, 1985 Pike 4,975,975 12, 1990 Filipski 4,577,343 3, 1986 Oura 4,977,598 12, 1990 Doddington et al. 4,586,158 4, 1986 Brandle 4,980,916 12, 1990 Zinser 4,587,670 5, 1986 Levinson et al. 4,985,924 1, 1991 Matsuura 4,589,022 5, 1986 Prince et al. 4,992,972 2, 1991 Brooks et al. 4,611,346 9, 1986 Bednar et al. 4,994,966 2, 1991 Hutchins 4,615,081 10, 1986 Lindahl 4,994,983 2, 1991 Landell et al. 4,618,984 10, 1986 Das et al. 5,001,774 3, 1991 Lee 4,642,790 2, 1987 Minshull et al. 5,003,577 3, 1991 Ertz et al. 4,653,021 3, 1987 Takagi 5,007,095 4, 1991 Nara et al. 4,654,875 3, 1987 Srihari et al. 5,007,098 4, 1991 Kumagai 4,655,233 4, 1987 Laughlin 5,010,574 4, 1991 Wang 4,658.425 4, 1987 Julstrom 5,016,002 5, 1991 Levanto 4,670,848 6, 1987 Schramm 5,020, 112 5, 1991 Chou 4,677,570 6, 1987 Taki 5,021,971 6, 1991 Lindsay 4,680,429 7, 1987 Murdock et al. 5,022,081 6, 1991 Hirose et al. 4,680,805 7, 1987 Scott 5,027,110 6, 1991 Chang et al. 4,686,522 8, 1987 Hernandez et al. 5,027.406 6, 1991 Roberts et al. 4,688,195 8, 1987 Thompson et al. 5,027,408 6, 1991 Kroeker et al. 4,692,941 9, 1987 Jacks et al. 5,029,211 7, 1991 Ozawa 4,698.625 10, 1987 McCaskill et al. 5,031,217 7, 1991 Nishimura 4,709,390 11, 1987 Atal et al. 5,032,989 7, 1991 Tornetta 4,713,775 12, 1987 Scott et al. 7, 1991 Bahl et al. 4,718,094 1, 1988 Bahl et al. 5,033,087 4,724,542 2, 1988 Williford 5,040.218 8, 1991 Vitale et al. 4,726,065 2, 1988 Froessl 5,046,099 9, 1991 Nishimura 4,727,354 2, 1988 Lindsay 5,047,614 9, 1991 Bianco RE32,632 3, 1988 Atkinson 5,047,617 9, 1991 Shepard et al. 4,736,296 4, 1988 Katayama et al. 5,050,215 9, 1991 Nishimura 4,750,122 6, 1988 Kaji et al. 5,053,758 10, 1991 Cornett et al. 4,754,489 6, 1988 Bokser 5,054,084 10, 1991 Tanaka et al. 4,755,811 T. 1988 Slavin et al. 5,057,915 10, 1991 Von Kohorn 4,759,070 T. 1988 Voroba et al. 5,067,158 11, 1991 Arjmand US 9,548,050 B2 Page 3 (56) References Cited 5,276,794 1, 1994 Lamb, Jr. 5,278.980 1, 1994 Pedersen et al. U.S. PATENT DOCUMENTS 5,282,265 1, 1994 Rohra Suda et al. 5,283,818 2, 1994 Klausner et al. 5,067,503 11, 1991 Stile 5,287,448 2, 1994 Nicol et al.