Programming by Demonstration An Inductive Learning Formulation Tessa A Lau and Daniel S Weld Department of Computer Science and Engineering University of Washington Seattle WA Octob er ftlau weldgcswashingtonedu ABSTRACT Applications that supp ort macros allow users to record a xed sequence of actions and later replay this Although Programming by Demonstration PBD has sequence using a shortcut such as a mouse clickora the p otential to improvethe pro ductivityof unsophis keypress Macros require little sophistication to cre ticated users previous PBD systems have used brittle ate the user simply records a normal interaction heuristic domainsp ecic approaches to executiontrace with the application however macros have limited generalization In this pap er we dene two application utility if the rep etitive task has minor variations indep endent metho ds for p erforming generalization that are based on wellundersto o d machine learning technol ogy TGen uses versionspace generalization and Scripting languages allow sophisticated users to write vs TGen is based on the FOIL inductive logic pro programs to control the application While this typ e foil gramming algorithm We analyze each metho d b oth of customization allows a user the most control it theoretically and empirically arguing that TGen has also requires programming exp erience and knowledge vs lower sample complexity but TGen can learn a of the p otentially complex scripting language and ap foil muchmoreinteresting class of programs plication interface Keywords Programming by Demonstration PBD aims to Programming by Demonstration machine learning in combine the simple interface of macros with the expres ductive logic programming version spaces siveness of a scripting language Like a macro recorder PBD allows users to construct a program by simply p er INTRODUCTION forming actions in the user interface with which they are already familiar Unlike macros however the learned Computer users are largely unable to customize mass programs can contain control structures suchas itera pro duced applications to t their individual tasks This tion conditionals and abstraction problem of enduser customization has b een addressed A complete PBD system consists of two ma jor parts in several ways The trace generalizer constructs a programmatic repre Adjustable preferences and defaults are simple to use sentation of the users actions by generalizing from an but limited to those options considered imp ortantby execution trace to a program capable of pro ducing that the application designer trace The interaction manager explains the resulting program to the user and gains authorization b efore ex Our researchwas greatly improved by discussions with ecuting the program on the users b ehalf and comments from Greg Badros Alan Borning Adam Example Supp ose a user p erformed the following Carlson William Cohen Pedro Domingos Michael Ernst Oren Etzioni Neal Lesh Alon Levy and Mike Perkowitz sequence of actions using an email application This researchwas funded byOceofNaval Research Grant N by National Science Foundation Grant Goto Message whose sender is Corey IRI and by ARPA Rome Labs grant F Goto Message whose sender is Oren Goto Message whose sender is Dan Forward the current message to Tessa Goto Message whose sender is Corey Goto Message whose sender is Dan Forward the current message to Tessa Goto Message whose sender is Corey Goto Message whose sender is Corey sequences Cima learns text editing commands by Given this execution trace a PBD trace generalizer example it relies on user hints rather than multiple ex might conjecture that the user is forwarding to Tessa amples to disambiguate p ossible programs Chimera all email messages sent from Dan and output the fol generalizes constraints b etween graphical ob jects from lowing pro cedure multiple snapshots and provides a macrobyexample fa For each message m in Mailbox cility for creating macros in a graphical editing domain Gotom Tinker supp orts programming Lisp by demonstra If Senderm is Dan tion but p erforms no inference instead relying on the Then Forwardm Tessa user to disambiguate the examples The PBD systems interaction manager would describ e These PBD systems all rely on heuristic rules to the pro cedure to the user accept any usersp ecied re nements to the program oer to execute the co de describ e when a generalization is appropriate These highlight the resulting eects and implement an undo heuristics make PBD systems like the rulebased ex p ert systems on which they are mo delled brittle lab o facility This pap er fo cuses on the problem of PBD trace gen rious to construct and dicult to extend For instance Eager is hardco ded to recognize a xed set of sequences eralization Previous PBD systems have used brittle over which a user may iterate suchas the email mes heuristic approaches In contrast we dene two ro bust applicationindependent metho ds for p erforming sages in a mailb ox or the days of the week Peridot uses a set of domaindep endent conditionaction rules to in generalization based on wellundersto o d machine learn fer constraints and lo ops Unfortunately these fragile ing technology heuristics must be handcrafted for each domain In Versionspace generalization TGen vs contrast weadvo cate using established machine learn ing techniques to p erform robust trace generalization Inductive logic programming TGen foil As we demonstrate in the remainder of this pap er our approach also yields low sample complexity TGen We theoretically and empirically analyze these two im vs and TGen are capable of generalizing from a small plemented systems oering a rational reconstruction of foil numb er of training examples asp ects of the Eager PBD system TGen can vs Others have considered applying domainindep endent learn with fewer examples but TGen can learn a foil algorithms to PBD Witten et al identify shortcom much more interesting class of programs such as pro ings in current machine learning approaches when ap grams with variablelength lo op iterations For exam plied to PBD such as an inability to take advantage ple TGen quickly learns a program equivalentto foil of domain knowledge or user hints Paynter sketches the one shown ab ove from the training examples shown out a generalpurp ose PBD framework using machine PRIOR WORK learning NevillManning and Witten de scrib e a lineartime algorithm for detecting hierarchical A complete PBD system is comp osed of trace general structure in sequences by generalizing a grammar from izer and interaction manager comp onents Generalizing rep eated subsequences in a single example Their al from an execution trace to a program involves among gorithm is elegantly simple but it is not obvious how other things determining the length of an iterativeloop to apply background knowledge to bias the generated noticing regular changes in each iteration of suchaloop grammars and recognizing conditional constructs The interaction Plan recognition typically requires a large li manager on the other hand explains the generaliza brary of p ossible plans whicharechecked against user tion to the user and obtains authorization for program actions This work maybeviewed as inductive learning execution Although prior PBD systems have made sig with a very strong bias the plan library Bauer nicantcontributions to b oth of these comp onents our presents algorithms for constructing plan libraries from work is restricted to trace generalization so wefocuson a corpus of logged user traces and optional action mo d that asp ect in this discussion els Weida and Litman use terminological sub Cypher describ es many PBD systems The Ea sumption with temp oral constraintnetworks to match ger system detects and automates rep etitive ac execution sequences with library plans Lesh and Et tions in Hyp ercard applications Each user action is zioni adapt the version space algorithm to p erform compared against the action history using a pattern goal recognition with an implicit planlibrary represen matching scheme Once the system detects a rep eti tation which is constructed by reasoning ab out action tive sequence it highlights its prediction of the next preconditions and eects action and oers to automate the remainder of the se Our work is similar to previous machine learning ef quence Peridot allows programmers to construct forts at developing selfcustomizing software Schlim user interfaces bydrawing them interactively It uses a mer and Hermens describ e a notetaking system set of conditionaction rules to determine when to infer that predicts what the user is going to write and fa constraints over widget placement and iterations over cilitates completion the system works by learning a matches anytwoaction sequence whose rst command nite state machine FSM mo deling note syntax and copies the sub ject into the clipb oard and whose second decision tree classiers for eachFSM state Maes and action pastes the clipb oard into some le Ahyp othesis Kozierok use nearestneighbor learning adjusted with k wildcards corresp onds to a PBD pro cedure with by user feedback to predict the users next action from a k nested lo op FindS requires a generaltosp ecic the action most recently executed but in contrast to order so wesay that hyp othesis h is equal to or more our work there is no attempt to learn lo ops general than h if and only if for each conjunct c in h i the corresp onding conjunct in h either is equal to c or i PBD AS AN INDUCTIVE LEARNING PROBLEM is the wildcard symbol By watching a users