AAAI 2008 Workshop Reports

University of Pennsylvania ScholarlyCommons

Technical Reports (CIS) Department of Computer & Information Science

5-2009

Mark Dredze University of Pennsylvania, [email protected]

Follow this and additional works at: https://repository.upenn.edu/cis_reports

Recommended Citation Mark Dredze, "AAAI 2008 Workshop Reports", . May 2009.

Sarabjot Singh Anand, Razvan Bunescu, Vitor Carvalho, Jan Chomicki, Vincent Conitzer, Michael T Cox, Virginia Dignum, Zachary Dodds, Mark Dredze, David Furcy, Evgeniy Gabrilovich, Mehmet H Göker, Hans Guesgen, Haym Hirsh, Dietmar Jannach, Ulrich Junker. (2009, April). AAAI 2008 Workshop Reports. AI Magazine, 30(1), 108-118. Copyright AAAI 2009. The copies do not imply AAAI endorsement of a product or a service of the employer, and that the copies are not for sale. Publisher URL: http://proquest.umi.com/ pqdlink?did=1680350011&sid=1&Fmt=2&clientId=3748&RQT=309&VName=PQD

This paper is posted at ScholarlyCommons. https://repository.upenn.edu/cis_reports/904 For more information, please contact [email protected]. AAAI 2008 Workshop Reports

Abstract AAAI was pleased to present the AAAI-08 Workshop Program, held Sunday and Monday, July 13-14, in Chicago, Illinois, USA. The program included the following 15 workshops: Advancements in POMDP Solvers; AI Education Workshop Colloquium; Coordination, Organizations, Institutions, and Norms in Agent Systems, Enhanced Messaging; Human Implications of Human-Robot Interaction; Intelligent Techniques for Web Personalization and Recommender Systems; Metareasoning: Thinking about Thinking; Multidisciplinary Workshop on Advances in Preference Handling; Search in Artificial Intelligence and Robotics; Spatial and Temporal Reasoning; Trading Agent Design and Analysis; Transfer Learning for Complex Tasks; What Went Wrong and Why: Lessons from AI Research and Applications; and Wikipedia and Artificial Intelligence: An vE olving Synergy.

Comments Sarabjot Singh Anand, Razvan Bunescu, Vitor Carvalho, Jan Chomicki, Vincent Conitzer, Michael T Cox, Virginia Dignum, Zachary Dodds, Mark Dredze, David Furcy, Evgeniy Gabrilovich, Mehmet H Göker, Hans Guesgen, Haym Hirsh, Dietmar Jannach, Ulrich Junker. (2009, April). AAAI 2008 Workshop Reports. AI Magazine, 30(1), 108-118.

Copyright AAAI 2009. The copies do not imply AAAI endorsement of a product or a service of the employer, and that the copies are not for sale. Publisher URL: http://proquest.umi.com/ pqdlink?did=1680350011&sid=1&Fmt=2&clientId=3748&RQT=309&VName=PQD

This technical report is available at ScholarlyCommons: https://repository.upenn.edu/cis_reports/904 Reports

AAAI 2008 Workshop Reports

Sarabjot Singh Anand, Razvan Bunescu, Vitor Carvalho, Jan Chomicki, Vincent Conitzer, Michael T. Cox, Virginia Dignum, Zachary Dodds, Mark Dredze, David Furcy, Evgeniy Gabrilovich, Mehmet H. Göker, Hans Guesgen, Haym Hirsh, Dietmar Jannach, Ulrich Junker, Wolfgang Ketter, Alfred Kobsa, Sven Koenig, Tessa Lau, Lundy Lewis, Eric Matson, Ted Metzler, Rada Mihalcea, Bamshad Mobasher, Joelle Pineau, Pascal Poupart, Anita Raja, Wheeler Ruml, Norman Sadeh, Guy Shani, Daniel Shapiro, Trey Smith, Matthew E. Taylor, Kiri Wagstaff, William Walsh, and Rong Zhou

I AAAI was pleased to present the AAAI-08 Workshop Program, held Sunday and Mon- T day, July 13–14, in Chicago, Illinois, USA. he Workshop on Advancements in POMDP Solvers brought to- The program included the following 15 gether active researchers in the area of solving partially observable workshops: Advancements in POMDP Markov decision processes (POMDPs). Participants discussed various Solvers; AI Education Workshop Colloqui- approaches to solving POMDPs, and discussed as well as potential re- um; Coordination, Organizations, Institu- tions, and Norms in Agent Systems, En- al real-world applications of the model. The AI Education Colloquium hanced Messaging; Human Implications of kicked off AAAI 2008’s AI Forum, a series of events on the teaching Human-Robot Interaction; Intelligent Tech- and learning of AI. The colloquium convened AI practitioners pas- niques for Web Personalization and Recom- sionate about improving both their students’ and their own appreci- mender Systems; Metareasoning: Thinking ation of our field’s compelling ideas. The goal of the workshop was to about Thinking; Multidisciplinary Work- examine and define the current state of the art research in agent sys- shop on Advances in Preference Handling; tems research related to coordination, organizations, institutions, and Search in Artificial Intelligence and Robotics; Spatial and Temporal Reasoning; Trading norming. The Enhanced Messaging workshop brought together re- Agent Design and Analysis; Transfer Learn- searchers from across the AI and computer science spectrum to discuss ing for Complex Tasks; What Went Wrong the state of research on e-mail and information overload. New con- and Why: Lessons from AI Research and Ap- nections between participants are driving forward work in this area plications; and Wikipedia and Artificial In- and building a new research community. The Human Implications of telligence: An Evolving Synergy. Human-Robot Interaction (HRI) workshop concerned aspects of HRI that particularly call for multidisciplinary research and dialogue, representing AI and robotics as well as disciplines such as psychology, theology, sociology, and philosophy. The Intelligent Techniques for Web Personalization and Recommender Systems workshop was scheduled as a joint event, bringing together researchers and practitioners from the fields of web personalization and recommender sys-

108 AI MAGAZINE Copyright © 2008, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602 Reports tems. It focused on current and emerg- oped 10 years ago were hardly able to traditionally handled when computing ing topics related to web intelligence, handle more than 10 states, while a POMDP policy. Participants agreed particularly its application to recom- modern solvers can handle domains that it is important to better under- mender systems. The goal of the with millions of states. New tech- stand such concerns in order to apply Metareasoning workshop was to ex- niques focus on computing approxi- POMDPs to real-world domains. plore the implications of a proposed mate policies of manageable complex- In the third part of the workshop, re- model for metareasoning by examin- ity, thus allowing us to handle these searchers presented new technical con- ing its aspects, its use as a model of larger and more complicated POMDPs. tributions in POMDP solvers. While self, and its role in single-agent and This advancement was achieved by a point-based methods and finite state multiagent applications. The Ad- few orthogonal approaches—the use of controllers still offer many opportuni- vances in Preference Handling work- point-based techniques, finite-state ties for scaling up and continue to shop highlighted recent progress in controllers, efficient model representa- present many interesting open ques- eliciting and exploiting preferences for tions, model compression techniques, tions, researchers also presented work computational tasks from artificial in- hierarchical decompositions, infer- in other directions. We heard interest- telligence, databases, and operations ence-based techniques, and improved ing ideas pertaining to multiagent sce- research. The Search in Artificial Intel- algorithms for online search. narios, POMDPs with continuous pa- ligence and Robotics workshop The first part of the workshop pro- rameters, and the integration of expert brought together search researchers to vided an overview of a number of knowledge into solutions. The discus- share their ideas and disseminate their these approaches. These tutorials at- sions throughout the meeting indicat- latest research results. It focused on tracted many researchers from nearby ed that POMDP researchers are inter- finding common ground between areas, such as planning, who were in- ested in strengthening the community search techniques used in artificial interested in learning about the new de- and its impact on the development of telligence and robotics with great suc- velopments in the field. We began by a autonomous systems. We will hence cess. The Workshop on Spatial and tutorial on point-based value iteration investigate several methods for sup- Temporal Reasoning brought together methods. These methods, which con- porting research in this area, such as related communities of researchers tributed much to the scaling up of offering a community web page, main- with an interest in the study of repre- POMDP solvers, compute a solution taining lists of active researchers, a bib- senting and reasoning about either over a subset of the belief space using liography of relevant papers, tutorials space or time—or both. The Trading the point-based backup operator. Next, and presentations, and links to rele- Agent Design and Analysis workshop we presented a tutorial on solving vant software. Finally, as many re- focused on the design and evaluation POMDPs through an online search searchers showed interest in addition- of trading agents. The Transfer Learn- over the belief space, starting at every al meetings, we decided to hold a sec- ing for Complex Tasks workshop cov- time step from the current belief state. ond workshop next year. This ered a wide range of topics, including The key challenges for online methods workshop will focus on bringing to- regression, classification, reinforce- include using efficient techniques for gether researchers that develop new al- ment learning, planning, Markov log- pruning the search space. We then dis- gorithmic contributions for POMDPs ic networks, and neural networks. The cussed policy iteration using finite and researchers that apply POMDPs for What Went Wrong and Why work- state controllers (FSCs), another im- real-world problems. shop at AAAI-08 was dedicated to the portant method that has shown the Joelle Pineau, Pascal Poupart, Guy propositions that insight often begins ability to scale up significantly. This tu- Shani, and Trey Smith organized the with unexpected results, and that clar- torial provided an overview of the workshop. The papers are available as ity arrives in the ensuing response. practical concerns of using FSCs, fo- AAAI Press Technical Report WS-08-02. The goals of the Wikipedia and Artifi- cusing on bounded memory con- cial Intelligence workshop were to in- trollers as a possible method for limit- AI Education Colloquium vestigate the mutually beneficial inter- ing the exponential growth of the FSC. action between Wikipedia and AI and The final tutorial reviewed methods for The AI Education Colloquium arose to foster a discussion on new applica- exploiting domain structure, focusing from the premise that teaching and tions and research directions that on factored POMDPs and algebraic de- learning AI is where we, from any disci- could benefit from this increasingly cision diagrams (ADDs) to efficiently pline within our broad community, important relationship. represent and compute policies. have the most to offer one another. The The second part of the workshop in- 11 talks and nine poster presentations Advancements in cluded presentations on several exam- spanned the full range of education’s POMDP Solvers ples of real-world applications of scale space: pedagogical strategies, cur- POMDPs, including assistance to elder- ricular innovations, and outreach to Over the past decade, much advance- ly people using robots and cognitive re- new audiences that leverage AI in inno- ment was achieved in the field of minders and a spoken dialog system vative ways. Taken together, this di- POMDP solvers. The size of POMDPs aimed at fixing Internet connectivity verse set of contributions suggests that that solvers can handle has increased problems. The demonstrations focused AI education and AI research are two by orders of magnitude. Solvers devel- on unexpected concerns that are not phrases describing a single pursuit.

SPRING 2009 109 Reports

In any room of educators, teaching ity—was pushed even further as the lem for the design of open complex tips will out. Suggestions included on- group asked, “How can AI education multiagent systems. line resources that enable students to serve more than just the AI communi- In recent years, social and organiza- contrast old and new AI systems, for ty?” In answer, participants presented tional aspects of agency have become example, Eliza and current chatbots. non-AI courses in which AI served as a a major issue in MAS research. Recent For material less easily “demoed,” em- central theme. Several used robots to applications of MAS on web services, ulating peer-reviewing techniques can motivate students to think computa- grid computing, and ubiquitous com- deepen student engagement with pri- tionally in early computer science or at puting enforce the need for using mary-source papers. “AI in the real the K–12 level. Another asked students these aspects in order to ensure social world” discussions are energizing— to write a MMORPG as a capstone CS 1 order within these environments. both in showing emerging applica- project. Perhaps the broadest-reaching Openness, heterogeneity, and scalabil- tions and when debunking overre- example came from Jim Marshall of ity of MAS pose new demands on tra- ported achievements. The group Sarah Lawrence College: a year-long ditional MAS interaction models. shared several nifty AI assignments seminar in which AI served as a touch- Therefore, the view of coordination that motivated students through a bal- stone for critical thinking, reading, and control has to be expanded to ance of peer cooperation and compe- and writing about the ideas in Ray consider not only an agent-centered tition. Eric Eaton of University of Kurzweil’s The Singularity Is Near. The perspective but societal and organiza- Maryland Baltimore County presented audience was entirely first-year liberal- tion-centered views as well. one memorable example of a next- arts students. To disseminate such re- The overall problem of analyzing generation Wumpusworld, in which sources—both for AI education and for the social, legal, economic, and tech- an image of Freeman Hrabowski (cus- education with AI—participants ex- nological dimensions of agent organi- tomizable, of course) executed stu- pressed interest in better leveraging the zations, and the coevolution of agent dents’ plans and responded to chang- AAAI’s AI Topics library; more general- interactions, provide theoretically de- ing conditions in order to save inhab- ly, the group welcomed opportunities manding and interdisciplinary re- itants of a search-and-rescue to consider AI education at future search questions at different levels of simulation. A mature set of online ap- AAAI venues. The colloquium’s discus- abstraction. The MAS research com- plets, AIspace, offers students accessi- sion concluded in agreement that, munity has addressed these issues ble interaction with many basic AI al- while energy invested in AI education from different perspectives that have gorithms. Two overarching pedagogi- does serve future generations of AI gradually become more cohesive cal goals emerged amid the many practitioners, it perhaps serves the cur- around the four notions that give title ideas: first, that students not only rent generation even more. to the workshop: coordination, organ- learn about AI but engage actively in Zachary Dodds, Kiri Wagstaff, and ization, institutions, and norms. it; second, that students perceive the Haym Hirsh served as cochairs of this Virginia Dignum and Eric Matson connection between their AI work and symposium. The papers are available were the cochairs of this workshop. current research. as AAAI Press Technical Report WS-08- The papers are published as an AAAI Technical Report WS-08-03. Strengthening ties with current re- 02. The event’s participant list, many search and applications of AI also mo- posters and slides, discussion sum- tivated much of the colloquium’s cur- maries, and supplementary material ricular innovation. Successful curricula have been archived at www.cs.hmc. Bringing Intelligence based on web crawling highlighted AI’s edu/aieducation. central role in modern life. A wide vari- to E-mail ety of game-based assignments rein- It has been more than a decade since forced AI’s ubiquity and added playful Coordination, the research community first became and creative opportunities for students Organizations, interested in e-mail overload. In that to express themselves while designing, Institutions, and Norms time, the HCI community has ex- implementing, and experimenting plored the effect of e-mail overload with agent behaviors. Reflecting on Multiagent systems (MASs) are often and tools for combating the problem. such approaches, the group felt it understood as complex entities where Five years ago, DARPA launched a ma- would serve the AI community to ar- a multitude of agents interact, usually jor research initiative under the per- ticulate a set of fundamental skills—as with some intended individual or col- sonal assistant that learns (PAL) pro- opposed to topics—that students lective purpose. Such a view usually gram, which encompasses the cogni- would build through an AI survey assumes some form of structure or set tive assistant that learns and organizes course. Such skill-based scaffolding of norms or conventions that articu- (CALO) and reflective agents with dis- would offer students a concrete and late or restrain interactions in order to tributed adaptive reasoning (RADAR) transferable grounding in “doing AI” make them more effective in attaining programs. These projects created intel- while allowing the leeway in approach, those goals, more certain for partici- ligent systems that assisted users in breadth, or depth balance exemplified pants, or more predictable. The engi- managing large amounts of informa- by these innovative curricula. neering of effective coordination or tion. At the same time, efforts have That flexibility—and AI’s applicabil- regulatory mechanisms is a key prob- emerged both at startups and in in-

110 AI MAGAZINE Reports dustry research labs to devise solutions ing procedures to complete user tasks, workshops has emphasized a distinc- to the e-mail overload problem. The creating an automated assistant. Melin- tive focus of interest within that do- result is an emerging community fo- da Gervasio and Thomas J. Lee demon- main. Our interest was summarized cused on intelligent tools for e-mail strated the ability of the CALO system concisely in a graphic that we used for analysis and assistance, spanning di- to learn these procedures from user ac- this year’s workshop call for papers verse research areas including AI and tions, automating common tasks stem- and onsite conference poster. Created HCI. Recognizing that new forms of ming from e-mail management. by Linda Pope, a United Methodist communication, such as instant mes- Finally, a difficult aspect of e-mail pastor, the graphic depicts a human saging and blogs, are becoming more research is a lack of resources due to looking into a hand mirror—but see- prevalent, e-mail overload is now the private nature of the data. Work- ing the face of a humanoid robot in more generally known as information shop participants discussed ideas for the mirror; the same robot, in an adja- overload. The Enhanced Messaging new data sources and annotations, cent frame, sees the human’s face in workshop brought together a diverse useful for applications such as sum- the mirror that it is holding. Immedi- group of researchers from academia marization (Jan Ulrich, Gabriel Mur- ately, of course, the graphic reminds and industry to discuss recent trends ray, and Giuseppe Carenini). us that an especially intimate and po- in messaging research and how we can In addition to strong participation tent dynamic of interaction emerges address the increasing problem of in- from the research community, several as AI achieves greater capability to formation overload. industry representatives in attendance equip robotic artifacts with lifelike be- Gabor Cselle and Greg Duffy of indicated an increased interest in de- havior. Indeed, the phenomenon of Xobni Inc, an e-mail startup, spoke ploying such technologies to combat HRI may progressively become a about their application, which im- overwhelmed users, a desire that has process of coevolution. proves e-mail organization, search, led to the recent founding of the In- Some familiar concepts of morality and navigation. They presented direct formation Overload Research Group. notably have illustrated this coevolu- feedback from end users about how in- The formation of this workshop indi- tionary possibility in each of our three telligent technologies improve the e- cates a continued interest in AI re- workshops. At least two of the presen- mail experience. search on information overload in e- tations in 2006, for example, effective- The 15 papers and posters at the mail as well as the desire for a transi- ly suggested that sustained HRI with workshop covered several broad tion from research into workable certain kinds of social robots could en- themes. Building intelligent interfaces technologies that can be deployed for courage humans to replace a historical for e-mail enables new tools for e-mail information workers. concept of categorically free moral ac- management was one of these themes. While the growing interest in e-mail tion with a much different notion of Intelligent e-mail tools have been the has left a fractured community spread purely deterministic moral behavior. focus of many researchers in recent through many subareas, the workshop This distinction was revisited in one of years, represented by papers on recipi- helped bridge this gap. Participants the 2008 presentations, which ob- ent recommendation and leak detec- from many subfields of AI as well as served that accepting a philosophical tion and suggesting reusable replies. the broader research community came thesis of epiphenomenalism sets aside Understanding e-mail as the mod- together to discuss the state and future intensionality worries that the distinc- ern information center yields an un- of the field. The workshop was an im- tion presumes. In any case, philosoph- derstanding of the modern informa- portant first step toward building a ical positions that we adopt apparent- tion worker. Such analyses are useful community structure that will open ly do strongly condition the ways in for understanding the workplace and channels of communication and col- which humans will interpret the moral for mining useful information about laboration as we move forward. status of robots—and themselves. an organization. For example, Christo- Mark Dredze (University of Pennsyl- Again, it has been noted in both the pher Diehl, Galileo Namata, and Lise vania), Vitor Carvalho (Microsoft Live AAAI-07 and AAAI-08 workshops that Getoor presented techniques for iden- Labs), and Tessa Lau (IBM Almaden Re- the enterprise of equipping robots to search Center) organized the work- tifying organizational relationships exhibit what humans regard as accept- shop. The papers were published as an based on e-mail activity, a key part to able moral behavior reflexively can AAAI Technical Report WS-08-03. understanding workplace roles. An- produce better understanding of our drew Lampert, Robert Dale, and Cécile own moral ideas. In fact, a machine Paris analyzed e-mails to learn how Human Implications of ethics presentation during the 2008 users make and commit to requests, workshop demonstrated the ability of which helps in understanding the na- Human-Robot Interaction an AI system to infer new ethical prin- ture of work and how users communi- This workshop claims a history of its ciples that are meaningful. cate about it. These techniques are al- own, having consecutively been an el- Another multidisciplinary topic so applicable to the emerging interest ement of the AAAI-06, AAAI-07, and that has appeared repeatedly in our in e-discovery, where legal teams are AAAI-08 conferences. Although study workshops is a concern with human responsible for analyzing millions of of HRI certainly was in progress prior acceptance of robots in specific HRI documents for relevance to litigation. to celebration of AI’s 50th birthday in scenarios. More than half of our AAAI- The PAL program focuses on learn- 2006, this particular series of recent 06 workshop presentations engaged

SPRING 2009 111 Reports this topic, reporting some results from stream data, web usage logs, or explic- ing, for client-side user monitoring to empirical studies. This theme of inter- it personalization rules. infer the user’s search intent, as well as est continued in 2007, with presenta- Recommender systems represent for community-based query expansion tions that explored acceptance in the one special and prominent class of and personalized document filtering. context of privacy issues raised by ro- personalized web applications. They In the personalization on the social botic butlers, as well as the case of di- focus particularly on the user-depen- web track, presenters addressed aspects minished human-to-human contact dent filtering and selection of relevant of search and navigation improvement within healthcare applications. The information and aim to support on- in social media platforms through tag AAAI-08 workshop also reflected inter- line users in the decision-making and clustering and tag-based user profiling, est in acceptance. Presentation of a pa- buying process. Recommender sys- as well as the issue of possible attacks per reviewing Masahiro Mori’s Bud- tems have already been a subject of ex- against such open tagging systems. Fi- dhist perspectives on robotics was fol- tensive research in AI over the past nally, in the technical track on recom- lowed by some discussion of his decade. With today’s soaring number mender systems technology, the focus uncanny valley hypothesis, according of e-commerce environments on the was on combining rating information to which human acceptance drops web, the demand for new approaches with additional knowledge about item sharply just when a robotic artifact be- to intelligent product recommenda- characteristics or about user relation- comes too lifelike. Results from cluster tion has become more pressing than ships in a social network, to improve analysis of survey data also were pre- ever. The web now has more online prediction accuracy. sented, examining patterns relating re- users, more online channels, more In keeping with the increasing us- sponses in the areas of ethics, religion, vendors, more products, and, most of age of social web and personalization and acceptance. Again, the 2008 work- all, more complex products and serv- features on media-sharing sites, this shop engaged human acceptance in ices. These recent developments in the year’s workshop also included an ex- the context of human trust of au- area of recommender systems generat- citing invited address by Paul Lamere tonomous systems. ed new research demands, in particu- of Sun Microsystems Labs, titled, “Mu- Perhaps it is not surprising that lar with respect to interactivity, adap- sic Recommendation Is Broken, and themes of ethics and acceptance have tivity, and user preference elicitation. Only You Can Fix It!” been conspicuous in this series of work- These research challenges, however, The workshop ended with an open shops. As our robotic creations become are not confined to the area of recom- discussion and a reflection on the state more lifelike, the dynamics of mutual mender systems only, but are also in of the art in the respective areas. The adaptation between human and ma- the focus of general web personaliza- exploitation of additional knowledge chine might naturally be expected to tion research. sources to improve personalization mirror those that take place among hu- In the face of this increasing overlap and recommendation was identified mans. After all, we commonly care very of the two areas, the aim of this work- as one of the central means for ad- much whether a stranger is morally shop was to bring together researchers vancing research in web personaliza- good and socially acceptable. and practitioners of both fields, to fos- tion and recommender systems. Such Lundy Lewis and Ted Metzler (chair) ter an exchange of information and new sources include content-related information, for example, in the form organized this workshop. The papers ideas, and to facilitate a discussion of of lexicons, semantic web ontologies, of the workshop were published as current and emerging topics related to or web 2.0 community knowledge. AAAI Technical Report WS-08-05 and web intelligence, particularly regard- They also include context-related in- are available from AAAI Press. ing its application to recommender systems. The workshop united topics formation, such as the user’s geospa- from web personalization and recom- tial position, which, for example, the Intelligent Techniques for mender systems after a successful pri- latest generation of location-aware Web Personalization and or event at AAAI-07. handheld devices can provide to the This year’s workshop attracted a personalization system. Recommender Systems The Workshop on Intelligent Tech- number of high-quality contributions niques for Web Personalization and Web personalization can be generally from 12 different countries. Of these, Recommender Systems was cochaired defined as the process of tailoring the seven papers (fewer than 40 percent) by Sarabjot Singh Anand, Bamshad content or visual presentation of a were accepted for full presentation, Mobasher, Alfred Kobsa, and Dietmar website, or the behavior of a web ap- with an additional three accepted for Jannach. The papers of the workshop plication, to the particular needs and short presentations. The accepted pa- were published as AAAI Press Techni- preferences of an individual user or a pers dealt with a variety of issues and cal Report WS-08-06. group of users. Automated personal- techniques for creating more intelli- ization systems typically accomplish gent personalization systems, but gen- this task through individualized infor- erally fell into three broad categories. Metareasoning: Thinking mation filtering and ranking, as well In the area of personalization in search about Thinking as through intelligent navigation sup- and navigation, new proposals were port. They thereby rely on different made for the ranking of personalized The two-day Workshop on Metarea- sources of knowledge such as click- search results based on machine learn- soning: Thinking about Thinking was

112 AI MAGAZINE Reports a sequel to the successful Workshop On day two Aaron Sloman of the such as web-based recommender sys- on Metareasoning in Agent-Based Sys- University of Birmingham in the UK tems, in automated problem solvers tems at AAMAS-07 last year. This year discussed varieties of metacognition in such as configurators, and in au- the cochairs wrote a brief manifesto natural and artificial systems. His talk tonomous systems such as Mars outlining a simple model of metarea- was based on the requirements analy- rovers. Nearly all areas of artificial in- soning and invited participants to sis in the CoSy robotics project and dis- telligence deal with choice situations compare and contrast their research to cussed various types of metacognition and can thus benefit from computa- the conceptualization. The contention in intelligent biological individuals. He tional methods for handling prefer- is that, like traditional reasoning that discussed the role of the environment ences. Preferences are also at the heart is composed of an action and percep- in the development of natural intelli- of social choice methods and are thus tion cycle, so too metareasoning has gence where the environment includes of importance for consensus methods, distinct control and monitoring com- the ways in which things outside that which are, for example, used in bioin- ponents. The resulting paper submis- organism, for example, types of inter- formatics, and for multiagent systems. sions were both exciting and novel, actions, relate to design features of the These new perspectives on prefer- falling into one of four categories. organism. Several interesting videos of ences led to the creation of a growing Some papers examined the metalevel metacognition in natural systems were community of researchers from artifi- control of reasoning, while others dis- screened, and the implications of these cial intelligence, databases, operations cussed introspective monitoring of metacognitive issues for architectures research, and other computational reasoning. Another set of papers re- and representations were discussed. fields who are interested in computa- ported research on distributed models The workshop participants dis- tional models of preferences and their of metareasoning and multiagent cussed the commonalities and need applications to computational tasks. metareasoning that considered the is- for differences among the various This workshop had an exciting pro- sues of coordination at the metalevel metareasoning architectures presented gram consisting of an invited talk, 23 and when and how agents could gain at the workshop. The discussions also technical presentations, a poster ses- a common metalevel context for prob- included whether a common model sion, and a panel discussion. lem solving. Finally, a large group of for metareasoning in the manifesto The topic of preference elicitation papers put everything together to talk was sufficient and the implications of found particular interest at the work- about computational models of self. enforcing such a model. There was al- shop. Craig Boutilier summarized ma- Day one organized paper sessions so discussion on the complexity of jor challenges and experiences about from the first two groups, whereas day solving various metareasoning issues. this topic in a fascinating invited talk. two contained the presentations for The group reached a general consen- A particular challenge consists in keep- the latter two groups. sus that metareasoning is an exciting ing the number of questions about In addition to the standard 25- area requiring more investigation. user preferences small while giving a minute presentation and question peri- There was also interesting discussion guarantee about the degree of optimal- ods, all sessions were followed by an in- on methods and metrics to evaluate ity of the recommendation produced teractive panel where each author pre- the effect of metareasoning on overall by an interactive system. As shown by sented views and comments regarding system performance. Boutilier and other participants, it is topics put forth by a select moderator. AAAI Press published the papers necessary to interleave elicitation and These sessions also gave the audience from this workshop as AAAI Technical problem solving to achieve good re- an opportunity to ask questions or Report WS-08-07. The collection is sults. A couple of other tasks discussed make comments on issues that spanned available in hard copy from the pub- preference elicitation in concrete sys- individual paper presentations. lisher and electronically from AAAI’s tems such as a smart home system that A special highlight of the workshop digital library. The cochairs of this adapts to changes in a resident’s pref- was the invited talks. On day one Don workshop were Michael T. Cox (BBN erences and multiagent systems for Perlis of the University of Maryland, Technologies) and Anita Raja (Univer- surveillance tasks. Preferences can also sity of North Carolina at Charlotte). College Park, spoke of two concep- be acquired by observing an agent’s be- tions of reflective reasoning. The first havior, in particular when the agent’s one he called “Meta-Strata,” referring Advances in environment is changed. to hierarchical architectures such as Another highlight of the workshop the one proposed in the workshop Preference Handling was the usage of preferences in game manifesto. Here one discrete layer lies Preferences are a central concept of de- theory. Milind Tambe’s group present- above another and reasons about its cision making. As preferences are fun- ed their work on Bayesian Stackelberg workings. In another arrangement, damental for the analysis of human games, which is used for security called “Meta-Loopy” (in deference to choice behavior, they are becoming of scheduling at the Los Angeles Interna- Doug Hofstadter), an architecture ex- increasing importance for computational Airport. These games distin- amines itself as a snake biting its own tional fields such as artificial intelli- guish different players, namely a tail. Much lively discussion erupted gence, databases, and human-comput- leader and followers, and pose partic- over the differences and similarities er interaction. Preference models are ular challenges for preference model- between these formulations. needed in decision-support systems ing. Game theory was also the topic of

SPRING 2009 113 Reports several other talks. Furthermore, Vin- continuous state spaces, how to trade June 13, the first of the two workshop cent Conitzer organized an excellent off between the run time and memory days of the 23rd AAAI Conference on panel about whether game theory is consumption of the search and the re- Artificial Intelligence (AAAI-08). It necessary or beneficial for research on sulting solution quality, how to select continued a tradition that started with multiagent preferences. The panelists between different search strategies, and a workshop 15 years ago at IJCAI-03 in were Craig Boutilier, David Parkes, how to focus the searches with sophis- Chambery, France, which led to a se- Yoav Shoham, and Milind Tambe. One ticated heuristics such as pattern data- ries of workshops with the common conclusion was that game theory is bases. Their results are published in dif- goal of bringing together related com- important but should be used with ferent conferences such as IJCAI, AAAI, munities of researchers with an inter- care, as it is essentially based on the ICAPS, NIPS, ICRA, and IROS. This est in the study of representing and notion of an equilibrium. workhop brought these researchers to- reasoning about either space or time— The workshop also covered topics gether to exchange their ideas, cross- or both. such as preference representations in fertilize the field, and combine various As is the case with many other rea- form of graphical models or soft con- search techniques that originated in soning techniques, spatial and tempo- straints, preference queries in databas- different research communities. ral reasoning is at home in many areas es and ontologies, and preference rep- The two-day workshop had more of artificial intelligence (and computer resentations in planning and in com- than 35 attendees, in part thanks to science in general), such as planning, binatorial auctions. It was interesting generous support from NSF for stu- robot control and guidance, natural to learn that preferences can also be dent participation. It featured an language understanding, assembly represented in the form of rules such overview that highlighted the similar- plant sequencing and scheduling, am- as those for change management in ities and differences of search in artifi- bient intelligence and smart homes, the service industry or for product cial intelligence and robotics and temporal databases, concurrent and configuration. Consensus methods in three invited talks (by Oliver Brock, distributed programming. The field of bioinformatics were illustrated for the Malte Helmert, and Maxim Likachev) spatial and temporal reasoning has problem of identifying sibling rela- on “Solving Hard Planning Problems progressed significantly over the recent tionships from genetic data. A poster in Robotics with Simple A*-like years, and some of the long-standing session allowed students to get feed- Searches,” “Automatically Deriving problems in the field have at least par- back about their thesis projects. Abstraction Heuristics,” and “Search tially been solved, in particular those The workshop was cochaired by Jan in Embodied Artificial Intelligence and related to tractability for spatial calculi, Chomicki, Vincent Conitzer, Ulrich Computational Biology.” explicit construction of models, char- Junker, and Patrice Perny. The papers The 15 oral presentations and more acterization of important subclasses of of the workshop were published as than 12 posters in a lively poster ses- relations, multidimensionality of spa- AAAI Press Technical Report WS-08-09. sion displayed the diversity of research tiotemporal calculi, and handling of on search and its applications, covering incomplete and imprecise informa- Search in Artificial topics such as abstraction, inconsistent tion. Despite all these successes, there heuristics, bounded suboptimality, per- is still a lack in a deeper understanding Intelligence and Robotics formance prediction, learning, symme- of the foundations of the field, which try, real-time search, moving-target Search is one of the few areas of artifi- might be the reason that it has not search, connections to probabilistic cial intelligence (and beyond) that found as much enthusiasm among the reasoning and applications to robotics, lack their own conference. This work- practitioners in artificial intelligence, machine learning, and diagnosis. shop therefore brought together re- computer science, and information Among the highlights of the workshop searchers interested in this topic to technology as it should have had. The were the presentations on the use of share their ideas and disseminate their aim of this workshop was to work to- heuristic search in the first- and sec- latest research results. It focused on wards overcoming this shortcoming. ond-place vehicles participating in the finding common ground between A total of seven presentations DARPA Urban Challenge. search techniques used in artificial in- spread over the whole day laid the ba- The organizing committee of the telligence and robotics. sis for the workshop. The presenta- symposium consisted of David Furcy, Heuristic search and related algo- tions were slightly biased towards spa- Sven Koenig, Wheeler Ruml, and Rong rithms are currently very active areas of tial reasoning (as opposed to temporal Zhou. The papers of the workshop research. For example, researchers in- reasoning), and one of them addressed were published as AAAI Press Techni- vestigate how to search in real time, both. However, due to the similarity of cal Report WS-08-10. how to search with limited (possibly spatial and temporal reasoning, most external) memory, how to search in of the presentations had an impact on parallel on several processors, how to both spatial and temporal reasoning. solve sequences of similar search prob- Spatial and They provided a solid basis for the dis- lems faster than with isolated searches, Temporal Reasoning cussion sessions, which were inter- how to improve the run time of the leaved with the presentation sessions. searches through randomization or The Workshop on Spatial and Tempo- The discussion sessions tied together learning techniques, how to discretize ral Reasoning was held on Sunday, the individual presentations into larg-

114 AI MAGAZINE Reports er themes and provided a forum for and one-off procurement contracts, information as possible is an effective clarification of ideas, exchange of and (2) a prediction challenge de- approach for an agent in one-shot sim- points of view, assessment of results signed to test price-prediction capabil- ulated auctions settings. Another pa- and methods, and suggestions for fu- ities of competing agents in both pro- per discussed how to classify bidding ture work. curement and sales markets. strategies in CAT and reported that us- Although progress was made in var- In contrast to the supply-chain sce- ing a hidden Markov model yields the ious areas of spatial and temporal rea- nario, which casts the competing best results. A second paper on CAT soning, it became obvious that there is agents as traders, the CAT scenario discussed how to design an effective e- still a large number of open problems. places agents in the role of competing market and provided an overview of In particular, there is still a gap be- exchanges. The CAT competition is the authors’ CAT agent. One paper dis- tween the theories of spatiotemporal motivated by the rise of independent cussed the problem of multiunit mul- reasoning and their applications to re- for-profit stock and commodity ex- tiattribute allocation through call auc- al-world scenarios, which requires a changes that compete for the attention tions. The paper demonstrated that an significant amount of future work in of traders. CAT agents compete by iterative bidding protocol can often the field. defining rules for matching buyers and overcome the limitations of bidding Hans Guesgen (Massey University, sellers and by setting commission fees language restrictions made to achieve New Zealand), Gérard Ligozat (LIMSI, for their services. Profitability is the ul- clearing tractability. Université Paris-Sud, France), and Rita timate measure of performance in both The first panel discussed the bene- V. Rodriguez (National Science Foun- the supply chain and CAT scenarios. fits and challenges of developing dation, USA) served as cochairs of this Both scenarios involve multiple mixed initiative variations of existing workshop. This report was written by rounds, each of which features a large scenarios, where human decision Hans Guesgen. The papers of the number of encounters aimed at captur- makers would compete with the sup- workshop were published as AAAI ing a broad range of market conditions. port of semiautonomous trading Press Technical Report WS-08-11. The workshop featured seven (six agents. The panel gave rise to lively extended and one short) paper presen- discussions and generally suggested Trading Agent Design tations, in-depth discussions of the that the trading community would performance of competing techniques likely welcome such a game, most and Analysis and agents, and two panel discussions. probably as an extension of TAC-SCM Research in trading agent technologies Paper presentations included in-depth procurement challenge. Panel partici- has gained increasing prominence analyses of the 2007 TAC-SCM pro- pants generally viewed the introduc- over the past decade, in part due to a curement and prediction challenges. tion of such a game as a possible way drive to partially automate trading de- Designers of the prediction challenge of engaging other subdisciplines both cisions in a number of different do- emphasized how their challenge within and outside of the AI commu- mains. This workshop focused on the makes it possible to isolate the predic- nity. A game like this would also like- design and evaluation of trading tion performance of different agents in ly be an excellent teaching tool in agents. Papers were invited on topics different areas, whereas the TAC-SCM business schools. Simply taking the in trading agent architectures, deci- scenario effectively allows agents to procurement challenge and requiring sion-making algorithms, theoretical compensate for inaccuracies in their a human decision maker to replace an analysis, and empirical evaluations of predictions by adjusting their procure- agent would not work, given the agent strategies in different negotia- ment and sales activities. The procure- amount of information the human tion scenarios. ment challenge presentation discussed would have to make sense of and the The workshop was held in conjunc- lessons learned from the 2007 edition number of decisions she would have tion with the finals of the 2008 Trad- and provided a comparison of the top to make. Instead a more promising ap- ing Agent Competition. Two game three entries and a discussion of proach would be to find ways to put scenarios and two challenge events at- changes introduced in the 2008 edi- human decision makers in control of tracted 42 entries. tion of the challenge. Another paper important decisions while allowing The supply-chain management presented a survey of agent designs in them to delegate more routine day-to- (TAC-SCM) scenario places six agents TAC SCM. The survey showed that, in day pricing, procurement, and rein the role of a personal-computer some areas such as modularity, there source allocation decisions to semiau- manufacturer. Each agent has to pro- are common themes emerging in how tonomous trading agents operating cure PC components and sell finished to design a successful trading agent, under their overall strategic guidance. goods in competitive markets while while in other areas, such as coordina- The day ended with a panel and managing inventory and production tion, there are strong differences in the group discussion on the future of trad- facilities. The baseline TAC-SCM com- designs. Another presentation re- ing agent research, and to what extent petition was also complemented by volved around an experimental study results from TAC are influencing cur- two additional challenge events: (1) A of bidding heuristics designed for the rent practice. Discussions suggested procurement challenge that requires TAC Travel game (a game that was not that TAC has started to make an im- agents to manage supply chain risk part of the 2008 competition), show- pact. Specifically, it appears that TAC- through a combination of long-term ing that using as much distributional like techniques are being implemented

SPRING 2009 115 Reports in several practical settings including One such open question is how to ics included differentiating transfer energy markets, flower auctions, and best determine task similarity au- from generalization and what goals procurement of electronic components. tonomously. In one of the three rein- are appropriate for deep transfer. Mixed-initiative interfaces are an im- forcement learning papers, Tom Croo- While no firm conclusions were portant issue in these deployments. nenborghs, Kurt Driessens, and Mau- reached, the consensus was that trans- Wolfgang Ketter, Norman Sadeh, rice Bruynooghe presented a novel fer learning is a relatively young field and William Walsh were organizers of approach to learn how state variables with many open questions. Current this workshop. The papers were pub- are related in different reinforcement results suggest that transfer can lead to lished as AAAI Technical Report WS- learning tasks. Research in transfer for substantial performance improve- 08-12. Additional information on the reinforcement learning typically lim- ments in many different machine- 2008 TAC competition can be found its transfer to tasks that have the same learning contexts. Going forward, we at www.sics.se/tac. state variables or relies on a human to expect that transfer will continue to specify how the tasks are related. In generate many questions, as well as Transfer Learning for contrast, this approach may allow an opportunities, for AI researchers. agent to transfer between very differ- Matthew E. Taylor wrote this report Complex Tasks ent tasks without requiring a human and was the primary contact for the workshop. The papers were published All machine-learning algorithms rein the loop. as AAAI Technical Report WS-08-13. quire data to learn, and often the Another long-term goal of transfer amount of data available is a limiting has been to enable successful knowl- factor. For instance, classification and edge reuse between very different What Went Wrong tasks. In their work, Lilyana Mihalko- regression require labeled data, which and Why may be expensive to obtain. Rein- va and Raymond Mooney show that forcement learning requires samples Markov logic networks can successful- Unfortunately, bugs, glitches, and fail- that must be collected through repeat- ly transfer between different domains. ures are rarely mentioned in academic ed interaction with an agent’s envi- For instance, their method can exploit discourse, so their role in informing ronment. Typically, a learning system similarities between the social organi- design and development is essentially or agent treats every problem as dis- zation in academia and that in the lost. The first What Went Wrong and tinct and must begin learning tabula movie industry to achieve transfer Why workshop addressed this gap at rasa. The insight behind transfer learn- when target domain data is severely the 2006 AAAI spring symposium by ing is that past experience may assist limited. Jesse Davis and Pedro Domin- inviting AI researchers and system de- learning a novel task, even if the tasks gos also use Markov logic networks, velopers to discuss their most reveal- are very different. While the idea of but explicitly focus on deep transfer, ing bugs and relate problems to les- transfer has long been explored in the where the domains transferred be- sons learned. Several of the articles psychological literature, it has only re- tween are even more different, such as and invited talks were published as a cently been gaining popularity as a using molecular biology data to learn special issue of the summer 2008 AI general machine-learning technique. better in an academic domain. Experi- Magazine. The second What Went In the transfer learning paradigm, ments show that learned network Wrong and Why workshop continued one typically uses a set of source tasks templates, representing concepts like this theme through a one-day pro- to help learn one or more target tasks. symmetry, transitivity, and homophi- gram at AAAI-08 that emphasized Successful transfer allows faster, or bet- ly, enable significant improvements methodological insights. It included ter, learning, when compared to learn- when learning in novel domains. invited talks by Kevin Ashley, Bruce ing without previous knowledge, even Will Bridewell and Ljupco Todorovs- Buchanan, Steve Chien, and Haym if the source and target task data orig- ki’s paper had a similar deep transfer Hirsch, plus four papers. inate from different distributions. goal, but in a very different setting. The Kevin Ashley focused on evaluating The primary goal of this workshop goal of inductive process modeling is to research in computational argumenta- was to bring together researchers produce a model that explains the be- tion. He incorporated computational working on different aspects of the havior of a dynamic system and pre- models of argumentation into his cur- transfer problem so that we could dis- dicts unseen data. After successfully riculum for first- and second-year law cuss current approaches and common learning on data from one ecosystem, students (whose business is [arguably] problems. Roughly 30 researchers at- the authors use transfer to learn a mod- to argue), and asked if existing models tended to discuss the 10 accepted pa- el of a second ecosystem, which had helped them learn argumentation pers. Papers covered a wide range of different characteristics and different skills. He discovered that learning in topics, including regression, classifica- organisms. Experiments showed that this realm was as hard to measure as tion, reinforcement learning, plan- transferred constraints could reduce evaluating argumentations. In the ning, Markov logic networks, and neu- search time by an order of magnitude process, however, he developed a suite ral networks. Despite the large number with little loss to model accuracy. of diagnostic tools that led to better-tar- of contexts, many of the same trans- After the last presentation, atten- geted pedagogical advice. This story is a fer-related questions were discussed by dees participated in a general debate reminder that the methodology can be the different presenters. on the relative merits of transfer. Top- as important as the intended result.

116 AI MAGAZINE Reports

Bruce Buchanan’s talk examined extraction tasks over distributed Wikipedia and Artificial what we learned from the expert sys- sources that are riddled with incorrect tem’s boom. After noting the power and incomplete data. Haym argued Intelligence: An Evolving and prevalence of such systems, he fo- that these elements transform the na- Synergy cused on cases where the reality of ture of the relevant research, and that As a large-scale repository of struc- commercialization forced developers we should review our examples in this, tured knowledge, Wikipedia has be- to address hard problems in AI and and other areas lest they blind us to a come a valuable resource for a diverse where pragmatic shortcuts let them changing world. set of AI applications. Major confer- avoid other problems. For example, Among the papers, Carl Hewitt dis- ences in natural language processing the expectation that nonexperts could cussed the history of logic program- and machine learning have recently write rules proved largely false, but led ming in terms of the issues and re- witnessed a significant number of new to the designation of knowledge engi- sponses characterizing its develop- approaches that use Wikipedia for neers‚ and clarified the target end user ment. He concluded that para- tasks ranging from text categorization for expert system shells. The lesson consistent logics are the new horizon, and clustering to word-sense disam- that some knowledge requires intri- as they can plausibly infer properties biguation, information retrieval, in- cate and subtle representational struc- of software written for practical do- formation extraction, and question tures clarified the need for new tech- mains, which is chock full of inconsis- answering. On the other hand, nology without diminishing the role tencies. Nestor Rychtyckyj and Alan Wikipedia greatly benefits from nu- of flat rule bases or their utility. Turski provided case studies in the de- merous algorithms and representation Steve Chien discussed lessons velopment of commercial expert sys- models developed during decades of learned for AI applications from au- tems and concluded that organiza- AI research, as illustrated recently in tonomous science craft, drawing on tional versus technical issues consti- tasks such as estimating the reliability his experiences at JPL/NASA. He noted tuted the key barriers to acceptance of authors’ contributions, automatic that work on such large-scale integrat- (for example, the presence/absence of linking of articles, and intelligent ed systems presents both organiza- software life-cycle support mecha- matching of Wikipedia tasks with po- tional and technical challenges. From nisms). Soumi Ray and Tim Oates pre- tential contributors. an organizational perspective, the key sented an unusual what went right Consistent with the aims of the issue is to balance autonomy, seen as and why story, which asked the audi- workshop, the paper presentations ad- risk, against benefits in the form of ence to help explain unreasonably fast dressed a highly diverse set of prob- cost reduction and scientific returns. convergence from a reinforcement lems, in which Wikipedia was seen ei- This trade-off is evaluated by hard- learning algorithm that randomly ther as a useful resource or as a target nosed engineering calculations but scaled Q-values. The resulting discus- for algorithms seeking to make it even has a social aspect as well. AI systems sion is still in process. Finally, Cindy better. As a rich knowledge source, in space applications need to build a Marling and David Chelberg discussed Wikipedia was shown to benefit appli- track record of trust before they can be an unsuccessful team’s entry into an cations in information extraction, ma- deployed. Steve showed the potential chine translation, summarization, on- international competition. The paper, for enormous returns: software on- tology mining and mapping, and in- titled “RoboCup for the Mechanically, board NASA’s Earth Observing One formation retrieval. We also learned of Athletically, and Culturally Chal- (EO-1) mission documented more interesting applications, most of them lenged,” noted that stuff blew up, than a 10-fold increase in science re- using machine learning, that could communication systems failed, the turn and more than $1M/yr in cost re- further enhance the breadth and the professor broke her ankle developing ductions. He concluded by noting that quality of Wikipedia, such as predict- domain expertise, and the principles AI software has flown on five missions, ing the quality of edits, vandalism de- and its successes are changing the ac- narrowly escaped arrest for shipping tection, infobox extraction, creation ceptability of spacecraft AI. compressed gas. Despite these set- of crosslingual links, and semantic an- Haym Hirsch examined what’s go- backs, the team concluded that failure notation. ing wrong in data mining from his is a catalyst for future progress. The workshop featured an invited perspective as a researcher and as the In summary, this workshop provid- talk by Michael Witbrock (Cycorp) on director of the Information and Intel- ed researchers and developers with an human-computer collaboration. Based ligent Systems Division at NSF. He ob- informal, valuable, and enjoyable op- on the premise that it is not always served that the examples motivating portunity to share their experiences possible to employ humans to verify large bodies of academic work have about What Went Wrong and Why. It the correctness of an ever increasing become dangerously out of date, and illustrated, once again, that WWWW number of Wikipedia submissions, shape research in inappropriate ways. experiences offer novel insights and Witbrock presented arguments for a For example, the Irvine machine- communicate interesting research les- large scale use of artificial agents that learning repository supports incre- sons in concise ways. Mehmet H. Gök- verify facts by repeatedly observing the mental improvements in classification er and Daniel Shapiro cochaired the behavior of others, assuming repeated algorithms, while application chal- workshop. The papers were published behaviors and facts to be correct. lenges concern terabyte information as AAAI Technical Report WS-08-14. The workshop concluded with an

SPRING 2009 117 Reports exciting panel discussion. Jamie Taylor Sarabjot Singh Anand is a senior lecturer Lundy Lewis is a professor and chair of the (Metaweb) gave a short presentation in the Department of Computer Science, Department of Computer Information on Freebase, a collaboratively edited University of Warwick. Technology, at Southern New Hampshire database of world knowledge that de- Razvan Bunescu is an assistant professor at University. rives its content from Wikipedia and the School of Electrical Engineering and Eric Matson is an assistant professor in the Computer Science, Ohio University. other knowledge bases. He then ar- Computer and Information Technology gued for a better understanding of se- Vitor R. Carvalho is a scientist at Microsoft Program, College of Technology, Purdue mantic issues within the Wikipedia Live Labs. University. community, as a necessary step towards turning Wikipedia into a se- Jan Chomicki is an associate professor of Ted Metzler is director of the Darrell W. computer science and engineering at the mantic network. He also pointed to Hughes Program for Religion and Science University at Buffalo. the challenge of transferring the cur- Dialogue at Oklahoma City University. Vincent Conitzer is an assistant professor rent techniques for mining Wikipedia Rada Mihalcea is an associate professor at of computer science and economics at to the significantly noisier World the Department of Computer Science and Duke University. Wide Web. Barney Pell (Powerset) Engineering, University of North Texas. drew attention to the rate of increase Michael T. Cox is a senior research scien- of Wikipedia submissions, which late- tist in the Intelligent Computing group of Bamshad Mobasher is a professor at the ly has been going down. As a possible BBN Technologies. School of Computing, DePaul University. solution, he proposed automatically Virginia Dignum is an assistant professor Joelle Pineau is an assistant professor of creating initial versions of stub articles at Utrecht University. computer science at McGill University. in order to motivate users to edit Zachary Dodds is an associate professor of them. He also emphasized the need to Pascal Poupart is an assistant professor at computer science at Harvey Mudd College. the School of Computer Science University improve the techniques for extracting of Waterloo. knowledge from Wikipedia and Mark Dredze is a fifth year Ph.D. student crosslinking it with other knowledge in computer science at the University of Anita Raja is an assistant professor in the Pennsylvania. sources. Michael Strube (EML Re- Department of Software and Information search) proposed going beyond the ex- David Furcy is an assistant professor at the Systems at the University of North Caroli- traction of factual knowledge from University of Wisconsin. na at Charlotte. Wikipedia in order to distill richer in- Evgeniy Gabrilovich is a senior research Wheeler Ruml is an assistant professor at formation such as opinions, procedur- scientist at Yahoo! Research. the University of New Hampshire. al knowledge, and even scientific insights. Daniel Weld (University of Mehmet H. Göker is the research director Norman Sadeh is a professor at Carnegie at PricewaterhouseCoopers Center for Ad- Washington) presented an approach Mellon University. vanced Research. to populating infoboxes with informa- Guy Shani is a researcher with Microsoft Hans Guesgen is a professor of computer tion extracted from Wikipedia articles Research. and the World Wide Web. He empha- science in the School of Engineering and Advanced Technology at Massey Universi- sized the importance of unobtrusively Daniel Shapiro is the executive director of ty, New Zealand. motivating users to contribute expert- the Institute for the Study of Learning and ise. In this context, he described the Haym Hirsh is a professor of computer sci- Expertise and president of Applied Reactiv- approach employed by his group ence at Rutgers University. ity, Inc. where online ads were placed on Dietmar Jannach is a professor at the De- Trey Smith is a systems scientist at Google, Yahoo, and MSN in order to partment of Computer Science, Dortmund Carnegie Mellon University West/NASA invite volunteers to verify automati- University of Technology. Ames Research Center. cally extracted facts. Other issues Ulrich Junker is a distinguished scientist at Matthew E. Taylor is a postdoctoral re- raised during the panel discussion in- ILOG. cluded integration of information search scientist at the University of South- from multiple pages when showing Wolfgang Ketter is an assistant professor at ern California the Rotterdam School of Management, search results, studying what users ac- Erasmus University. Kiri Wagstaff is a researcher in the Ma- tually want from Wikipedia, allowing chine Learning Systems Group at the Jet contributors to define and annotate Alfred Kobsa is a professor in the Donald Propulsion Laboratory. relations in Wikipedia, and more gen- Bren School of Information and Computer erally assisting contributors to provide Sciences of the University of California, William Walsh is a senior research scien- Irvine. tist at CombineNet. deeper semantic annotations. Razvan Bunescu, Evgeniy Sven Koenig is an associate professor of Rong Zhou is a researcher at the Palo Alto Gabrilovich, and Rada Mihalcea served computer science at the University of Research Center. as cochairs of the workshop. The pa- Southern California. pers of the workshop were published as Tessa Lau is a research staff member at IBM AAAI Press Technical Report WS-08-15. Almaden Research Center.

118 AI MAGAZINE Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.