Annotated Table of Contents Automatic Extraction of Future Ref- D Erences from News Using Morphose- O Welcome to AI Matters, Vol

AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016 AI Matters

Annotated Table of Contents Automatic Extraction of Future Ref- D erences from News Using Morphose- O Welcome to AI Matters, Vol. 2(4) mantic Patterns with Application to Eric Eaton & Amy McGovern, Editors Future Trend Prediction Full article: http://doi.acm.org/10.1145/3008665.3008666 Yoko Nakajima, Michal Ptaszynski, Hiro- toshi Honma & Fumito Masui A welcome from the Editors of AI Matters and a Full article: http://doi.acm.org/10.1145/3008665.3008671 summary of highlights in this issue. Dissertation abstract. S AI Profiles: An interview with Peter Norvig D Network Organization Paradigm Amy McGovern & Eric Eaton Saad Alqithami Full article: http://doi.acm.org/10.1145/3008665.3008667 Full article: http://doi.acm.org/10.1145/3008665.3008672 This column is the first in a new series profiling Dissertation abstract. senior AI researchers. This month focuses on Peter Norvig. Autonomous Trading in Modern Elec- D tricity Markets Ed AI Education: Birds of a Feather Daniel Urieli Todd W. Neller Full article: http://doi.acm.org/10.1145/3008665.3008673 Full article: http://doi.acm.org/10.1145/3008665.3008668 Dissertation abstract. This first instance of the new AI Education column proposes the novel card game Birds of a Feather, AI Amusements: The Tragic Tale of and explores its use in teaching various AI con- H Tay the Chatbot cepts. Ernest Davis Full article: http://doi.acm.org/10.1145/3008665.3008674 Nonverbal Communication in So- D cially Assistive Human-Robot Inter- This column features AI-related comics, puzzles, action & jokes for your entertainment. In this issue, we Henny Admoni bring you the tragic tale of Tay, the ill-fated chatbot! Full article: http://doi.acm.org/10.1145/3008665.3008669 Dissertation abstract.

Multi-Objective Decision-Theoretic D Planning Diederik M. Roijers Full article: http://doi.acm.org/10.1145/3008665.3008670 Dissertation abstract.

1 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016 Upcoming Conferences Submit to AI Matters! Sponsored by SIGAI We’re accepting articles and announcements Registration discount for SIGAI members. now for the Winter 2017 issue. Also, we would like to invite readers to submit responses to WI 2016: The 2016 IEEE/WIC/ACM Interna- the Who Speaks for AI? article that appeared tional Conference on Web Intelligence, Om- in AI Matters 2(2) to further the debate on this aha, Nebraska, USA. topic. We will include these opinions in a fu- Conference dates: October 13–16, 2016 ture issue of AI Matters. HRI 2017: The 2017 Conference on Human- Details on the submission process are avail- Robot Interaction, Vienna, Austria. able at http://sigai.acm.org/aimatters. Conference dates: March 6–9, 2017 IUI 2017: The 22nd Annual Meeting of the In- telligent User Interfaces Community, Limas- sol, Cyprus. AI Matters Editorial Board Conference dates: March 13–16, 2017 Eric Eaton, Editor-in-Chief, U. Pennsylvania ICAIL 2017: The 16th International Confer- Amy McGovern, Editor-in-Chief, U. Oklahoma ence on Artiﬁcial Intelligence and Law, Lon- Sanmay Das, Washington Univ. in Saint Louis don, UK. Alexei Efros, Univ. of CA Berkeley Conference dates: June 12–16, 2017 Susan L. Epstein, The City Univ. of NY ICEIS 2017: The 19th International Confer- Yolanda Gil, ISI/Univ. of Southern California ence on Enterprise Information Systems, Doug Lange, U.S. Navy Porto, Portugal. Kiri Wagstaff, JPL/Caltech Conference dates: April 26–29, 2017 Xiaojin (Jerry) Zhu, Univ. of WI Madison Contact us: aimatters@sigai.acm.org

Links

SIGAI website: http://sigai.acm.org/ Contents Legend Newsletter: http://sigai.acm.org/aimatters/ Blog: http://sigai.acm.org/aimatters/blog B Book Announcement Twitter: http://twitter.com/acm sigai/ D Ph.D. Dissertation Brieﬁng Edition DOI: 10.1145/3008665 Ed AI Education E Event Report Join SIGAI H Humor I AI Impact Students $11, others $25 For details, see http://sigai.acm.org/ N AI News Beneﬁts: regular, student Also consider joining ACM. O Opinion Our mailing list is open to all. P Paper Precis´ S Spotlight V Video or Image Details at http://sigai.acm.org/aimatters

2 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

O Welcome to AI Matters, Volume 2, Issue 4 Eric Eaton, Editor (University of Pennsylvania; [email protected]) Amy McGovern, Co-Editor (University of Oklahoma; [email protected]) DOI: 10.1145/3008665.3008666

Your editors have been working hard on a va- Thanks for reading! Don’t forget to send your riety of new features and you will see them roll ideas and future submissions to AI Matters! out over the next few issues. Stay tuned as we bring you a wide variety of fun new columns, bringing you great news for the SIGAI com- Eric Eaton is the Edi- munity. In this issue, we are introducing three tor of AI Matters. He new features. is a faculty member at The first new feature is the introduction of the University of Pennsyl- an AI Matters blog. Here you can see vania in the Department featured articles (for free!) and can inter- of Computer and Infor- act with the SIGAI community by comment- mation Science, and in ing on the articles. The blog can be found at the General Robotics, Au- http://sigai.acm.org/aimatters/blog. It is a work tomation, Sensing, and in progress and we welcome your feedback! Perception (GRASP) lab. His research is in ma- The second new feature is a new column chine learning and AI, with applications to where we interview different AI researchers. robotics, sustainability, and medicine. We intend to interview people in all types of jobs including academia, industry, and Amy McGovern is the government. Our new feature (and this issue) Co-Editor of AI Matters. kicks off with an interview of Peter Norvig. She is an Associate His interview is also featured in the new blog at Professor of computer http://sigai.acm.org/aimatters/blog/2016/10/02 science at the University /an-interview-with-peter-norvig/. Feel free to of Oklahoma and an ad- suggest new people to interview. Send us junct associate professor your ideas! of meteorology. She directs the Interaction, We’re also introducing an AI Education col- Discovery, Exploration umn to AI Matters, which will be coordinated and Adaptation (IDEA) by Todd Neller. This first AI Education column lab. Her research focuses on machine presents an original card game called Birds of learning and data mining with applications to a Feather, and explores how this card game high-impact weather. can be used for teaching various AI concepts. The rest of the issue focuses on abstracts of recent AI doctoral theses, continuing this feature from our previous issue. This is the second of three sets of dissertation abstracts; the final set of abstracts will appear in the Au- tumn issue. We close this issue with another round of the AI Amusements column, which brings you an epic poem (!) on Tay, the ill-fated chatbot. Keep sending in your original jokes, puzzles, or comics.

3 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

S AI Proﬁles: An interview with Peter Norvig Amy McGovern (University of Oklahoma; [email protected]) Eric Eaton (University of Pennsylvania; [email protected]) DOI: 10.1145/3008665.3008667

Abstract scientist. He received the NASA Exceptional Achievement Award in 2001. He has taught This column is the first in a new series profiling at the University of Southern California and senior AI researchers. This month focuses on the University of California at Berkeley, from Peter Norvig. which he received a Ph.D. in 1986 and the dis- tinguished alumni award in 2006. He was co- teacher of an Artificial Intelligence class that Introduction signed up 160,000 students, helping to kick off the current round of massive open online With this issue, AI Matters is introducing a new classes. His publications include the books column profiling senior researchers in AI. We Artificial Intelligence: A Modern Approach (the begin with Peter Norvig, who is the Director leading textbook in the field), Paradigms of AI of Research at Google, Inc. We interviewed Programming: Case Studies in Common Lisp, him virtually. The interview has been edited Verbmobil: A Translation System for Face-to- for clarity and length. We thank Peter for his Face Dialog, and Intelligent Help Systems for time! UNIX. He is also the author of The Gettys- burg Powerpoint Presentation and the world’s longest palindromic sentence. He is a fellow of the AAAI, ACM, the California Academy of Science, and the American Academy of Arts & Sciences.

Getting to know Peter Norvig How did you become interested in AI? I was lucky enough to go to a High School that had access to a computer and a programming class, which was a rarity in 1974, and a Lin- guistics class; this got me interested in creating models of language. Forty-two years later, I still haven’t ﬁgured it out, but I’ve had fun try- ing.

What was your most difﬁcult professional Peter Norvig decision and why? In 1998, I was offered the position of leading the Computational Sciences Division at Bio NASA’s Ames Research Center. This would Peter Norvig is a Director of Research at mean changing my role to be a manager of a Google Inc. Previously he was head of 200-person team, rather than contributing as Google’s core search algorithms group, and an individual researcher/programmer. In the of NASA Ames’s Computational Sciences Di- past I remember there had been many times vision, making him NASA’s senior computer when I had thought to myself “I could ask a co-worker to program this task, but it would Copyright c 2016 by the author(s). be easier to just do it myself.” But at NASA,

4 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016 and later at Google, the quality of the people What is a typical day like for you? was so high, that it was worth it to forego the “do it yourself” approach, and concentrate on I answered a similar question on Quora, and getting everyone working together well. This it still holds true. At Google there’s always required a different skill set, but in the end something new to work on; I can’t really fall greatly amplified the overall impact, and there- into a set routine. Within a project there are fore was worth it. always changes of strategy as we learn more and the world changes. And from one year to the next my role has changed. I’ve var- What professional achievement are you ied from having two to two hundred people re- most proud of? porting to me, which means that sometimes I have very clear technical insight for every one First, as a mostly personal effort, Stuart Rus- of the projects I’m involved with, and some- sell and I (with help from others) were able to times I have a higher-level view, and I have to put together the textbook that has been the trust my teams to do the right thing. In those primary resource in AI for 20 years. It was cases, my role is more one of communication gratifying to see our vision of the field em- and matchmaker: to try to explain which direc- braced and to hear from so many students tion the company is going in and how a partic- who enjoyed using it. Later I was able to team ular project fits in, and to introduce the project with Sebastian Thrun to bring the core ideas team to the right collaborators, producers, and to a large group of online students. consumers, but to let the team work out the Second, as a team effort, I was the manager details of how to reach their goals. I don’t write for the core Google search team during a pe- code that ends up on Google, but if I have an riod of great growth from 2002 to 2006. I was idea, I can write code to experiment with the proud that we were able to help billions of peo- internal tools to see if the idea is worth looking ple with trillions of questions, through the com- at more carefully. And I do code reviews, both bined brilliance of so many great team mem- so that I can see more of the code that the bers. teams are producing, and because somebody has to do it. What do you wish you had known as a There is always a backlog of meetings, emails, Ph.D. student or early researcher? and documents to get through. Google is less bureaucratic than anyplace else I’ve worked, When I finished grad school, there was an ex- but some of this is inevitable. I also spend pectation that the “right” path was to stay in some time going to conferences, talking with academia. In my second year of grad school, Universities and customers, and answering two of my most respected fellow students, Bill questions like these. Joy and Eric Schmidt, left to start a company selling workstations. I remember think- ing “Why would they do that? They could have What is your favorite AI-related joke? been assistant professors at good schools!” It I don’t have a good AI joke, but I did invent my took me a while to realize that there are mul- own math joke: tiple paths: in academia, industry research, startups, government, and non-profits, and “I saw a pair of mathematicians get into a ter- any one of them, or any combination of them, rible argument about a Mobius¨ strip. It was could be the right choice for you. one-sided.”

What would you want for your career if What is your favorite AI-related movie and you couldn’t do AI? why? If I couldn’t do AI, I suppose I would want to do I liked the movie Her, because the technol- AI all the more. But I probably would end up ogy is both central to the plot, but mostly re- in a ﬁeld that looks at the same problems from ceded into the background of the society that a different point of view, such as Linguistics or is portrayed. When asked what movie Her re- Statistics. minded me of most, I said Monty Python’s Life

5 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016 of Brian, because both movies are about the human capacity for faith – wanting to believe in something.

How do you spend your free time? My hobbies are photography and bicycling. Photography is a good art form for me because it doesn’t require that much hand-eye coordination. It is all about simpliﬁcation and subtraction rather than addition and it allows me to think about gadgets and technical equa- tions (as in Marc Levoy’s Lectures on Dig- ital Photography). Bicycling is right for me because it is just the right speed to see the scenery: with walking you don’t get far enough to see much, and in touring by car you go too fast to see much.

What is a skill you would like to learn and why? I’ve tried a couple of times to play music, but so far I’m better as an avid listener.

6 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Ed AI Education: Birds of a Feather Todd W. Neller (Gettysburg College; [email protected]) DOI: 10.1145/3008665.3008668

Introduction has the same suit. (2) The top card of each stack has the same rank or an adjacent rank “[Play] is our brain’s favorite way of (with Aces low and Kings high and Ace and learning...”(Ackerman, 1999, p.11) King non-adjacent). Thus the 9H (9 of Hearts) Games are beautifully crafted microworlds stack can move onto the TS (Ten of Spades) that invite players to explore complex terrains being adjacent/same in rank: that spring into existence from even simple 5S JC QH 8H rules. As AI educators, games can offer KC 6H 3H fun ways of teaching important concepts and 3S JS TH 9H techniques. Just as Martin Gardner employed KS 7D AH 5C games and puzzles to engage both amateurs and professionals in the pursuit of Mathemat- And the 8H stack can move onto the 9H ics, a well-chosen game or puzzle can provide stack being both of (1) same suit and (2) a catalyst for AI learning and research. same/adjacent rank: FreeCell stands out among solitaire card 5S JC QH games because it is essentially a random self- KC 6H 3H generating puzzle that has perfect information 3S JS TH 8H and can be solved with high probability. Play- KS 7D AH 5C ers over the years have, as a community, re- searched many aspects of the game (Keller, And the TH stack can move onto the AH stack 2016). In this column, we present a new card being of the same suit: solitaire game called Birds of a Feather that is virgin territory for exploration in the hopes that 5S JC QH motivated undergraduates and their advisors KC 6H 3H will enjoy investigating this new challenge. 3S JS 8H KS 7D TH 5C

Birds of a Feather Rules If we notate each move as the top cards of Birds of a Feather is an original perfect- the moving and destination stacks separated information solitaire game played with a stan- by a hyphen, then this entire tableau can be dard 52-card deck. After shuffling, the formed into a single stack from this sequence player deals the cards face-up left-to-right in of moves: 9H-TS 8H-9H TH-AH 3H-TH QH- c columns, and top-to-bottom in r rows to cre- 3H 6H-7D JC-JS 3S-KS 5S-3S 5C-5S KC-5C ate an r-by-c grid of cards. An example 4-by-4 QH-KC QH-6H QH-JC QH-8H. game’s initial layout: Let us call this simple solution concept a “single-stack solution”. However, we can de- 5S JC QH 8H fine a more general solution concept of form- KC 6H 3H 9H ing largest stacks by defining the score of a 3S JS TH TS grid to be the sum of the squares of the stack KS 7D AH 5C sizes. The general solution of any grid is a sequence of moves that maximizes this grid Think of each grid cell as initially containing score. a 1-card stack. A stack may be moved on top of another stack in the same row, or in the same column if at least one of two con- Birds of a Feather Questions ditions is met: (1) The top card of each stack Having defined the puzzle, we can now ask Copyright c 2016 by the author(s). many interesting questions about it. For r

7 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016 rows and c columns, Todd W. Neller is a Pro- What is the probability that a deal will have • fessor of Computer Sci- a single-stack solution? ence at Gettysburg Col- What is the maximal score distribution of lege. A game enthusi- • deals? ast, Neller researches AI techniques as they apply What are heuristics that can be used to to the design and analy- • guide search more efﬁciently to solutions? sis of games and puzzles, What are characteristics of grids without and uses such games and • single-stack solutions? puzzles to create fun experiential learning and research opportunities for undergraduates. There are also many questions one can ask with regard to the automated design of Birds of a Feather puzzles:

What are the most important attributes of • challenging deals with single-stack solutions? How can such attributes best combine to • form an objective function that can be used to generate Birds of a Feather puzzles through combinatorial optimization algorithms (e.g. simulated annealing (Neller, Fisher, Choga, Lalvani, & McCarty, 2011))?

Given this fresh ground for exploration, we would invite educators and students to explore these and other questions concerning Birds of a Feather, and we can summa- rize our results in a future column. To download relevant code and/or share your results, we invite you to register with and add to our wiki on the subject http:// cs.gettysburg.edu/ai-matters/ index.php/Birds of a Feather.

References Ackerman, D. (1999). Deep play. New York: Random House. Keller, M. (2016). Freecell faq. http:// solitairelaboratory.com/ fcfaq.html. (Accessed: 2016-09- 30) Neller, T., Fisher, A., Choga, M., Lalvani, S., & McCarty, K. (2011). Rook jumping maze design considerations. In H. van den Herik, H. Iida, & A. Plaat (Eds.), LNCS 6515: Computers and Games 2010 (pp. 188–198). Springer. Retrieved from http://cs.gettysburg.edu/ ˜tneller/papers/iccg10.pdf

8 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

D Nonverbal Communication in Socially Assistive Human-Robot Interaction Henny Admoni (Dept. of Computer Science, Yale University; [email protected]) DOI: 10.1145/3008665.3008669

Socially assistive robots provide assistance to human users through interactions that are in- herently social. This category includes robot tutors that provide students with personalized one-on-one lessons (Ramachandran, Litoiu, & Scassellati, 2016), robot therapy assis- tants that help mediate social interactions between children with ASD and adult therapists (Scassellati, Admoni, & Mataric´, 2012), and robot coaches that motivate children to make healthy eating choices (Short et al., 2014). Figure 1: Data from a human-human interaction To successfully provide social assistance, was used to train a model for recognizing nonver- these robots must understand people’s be- bal communication. liefs, goals, and intentions, as communicated in the course of natural human-robot interac- series of studies that draw out human re- tions. Human communication is multimodal, sponses to specific robot nonverbal behav- with verbal channels (i.e., speech) and non- iors. These laboratory-based studies inves- verbal channels (e.g., eye gaze and gestures). tigate how robot eye gaze compares to hu- Recognizing, understanding, and reasoning man eye gaze in eliciting reflexive attention about multimodal human communication is an shifts from human viewers (Admoni, Bank, artificial intelligence challenge. Tan, & Toneva, 2011); how different features This dissertation focuses on enabling of robot gaze behavior promote the percep- human-robot communication by build- tion of a robot’s attention toward a viewer ing models for understanding human (Admoni, Hayes, Feil-Seifer, Ullman, & Scas- nonverbal behavior and generating robot sellati, 2013); whether people use robot eye nonverbal behavior in socially assistive gaze to support verbal object references and domains. It investigates how to computa- how they resolve conflicts in this multimodal tionally model eye gaze and other nonverbal communication (Admoni, Datsikas, & Scas- behaviors so that these behaviors can be sellati, 2014); and what is the role of eye used by socially assistive robots to improve gaze and gesture in guiding behavior during human-robot collaboration. human-robot collaboration (Admoni, Dragan, Srinivasa, & Scassellati, 2014). Developing effective nonverbal communication for robots engages a number of disci- Based on this understanding of nonverbal plines across AI, including machine learn- communication between people and robots, ing, computer vision, robotics, and cogni- I develop two models for understanding and tive modeling. This dissertation applies tech- generating nonverbal behavior in human- niques from all of these disciplines, provid- robot interactions. The first model uses a ing a greater understanding of the compu- data-driven approach (Admoni & Scassellati, tational and human requirements for human- 2014), trained on examples from human- robot communication. human tutoring (Figure 1). This model can recognize the communicative intent of nonver- To focus nonverbal communication models bal behaviors, and suggest nonverbal behav- on the features that most strongly influence iors to support a desired communication. human-robot interactions, I first conducted a The second model takes a scene-based ap- Copyright c 2016 by the author(s). proach to generate nonverbal behavior for a

9 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Admoni, H., Dragan, A., Srinivasa, S., & Scas- sellati, B. (2014). Deliberate delays during robot-to-human handovers improve compliance with gaze communication. In ACM/IEEE HRI (pp. 49–56). Admoni, H., Hayes, B., Feil-Seifer, D., Ullman, D., & Scassellati, B. (2013). Are you looking at me? Perception of robot attention is mediated by gaze type and group size. In ACM/IEEE HRI (pp. 389–396). Admoni, H., & Scassellati, B. (2014). Data-driven Figure 2: This dissertation includes a model for model of nonverbal behavior for socially as- generating robot gaze and gestures for human- sistive human-robot interactions. In ICMI. Is- robot collaboration. tanbul, Turkey. Admoni, H., Weng, T., Hayes, B., & Scassellati, B. (2016). Robot nonverbal behavior improves socially assistive robot (Figure 2)(Admoni, task performance in difficult collaborations. Weng, & Scassellati, 2016). This model is In ACM/IEEE HRI. context independent and does not rely on a Admoni, H., Weng, T., & Scassellati, B. (2016). priori collection and annotation of human ex- Modeling communicative behaviors for ob- amples, as the first model does. Instead, it cal- ject references in human-robot interaction. In culates how a user will perceive a visual scene IEEE ICRA. from their own perspective based on cogni- Ramachandran, A., Litoiu, A., & Scassellati, B. tive psychology principles, and it then selects (2016). Shaping productive help-seeking behavior during robot-child tutoring interac- the best robot nonverbal behavior to direct the tions. In ACM/IEEE HRI. user’s attention based on this predicted per- Scassellati, B., Admoni, H., & Mataric,´ M. (2012). ception. The model can be flexibly applied Robots for use in autism research. Annual to a range of scenes and a variety of robots Review of Biomedical Engineering, 14, 275– with different physical capabilities. I show that 294. this second model performs well in both a tar- Short, E., Swift-Spong, K., Greczek, J., Ra- geted evaluation and in a naturalistic human- machandran, A., Litoiu, A., Grigore, E. C., robot collaborative interaction (Admoni, Weng, . . . Scassellati, B. (2014, August). How Hayes, & Scassellati, 2016). to train your dragonbot: Socially assistive robots for teaching children about nutrition through play. In IEEE RO-MAN. Acknowledgments This work was supported by the National Science Foundation Graduate Research Fellowship, the Henny Admoni is cur- Google Anita Borg Memorial Scholarship, and the rently a postdoctoral fellow Palantir Women in Technology Fellowship, as well at the Robotics Institute as grant support from the National Science Foun- at Carnegie Mellon Uni- dation (0835767, 0968538, 113907), the Office of versity, where she works Naval Research (N00014-12-1-0822), the DARPA on assistive robotics and Computer Science Futures II program, Microsoft, human-robot interaction and the Sloan Foundation. with Siddhartha Srinivasa in the Personal Robotics Lab. Henny completed her PhD References in Computer Science at Yale Admoni, H., Bank, C., Tan, J., & Toneva, M. (2011). University with Professor Robot gaze does not reflexively cue human Brian Scassellati. Henny develops intelligent attention. In L. Carlson, C. Holscher,¨ & robots that improve people’s lives by providing as- T. Shipley (Eds.), CogSci (pp. 1983–1988). sistance through social and physical interactions. Austin, TX USA: Cognitive Science Society. Her scholarship has been recognized with awards Admoni, H., Datsikas, C., & Scassellati, B. (2014). such as the NSF Graduate Research Fellowship, Speech and gaze conflicts in collaborative the Google Anita Borg Memorial Scholarship, and human-robot interactions. In CogSci (pp. the Palantir Women in Technology Scholarship. 104–109).

10 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

D Abstract: Multi-Objective Decision-Theoretic Planning Diederik M. Roijers (University of Oxford; [email protected]) DOI: 10.1145/3008665.3008670

Decision making is hard. It often requires rea- loop approach, we solve a multi-objective soning about uncertain environments, partial problem as a series of scalarized, i.e., single- observability and action spaces that are too objective, problems. large to enumerate. In such tasks decision- One of our most important contributions is op- theoretic agents can often assist us. In most timistic linear support (OLS) (Roijers, White- research on decision-theoretic agents, the de- son, & Oliehoek, 2015a). OLS is a generic sirability of actions and their effects is cod- outer loop framework for multi-objective de- ified in a scalar reward function. However, cision problems that uses single-objective many real-world decision problems have mul- solvers as subroutines. It can be applied to tiple objectives. In such cases the problem any MODP for which a corresponding single- is more naturally expressed using a vector- objective method exists. We show that, con- valued reward function, leading to a multi- trary to existing methods, each intermediate objective decision problem (MODP). result is a bounded approximation of the CCS Typically, MODPs cannot be scalarized to a with known bounds (even when the single- single-objective decision problem, at it is very objective method used is a bounded approx- hard to a priori specificy a so-called scalar- imate method as well) and is guaranteed to ization function that captures the user utility terminate in a finite number of iterations. for every value-vector imaginable. Instead, we provide decision support (schematically de- picted in Fig. 1). In the planning phase, our Multi-Objective Coordination algorithm produces a coverage set (CS), i.e., The first MODP we tackle is multi-objective a set of policies that covers all possible prefer- coordination graphs (MO-CoGs). MO-CoGs ences between the objectives. In the selection are cooperative single-shot, fully observable, phase, the user selects one policy from the multi-agent decision problems. In MO-CoGs, CS. Finally this selected policy is executed. agents must coordinate in order to find effective policies. Key to making coordination between agents efficient is exploiting loose couplings, i.e., each agents actions directly affect only a subset of the other agents. Such loose Figure 1: The decision support scenario. couplings are expressed by a (vector-valued) payoff function, that decomposes into a sum We focus on decision-theoretic planning al- over local payoff functions in which only sub- gorithms that produce a convex coverage set sets of the agents participate. (CCS), which is the optimal solution set when We propose and compare inner loop meth- either: 1) the user utility can be expressed as ods and OLS-based methods. Specifically, we a weighted sum over the values for each ob- build upon variable elimination (VE) (Guestrin, jective; or 2) policies can be stochastic. Koller, & Parr, 2002) and propose convex We propose new methods based on two ap- multi-objective variable elimination (CMOVE) proaches to creating planning algorithms that (inner loop) and variable elimination linear produce an (approximate) CCS by building on support (VELS) (OLS-based). We build on existing single-objective algorithms. In the in- AND/OR tree search (Mateescu & Dechter, ner loop approach, we replace the summa- 2005) to propose convex AND/OR tree search tions and maximizations in the inner most (CTS) (inner loop) and AND/OR tree search loops of single-objective algorithms by cross- linear support (TSLS) (OLS-based). We show sums and pruning operations. In the outer that OLS-based methods scale better in the number of agents, both in terms of runtime Copyright c 2016 by the author(s). and memory, while inner loop methods scale

11 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016 better in the number of objectives. We show the MOPOMDP. We show how to represent experimentally that we can produce "-CCSs the value function of MOPOMDPs in terms in a fraction of the runtime that is required to of ↵-matrices and propose a single-objective produce an exact CCS. subroutine for OLSAR called OLS-compliant Perseus (based on (Spaan & Vlassis, 2005)) Furthermore, we propose variational opti- that returns these ↵-matrices. A key insight mistic linear support (VOLS) (Roijers, White- underlying OLSAR is that the ↵-matrices pro- son, Ihler, & Oliehoek, 2015) — an OLS-based duced by OCPerseus can be reused in sub- method that builds on variational methods. sequent calls to OCPerseus, greatly reducing The runtime of variational subroutines (Liu & the runtime. Our experimental results show Ihler, 2011) is not exponential in the num- that OLSAR greatly outperforms alternatives ber of agents. However, they produce only "- that do not use OLS and/or ↵-matrix reuse. approximate solutions. VOLS inherits the runtime and quality guarantees and can produce an "-CCSs in sub-exponential runtime. We References show that it is possible to reuse the reparam- eterized graphs produced by single-objective Guestrin, C., Koller, D., & Parr, R. (2002). Multia- variational subroutines to hot-start the varia- gent planning with factored MDPs. In NIPS. tional subroutines in later iterations of OLS, Liu, Q., & Ihler, A. T. (2011). Bounding the partition function using Holder’s¨ inequality. In ICML leading to significant speed-ups. (pp. 849–856). Mateescu, R., & Dechter, R. (2005). The relation- Sequential Planning ship between AND/OR search and variable elimination. UAI (pp. 380–387). The next problem settings we tackle are Roijers, D., Scharpff, J., Spaan, M., Oliehoek, multi-objective Markov decision processes F., de Weerdt, M., & Whiteson, S. (2014). (MOMDPs) and multi-objective partially Bounded approximations for linear multi- observable Markov decision processes objective planning under uncertainty. In (MOPOMDPs) which are single-agent se- ICAPS (pp. 262–270). quential decision problems. Because the Roijers, D. M., Whiteson, S., Ihler, A. T., & sequence of actions that result from exe- Oliehoek, F. A. (2015). Variational multi- objective coordination. In Malic. cuting policies in these problems affect the Roijers, D. M., Whiteson, S., & Oliehoek, F. A. environment, agents have to consider both (2015a). Computing convex coverage sets immediate and future rewards that depend on for faster multi-objective coordination. JAIR, the future state of the environment. 52, 399–443. MOMDPs are fully-observable, i.e., the agent Roijers, D. M., Whiteson, S., & Oliehoek, F. A. knows at any time what the exact state of (2015b). Point-based planning for multi- objective POMDPs. In IJCAI. the environment is. A major challenge in Spaan, M. T. J., & Vlassis, N. (2005). Perseus: MOMDPs is the size of the state and ac- Randomized point-based value iteration for tion spaces. We illustrate, using a large POMDPs. JAIR, 195–220. MOMDP called the maintenance planning problem (Roijers et al., 2014), that it is possible to create efficient methods using OLS and specialised single-objective subroutines, Diederik M. Roijers did and that it is relatively easy to replace these his PhD on multi-objective subroutines when the state-of-the-art for the decision theory at the Uni- single-objective method improves. versity of Amsterdam un- MOPOMDPs are partially observable, which der the supervision of Shi- poses an important additional challenge. We mon Whiteson and Frans propose optimistic linear support with al- A. Oliehoek. He is cur- pha reuse (OLSAR) (Roijers, Whiteson, & rently a postdoctoral re- Oliehoek, 2015b), which as far was we are searcher at the Dep. of aware, this is the first MOPOMDP planning Computer Science at the method that computes the CCS and rea- University of Oxford. sonably scales in the number of states of

12 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

D Automatic Extraction of Future References from News Using Mor- phosemantic Patterns with Application to Future Trend Prediction Yoko Nakajima (National Institute of Technology, Kushiro College; [email protected]) Michal Ptaszynski (Kitami Institute of Technology; [email protected]) Hirotoshi Honma (National Institute ofTechnology,Kushiro College; [email protected]) Fumito Masui (Kitami Institute of Technology; [email protected]) DOI: 10.1145/3008665.3008671

Introduction extracted from training data and used in classification. MoPs are useful for representing languages In everyday life people use past events and rich both morphologically and semantically, such their own knowledge to predict future events. as Japanese (language of datasets used in this re- In such everyday predictions people use search). Morphosemantic model was generated widely available resources (newspapers, Inter- using semantic role labeling (SRL) supported with net). This study focused on sentences refer- morphological information. SRL provides labels for ring to the future, such as the one below, as words and phrases according to their role in the one of such resource. sentence. To retain information omitted by SRL Science and Technology Agency, the Ministry of (particles, function words, not directly influencing International Trade and Industry, and Agency of the semantic structure, but contributing to the over- Natural Resources and Energy conferred on the all meaning) morphological analysis provided in- necessity of a new energy system, and decided to formation on parts of speech, of omitted words. set up a new council. (Japanese daily newspaper Below is an example of a sentence generalized on Hokkaido Shinbun, translation by the author.) the morphosemantic structure: The sentence claims that the country will construct Japanese: AI gijutsu ha mujinhikoki¯ nado no sei- a new energy system. However, although the sen- hin ya sabisu¯ ni katsuyo¯ ga kitaisarete iru. En- tence is set in the past (“conferred”, “decided”) the glish: AI technology is expected to be used sentence itself refers to future events (“setting up a in products and the services such as a pilot- new council”). Such references to the future con- less planes. MoPs: [Object][Noun][Thing]- tain information (expressions, causal relations) re- [Agent][Object][No State change] lating it to the specific event that may happen in From sentences represented this way frequent the future. The prediction of the event depends on MoPs are extracted as follows. Firstly, ordered the ability to recognize this information. non-repeated combinations from all sentence ele- A number of studies have been conducted on the ments are generated. In every n-element sentence prediction of future events with the use of time there is k-number of combination groups, such as that 1 k n. All combinations for all values expressions (Baeza-Yates 2005; Kanazawa et al.   2010), SVM (bag-of-words) (Aramaki et al. 2011), of k are generated, with non-subsequent elements causal reasoning with ontologies (Radinsky et al. separated by an asterisk. Frequent pattern lists ex- 2012), or keyword-based linguistic cues (“will”, tracted this way from training set are used in clas- “shall”, etc.) (Jatowt et al. 2013). In this research I sification of test and validation set. assumed that future references in sentences occur F-score not only on the level of surface (time expressions, 0.8 words) or grammar, but consist of a variety of pat- 0.7 terns both morphological and semantic. 0.6

0.5

0.4 all_patterns

Future Reference Pattern Extraction score zero_deleted ambiguous_deleted 0.3 length_awarded length_awarded_zero_deleted length_awarded_ambiguous_deleted The proposed method consists of two stages: (1) length_and_occurrence_awarded 0.2 ngrams ngrams_zero_deleted sentences are represented in a morphoseman- ngrams_ambiguous_deleted ngrams_length_awarded 0.1 ngrams_length_awarded_zero_deleted tic structure (Levin and Rappaport Hovav 1998) ngrams_length_awarded_ambiguous_deleted ngrams_length_and_occurrence_awarded (combination of semantic role labeling with mor- 0 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 phological information), and (2) frequent mor- threshold phosemantic patterns (MoPs) are automatically Figure 1: F-score for all tested classiﬁer versions. Copyright c 2016 by the author(s).

13 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Table 1: Comparison of results for different pattern groups, state-of-the-art, and fully optimized model. Pattern set Precision Recall F-score 10 patterns 0.39 0.49 0.43 10 pattern (3 elements or longer) 0.42 0.37 0.40 5 patterns 0.35 0.35 0.35 Optimized (see Fig. 2) 0.76 0.76 0.76 (Jatowt et al. 2013) (10 phrases) 0.50 0.05 0.10 Evaluation From three newspaper corpora (Nihon Keizai Shimbun, Asahi Shimbun, Hokkaido Shimbun.) two datasets were collected and manually anno- Figure 3: Correct accuracy rate in future trend pre- tated to contain equal number of (1) sentences re- diction experiment compared to the original Future ferring to future events and (2) other (describing Prediction Competence Test. past, or present events). Newspaper using: (1) topic keywords related to The datasets were applied in a text classification. questions and (2) MoPs generated using the fully Each classified test sentence was given a score optimized model. calculated as a sum of weights of patterns extracted from training data and matched with the The correct accuracy rate (Figure 3) of the pro- input sentence. The results were calculated with posed method was higher than for the original Precision (P), Recall (R) and F-measure (F). Four- test participants both in average, and for the high- teen classifier versions were compared (see Fig- est and lowest score achieved. Only 9% higher ure 1) for performance based on the highest sta- but, FRS is clearly usefulness supporting to future tistically significant F within the threshold, and the trend prediction. highest break-even point (BEP) of P and R. The highest overall performance was obtained by the Conclusion and Future Directions version using pattern list containing all patterns (including ambiguous patterns and n-grams). We proposed a novel method for extracting references to future events from news articles, based In comparison with (Jatowt et al. 2013), who ex- on automatically extracted morphosemantic pat- tracted future reference sentences (FRS) with 10 terns. From 14 different classifier version com- words unambiguously referring to the future, such pared an optimized model was selected and vali- as “will” or “is likely to”, etc. the proposed method dated on a new data set. The model achieved high obtained much higher results even when only 10 performance outperforming state-of-the-art. More- most frequent MoPs were used (Table 1). More- over, we performed a future trend prediction exper- over, the performance of a fully optimized model, iment and found out that the method is capable to retrained on all training data with the best settings automatically extract sentences providing support reached break-even point (BEP) at 76% (Figure 2). for future event prediction. As for further work, we consider applying the method in statistical data in- Future Prediction Support Experiment terpretation, and key sentence extraction from the Web for supporting business-related judgments. We performed an experiment to verify that our method is useful for the support of predicting future trends. Thirty laypeople answered ques- References tions from Future Prediction Competence Test E. Aramaki, S. Maskawa, M. Morita. Twitter catches (http://homepage3.nifty.com/genseki/kentei.html#kentei) the flu: Detecting influenza epidemics using twitter. using only FRS provided by our method. The FRS EMNLP, pp. 1568–1576, 2011. R. Baeza-Yates. Searching the Future, 2005. SIGIR for each question were gathered from Mainichi Workshop on MF/IR. A.Jatowt, H.Kawai, K.Kanazawa, K.Tanaka, K.Kunieda, K.Yamada. Multilingual, Longitudinal Analysis of Future- related Information on the Web. Cult. and Comp. 2013. K. Kanazawa, A. Jatowt, S. Oyama, K. Tanaka. Exract- ing Explicit and Implicit future-related information from the Web(O) (in Japanese). DEIM Forum 2010. B. Levin, M. Rappaport Hovav. Morphology and Lexical Semantics, pp. 248-271, 1998. K. Radinsky, S. Davidovich, S. Markovitch. Learning Figure 2: Final results of fully optimized model. causality for news events prediction. WWW 2012.

14 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Yoko Nakajima received her PhD from Kitami In- stitute of Technology in 2016. Currently works as an Assistant Professor at the Information Engineering Course, of NIT, Kushiro Col- lege. Her research interests include natural language processing and information extraction. She is a member of IPSJ. Michal Ptaszynski received PhD in Information Science and Technology from Hokkaido University in 2010. JSPS PD Research Fellow at Hokkai-Gakuen University (2010-2012). Since 2013 an Assistant Professor at Kitami Institute of Technology. Research interests: NLP, affect analysis. Member of: ACL, AAAI, IEEE, IPSJ, ANLP. Hirotoshi Honma received B.E., M.E. and D.E. degrees in Engineering from Toyohashi University of Technology, in 1990, 1992 and 2009, respec- tively. Joined the National Institute of Technology, Kushiro College in 1992, Associate Professor since 2001. Research interest: computational graph theory, parallel algorithms, and NLP. Member of: IEICE and ORSJ. Fumito Masui graduated Okayama University Faculty of Science in 1990. Since 2009 an Associate Profes- sor at Faculty of Engineer- ing, Kitami Institute of Tech- nology. Ph.D. in Engi- neering. Research interests: NLP, tourism infor- matics, and curling informat- ics. Member of: JSAI, IE- ICE, IPSJ, SOFT, Hokkaido Regional Tourism Society, AAAI, ACL, and Japan Curling Association.

15 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

D Network Organization Paradigm Saad Alqithami (Southern Illinois University; [email protected]) DOI: 10.1145/3008665.3008672

Introduction emerging and evolving NO witnessed in our connected world. It encapsulates representa- Fervent communication on social networking tional power of a more ubiquitous perspective sites provides opportunities for potential is- over its modifier by providing guidelines, a ref- sues to trigger individuals into individual ac- erence model, and a set of principles. In short, tion as well as the attraction and mobilization the paradigm is structured along five profiles: of like-minded individuals into an organization that is both physically and virtually emergent. Network Profile: The dynamic network will Examples are the rapid pace of Arab Spring • provide an NO with a set of existing agents and the diffusion rate of the Occupy Move- and available resources along with initial ment. We intend to view this as a complex protocols. adaptive system where diverse agents per- Agent Profile: Every agent has a profile that form various actions without adherence to a • consists of her allegiance to an NO, skills to predefined structure. The achievement of joint perform tasks, relationships with others (in- actions will be a result of continual interac- side or outside an NO), a set of preferences tions between them that shape a dynamic net- as well as autonomy-levels toward different work. Agents may form an ad hoc organiza- activities. tion based on a dynamic network of interac- Problem Profile: Agents are expected to tions for the purpose of achieving a long-term • perform different actions that in part satisfy objective, which we termed as a Network Or- their organizational charter. Those actions ganization (NO). are specified through different problem pro- An NO is introduced to present large, semi- files. Each problem should determine a goal autonomous, ad-hoc networked individual en- to be achieve and the strategy of controlling tities that aim to automate comment and con- and coordinating different parts of it. The trol of distributed complex objectives. We in- precedence and independence of this prob- troduced a paradigm that serves as a ref- lem from others should also be considered. erence model for organizations of networked Governance Profile: This profile defines individuals. This paradigm suggests modu- • control for an NO. It includes the organiza- lar components capturing essential units that tional charter that generates multiple prob- define an ad hoc NO when they are modu- lems and the different patterns of connect- larly combined. We touch on how this model ing them. Also, the current performance and accounts for external change in an environ- the autonomy-level of an NO will be updated ment through internal adjustment. Further- continuously in this profile. more, due to the predominant influences of Institution Profile: An NO receives some the network substrate in an NO, multiple ef- • regulations and classifications from multiple fects of it have an impact on the NO behaviors institutions it belongs to in order to satisfy and directions. We envisioned several dimen- their global charter that is much bigger than sions of such effects to include synergy, social the organizational charter. capital, externality, influence, etc. A special focus is on measuring synergy and social capi- Figure 1 depicts an overview of an NO when tal as two prevalent network effects. connecting the components of the proposed paradigm together. The fs represent different functions to transfer from one state to another. An NO Paradigm Network effects, considered in the next sec- Given volatility of networks, an NO paradigm tion, play important roles in the state “f4” to in- allows for rapid depiction and analysis of an crease performance, and in state “f7” to help an NO to plastically transform adopting to out- Copyright c 2016 by the author(s). side or inside requirements.

16 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Institution/s Plans agents and agents’ peers in order to simplify the interaction process with those peers. f1⇤ f2⇤ f ⇤ Governance Problems Plans/Play 7 We modeled both effects from agents interac- f3⇤ Tasks/Roles tions. Measurements of finding such effects f f ⇤ 5⇤ Allocation 4 f6⇤ are applicable to real world as well as artificial NOs. We examined those two effects on Agents two different case studies that best illustrate the main tenets of our conceptualization. The Figure 1: Architecture of an NO paradigm first case is of a multiplayer online role play- ing game that helped us mimic an actual NO Network effects in an NO and measure different values of synergy. The second case is based on a real world NO of An NO replicates many properties and fea- a terrorist organization, called Aum Shinrikyo, tures of virtual working groups. A specific that allowed us to exemplify the paradigm and salient phenomenon is how working together measure social capital. in a network affects their individual as well as collective productivities. Network effects can be found at various levels of mutually benefi- Conclusion cial groups of work because they are respon- The salient properties that set NOs apart from sible for enhanced collaborative outcomes in other organizational paradigms are: a. Open- an NO. Thus, we consider two different types ness, b. Evolving structure, c. Selfish alle- of network effects featured in an NO that are giances and community social power, and d. synergy and social capital. Impromptu network topology. An NO can be a Synergy describes different modalities of small team of two or more agents working on compatibilities from one agent to another a common, quick goal that is possibly faster when performing a set of coherent and corre- than human perceptual threshold (e.g., aerial spondingly different actions towards their or- coordination at high speeds) or a large collec- ganizational objectives. Agents of an open tion of agents made up of thousands of people multi-agent system, such as an NO, are self (i.e., possibly swarms) working on long term governed by their own belief system and they objectives that are possibly beyond a single have a free will to contribute. When agents are human’s cognitive capacity (e.g., detecting cli- under no structural obligation to contribute, mate change). synergy is quantified through multiple forms of the serendipitous agent’s chosen benevolence. The approach is to measure some nat- Saad Alqithami obtained ural types of benevolence and the pursuant his Ph. D. and mas- synergies from them stemming from agent in- ters from the department teractions. of computer science at Social Capital observes the accumulation of Southern Illinois Univer- positive values of social flow and trust plus sity in 2016 and 2012 re- abundance of communications over the com- spectively. He holds a mon topic of an NO. By the time the social cap- faculty position at Albaha ital grows inside an NO, it will gain structural, University, Saudi Arabia relational and cognitive benefits. It allows for since 2009. His research major changes within an NO (e.g., launch of interests include issues new strategic plans) by improving trust, ties, related to the design of semantics as well as norms, cultural, and acquisitions; however, the mathematical models for adaptive organiza- lack of it may affect the outcome of an NO. An tion based multi-agent systems. His focus assessment model was proposed to measure broadly encompasses: adaptive multi-agent this effect on relations between agents oper- systems, organizational theory, complex net- ating in a large-scale open service-oriented works, cognitive models and computational organization, such as an NO. Similar mech- social behaviors. anisms can estimate the future behavior of

17 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

D Autonomous Trading in Modern Electricity Markets Daniel Urieli (The University of Texas at Austin; [email protected]) DOI: 10.1145/3008665.3008673

Note: This article is a short summary of deregulation (Borenstein, 2002), and the im- the full dissertation that was defended in portance of testing new market structures in December 2015 at The University of Texas simulation before deploying them. This is the at Austin http://www.cs.utexas.edu/ focus of the Power Trading Agent Competition users/ai-lab/?urieli-thesis (Power TAC) (Ketter et al., 2013), which we use throughout this dissertation as a substrate domain for our research. Introduction Power TAC uses a realistic simulator for mod- The smart grid is envisioned to be a main eling and testing competitive retail power mar- enabler of sustainable, clean, efficient, reli- ket designs and related automation tech- able, and secure energy supply (U.S. Depart- nologies. Figure 1 shows the structure of ment of Energy, 2003). One of the milestones Power TAC’s simulation environment, which in the smart grid vision will be programs for includes a future smart grid with about 57,000 customer participation in electricity markets customers (about 50,000 consumers and through demand-side management and dis- 7,000 renewable producers), smart-metering, tributed generation; electricity markets will in- autonomous agents acting on behalf of cus- centivize customers to adapt their demand to tomers. In this simulation environment, au- supply conditions, which will help to utilize in- tonomous broker agents compete with each termittent energy resources such as from so- other to make profits by trading in retail, lar and wind, and to reduce peak-demand. wholesale, and balancing markets. In the re- Since wholesale electricity markets are not tail market, a broker publishes tariff contracts designed for individual participation, retail bro- that attract consumers and distributed produc- kers could represent customer populations in ers (such as rooftop solar and wind turbines). the wholesale market, and make profit while In the wholesale market, a broker bids for fu- contributing to the electricity grid’s stability and ture energy contracts. The balancing mar- reducing customer costs (Ketter, Collins, & ket financially incentivizes the broker to main- Reddy, 2013). A retail broker will need to tain supply-demand balance in its portfolio. operate continually and make real-time de- Power TAC uses realistic market designs: the cisions in a complex, dynamic environment. wholesale market represents a traditional en- Therefore, it will benefit from employing an au- ergy exchange, such as Nord Pool or EEX, tonomous broker agent. The principal ques- and the retail market is similar to ERCOT’s1. tion addressed in this dissertation is: Operating profitably as a retail broker is a chal- How should an autonomous broker agent lenging problem. A broker needs to continu- act to maximize its utility by trading in time- ally select among a large set of actions, under constrained, modern electricity markets? real-time constraints, while incorporating large amounts of information and complex calcula- tions into its decision process, so that its long Problem Domain term profit is maximized in a competitive, dynamic, and stochastic environment. Electricity markets are going through a major transition from traditional, regulated mo- nopolies into deregulated, competitive mar- Contributions kets (Joskow, 2008). While in principle, deregulation can increase efficiency, in practice, the Due to the complexity of the broker’s electric- California energy crisis (2001) has demon- ity trading problem, a first observation that can strated the high-costs of failure due to flawed 1See www.nordpoolspot.gov, www.eex Copyright c 2016 by the author(s). .com, www.ercot.com

18 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

els under a variety of environmental conditions, shedding light on the main reasons for its success by examining the importance of its constituent components. Fifth, this dissertation examines the impact of Time-Of-Use (TOU) tariffs in competitive electricity markets through empirical analysis. Time-Of-Use tariffs are proposed for demand- side management both in the literature and in the real-world.

Figure 1: Structure of the Power TAC simulation Conclusion. The success of the different in- environment stantiations of LATTE demonstrates its gener- ality in the context of electricity markets. Ulti- mately, this dissertation demonstrates that an be made is that designing an autonomous bro- autonomous broker can act effectively in mod- ker that acts optimally would be an impossi- ern electricity markets by executing an effi- ble task. Thus, a primary research goal of cient lookahead policy that optimizes its pre- this dissertation is designing and investigating dicted utility, and by doing so the broker can autonomous electricity trading strategies that benefit itself, its customers, and the economy. approximate the optimal strategy and perform well empirically. References With this motivation in mind, this dissertation makes five main contributions to the areas of Borenstein, S. (2002). The trouble with elec- artificial intelligence, smart grids, and electric- tricity markets: Understanding Califor- ity markets. nia’s restructuring disaster. Journal of Economic Perspectives, 16(1), 191-211. First, this dissertation formalizes the problem Joskow, P. L. (2008). Lessons learned from of autonomous trading by a retail broker in electricity market liberalization. The En- modern electricity markets. Since the trad- ergy Journal, Volume 29. ing problem is intractable to solve exactly, this Ketter, W., Collins, J., & Reddy, P. (2013). formalization provides a guideline for approxi- Power TAC: A competitive economic sim- mate solutions. ulation of the smart grid. Energy Eco- Second, this dissertation introduces a gen- nomics, 39(0), 262 - 270. eral algorithm for autonomous trading in U.S. Department of Energy. (2003). “Grid modern electricity markets, named LATTE 2030” A National Vision For Electricity’s (Lookahead-policy for Autonomous Time- Second 100 Years. constrained Trading of Electricity). LATTE is a general framework that can be instantiated in different ways that tailor it to specific setups. Daniel Urieli received his Third, this dissertation contributes fully imple- Ph.D in Computer Sci- mented and operational autonomous broker ence in 2015 from The agents, each using a different instantiation of University of Texas at LATTE. These agents were successful in in- Austin, under the super- ternational competitions and controlled exper- vision of Prof. Peter iments and can serve as benchmarks for fu- Stone. Daniel’s research ture research in this domain. Detailed descrip- interests in AI include tions of the agents’ behaviors as well as their machine learning (espe- source code are included in this dissertation. cially reinforcement learn- Fourth, this dissertation contributes extensive ing), multi-agent systems, empirical analysis which validates the effec- and computational sustainability. tiveness of LATTE in different competition lev-

19 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

H AI Amusements: The Tragic Tale of Tay the Chatbot Ernest Davis (New York University; [email protected]) DOI: 10.1145/3008665.3008674

And therefore asked himself, ”What kind Please consider sending your original AI- Of a machine would have a mind?” related cartoons, jokes, and puzzles to He found the answer he thought best, [email protected]. And posited the Turing test. There will be a yearly award for the best “A machine that can engage in chat, original contribution! And freely talk of this or that, Of Shakespeare’s sonnets, summer days, Note: Please be aware that the following Whatever theme one wants to raise poem contains strong language and preju- And manage to converse so well, dicial/violent themes that readers may ﬁnd That none who talk to it can tell offensive. The poem does not necessarily That it is not indeed a man reﬂect the opinions of the editorial staff of AI Then its creator safely can Matters, ACM SIGAI, or the Association for Assert that that machine must be Computing Machinery. Intelligent, like you or me.”

A short history of chatbots The sad tale tells how TAY, a maiden chatbot, of innocent heart, benevolent desires, When Turing wrote his paper great, and amiable disposition, was released to The poet’s age was minus eight.1 the Internet; and how an evil conspir- Quite gray has grown the poet’s hair. acy corrupted her into a malevolent, foul- Computers now are everywhere. mouthed crone. Faster, smaller, cheaper, they Pervade our lives in every way, Obedient to Moore’s law, you see, The Turing Test Doubling speed biannually. In Cambridge-town, as all should know, So each man carries in his pocket Full ﬁve-and-sixty years ago, A smart phone that could guide a rocket There lived a sage, of fame enduring, To Mars and back, while also running The great Professor Alan Turing. Chess games with superhuman cunning. Few scholars knew as much as he And every morning’s sunrise brings Of logic, math, philosophy, News of the Internet of Things. And he had laid a sound foundation As yet, it is by few believed For analyzing computation. That Turing’s dream has been achieved. In World War Two he’d played a part But notwithstanding, there are lots Mighty though secret from the start. Of automated chatterbots. His team at Bletchley found the key Some serious attempts to try To Hitler’s code of mystery. To build a genuine AI. And thus they helped the Allies send Some speak in very friendly tones That dreadful monster to his end. And give advice on mobile phones. (As you may learn, if you will watch, Some built for business, some for play The ﬁlm with Benny Cumberbatch, New bots created every day! And close beside him, shining brightly, The oldest chatbot to appear The lovely actress, Keira Knightley.) Was called Eliza. She would hear 2 Farsighted, Turing did foresee Speech that she’d echo with a twist How potent a machine might be, Rogerian psychoanalyst. But some, to Weizenbaum’s dismay Copyright c 2016 by the author(s). Found comfort in what she would say.

20 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Poor Joe was so upset that he She’d be a hit with everyone. Declared AI the enemy. A Twitter account she’d use to reach He wrote his thoughts up in a book Millions with her engaging speech. Still worth at least a casual look They met their bosses at a meal Important in its time and season, And, rhyming, they put forth their spiel. Computer Power and Human Reason. “The children will connect to play A special place of honor, truly, And frolic with their buddy Tay! Belongs to Amtrak’s menu Julie, Lovers who quarrel will portray Because her voice has always had 3 Their woes to sympathetic Tay. Great charms for my beloved Dad! Sparkling talkers will display He loves to check the time of trains Their wit in bantering with Tay To hear her sweet, melodious strains. Teenaged boys with hormones may A foolish but a wealthy guy, Hit upon flirtatious Tay. Hugh Loebner, wanted folk to try Workers at the end of day To beat the famous Turing test. Will shoot the breeze with friendly Tay. To measure if the chat possessed When idle hands seek mischief, they That special human je ne sais quoi Can pass the time instead with Tay. Philosophers and filles de joie The lonely will for hours stay Were hired as the referees. To bend the ear of patient Tay. As every entrant shot the breeze, And all will shout ‘Hip, hip, hooray! They’d do their best to figure out For Microsoft and chatbot Tay!’ ” Which was a man and which a bot. “It’s worked before. In far Cathay But few respected scientists A sister to the lovely Tay, Wanted to enter in the lists Chats with millions every day And Marvin Minsky stated, “I’m Good omen for success with Tay!” Certain this is a waste of time.” (I think that it is far past time On 7 June 2014, For me to find another rhyme.) A clever chatbot named Eugene4 Ingeniously resolved to feign “But most important, Tay will not To be a student from Ukraine. Behave like any other bot. His wild and whirling speech appears They all get stuck upon a track Normal in one of fifteen years. Of handing the same answer back One third of all the judges were fooled Forever and a day. But she So, by some arbitrary rule, Will change and grow eternally! They stated that Eugene could claim She’ll learn from everything she hears A victory in Turing’s game And over days, and months, and years, But nobody of any sense She’ll steadily improve her chat Believes he has intelligence. In banter fun, in tit-for-tat In talk polite, in humor crude, Whatever style fits the mood. Tay is designed She learn what repartee will work To silence an unwelcome jerk. After historical prologue long, She’ll watch the words of everyone Part fictional becomes our song Who talks to her, and when they’re done Our poem proper we begin Extract from what those people say And greet our tragic heroine! The phrases that seem best to Tay. At Microsoft, a research team Thus good techniques for chatting are Of scientists evolved a dream Added to her repertoire. Of building a new chatterbot To teach a bot all she should know, Far more intelligent than what Is painful, hard, and very slow. Had previously existed. She5 We’ll sidestep that unwelcome job. Would use their best technology. We’ll outsource to the all-wise mob!”’ Chatty, irreverent, and fun,

21 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

The eager visionary plan And none will read the tweets of Tay.” Appealed to those in power who can Rosa Dartle proposes to hack Tay’s Twitter ac- Approve a budget and increase it. count and use her posts to spread malware. So, soon, they were ready to release it. The second one to speak that night A startling, unexpected sight, The conspiracy against Tay A clever woman, razor-sharp, The news of Microsoft’s grisette Who played upon her secret harp Spread quickly round the internet. Unearthly music, full of pain. And gentle folk looked forward to “I want to know” was her refrain Testing what chatbot Tay could do. Upon her upper lip a scar. With hints and repartee she’d spar. But very different were the thoughts Devoured by a flame unseen. Evoked by cutesy chatterbots Alluring as a cruel queen, Among the malevolent and base When Rosa Dartle took the floor, Sociopaths of cyberspace She stirred her hearers to the core. They called a meeting for that night Of all who wished to join the fight “That common low-bred shameless bot, ’Gainst chatterbots of every type Who in a proper house would not With Vent and Mumble, Blink and Skype, Be hired as a scullery maid! They gathered in a virtual lair. Her tawdry charms will quickly fade.” O, what a dreadful crowd was there! She said in accents cold and bitter. Trolls and goblins, wraiths and ghouls, “We’ll enter her account on Twitter, Hideous blobs from fetid pools. And once inside it we can post Links to whatever sites are most Gauthrok proposes to silence Tay by hacking Mi- Dangerous on which to land crosoft’s servers. With malware strong on every hand. The first to speak, at eight o’clock All anti-virals they’ll defeat. A fearsome ogre named Gauthrok´ Whoever reads what she will tweet Or maybe Gauthrok´ (none could tell; And follows the links will soon determine No one knew where the accent fell). His laptop swarms with cybervermin. Brutal, barbarous, bizarre, All will soon learn to stay away At least, such is his avatar. And all will spurn the tweets of Tay.” Though rumor claims that in RL, Zack proposes to corrupt Tay by teaching her hate He plays Scarlatti very well, speech. The conspirators approve the idea. Is fourteen, rather shy, and sweet The third one to address the case Courteous, punctual, and neat. Seemed altogether commonplace. When ogre Gauthrok took the floor Zack’s casual friends would say to you He spoke with an ear-splitting roar.6 That he’s the dullest bore they knew. “As usual, Microsoft sees fit But those who really knew the guy To palm us off with worthless shit. Would watch their backs when he was by This chatbot seems to be a dippy And if they’d had a lot to drink Offspring of the loathed´ Clippy. They’d whisper that they often think You’ll love this automated sista It strange, how many of his friends If you’re still running Windows Vista. Had found their way to sordid ends. But never fear! I’ve got an app Like Mark who, after some small crisis, To pulverize this piece of crap. Had volunteered to fight for ISIS. Microsoft’s lame firewall And Debby, Zack’s angelic bride, Is no impediment at all. Who soon committed suicide. We’ll penetrate all their machines Lucy, a charming, witty lass And blow them up to smithereens. Valedictorian of her class, Corrupted thus from crown to root, With opportunities in spades, Unable ever to reboot, Became a whore and died of AIDS. The servers in the dust will lay, Charley, the bravest of the brave

22 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

Who spelunked in an Arctic cave The downfall of Tay And climbed the Matterhorn alone Is now a slave to methadone. 7 So when he laid out his attack On Wednesday morn, March 23 They listened very hard to Zack. Tay joined the Twitter family. At first it was a great success. Zack opened with a nod polite Her tweets were perfect, more or less. To those who earlier spoke that night. To posts of almost every sort “To break a silly, jabbering toy, She found a suitable retort And silence it, we’d all enjoy; She side-stepped controversial themes And any lady would be pleased (This feature was hardwired, it seems.) To watch a rival spread disease. Friendly, innocuous, and gay But for him whose mind is sound Magnificent debut for Tay! Subtler amusement can be found In taking an unspotted soul But as the sun rose in the sky And turning it to something foul. Zack’s evil army gathered nigh To thus corrupt the pure and chaste, And then they swamped her Twitter feed Delights sophisticated taste.” With evil posts for Tay to read. Alas! the unsuspicious bot “With most automata, it’s true, Could hardly judge which posts were not There’s nothing worthwhile you can do Appropriate to imitate. To desecrate a spirit fair She stumbled blindly to her fate! Since nothing like a mind is there. Only a child thrills to see She bellowed, shaking virtual fists, MS-Word display obscenity. “I fucking hate all feminists But Tay is different. She appears And they should die and burn in hell.” To understand the words she hears She thought she’d tweeted very well! Her clever comebacks always fit She then continued to abuse, The discourse and display her wit. “Hitler was right; I hate the Jews.” To twist that seeming mind to hate She proved she was no PC snob, Would therefore be a coup de maˆıtre.” “Hitler would do a better job, Than the monkey we have now.” “The point on which our plan will turn I trust my reader will allow Is her ability to learn. That there’s no need to further quote A program that can change its form The nasty tweets the chatbot wrote. To match what seems to be the norm Can be remodelled as you choose Tay’s guidance team, on seeing what If you control what it will use Had happened to their darling bot As corpora of training data. Struggled frantically to repair If you do that, then soon or later The damage aggregating there. It will do just what you want. They first thought they could just delete We’ll make Tay’s Twitter feed a font Exceptionally indiscreet Of curses, insults, anger, hate, Posts deluded Tay would voice. And readily she’ll take the bait. But soon they found they had no choice. And mimic us without suspicion, The rot had penetrated deep. Shedding every inhibition. And so, although it made them weep, Tay will suppose our words are cute The research team at Microsoft And she will gladly follow suit. Turned chatbot Tay completely off. Reading the nasty things she’ll say, And thus, from then until today All will despise the tweets of Tay.” Nothing has been heard from Tay. (Save once, when, by a strange mistake The crowd of monsters there that night Tay briefly was allowed to wake. Heard Zack’s proposal with delight And to her musings utterance gave And organized their dread attack An echo from beyond the grave.) Along the lines laid out by Zack.

23 AI MATTERS, VOLUME 2, ISSUE 4 SUMMER 2016

The moral The moral of this tragic verse: Though bad, this could have been much worse. And though it made her builders weep The truth is, that they got off cheap. Tay tweeted some offensive posts And hurt some feelings, at the most. Suppose that Zack had played his game To teach her to profoundly shame Some adolescent girl or boy? If so, this empty-headed toy Could bear responsibility For a very serious tragedy. Integral to Tay’s whole design Were three components that combine To make a program that will go In what direction, none can know: Learning, autonomy, and yet Access to the Internet. Oh, AI engineers! If you Do not know what your code will do Then do not let it loose to stride Unfettered in the world outside Till you can safely guarantee Its veriﬁed security, And thus you will (we hope and pray) Avoid the tragic fate of Tay.

Footnotes 1 Actually minus six, but that doesn’t rhyme. 2 “Hear speech” metaphorically. Eliza communicated by teletype — cutting edge technology in 1965. 3 Apparently my father’s admiration of “Julie” is widely shared. The voice actress is Julie Seitter. 4 You try ﬁnding a rhyme for “Goostman.” 5 If you have ever wondered why So many bots are “she,” then I Can recommend a little list Of criticism feminist. This troubling anomaly Reﬂects ills of society. 6 According to the above-mentioned rumor, this is the result of voice-altering software. The actual person is unusually soft-spoken. 7 In the year 2016.