Harmon on BPM Paul Harmon November 5 2019 Google's

Harmon on BPM Paul Harmon November 5 2019

Google’s DeepMind and StarCraft 2

We’ve all gotten used to stories about how Artificial Intelligence (AI) is doing exciting things. Process practitioners at leading companies are increasingly engaged in revising or creating new processes to accommodate processes that include AI software applications. Hopefully BPTrends readers understand that AI is more complex and at the same time simpler than all the hype. AI is just a term for a large collection of computer techniques that can be used to get computers to do new things.

The underlying set of AI techniques on which most attention is focused at the moment are neural networks that can learn new patterns by practicing. Obviously there are lots of limits on this approach. Someone, for example, has to work with the computer application, telling it whether specific instances are an example of a correct or incorrect response. Even relatively simple learning tasks require thousands of training rounds before the computer system begins to perform as desired.

There is no question about AI techniques being used – its just another way of saying we want our computers to perform better. There are already hundreds of applications that have incorporated a bit of AI in their programming, and more are being released everyday. For business people, the important thing is to observe what can be done, what specific problems can be better solved using specific techniques, and to then decide if your organization has a similar problem that you need to solve.

Many companies developing AI applications are reluctant to talk about the details of their use of AI. They hope the new technology will give them a temporary advantage, and so they are reluctant to share information about what they did or how they did it. This reluctance had led market watchers to pay particular attention to “toy” applications, which are often cutting edge demonstrations of AI use.

One thinks of IBM’s use of Watsons/Deep Blue to defeat the world chess champion, Garry Kasparov, in 1997, of Watson’s specular defeat of three Jeopardy! Champions on TV in 2010. Similarly, there was the use of Google’s AlphaGo application to defeat international Go grandmasters in 2015-16. The latest in this tradition is this year’s success of DeepMind’s AlphaStar in online StarCraft II play.[1]

StarCraft II is a real-time video strategy game developed by Blizzard Entertainment. One player plays against another online. The game allows simultaneous play, which

BPTrends – March 2018 RPA is to say that players do not need to wait for another player to move, but can make moves without regard to what their opponent is doing. [2] No user can see the complete “map” of the game at any one time, and thus, unlike chess or Go, each player is playing with incomplete information about what other players might be doing.

The original version, StarCraft was released in 1998, and StarCraft was released in 2010. Extensions have been released since then with the latest, Nova Covert Ops released in March of 2016. The game involves an ongoing war between Terrans (humans), the Zerg (a super human life form) and the Protoss (a technological advanced species.). The game is extremely popular. In the first month after its release, StarCraft II was acquired by over 3 million users, worldwide. Each release is, in effect, a different campaign with different goals.

One way to think of the complexity involved is to consider the options one faces when one makes a move. In chess, the player making the first move can move pawns or knights and has 20 initial possible moves. His opponent faces the same 20 options, on the opposite site of the board. In Go, which is played on a board with 19 by 19 lines, creating 361 intersections, the initial player (facing an equally rated master player) can choose to place his “stone” on any one of the 261 intersection points. His opponent can choose to place a stone on any one of the 260 remaining points. In both games, as the games progress, options rise or fall. In StarCraft II there are 1026 possible moves, every move. This, coupled with imperfect information about your opponent’s moves generates a game of infinitely more complexity.

After DeepMind’s AlphaGo program beat the world grand master, the team began to consider other possibilities, and quickly settled on StarCraft II as their next challenge. AlphaStar was developed from scratch, using the same basic technologies employed in AlphaGo. A set of neural networks were created and trained for StarCraft II play. The DeepMind team created three players (agents), one to play as a human, one to play as a Zerg and one to play as a Protoss. Each player began by supervised play. In effect the player played against a human and was told, as it played, whether each play was a good or bad play. Later it evolved and was reinforced simply for winning or losing the game. After a period of supervised play, individual players were duplicated and set to play each other. Two copies of the Terrian player, playing at top speed could complete thousands of games in a day and learn as much as a human player would learn with years of practice.

One of the unique techniques used in training AlphaStar involved the development of specialized players that were designed by the team to be especially good at specific problems. These players, called “exploiters” because they were designed to exploit specific weaknesses, each played AlphaStar core agents until the core agent could consistently beat them, in effect, eliminating a specific weakness.

In the end, AlphaStar mastered the game after 44 days of training, and achieved the games highest rank (Grandmaster League). It can currently outperform 99.8% of all players, and it will continue to improve as it plays other master players and itself, thousands of additional times.

In January DeepMind announced that AlphaStar was available for play and several human Grandmaster players attempted to play against it, giving AlphaStar more practice. By June DeepMind opened the program to invites and rapidly became

BPTrends – March 2018 RPA among the most sophisticated players. Data suggests that today there are roughly 0.2% of players, worldwide, who are capable of defeating AlphaStar. Naturally the program will keep playing and getting better and it is assumed that in a short period it will be able to defeat any human player.

To sum up. Game playing AI applications often point to new trends in AI software applications. Previous game playing applications have demonstrated the ability to search vast online databases very rapidly, and to interface with users using natural languages (Jeopardy!), playing human experts where complex logical patterns are involved (Go), and bluffing (poker). All have emphasized the use of neural networks and algorithms that stressed reinforced learning as ways of quickly training the systems to high levels of competence. DeepMind’s latest achievement, with AlphaStar, shows AI systems, using today’s now standard techniques, can be rapidly developed (44 days!) to handle incredibly complex games involving real time simultaneous play. This is the kind of capability one will need to develop robots who can learn and systems that can drive cars. [3]

As we have always emphasized, the challenge for process professionals remains unchanged. They don’t have to understand the internals of the AI application – it’s ultimately simply another software application that will automate another business process. Process professionals need to figure out how the program will fit with other business processes and how to explain it to fellow workers and customers. What AlphaStar does suggest, however, is that new increasingly intelligent software applications are going to keep coming and the need to change key business processes to keep up with the competition is going to be ever more challenging. The future keeps coming faster all the time.

----- DeepMind was started by a group of UK AI gurus in 2010. In 2014 DeepMind was acquired by Google. The core group at DeepMind, located in London, England, remains focused on research and development, but as specific groups develop applications of commercial, they apparently are shifted to Google divisions. Thus, DeepMind’s health care project team was recently moved from DeepMind to Google for commercialization.

1. DeepMind has submitted an article providing detailed technical information on the development of AlphaStar to Nature magazine and that article will be published in the near future, providing AI researchers with more information about AlphaStar’s development.

2. Because of the simultaneous play, to keep the game fair, DeepMind has limited the speed at which AlphaStar can play to 22 non-duplicated actions every five second – the speed at which top human players typically move.

3. DeepMind has publically announced that it will never undertake military work. It points out that Starcraft II is not a realistic war simulation. They emphasize that AlphaStar relies on extensive data on games previously played and training with skilled players and that neither are available for real conflicts, as in Syria or Yemen, for example. Never-the-less, the US government has determined that war game simulation is of great potential and AlphaStar’s approach to strategy development will undoubtedly be studied by military gamers.

BPTrends – March 2018 RPA

Author

. Bio

. Latest Posts

Paul Harmon

Executive Editor and Founder, Business Process Trends In addition to his role as Executive Editor and Founder of Business Process Trends, Paul Harmon is Chief Consultant and Founder of BPTrends Associates, a professional services company providing educational and consulting services to managers interested in understanding and implementing business process change. Paul is a noted consultant, author and analyst concerned with applying new technologies to real-world business problems. He is the author of Business Process Change: A Manager's Guide to Improving, Redesigning, and Automating Processes (2003). He has previously co-authored Developing E-business Systems and Architectures (2001), Understanding UML (1998), and Intelligent Software Systems Development (1993). Mr. Harmon has served as a senior consultant and head of Cutter Consortium's Distributed Architecture practice. Between 1985 and 2000 Mr. Harmon wrote Cutter newsletters, including Expert Systems Strategies, CASE Strategies, and Component Development Strategies. Paul has worked on major process redesign projects with Bank of America, Wells Fargo, Security Pacific, Prudential, and Citibank, among others. He is a member of ISPI and a Certified Performance Technologist. Paul is a widely respected keynote speaker and has developed and delivered workshops and seminars on a wide variety of topics to conferences and major corporations through out the world. Paul lives in Las Vegas. Paul can be reached at [email protected] FacebookTwitter You’re Invited! Join the BPTrends Community