Winning in Starcraft 2 an Analysis of Skill Using SPL and MLTK

Winning in StarCraft 2 An Analysis of Skill Using SPL and MLTK

During the course of this presentation, we may make forward‐looking statements Forward- regarding future events or plans of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us Looking and that actual events or results may differ materially. The forward-looking statements made in the this presentation are being made as of the time and date of its live Statements presentation. If reviewed after its live presentation, it may not contain current or accurate information. We do not assume any obligation to update any forward‐looking statements made herein.

In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionalities described or to include any such feature or functionality in a future release.

Splunk, Splunk>, Turn Data Into Doing, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2019 Splunk Inc. All rights reserved. Use this if there will be two speakers for your session.

Shawn Routhier I play Protoss

Blizzard Entertainment for 4 years • Sr. Security Engineer

Defcon25 Black Badge (Uber) Winner – Telephreak

NPC In World of Warcraft • “Shawn” in Nagrand (Outland)

Security Guy for 10+ years… and counting… • Booz Allen Hamilton, MIT – Lincoln Laboratory, Blizzard Entertainment

B.S. in Network Security (2009)

Splunk Trust from 2017 to present

Splunking since v4.1 (2011) • MIT – Lincoln Laboratory, Aplura, Dell, GitHub

Zerg since 1999 • Highest League - Master

B.S. Computer Science – Wentworth Institute of Technology

What is StarCraft?

StarCraft 2 • Wings of Liberty (2010), Heart of the Swarm (2013), Legacy of the Void (2015)

Real-Time Strategy (RTS) • Player input is read and actioned immediately (real-time) • Micro: Ability to control units effectively to reduce waste • Macro: Ability to gather resources and maintain production of units, buildings, and upgrades

Three opposing races: • Terran: Future human descendants from a resource-depleted Earth • Protoss: Sapient humanoid species of advanced technology and psionic ability • Zerg: Xenomorphic superorganisms assimilating other races genetics to their own © 2019 SPLUNK INC. © 2019 SPLUNK INC.

How Do You Play? Explained in 20 seconds or less! Objective • Kill the opponent’s units before they kill yours!

How StarCraft Ranks Players When I’m Grand Master, I will play faster…

Player Ranking System – “Leagues” • Progress within a League by winning • Leagues progress from Bronze to Grandmaster

Performance measured by Match Making Rating (MMR) • MMR’s goal is to match ranked players for a 50% win ratio

Data Ingest

Turning Binary Data Into A Splunk Parsable Structure Ingesting a Binary File It’s as easy as “NO_BINARY_CHECK = True”! .SC2REPLAY

• Proprietary binary format

How do you Splunk a binary file?

Extracting Data From .SC2REPLAY Not really as easy as “NO_BINARY_CHECK = True”… Sorry

We stand on the shoulders of the community.

Blizzard Supported API for communicating with the game client: https://github.com/Blizzard/s2client-proto • Nice, but we don’t need the full client

Blizzard Supported SDK for the s2protocol: https://github.com/Blizzard/s2protocol • This does a ton of the legwork for parsing the binary data structures.

GGGTracker: https://github.com/ccbrown/gggtracker • Community supported replay parsing site: https://gggtracker.com/ • Lots of great examples on how to use s2protocol && MIT Licensed! © 2019 SPLUNK INC.

Wrote Some Code Bridging gaps and exporting in JSON

~1300 lines of python • Again, standing on the shoulders of giants • Incredible wealth of data within the replay files

Compressed (.SC2Replay) == ~50KB

After “uncompressing” == ~60MB © 2019 SPLUNK INC. Some Code & Data Challenges …Each causing a refactor of code and rebuild of our dataset Recursion is fun … Until it’s infinite • When a Unit data type is found, the get_unit function is called • This works great, until a Unit is killed by a Unit…. And so on.

Complex data structure • Feature creep • We kept finding interesting pieces of data to parse out of the replay.

Player ID weirdness • An “event” can be owned by somebody other than the player it references. • Comes into play during matches with observers

How Did We Collect Game Replays Hint: the community

Tournament replay packs released by Blizzard and curated by the community – https://lotv.spawningtool.com/replaypacks/

Challenges With Sample Size // Bias Samples show bias against slow drivers in the left lane

Original sample included several 2v2, 3v3, 4v4 matches • Identified and “| delete” those from Splunk

Difficulty finding bulk sources of replays for anything except pro matches

Difficulty finding Korean Professional replays • Local Tournaments are not usually publicly shared

Attributes

Usable Metrics We Pulled From The Replays © 2019 SPLUNK INC. Key Datapoints (Observations) Observers cost 25 minerals & 75 gas Player Stats • Actuations Per Minute (APM), Effective Actions Per Minute (EAPM) • Macro actions, Micro actions • Camera actions*, Control Groups • League Rank

Base Metrics • Worker Saturation, Gas Timing, Idle Workers, Supply/Resource Blockage & Resolution • Expansion Rate

Match Progress • Resource Accrual & Expenditure, Military Value, Units & Structures Killed/Lost

Two key event_type: • ReplayMeta (overall match) • PlayerStatsEvent (frame by frame stats)

Resource current/used/killed

Food made/used; “Supply”

Testing Attributes | analyzefields is OP; needs a nerf next patch

Tests numerical fields against “classfield”

Co-occurrence • The ratio of results with field present

Scoring The Model Seed=1337 was essential to our success

Ensure a consistent testing methodology • | sample with a consistent seed

| multireport provided by MLTK GUI • “Open in Search”

Scoring measurements © 2019 SPLUNK INC. We Needed to Scale Our SPL Updates took as long as a Bronze vs Bronze match Our SPL was getting out of hand • Most lines in one search was 624 • Took 20+ minutes to modify when we needed new data

We created a few macros! • Using “| appendcols” to include new features to the dataset © 2019 SPLUNK INC. pstats_delta_minute($field$) More macro than in a Bronze match YOU MUST CONSTRUCT ADDITIONAL PYLONS! © 2019 SPLUNK INC. Supply Block

Everyone gets supply blocked at 14/15 supply

0:00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00

SPL Sidebar Data normalization is extremely important! SC2 engine runs at 16 fps (tick-rate) • PlayerStatsEvent is summarized every 10 seconds

Current Army Value Total Minerals & Vespene cost of your army hivemind

Calculated Effective Micro Yin Micro: Ability to control units effectively to reduce waste

Calculated Effective Macro Yang Macro: Ability to gather resources and maintain production of units, buildings, and upgrades

• SC2 maps are played in an x,y coordinate plane • Distance was calculated with streamstats & eval:

Actuations Per Minute (APM) Gotta go fast!

Highest correlation to Winrate • | analyzefields @ 0.642

APM can be inflated

“It is rarely a mysterious technique that drives us to the top, but rather a profound mastery of what may well be a basic skill set.”

Predicting A Winner

Our Findings Backed by DATA!

A Skilled Player Will: •Resolve supply blocks quickly •Tend to both economy and army •Utilize the map •Know when to commit to a fight •Naturally have a higher APM

Predicting A Winner As Early As Possible We can kind of see the future!

Fun Slides

Q&A

Thank You

Go to the .conf19 mobile app to ! RATE THIS SESSION