If you didn’t catch it, last Thursday January 24, 2019 DeepMind’s new AI, AlphaStar beat a Team Liquid player Grzegorz Komincz, also known as MaNa in a professional game of StarCraft II.

What are we talking about? StarCraft II is a real-time strategy game by Blizzard Entertainment, which has become a staple of the eSports community. Professional play has been ongoing since the game’s initial release in 2010, following on the success of StartCraft and StarCraft: Brood Wars, both of which had substantial eSports fan bases.

That is, StarCraft II is a computer game which has been played competitively for ~ 20 years. Major leagues in this include the Korean eSports Association (KeSPA), Intel Extreme Masters (IEM) World Championships and BlizzCon, with professionals making 250K+ on prize money alone. Find the global rankings here.

DeepMind's latest/greatest AI AlphaStar, just beat a top ranked StarCraft II player. This post is about what happened, what it means and how to think about AI sensibly. AlphaStar is the evolution of other Alpha- projects,AlphaZero for Chess and AlphaGo for Go.

StarCraft II - Basic Rules for Humans and Machines

The StarCraft franchise has a large/complex storyline (lore in gamer-speak) which sets up the games, but that isn’t terribly important. Here is what you need to know:

• There are three main factions, Protoss (advanced alien), Terran (future human) and Zerg (bugs ala Starship Troopers), each with unique units. The way to think of this from a game theory perspective is as a complex version of Rock-Paper-Scissors.
  
• The play is generally between 2 players and takes places on tournament maps. Tournament maps are generally modified to include multi level terrain with high ground advantage, strategic choke points, terrain modifiers which impact unit speed, range and hit points among others. A standard complexity 1v1 map is shown below.
  
• There are two basic resources, minerals and gas. Some maps also have a special minerals which translates to quicker unit production. The other major issue is fog of war which translates to incomplete knowledge of the map, necessitating scouting, cheesing and other assorted mind-games.
• The players actions are basically, build mining units to get resources, use resources to make weapon units, and kill other players weapon units while surviving yourself. As such, the build order (the order in which you build units and structures to maximise chances of winning) becomes super important, not least because the times for specific strategies are well known, leading to timing attacks.

StarCraft II - Micro and Macro Game

• Micro refers to the ability of the player to perfectly control their units. Units in StarCraft II often have abilities. A simple example is something like a MedEvac unit allows other units to load up, and also heals. Being proficient in micro allows a player to perform better in ground level engagements.
• Macro refers to the ability of the player to match their opponent in building bases and drones, the idea being to successfully min/max unit production.
• Another consideration around macro refers to strategy transitions. Each race whether Protoss, Terran or Zerg has a technology tree which allows players to make unit composition choices. An example of this in Terran, is the choice between bio - which refers to units like marines and snipers (Ghost) or mech - which refers to tanks etc.
• To understand the differences, please have a look at this guide. This is important, because to think about the relative strengths of the AI, the real test is AlphaStar's ability to balance Micro and Macro game.
• Visually in the below example, red goes hard on unit production early attempting to win with good micro, but blue focuses on macro game, and wins. Note, the graph is of Army Value (think unit count hence micro focus).
  

AlphaStar - Abilities + Limitations

Well described in the original blog post, are the challenges that the DeepMind team faced when training their AI. These include: game theory - no single best strategy, imperfect information, long term planning, real-time and large action space. Also included in that post is the basic process of how the training worked. What you need to know:

• The AI was trained using an advanced generative adversarial network (GAN) AI. This is a fancy way of saying they took multiple AI's (agents) and made them play against each other. So the final AI has the equivalent of 200 years of StarCraft II gaming experience.
• The AI was trained on (and can only play) Protoss vs Protoss - that is one faction against itself only, and on Catalyst shown below.
• 
• The AI was limited to control the units in a way which emulates human limitations. So the AI was limited to under 300 APM (actions per minute). This is on the low end compared to professional players, the world champion Serral averages ~450 APM.
• Additionally, the AI is also limited see the map like a human, it cannot just flick around and see everything, it is limited using a screens per minute to simulate human reaction times. It must virtually click around to trigger actions and look at different places on the map. The primary role of AlphaStar is to look around, prioritize its actions and engage in unit control in engagements.

What Happened - Cheeses + Mind Games

AlphaStar beat MaNa 5-0. Ok - so this was not totally unexpected. It should also be noted, that the human players were NOT playing their preferred faction in StarCraft II, because AlphaStar currently can only play Protoss v Protoss, on Catalyst LE. It can’t do anything else.

Cheese most often refers to an unexpected strategy that relies in large parts on lack of information and/or psychological impact on the opponent. Cheese build orders typically revolve around an early attack that, if undetected, is more difficult to defend than execute.

Have a look at this to find examples of more cheeses. Cheesing is most effective when mixed with mind games, most of which reflect imperfect knowledge about what your opponent is doing, relying on fog of war.

In this clip, (i) TLO sends over a probe to scout, (ii) AlphaStar sees the probe detect a stargate, (iii) waits until the probe no longer has visibility, and (iv) cancels the stargate.

This behavior is very human like, but why it is doing some of these things is not entirely clear. For example, seen here something interesting happens. The AI overproduces probes on the first base (that is, it saturates the base and hence the base efficiency is lower) where as a human would not do this and you would think that a machine would also not do this, seeing as the AI is likely to understand going over 100% reduces efficiency - but AlphaStar does any way.

Conclusion

• On watching the stream, and you should totally check it out, I was impressed, intrigued and a little bit terrified. Which seems like an appropriate response.
• AlphaStar seemed to have a lot of trouble with macro transitions. So a fairly textbook strategy for Terran would be, (i) start with bio - marines/marauders, (ii) transition to mech by mid game and, (iii) then move to air units for late game. AlphaStar cannot (from my understanding of its training regime) respond to this kind of play. AlphaStar tends to start with a composition, and then make minor changes. This makes sense when you think about how the neural network was trained.
• The main place where AlphaStar absolutely blew me away, was its micro control. So there were large parts of the games where the Actions Per Minute (APM) for AlphaStar was in the 60's, whereas its human opponent was at 300+. But looking at the play, the micro was amazing. AlphaStar is obviously a computer, so you would expect it to NOT make mistakes clicking around, but it is still very impressive.
• Will AI’s like AlphaStar dominate StarCraft II going forward - absolutely not. StarCraft maps, to provide just one example, are evolving towards allowing players to using deeper macro strategy - such as, where should my 3rd base be is less of a tactical decision and more of a strategic decision going forward, which makes it immeasurably harder for an AI to win.

However, the longer term implication is pretty clear, DeepMind took an AI and is teaching it to mimic a gestalt consciousness (Zerg is an exo-galactic hive mind in StarCraft II lore). This might not end well 😃

Extra

You should totally check out the LowkoTV and WinterStarcraft channels if this was interesting. They are both good guys and would appreciate the custom.

Please connect with us on LinkedIn, YouTube or Facebook for any comments, questions, or just to keep up to date with the work we do!