Bleeding Edge AI: How A DeepMind Just Beat Team Liquid At StarCraft II
Table Of Contents
What are we talking about?
StarCraft II is a real-time strategy game by Blizzard Entertainment, which has become a
staple of the eSports community. Professional play has been ongoing since the game’s initial release in 2010, following on the success of StartCraft and StarCraft: Brood Wars, both of which had substantial eSports fan bases.
StarCraft II is a computer game which has been played competitively for ~ 20 years. Major leagues in this include the Korean eSports Association (KeSPA), Intel Extreme Masters (IEM) World Championships and BlizzCon, with professionals making 250K+ on prize money alone. Find the
global rankings here.
DeepMind's latest/greatest AI AlphaStar, just
beat a top ranked StarCraft II player. This post is about what happened, what it means and how to think about AI sensibly. AlphaStar is the
evolution of other Alpha- projects,
AlphaZero for Chess and
AlphaGo for Go.
StarCraft II - Basic Rules for Humans and Machines
The StarCraft franchise has a large/complex storyline (
lore in gamer-speak) which sets up the games, but that isn’t terribly important. Here is what
you need to know:
There are three main factions, Protoss (
advanced alien), Terran (
future human) and Zerg (
bugs ala Starship Troopers), each with unique units. The way to think of this from a game theory perspective is as a complex version of
The play is generally between 2 players and takes places on tournament maps. Tournament maps are generally modified to include
multi level terrainwith
high ground advantage,
strategic choke points,
terrain modifiers which impact unit speed, range and hit pointsamong others. A standard complexity 1v1 map is shown below.
There are two basic resources,
gas. Some maps also have a
special mineralswhich translates to quicker unit production. The other major issue is
fog of warwhich translates to incomplete knowledge of the map, necessitating
cheesingand other assorted
The players actions are basically,
build mining units to get resources,
use resources to make weapon units, and
kill other players weapon units while surviving yourself. As such, the
build order(the order in which you build units and structures to maximise chances of winning)
becomes super important, not least because the times for specific strategies are well known, leading to
StarCraft II - Micro and Macro Game
Micro refers to the ability of the player to perfectly control their units. Units in StarCraft II often have
abilities. A simple example is something like a
MedEvac unit allows other units to load up, and also heals. Being
proficient in micro allows a player to perform betterin ground level engagements.
Macro refers to the ability of the player to match their opponent in building bases and drones, the
idea being to successfully min/max unit production.
Another consideration around macro refers to
strategy transitions. Each race whether Protoss, Terran or Zerg has a technology tree which
allows players to make unit composition choices. An example of this in Terran, is the choice between
bio - which refers to units like marines and snipers(Ghost) or
mech - which refers to tanks etc.
To understand the differences, please have a look at this guide. This is important, because to think about the relative strengths of the AI,
the real test is AlphaStar's ability to balance Micro and Macrogame.
Visually in the below example,
red goes hard on unit production early attempting to win with good micro, but blue focuses on macro game, and wins. Note, the graph is of Army Value (think unit count hence micro focus).
AlphaStar - Abilities + Limitations
Well described in the original blog post, are the challenges that the DeepMind team faced when training their AI. These include:
game theory - no single best strategy,
long term planning,
large action space. Also included in that post is the basic process of how the training worked. What you need to know:
The AI was trained using an advanced generative adversarial network (GAN) AI. This is a fancy way of saying they took
multiple AI's (agents) and made them play against each other. So the final AI has the equivalent of
200 years of StarCraft II gaming experience.
The AI was trained on (and
can only play) Protoss vs Protoss - that is
one faction against itselfonly, and on
The AI was limited to
control the unitsin a way which emulates human limitations. So the AI was limited to
under 300 APM (actions per minute). This is on the
low end compared to professional players, the
world champion Serral averages ~450 APM.
Additionally, the AI is also limited
see the maplike a human, it cannot just flick around and see everything, it is limited using a
screens per minuteto simulate human reaction times. It must virtually
look at different placeson the map. The
primary role of AlphaStaris to look around,
prioritize its actionsand
engage in unit controlin engagements.
What Happened - Cheeses + Mind Games
AlphaStar beat MaNa 5-0. Ok - so this was not totally unexpected. It should also be noted, that
the human players were NOT playing their preferred factionin StarCraft II, because
AlphaStar currently can only play Protoss v Protoss, on Catalyst LE. It can’t do anything else.
Cheese most often refers to an unexpected strategythat relies in large parts on lack of information and/or psychological impact on the opponent. Cheese build orders typically revolve around an
early attack that, if undetected, is more difficult to defend than execute.
Have a look at this to find examples of more
cheeses. Cheesing is most effective when
mixed with mind games, most of which reflect imperfect knowledge about what your opponent is doing, relying on
fog of war.
In this clip,
(i) TLO sends over a probe to scout, (ii) AlphaStar sees the probe detect a stargate, (iii) waits until the probe no longer has visibility, and (iv) cancels the stargate.
behavior is very human like, but why it is doing some of these things is not entirely clear. For example, seen here something interesting happens.
The AI overproduces probeson the first base (that is,
it saturates the base and hence the base efficiency is lower) where as a human would not do this and
you would think that a machine would also not do this, seeing as the
AI is likely to understand going over 100% reduces efficiency- but AlphaStar does any way.
- On watching the stream, and you should totally check it out, I was
impressed, intrigued and a little bit terrified. Which seems like an appropriate response.
- AlphaStar seemed to have a
lot of trouble with macro transitions. So a fairly textbook strategy for Terran would be, (i)
start with bio - marines/marauders, (ii)
transition to mech by mid gameand, (iii) then
move to air units for late game. AlphaStar cannot (from my understanding of its training regime) respond to this kind of play. AlphaStar tends to start with a composition, and then make minor changes. This makes sense when you think about how the neural network was trained.
- The main place where
AlphaStar absolutely blew me away, was its micro control. So there were large parts of the games where the
Actions Per Minute (APM) for AlphaStar was in the 60's, whereas its human opponent was at 300+. But looking at the play, the micro was amazing. AlphaStar is
obviously a computer, so you would expect it to NOT make mistakes clicking around, but it is still very impressive.
- Will AI’s like
AlphaStar dominate StarCraft II going forward - absolutely not. StarCraft maps, to provide just one example, are evolving towards allowing players to using deeper macro strategy - such as, where should my 3rd base be is less of a tactical decision and more of a strategic decision going forward, which makes it immeasurably harder for an AI to win.
the longer term implication is pretty clear, DeepMind took an AI and is teaching it to mimic a gestalt consciousness (Zerg is an exo-galactic hive mind in StarCraft II lore). This might not end well :)