Google shows how DeepMind learns Quake III Arena

DeepMind, Google’s sister company that works on artificial intelligence, has published a paper detailing how it has trained bots to learn how to play Quake III Arena on their own and without any explanation.

DeepMind has created bots, agents the company calls them, that have learned how to Capture the Flag to play, one of the game modes in Quake III Arena, the shooter released by id Software in 1999. The agents do not receive an explanation of the rules in advance, but now perform so well that they can compete with good human players.

DeepMind also wanted to make things even more difficult for the bots and developed a tool for the research that can generate random maps for Capture the Flag, so that the self-learning agents cannot rely on map knowledge. In addition, the agents were made to only respond to visual information, not to other game data. So they can’t see through walls like some bots created by game developers. The makers wanted the bots to understand the rules of the game. In addition, they wanted the bots to learn how to both work together in a group and fight opponents. That worked out wonderfully. The agents even developed recognized humane strategies, such as camping in the enemy base, defending one’s own base, and escorting the flag bearer.

Although the agents were not given an explanation of the rules beforehand, they quickly learned to understand the concept of the game format. For example, DeepMind employees found neurons in the code that were activated when their own flag was stolen or when an agent’s team member held the enemy’s flag. Not only did the agents perform well in the game, their contribution was also highly appreciated by human players. In an evaluation after a number of mixed games, in which agents and human opponents played together, the contribution of the agents was rated higher, because they would work better together. DeepMind has previously researched bots learning to play StarCraft II.