Diplomacy is a classic military strategy board game, famous for taking ages to play and ruining friendships. Unlike other strategic military board games, Diplomacy is quite light on rules; the meat of the game is in talking to the other players, making alliances and breaking promises. This makes it a poor fit for Alpha*-style self-play approaches; even if you make a very skilled agent that beats itself at diplomacy, if it doesn’t speak a human language or act in line with human norms, humans won’t cooperate with it and it will lose the game every time.
In 2022, the FAIR team released Cicero, an AI that convincingly plays Diplomacy at a human level. Cicero won a 40-game online blitz league, and none of the humans it played against realized it was a bot. This breakthrough, an AI that can do human-level strategic reasoning, was scary to many! Surely the first country to hook Cicero up to a drone swarm would dominate the earth?
This week we’ll study the Cicero paper, deciphering the technical details, cutting through the hype. What are the real safety risks from this kind of research?
The game Diplomacy has been a major challenge for artificial intelligence (AI). Unlike other competitive games that AI has recently mastered, such as chess, Go, and poker, Diplomacy cannot be solved purely through self-play; it requires the development of an agent to understand other players’ motivations and perspectives and to use natural language to negotiate complex shared plans. The Meta Fundamental AI Research Diplomacy Team (FAIR) et al. developed an agent that is able to play the full natural language form of the game and demonstrates performance well above the human average in an online Diplomacy league. The present work has far-reaching implications for the development of cooperative AI and language models for communication with people, even when interactions involve a mixture of aligned and competing interests.