Dev Diary #39 - AI Players 🤖
What's happening / TLDR: Developer diaries introduce details of Espiocracy - Cold War strategy game in which you play as an intelligence agency. You can catch up with the most important dev diary (The Vision) and find out more on Steam page.
---
Dev diaries of Espiocracy often explore the nature of (good) strategy games. The yearning for crafting own agency during gameplay, for the rocky marriage of immersion and strategic decisions, for interesting emergent simulation. It's time to wrestle another hallmark of the genre: challenges generated by other artificial players.
It's not a feature of all strategy games - colony managers and city builders pose the player mainly against the environment. Espiocracy harbors rich dose of such environmental challenges, as should be the case for the Cold War and beyond, to the point of implementing a kind of AI for the most important cogs (actors). However, the main challenge in this game stems from vigorous competition between players: hunting spies, stealing assets, ambushing operations, or even turning operatives against a player. This is further complicated by a game of information with covers, traps, bluffs, and the entire geopolitical poker. Artificial intelligence for THAT surely requires special approach.
Indeed, it does.
[h2]The Player[/h2]
To allow precise discussion, let's begin with a quick recap on the player persona. In Espiocracy, you play as an intelligence community (a set of intelligence agencies) of a single nation.
As the first dev diary proposed, a player here is the most resilient entity of the period, right in the fascinating middle between playing as an entire nation and playing as a small group of characters. For the player ecology, it means that the game harbors secondary (to international) layer of diplomacy: non-kinetic interactions between intelligence communities. Following DD#8 (to be refreshed and expanded in another dev diary), these include communication, alliances, deals, or partially controlling another player. This is further exacerbated by the fact that every player can try to get a foothold on the ground held by another player, in what can be described as a state of constant war. With this unusually competitive landscape in mind, let's jump straight into AI of such players.
[h2]Climbing the Shoulders of Giants[/h2]
Implementation of AI players in Espiocracy follows the principles of chess programming.
For 70 years, chess programmers have been writing engines that can competitively play chess, a game with tree complexity reaching 10 with 123 zeros. They have been working on outsmarting chess grandmasters for the entire Cold War and succeeded just a few years after its end, which is yet another topical and historical parallel that permeates development of Espiocracy.
This niche field generously offers many clever solutions that crave to be implemented in strategy games. Naturally, there are many reasons why they have not been (largely) included in games up to this point. First and foremost, they require counterintuitive approach to development. Instead of writing AI five minutes before midnight when a game is complete, Espiocracy features AI since early builds. In a nuanced way, parts of the code have to be explicitly designed to allow the development of appropriate AI - similarly to architecture requirements posed by multiplayer. Sometimes, even player-facing mechanics have to be (re)designed for AI. As a testament to this design lens, in what began as a hermetic joke and ended up as actually useful implementation, the player is currently served with interfaces corresponding to the dynamics of a chess game:
What principles, precisely?
This framework sets up the stage for developing AI players that utilize decades of lessons in chess programming. In the most universal approach, AI regularly evaluates positions and finds a combination of moves that will improve the position by the highest value.
There are many exciting solutions that flesh out the details (piece-square tables, alpha-beta search, even a counterpart of opening books). For a nice and easily understandable demonstration of the ingenuity of chess programmers, take a look at mobility. Mobility is the number of available legal moves. In chess, statisticians found a definite correlation between higher mobility and victories - better players usually create more opportunities for themselves. This point of view makes a lot of sense in Espiocracy which, much like immensely complex chess, can be meaningfully reduced to mobility (a single number!), or as we know it here, player agency. With a rather simple algorithm, AI players can find a series of moves that increase mobility, which already leads to pretty sophisticated gameplay such as accepting small losses for larger long-term gains, expanding in preparation for invasive actions, or eliminating enemies that may limit future expansion. This is only a single fascinating solution among hundreds available. And it's not the end of innovation in this department...
[h2]Open AI[/h2]
The gravity pull of chess programming, the one which worked for 70 years and still draws people to this day, may be slightly difficult to understand from the outside. If I would have to explain it, I would compare it to a boxing championship for programmers. Thankfully, I don't have to, because the modding scene of strategy games speaks for itself - given the ability to mod, almost every strategy game receives AI mods, some of which are developed with immense dedication for years. Their authors develop them often despite heavy limitations or technical hostility of the game to modifications in AI behavior.
In the spirit of chess programming, Espiocracy will be fully open to multiple external AIs via dedicated API in C#.
The game will be released with a few AIs implemented by yours truly, using chess programming lessons to the best of my ability. However, that ability is severely limited not only by my brain and time but also perspective. Critically and perhaps controversially, I think that too many strategy games suffer from (relatively) poor AI because developers cannot unsee the implementation of mechanics. As laid out in the introduction, solid AI is too important for this game to leave it solely in the hands of the developer(s). By developing both a fish and a fishing rod, I have a sincere hope that one day you could play Espiocracy with easily developed AI mods authored by people much smarter than me.
To allow a truly competitive nature of AI development, the game is already set up to allow multiple different AIs assigned to different artificial players. In a dream-come-true difficulty settings, instead of deciding between degrees of handicap, you can now choose to play against artificial players who can be actually more stupid or more intelligent.
As you can probably guess at this stage in the hereby honeypot for programmers disguised as a dev diary, AI configuration is used not only for difficulty management or for recording timelapses of USA with AI_X vs USSR with AI_Y. In parallel, I'm working on tools that would measure and compare AIs against each other, have a counterpart of strategic test suite, and maybe (I didn't go that far yet) even a multipolar analogue of Elo rating.
Last but not least, I see this openness as an investment into the unknown future of new tools, about which we hear in recent months. By implementing open API, I'm leaving the door open for the day when our computers will have enough power to run deep learning algorithms with strategic reasoning skills.
[h2]Behind The Scenes[/h2]
â–º For the folks curious about the last remark, we may be surprisingly close. You can already run a 25x smaller equivalent of ChatGPT with a speed of ~2s per sentence on an 8-core CPU. It's large enough to exhibit pretty complex reasoning and planning. Naturally, it's still too slow to power ~100 players in parallel, but once the mechanics are more complete, I can honestly see an attempt at managing an American/Soviet player with its faster iteration (which would probably require most engineering around translation of the quantified position into text description - tinkering with the representation of position is a common trope for many chess engines).
â–º Speaking of efficiency, this is also a significant chunk of the chess programming legacy. In addition to their numerous tricks, I'm testing unusual distributed approaches (such as... queuing reasoning of AI players to the moment when the game is paused) and explicit computation management. In an ideal case, AI players would receive a limited computational budget, much like in chess engine competitions, which would be set up in game rules. Moreover, comparative tools lend themselves to easy measurement of performance which also should be communicated to the player (e.g. consciously choosing more intelligent AI that will slower ticks by X ms).
â–º If you want to begin scratching that chess programming itch, have a look at 30 weird chess algorithms, simple & strong chess engine written in 131 lines of Python, elegant pseudocode of Negamax C*, and then lose yourself in clever takes on finding good moves.
[h2]Final Remarks[/h2]
The next dev diary will be posted on April 1st.
If you're not already wishlisting Espiocracy, consider doing it
https://store.steampowered.com/app/1670650/Espiocracy/
There is also a small community around Espiocracy:
---
"Turochamp, a chess program by Alan Turing and David Champernowne developed in 1948 as chess playing algorithm, implemented as . Since there was no machine yet that could execute the instructions, Turing acted as a human CPU requiring more than half an hour per move. One game from 1952 is recorded, which Turochamp lost to one of Turing's colleagues, Alick Glennie. (...) The system anticipated much that has become standard in chess programming." - Chess Programming Wiki & Jack Copeland
---
Dev diaries of Espiocracy often explore the nature of (good) strategy games. The yearning for crafting own agency during gameplay, for the rocky marriage of immersion and strategic decisions, for interesting emergent simulation. It's time to wrestle another hallmark of the genre: challenges generated by other artificial players.
It's not a feature of all strategy games - colony managers and city builders pose the player mainly against the environment. Espiocracy harbors rich dose of such environmental challenges, as should be the case for the Cold War and beyond, to the point of implementing a kind of AI for the most important cogs (actors). However, the main challenge in this game stems from vigorous competition between players: hunting spies, stealing assets, ambushing operations, or even turning operatives against a player. This is further complicated by a game of information with covers, traps, bluffs, and the entire geopolitical poker. Artificial intelligence for THAT surely requires special approach.
Indeed, it does.
[h2]The Player[/h2]
To allow precise discussion, let's begin with a quick recap on the player persona. In Espiocracy, you play as an intelligence community (a set of intelligence agencies) of a single nation.
As the first dev diary proposed, a player here is the most resilient entity of the period, right in the fascinating middle between playing as an entire nation and playing as a small group of characters. For the player ecology, it means that the game harbors secondary (to international) layer of diplomacy: non-kinetic interactions between intelligence communities. Following DD#8 (to be refreshed and expanded in another dev diary), these include communication, alliances, deals, or partially controlling another player. This is further exacerbated by the fact that every player can try to get a foothold on the ground held by another player, in what can be described as a state of constant war. With this unusually competitive landscape in mind, let's jump straight into AI of such players.
[h2]Climbing the Shoulders of Giants[/h2]
Implementation of AI players in Espiocracy follows the principles of chess programming.
For 70 years, chess programmers have been writing engines that can competitively play chess, a game with tree complexity reaching 10 with 123 zeros. They have been working on outsmarting chess grandmasters for the entire Cold War and succeeded just a few years after its end, which is yet another topical and historical parallel that permeates development of Espiocracy.
This niche field generously offers many clever solutions that crave to be implemented in strategy games. Naturally, there are many reasons why they have not been (largely) included in games up to this point. First and foremost, they require counterintuitive approach to development. Instead of writing AI five minutes before midnight when a game is complete, Espiocracy features AI since early builds. In a nuanced way, parts of the code have to be explicitly designed to allow the development of appropriate AI - similarly to architecture requirements posed by multiplayer. Sometimes, even player-facing mechanics have to be (re)designed for AI. As a testament to this design lens, in what began as a hermetic joke and ended up as actually useful implementation, the player is currently served with interfaces corresponding to the dynamics of a chess game:
What principles, precisely?
- State of the game from the perspective of a player (resources, intelligence, alliances, politics, other players, and so on) is approximated in a multi-dimensional vector, an analogue to chess position. As a must for an espionage game, this point also implements absolute no-cheating paradigm (AI players do not have any more intelligence than a human player would have in the same situation).
- From such position, the game generates a (limited/filtered) list of legal moves such as launching an operation against a particular target, establishing a station, nudging an actor towards an action, or directly attacking other players (captures).
- An AI player can ask the game to simulate making a move, own move or other players', akin to asking: what will happen after I arrest this operative? The game responds with an updated position which, importantly, stems from internal rules that approximate changes in the world in a way easily understood by a human in a single sentence (e.g. establishing an intelligence station will subtract money and free operatives, but it will establish local spy network in the future and improve outcomes of local operations). The new position can be used further for analysis or to repeat the cycle of generating and making moves.
This framework sets up the stage for developing AI players that utilize decades of lessons in chess programming. In the most universal approach, AI regularly evaluates positions and finds a combination of moves that will improve the position by the highest value.
There are many exciting solutions that flesh out the details (piece-square tables, alpha-beta search, even a counterpart of opening books). For a nice and easily understandable demonstration of the ingenuity of chess programmers, take a look at mobility. Mobility is the number of available legal moves. In chess, statisticians found a definite correlation between higher mobility and victories - better players usually create more opportunities for themselves. This point of view makes a lot of sense in Espiocracy which, much like immensely complex chess, can be meaningfully reduced to mobility (a single number!), or as we know it here, player agency. With a rather simple algorithm, AI players can find a series of moves that increase mobility, which already leads to pretty sophisticated gameplay such as accepting small losses for larger long-term gains, expanding in preparation for invasive actions, or eliminating enemies that may limit future expansion. This is only a single fascinating solution among hundreds available. And it's not the end of innovation in this department...
[h2]Open AI[/h2]
The gravity pull of chess programming, the one which worked for 70 years and still draws people to this day, may be slightly difficult to understand from the outside. If I would have to explain it, I would compare it to a boxing championship for programmers. Thankfully, I don't have to, because the modding scene of strategy games speaks for itself - given the ability to mod, almost every strategy game receives AI mods, some of which are developed with immense dedication for years. Their authors develop them often despite heavy limitations or technical hostility of the game to modifications in AI behavior.
In the spirit of chess programming, Espiocracy will be fully open to multiple external AIs via dedicated API in C#.
The game will be released with a few AIs implemented by yours truly, using chess programming lessons to the best of my ability. However, that ability is severely limited not only by my brain and time but also perspective. Critically and perhaps controversially, I think that too many strategy games suffer from (relatively) poor AI because developers cannot unsee the implementation of mechanics. As laid out in the introduction, solid AI is too important for this game to leave it solely in the hands of the developer(s). By developing both a fish and a fishing rod, I have a sincere hope that one day you could play Espiocracy with easily developed AI mods authored by people much smarter than me.
To allow a truly competitive nature of AI development, the game is already set up to allow multiple different AIs assigned to different artificial players. In a dream-come-true difficulty settings, instead of deciding between degrees of handicap, you can now choose to play against artificial players who can be actually more stupid or more intelligent.
As you can probably guess at this stage in the hereby honeypot for programmers disguised as a dev diary, AI configuration is used not only for difficulty management or for recording timelapses of USA with AI_X vs USSR with AI_Y. In parallel, I'm working on tools that would measure and compare AIs against each other, have a counterpart of strategic test suite, and maybe (I didn't go that far yet) even a multipolar analogue of Elo rating.
Last but not least, I see this openness as an investment into the unknown future of new tools, about which we hear in recent months. By implementing open API, I'm leaving the door open for the day when our computers will have enough power to run deep learning algorithms with strategic reasoning skills.
[h2]Behind The Scenes[/h2]
â–º For the folks curious about the last remark, we may be surprisingly close. You can already run a 25x smaller equivalent of ChatGPT with a speed of ~2s per sentence on an 8-core CPU. It's large enough to exhibit pretty complex reasoning and planning. Naturally, it's still too slow to power ~100 players in parallel, but once the mechanics are more complete, I can honestly see an attempt at managing an American/Soviet player with its faster iteration (which would probably require most engineering around translation of the quantified position into text description - tinkering with the representation of position is a common trope for many chess engines).
â–º Speaking of efficiency, this is also a significant chunk of the chess programming legacy. In addition to their numerous tricks, I'm testing unusual distributed approaches (such as... queuing reasoning of AI players to the moment when the game is paused) and explicit computation management. In an ideal case, AI players would receive a limited computational budget, much like in chess engine competitions, which would be set up in game rules. Moreover, comparative tools lend themselves to easy measurement of performance which also should be communicated to the player (e.g. consciously choosing more intelligent AI that will slower ticks by X ms).
â–º If you want to begin scratching that chess programming itch, have a look at 30 weird chess algorithms, simple & strong chess engine written in 131 lines of Python, elegant pseudocode of Negamax C*, and then lose yourself in clever takes on finding good moves.
[h2]Final Remarks[/h2]
The next dev diary will be posted on April 1st.
If you're not already wishlisting Espiocracy, consider doing it
https://store.steampowered.com/app/1670650/Espiocracy/
There is also a small community around Espiocracy:
---
"Turochamp, a chess program by Alan Turing and David Champernowne developed in 1948 as chess playing algorithm, implemented as . Since there was no machine yet that could execute the instructions, Turing acted as a human CPU requiring more than half an hour per move. One game from 1952 is recorded, which Turochamp lost to one of Turing's colleagues, Alick Glennie. (...) The system anticipated much that has become standard in chess programming." - Chess Programming Wiki & Jack Copeland