Dev blog 26 - Enemy AI
Hi there.
This week I wanted to talk about how we made the enemy AI.
The requirements for the AI system were apparent a long time ago before we moved to the Serious Engine. Our units each have a list of abilities they can use, and each of those abilities can be used in an area around the unit, determined by the unit range attribute and the ability range configuration. As humans, we can immediately know whether using an ability on a tile makes no sense because we know instinctively it would have no effect - like using a damage ability on a tile where there are no units. Unless it’s an area effect thing, but we can also intuit that very fast.
The AI has to evaluate using an ability on each tile until it can safely determine that “move” makes no sense. I use the term “move” here because I started thinking of our AI as a chess program. A single unit will have many abilities and has to evaluate each one being used on every tile in range to decide if that makes sense, and find the “best move”. We do this for every AI unit that can act, so the data set of [unit, ability, tile] (you can read this as “unit using ability on tile”) is evaluated, given a score, and stored. After all possible combinations of that data set are calculated, the AI picks the best one and runs with it.
Our entire AI is data-driven through the configuration of the underlying utility AI system. We call this the Brain of the unit, and it contains descriptions on how to use a specific ability. The unit might know how to use an ability, but it can only use it if it also HAS that ability (as a side note, this allows us to tell a unit how to use many abilities, but only give it a subset of those abilities, and give the full suite of abilities to an advanced version of the enemy). This means that when we invoke the AI to calculate the best move, every AI unit will attempt to use every ability it has and knows how to use on every tile in range of that ability. Every tile is evaluated using a list of considerations that describe the scoring of that ability. These considerations produce a score between 0 and 1 which defines the utility score of using the ability on that tile. There can be multiple considerations evaluated per tile, and the score of each of these is multiplied with each other to produce the final score.
If all considerations produce a score of 1, it would imply that is the best move. If there is more than one action with the same best score, we pick a random one amongst them, since we assume it doesn’t matter which we pick, since all of them are equally good. Once an action is chosen, it is executed, and then the entire process repeats. We have to repeat the entire process because, after every action, there can be changes to the state of the game that might affect the thought process for other units. For instance, if unit A prefers attacking enemies that have lost health (because it can smell blood or whatever), the score for using that ability would be low for all units that have full health. If another action with a greater (or not penalized) score is executed first and has the outcome of damaging the health of an enemy, then that changes how unit A thinks about using its blood-seeking ability. Another simpler example is having units move around the area, affecting whether they can now attack enemies or not.
This system allows us to create enemies that prefer specific rules, but are not forced to use them or bust. It creates a feeling of uncertainty for what the enemy will do, because it’s dependant on the state of the game that’s not immediately clear, but is still learnable with enough observation.
Thanks for reading! This is just a glimpse into how our utility AI works, the topic is very complex, and if you’re interested in learning more about the general concept of utility AI, I recommend watching some Dave Mark’s videos on the topic. If you’ve got more in-depth questions about our implementation, join our Discord and ask about it, I’m happy to go into much more detail than I’ve written here. You could also ask via Twitter if that’s your preferred medium.
MarkoP
This week I wanted to talk about how we made the enemy AI.
The requirements for the AI system were apparent a long time ago before we moved to the Serious Engine. Our units each have a list of abilities they can use, and each of those abilities can be used in an area around the unit, determined by the unit range attribute and the ability range configuration. As humans, we can immediately know whether using an ability on a tile makes no sense because we know instinctively it would have no effect - like using a damage ability on a tile where there are no units. Unless it’s an area effect thing, but we can also intuit that very fast.
The AI has to evaluate using an ability on each tile until it can safely determine that “move” makes no sense. I use the term “move” here because I started thinking of our AI as a chess program. A single unit will have many abilities and has to evaluate each one being used on every tile in range to decide if that makes sense, and find the “best move”. We do this for every AI unit that can act, so the data set of [unit, ability, tile] (you can read this as “unit using ability on tile”) is evaluated, given a score, and stored. After all possible combinations of that data set are calculated, the AI picks the best one and runs with it.
Our entire AI is data-driven through the configuration of the underlying utility AI system. We call this the Brain of the unit, and it contains descriptions on how to use a specific ability. The unit might know how to use an ability, but it can only use it if it also HAS that ability (as a side note, this allows us to tell a unit how to use many abilities, but only give it a subset of those abilities, and give the full suite of abilities to an advanced version of the enemy). This means that when we invoke the AI to calculate the best move, every AI unit will attempt to use every ability it has and knows how to use on every tile in range of that ability. Every tile is evaluated using a list of considerations that describe the scoring of that ability. These considerations produce a score between 0 and 1 which defines the utility score of using the ability on that tile. There can be multiple considerations evaluated per tile, and the score of each of these is multiplied with each other to produce the final score.
If all considerations produce a score of 1, it would imply that is the best move. If there is more than one action with the same best score, we pick a random one amongst them, since we assume it doesn’t matter which we pick, since all of them are equally good. Once an action is chosen, it is executed, and then the entire process repeats. We have to repeat the entire process because, after every action, there can be changes to the state of the game that might affect the thought process for other units. For instance, if unit A prefers attacking enemies that have lost health (because it can smell blood or whatever), the score for using that ability would be low for all units that have full health. If another action with a greater (or not penalized) score is executed first and has the outcome of damaging the health of an enemy, then that changes how unit A thinks about using its blood-seeking ability. Another simpler example is having units move around the area, affecting whether they can now attack enemies or not.
This system allows us to create enemies that prefer specific rules, but are not forced to use them or bust. It creates a feeling of uncertainty for what the enemy will do, because it’s dependant on the state of the game that’s not immediately clear, but is still learnable with enough observation.
Thanks for reading! This is just a glimpse into how our utility AI works, the topic is very complex, and if you’re interested in learning more about the general concept of utility AI, I recommend watching some Dave Mark’s videos on the topic. If you’ve got more in-depth questions about our implementation, join our Discord and ask about it, I’m happy to go into much more detail than I’ve written here. You could also ask via Twitter if that’s your preferred medium.
MarkoP