Day 3: Starting on AI improvements
Alright, let's start with the most important note: There's an updated build in the PREVIEW branch on Steam. It features some minor bugfixes and improvements to the AI.
If you want to give that a try simply right click on the game in your library, select properties, then Betas and then use the dropdown to select the Preview beta.
Here's how this looks:
With that out of the way, let's talk a little bit about today and this preview build...
The AI in Nowhere Prophet works by simulating all manner of moves from its current point of view and then picking the best one (MCTS for the more technically inclined among you).
This has a few ramifications:
Since this takes time and each player in Nowhere Prophet has a lot of moves (move unit to tile a, b, c..., attack with unit to target a, b c... play card on target a, b, c... you get the drift) this means that I can't simulate all the moves but need to do some clever prioritization to determine which move has potential, before simulating a fixed number. Theoretically, if I players were willing to wait a few minutes for each of the AI's moves, then it could always play near optimally, but there's a compromise to be made between the depth of the simulation and the wait time.
Also the AI needs some way to evaluate the game's current state after each move to understand if its actions made the situation better or worse.
Both these things can be terribly opaque so I spent a lot of time today making the AI's thoughts and state evaluation more transparent. I need to understand what's happening to judge where it's going wrong and how to improve things.
Here's a look at some of that debug information:
And just by doing that I found a little bug in the decision system. What was happening was when the AI was looking for possible moves, the game state was changed by mistake!
When checking for all possible moves with cards it inadvertently put a reference to all the enemy's leader cards in its convoy hand. So for the first move in a simulated chain of moves, it had the right amount of cards. For the second move, the convoy hand was suddenly 7 because the 3 cards from the leader card were added to the convoy hand. Then 10, 13 etc. etc.
This lead to the value of the game state improving continuously (more cards = better game state), which meant the AI evaluated a branch of actions a lot higher than it should have. Even useless actions produced value, simply by virtue of being a step in the chain of actions, they created cards out of thin air.
This bug snuck in 6 weeks ago when I optimized the AI to run faster so I could run more iterations in the simulation. Unfortunately I made it faster, but also dumber.
But that one's now fixed in the PREVIEW build. Let me know if you can feel the change. I believe that the AI should be doing less "obviously stupid blunders" now. This may make the game a bit harder too so let me know how that feels as well!
Here's the changelog:
But that's not all. Beyond that I'm planning to make some more tweaks to the AI, specifically when it comes to pre-selecting and prioritizing AI moves since that allows me to get the best out of the simulation iterations I have available in the time given. But this is something for tomorrow and later in the week.
Best,
Martin
If you want to give that a try simply right click on the game in your library, select properties, then Betas and then use the dropdown to select the Preview beta.
Here's how this looks:
With that out of the way, let's talk a little bit about today and this preview build...
How the enemies think
The AI in Nowhere Prophet works by simulating all manner of moves from its current point of view and then picking the best one (MCTS for the more technically inclined among you).
This has a few ramifications:
Since this takes time and each player in Nowhere Prophet has a lot of moves (move unit to tile a, b, c..., attack with unit to target a, b c... play card on target a, b, c... you get the drift) this means that I can't simulate all the moves but need to do some clever prioritization to determine which move has potential, before simulating a fixed number. Theoretically, if I players were willing to wait a few minutes for each of the AI's moves, then it could always play near optimally, but there's a compromise to be made between the depth of the simulation and the wait time.
Also the AI needs some way to evaluate the game's current state after each move to understand if its actions made the situation better or worse.
AI Brain Scan
Both these things can be terribly opaque so I spent a lot of time today making the AI's thoughts and state evaluation more transparent. I need to understand what's happening to judge where it's going wrong and how to improve things.
Here's a look at some of that debug information:
And just by doing that I found a little bug in the decision system. What was happening was when the AI was looking for possible moves, the game state was changed by mistake!
When checking for all possible moves with cards it inadvertently put a reference to all the enemy's leader cards in its convoy hand. So for the first move in a simulated chain of moves, it had the right amount of cards. For the second move, the convoy hand was suddenly 7 because the 3 cards from the leader card were added to the convoy hand. Then 10, 13 etc. etc.
This lead to the value of the game state improving continuously (more cards = better game state), which meant the AI evaluated a branch of actions a lot higher than it should have. Even useless actions produced value, simply by virtue of being a step in the chain of actions, they created cards out of thin air.
This bug snuck in 6 weeks ago when I optimized the AI to run faster so I could run more iterations in the simulation. Unfortunately I made it faster, but also dumber.
But that one's now fixed in the PREVIEW build. Let me know if you can feel the change. I believe that the AI should be doing less "obviously stupid blunders" now. This may make the game a bit harder too so let me know how that feels as well!
Here's the changelog:
Originally posted by changelogChangelog
Balancing
• Unconquered Ascetic: Lost Shielded keyword
• Scion Bloodhand deck adjusted: Fewer charge followers in early maps
• Reduced rest at initial crash site to 3 uses (was 5)
• Finally added rest options to the Crypt Gate (3 uses). You can now take a breather before diving in.
Interface
• Improved visual quality of intro logos
Bugfixes
• AI evaluation bug fixed: Later moves in a sequence had inflated scores, making even bad moves look useful
• Soothing Shadows no longer triggers on moved obstacles or constructs
• Defy Death is depleted on trigger, not on turn start
• Hedgehog Drone is depleted on trigger, not on turn start
• Battery mutator no longer has second player start with 10 Energy
• Wounded followers on the enemy side now start with 1 damage
• Rest button no longer vanishes on 4K displays
But that's not all. Beyond that I'm planning to make some more tweaks to the AI, specifically when it comes to pre-selecting and prioritizing AI moves since that allows me to get the best out of the simulation iterations I have available in the time given. But this is something for tomorrow and later in the week.
Best,
Martin
