Dev Diary: The Game Plays Itself

(it literally does play itself here)

Testing is probably the biggest hidden bottleneck for solo devs and small indie teams.

I obviously dry test every new feature and piece of content I add, but there's time where I just have to sit down and play the damn game normally. Especially when I'm almost done with an update and need to check how everything fits together.

I do two complete runs for every major release. A fresh playthrough, and one using my beloved save file that I've kept since the very first release to test compatibility and balance from the perspective of an existing player. It currently takes about 60 hours all in all. Including on the fly fixes.

That wouldn't be an issue in the studio I work for at my day job. We've got several full-time QA folks who do nothing but play the project every day. But for Bonfire, well, it's just me. And I needed a better solution.

I had this particularly difficult moment during Solstice development, when I realised I'll go crazy if I have to read through the same story again for the hundredth time, but I still needed to test every route. So I wrote myself a bot that did it for me.

Automated test are nothing new, especially in the commercial software world. For games it's less prevalent. Part of it is the complexity of "teaching" a bot to believably act as a player in a mess of complex interacting systems. Super easy to do for something like Solstice. It's a visual novel. Just click through the text and make choices. Done. But Bonfire is a tactical RPG with combos, metagame, resource management, etc. It would require a fairly complex AI to make it work.

Except turns out I already wrote that AI. There's this rare encounter where you fight shadows of your previously failed run, which use the exact same party composition and stats that you've had at that point. These shadows mimic the way an actual player plays, including specific cross-class combos and wider strategies. So hey, I just had to repurpose this for the bot and only add the decision-making layer for picking quests, upgrading characters, and managing food and gold. Right?

Well, no. It took a bit more than that, rather unsurprisingly. But having this initial boost was enough to convince me to give it a go and it was very much worth it. I can have the game play itself at ludicrous speed in a tiny window in the corner of my screen pretty much at all times, racking up hundreds of hours of playtime.

This obviously doesn't make human testing obsolete. AI can play the game, look for bugs & crashes, even test long term balance, but it won't tell me I've made a typo or if the pacing is just off. It's also... too good at the game. Not as good as an experienced player, but much better than a new one. It doesn't make mistakes, it doesn't experiment with new items or abilities, and it always knows what to do and where to go.

Still, it's massively helpful. While writing this post, it simulated nearly 6 hours of playtime and discovered two crashes (one of which I already fixed, the other looks like major PITA). So yeah. If you're a small developer, I very much encourage you to give this approach a try. And if you're a player, I hope this was an interesting peek into how games are made.

Cheers!
-Tom