Game Health - A deep dive into QA and how YOU can help
[p]Hello Responders,[/p][p]Last week we revealed that 1.0 will be coming in 2026, and we also touched upon a production pivot that has us focusing more on perfecting the current in-game features and offering players an optimized and polished gameplay experience.[/p][p]September will bring a health focused update, similar to our Reinforcement update last November where we highly prioritize the health of the game over adding new features to NMRiH2. Game health is a mixture of optimization — the art of delivering the highest quality game in the smallest possible package — and polishing of the gameplay experience by removing bugs and confusing or unintended issues.[/p][p]This first deep dive is all about our QA team, the work they’ve put into the game over the past 10 months to get the game to where it is today, and our ambitious polishing objectives for 1.0 next year. Every Support ticket and crash report is a key component to game health, and offers a tremendous amount of help to our relatively small QA Team.[/p][p][/p][p]![]()
[/p][h3]What’s QA?[/h3][p]Every Quality Assurance (QA) team at every development studio works differently, but they all generally have the same mission: supporting the team in delivering the best possible product to its players. At Torn Banner, our QA team wears many hats, which include but are not limited to:[/p]
The Art of Triage[/h3][p]Torn Banner is a small studio, with a relatively small QA team, and that makes managing a large list of tasks difficult, No More Room in Hell 2 being an 8 player game makes everything more complicated, because so much testing has to be done in groups. We cannot be everywhere at once, and that makes triaging issues the most important part of the job.[/p][p] We have to take a few things into account:[/p]
[/p][p]“Severity is our shorthand to assess: “how bad is this bug going to be for a player?” As a QA team it’s important for us to be in alignment and test the most severe potential problems first, and it’s also key for us to communicate each bug’s Severity to the Production team and other departments, to understand the impact that any bug is having on the experience.”[/p][p]—TornSheep, QA Tester[/p][p][/p][h3]A look at QA in motion:[/h3][p]Figuring out the cause of every bug can be an extremely difficult and time consuming task. Sometimes it takes a lot of investigation, or just plain dumb luck, to recreate a bug that a player has seen and get the data we need to fix it.[/p][p][/p][p]Critical Objective bug:[/p][p]For a while now players have been running into a severe issue where players on Pottsville were unable to interact with a transformer that was critical for opening up the next segment of the map. Through tedious trial and error the team eventually figured out that if you collected the objectives under CRC Responder body 2 & 3 at the exact same time it would cause the bug. [/p][previewyoutube][/previewyoutube][p][/p][p]When testing maps, the stereotype is often "Time to run into this wall 300 times to see if it breaks!" - and while this absolutely is a part of what we do, we also have to look at objectives, replication (ensuring the game state is shown equally to all players on a server), and more subjective QA work like loot and zombie spawn balance. The biggest new content we add to the game are our maps, and they get developed and iterated on over the course of many months. Because of their size, there are *hundreds* of map-specific bugs that get found and fixed, but some bugs are rare, or introduced late in the development cycle. Unfortunately, no studio in the world can match the sheer number of in-game hours that thousands of live users will put in on a content launch day. There's always one missed hole in the wall, place someone can get stuck, or rare misbehaving objective.[/p][p]— TornPotassium, QA Tester[/p][p][/p][p]This is a prime example of the type of “meat and potatoes” investigation and problem solving QA does on a daily basis, and it also shows how seemingly simple issues can be extremely difficult to solve. This specific bug has a fix that will go live in our September health update.[/p][p][/p][p]“Almost every single bug report — whether that comes from someone inside the QA team in our morning smoke test, from a developer in a weekly full-team playtest, or from a player reaching out on Discord or our ticketing system — gets a QA ticket for investigation. Sometimes these problems are easy to find and ping a developer to fix, but other times when we need server logs and performance profiles and the steps to recreate the bug are so rare, it can be a real collaborative effort to solve the problem. It can be like feeling around a dark room for a light switch.”[/p][p]—TornSheep, QA Tester[/p][p][/p][p]Hit Registration:[/p][p]Another really important, and complicated issue that our QA and Programmers are constantly dealing with is hit registration. A lot of independent variables go into hit registration, but the biggest is ping and servers.[/p][p][/p][previewyoutube][/previewyoutube][p][/p][p]To try and get a deeper understanding of the issue our QA team started the tenuous task of testing each and every weapon in various settings to gain a deeper understanding of how they all interact with the game and our servers. Through this, we’ve already fine tuned multiple factors related to combat and hit registration (we hope you’ve noticed!) but we’re not done yet.[/p][p]At the end of the day, your ping causes latency, and that latency causes hit registration “mis-hits”. We’re hoping to be able to further refine and update the way our servers interact with players to trim down players ping and lower the amount of hit registration issues.[/p][p][/p][h3]The Path Forward:[/h3][p]Our QA team’s primary goal still remains finding critical bugs and helping developers to remove them from the game, but we’re also starting to put a stronger emphasis on “polish” bugs, those that take away from players immersion but don’t necessarily impact the core gameplay loop of the game. [/p][p]Often in the gaming community there can be a sentiment that bugs get into the live game because QA testers ‘aren’t doing their jobs.’ But our testing team has a lot of ground to cover every day, and we have to make trade-offs about which bugs to investigate and log. Knowing about as many bugs as possible is just the first step. Helping developers and producers understand the most important fixes to make first is the other half of the battle. On top of it all, it’s very hard to predict exactly how complicated and time consuming a given bugfix can be! A lot of hands touch every single bug – from Community team members, to QA, to Production, to Developers, and back to QA again.[/p][p]—TornSheep, QA Tester[/p][p]The “polishing” and focus on our game health works hand in hand with our goals to prioritize getting the current features in No More Room in Hell 2 just right, and offering players a fully immersive, engaging, and replayable 1.0 experience.[/p][p][/p][h3]What we need from you:[/h3][p]During this Early Access period your help with tickets and technical support feedback is more important than ever. While our priority needs to remain triaging tickets and targeting critical game breaking issues, we want to use this newly earned time to drop the overall “sitting” ticket number, and polish No More Room in Hell 2 until it’s in a place where it truly shines.[/p][p]A reminder of what to add in your tickets:[/p]
[/p]
- [p]Finding as many bugs as possible in as many iterations of each upcoming feature and update as possible.[/p]
- [p]Troubleshooting those bugs and finding the root cause, then giving as much information as possible to the correct developer to fix the issue.[/p]
- [p]Working with our Community team to track customer support tickets, feedback on platforms with Twitter, Reddit, and Discord, and triage and troubleshoot issues as they come up.[/p]
- [p]Entrench themselves within different development groups and support feature development as it happens[/p]
- [p]Support and run internal and external playtests every week[/p]
- [p]Generate game analytics reports & QA reports[/p]
The Art of Triage[/h3][p]Torn Banner is a small studio, with a relatively small QA team, and that makes managing a large list of tasks difficult, No More Room in Hell 2 being an 8 player game makes everything more complicated, because so much testing has to be done in groups. We cannot be everywhere at once, and that makes triaging issues the most important part of the job.[/p][p] We have to take a few things into account:[/p]
- [p]How bad is a bug if it happens (total blocker, there are workarounds, just a blemish)[/p]
- [p]How often a bug happens (every single game, some players might see it, you really have to look for it)[/p]
- [p]Balancing our time between live bugs and bugs in upcoming features [/p]
[/p][p]“Severity is our shorthand to assess: “how bad is this bug going to be for a player?” As a QA team it’s important for us to be in alignment and test the most severe potential problems first, and it’s also key for us to communicate each bug’s Severity to the Production team and other departments, to understand the impact that any bug is having on the experience.”[/p][p]—TornSheep, QA Tester[/p][p][/p][h3]A look at QA in motion:[/h3][p]Figuring out the cause of every bug can be an extremely difficult and time consuming task. Sometimes it takes a lot of investigation, or just plain dumb luck, to recreate a bug that a player has seen and get the data we need to fix it.[/p][p][/p][p]Critical Objective bug:[/p][p]For a while now players have been running into a severe issue where players on Pottsville were unable to interact with a transformer that was critical for opening up the next segment of the map. Through tedious trial and error the team eventually figured out that if you collected the objectives under CRC Responder body 2 & 3 at the exact same time it would cause the bug. [/p][previewyoutube][/previewyoutube][p][/p][p]When testing maps, the stereotype is often "Time to run into this wall 300 times to see if it breaks!" - and while this absolutely is a part of what we do, we also have to look at objectives, replication (ensuring the game state is shown equally to all players on a server), and more subjective QA work like loot and zombie spawn balance. The biggest new content we add to the game are our maps, and they get developed and iterated on over the course of many months. Because of their size, there are *hundreds* of map-specific bugs that get found and fixed, but some bugs are rare, or introduced late in the development cycle. Unfortunately, no studio in the world can match the sheer number of in-game hours that thousands of live users will put in on a content launch day. There's always one missed hole in the wall, place someone can get stuck, or rare misbehaving objective.[/p][p]— TornPotassium, QA Tester[/p][p][/p][p]This is a prime example of the type of “meat and potatoes” investigation and problem solving QA does on a daily basis, and it also shows how seemingly simple issues can be extremely difficult to solve. This specific bug has a fix that will go live in our September health update.[/p][p][/p][p]“Almost every single bug report — whether that comes from someone inside the QA team in our morning smoke test, from a developer in a weekly full-team playtest, or from a player reaching out on Discord or our ticketing system — gets a QA ticket for investigation. Sometimes these problems are easy to find and ping a developer to fix, but other times when we need server logs and performance profiles and the steps to recreate the bug are so rare, it can be a real collaborative effort to solve the problem. It can be like feeling around a dark room for a light switch.”[/p][p]—TornSheep, QA Tester[/p][p][/p][p]Hit Registration:[/p][p]Another really important, and complicated issue that our QA and Programmers are constantly dealing with is hit registration. A lot of independent variables go into hit registration, but the biggest is ping and servers.[/p][p][/p][previewyoutube][/previewyoutube][p][/p][p]To try and get a deeper understanding of the issue our QA team started the tenuous task of testing each and every weapon in various settings to gain a deeper understanding of how they all interact with the game and our servers. Through this, we’ve already fine tuned multiple factors related to combat and hit registration (we hope you’ve noticed!) but we’re not done yet.[/p][p]At the end of the day, your ping causes latency, and that latency causes hit registration “mis-hits”. We’re hoping to be able to further refine and update the way our servers interact with players to trim down players ping and lower the amount of hit registration issues.[/p][p][/p][h3]The Path Forward:[/h3][p]Our QA team’s primary goal still remains finding critical bugs and helping developers to remove them from the game, but we’re also starting to put a stronger emphasis on “polish” bugs, those that take away from players immersion but don’t necessarily impact the core gameplay loop of the game. [/p][p]Often in the gaming community there can be a sentiment that bugs get into the live game because QA testers ‘aren’t doing their jobs.’ But our testing team has a lot of ground to cover every day, and we have to make trade-offs about which bugs to investigate and log. Knowing about as many bugs as possible is just the first step. Helping developers and producers understand the most important fixes to make first is the other half of the battle. On top of it all, it’s very hard to predict exactly how complicated and time consuming a given bugfix can be! A lot of hands touch every single bug – from Community team members, to QA, to Production, to Developers, and back to QA again.[/p][p]—TornSheep, QA Tester[/p][p]The “polishing” and focus on our game health works hand in hand with our goals to prioritize getting the current features in No More Room in Hell 2 just right, and offering players a fully immersive, engaging, and replayable 1.0 experience.[/p][p][/p][h3]What we need from you:[/h3][p]During this Early Access period your help with tickets and technical support feedback is more important than ever. While our priority needs to remain triaging tickets and targeting critical game breaking issues, we want to use this newly earned time to drop the overall “sitting” ticket number, and polish No More Room in Hell 2 until it’s in a place where it truly shines.[/p][p]A reminder of what to add in your tickets:[/p]
- [p]As much context as possible[/p]
- [p]What happened?[/p]
- [p]With who?[/p]
- [p]What map?[/p]
- [p]What location of the map?[/p]
- [p]Any other pieces of information that help further flesh out the situation you found yourself in[/p]
- [p]A description of your rig and technical situation[/p]
- [p]GPU and CPU[/p]
- [p]Ping and the region you’re playing in[/p]
- [p]~FPS[/p]
- [p]Video or photo – if possible, they help a lot! – especially videos of things you did leading up to the bug, not just a video of the result![/p]
[/p]