Hot Fix: Testing and Bug Fixing
Anyone who plays video games encounters program errors sooner or later.These types of errors have hindered players since the emergence of the gaming industry. We are used to them and not-so-affectionately nickname them "bugs." Bugs can be found everywhere, and World of Warships is no exception. Our project comprises millions of constantly updated lines of code, a huge amount of content, and a complex system of interactions between players' computers and our servers. All this creates a multitude of relationships in which mistakes can occur. Today, you will learn about how we reduce the number of bugs that reach the main server and how we fix those that are overlooked.
[previewyoutube][/previewyoutube] [hr][/hr]
[h2]What Is a Bug?[/h2]
The term "bug" is probably familiar to most gamers.
A bug is an error that causes the actual result of a program's operation to differ from the expected one.
In the early days of the game, we divided bugs into two main types: bugs in the game content and bugs in the code. Now, there are many more types.
[hr][/hr]
[h2]How Is Testing Performed?[/h2]
Quality assurance (QA) teams are responsible for finding and fixing bugs in World of Warships.
QA departments exist in most World of Warships development teams. For example:
- Content: Checking models, sounds, visuals, and more.
- Game logic: Checking the correct operation of the game mechanics.
- Server core: Checking the correct operation of the game servers.
Both negative and positive tests are carried out when testing the program:
- Positive testing uses scenarios that match the normal, expected behavior of the program. For example, checking that a user can enter the game client upon entering the correct password.
- Negative testing involves performing "incorrect" actions deliberately (abnormal scenarios). For example, checking if it is possible to log in to the game client by entering an incorrect password or leaving the password input field blank.
Other types of checks are carried out if deemed necessary: performance tests, user friendliness tests, and others.
In World of Warships, testing can be simplified into several main stages.
Internal testing:
- Various types of functionality testing at all stages of functionality creation, such as:
- Ideation
- Design
- Development
- Ideation
- Integration testing, or checking that all the individual parts of the game work correctly when put together and added to the main game code.
Testing an assembled build for obvious defects before an external test.
A build is a specific version of the game client that can also be used for testing.
Tracking absolutely all relationships and their results in various elements of such a large project as World of Warships is not possible even for the largest QA teams.
Therefore, external testing begins in parallel with the final stages of internal tests:
- Testing of a pre-release build of the game by a small group of volunteers (Supertest and Clan Test).
- Public testing, which is open to everyone.
External testing also allows us to check the operation of the game on a wide variety of computer and software configurations.
The World of Warships development team is also involved in external testing. We have a mechanism called "Traffic lights" in the studio. This is a special page that all teams fill out after playing on the Public Test Server. On this page, staff can leave feedback about each new tested version feature, as well as give it an assessment that is later taken into account when summing up the testing results.
Based on the results of external testing, we fix the last identified errors in the current build of the game. If it is impossible to fix the errors, we may even have to remove certain features from the coming update.
This was the case with Asymmetric Battles in Update 0.10.9. A bug that led to incorrect operation of the mode and required additional time to fix was discovered during the Public Test stage. As a result, the mode was removed from the update.
[hr][/hr]
[h2]Occurrence of Bugs[/h2]
Why do bugs occur on the Live Server? Bugs occur for a variety of reasons and at different development stages. From commonplace errors in the program code and the human factor to non-obvious interactions and errors at the stage of creating a design document for a new version.
An error may be present, for example, already at the stage of developing an idea and will still be considered a bug.
For example, the mechanics of temporary invulnerability of an aircraft carrier squadron immediately after an attack, which could earlier be used to overcome the enemy's air defense zone, can also be attributed to bugs. This particular use of the mechanic was not foreseen in the design stage of the aircraft carrier update and was later fixed.
To prevent similar situations, we try to start testing as early as at the stage of discussing new ideas.
In addition, game elements that work perfectly in isolation can "break" when combined with other game elements due to various interconnections and conflicts that arise.
For example, in one of the internal tests of the Ranked Battles update, we found that they caused errors in the operation of Clan Battles—players were not able to start a battle because matchmaking took an infinitely long time. The cause of this was not the most obvious relationship, but it was fortunately found and did not reach the Live Server.
Another cause of bugs—one that is rather difficult to track down—is the huge variety of our users' computer configurations, which, together with the complex client-server architecture, can lead to unexpected behavior of the game code on specific computers.
For example, in one update, due to the date format in Japan being different to the format used in Europe or America, the game code was not able to process this information, and the post-battle statistics were not displayed for some players.
What players consider to be a bug can sometimes not be a bug at all because the program behavior seen by players is actually the expected outcome. In such situations, it is important to analyze player feedback and, possibly, change something. The opposite can also be true—a bug can sometimes become part of the game and the decision not to fix it can be deliberate.
For example, the unique sound of
X Shikishima's guns had an error in the audibility range setting (it could be heard at a greater range than was intended), and this error was fixed. However, you noticed that the sound lost its unique quality due to the fix, so we returned this "error" to the game.Bugs can also be tentatively attributed to technical problems in data centers or Internet providers. Unfortunately, there is little that we can do in such situations, and we cannot always help players, regardless of whether we know the source of the problem.
It should be understood that it is impossible to track down absolutely all bugs. The result of the QA team's work is that a new functionality works as intended, without critical bugs and flaws, and that it can be released on the Live Server.
[hr][/hr]
[h2]Fixing[/h2]
Why do some bugs take so long to fix? The required time period for correcting a particular error depends primarily on its nature. Some defects require a significant investment of time as early as the error analysis stage. Indeed, in order to correct an error, we need to reproduce it accurately and understand the reason or reasons why this error occurred.
For example, a recent bug caused torpedoes to launch incorrectly when the launching ship was turning. This was a mistake in the game logic, and any changes and fixes to the game logic are always very risky because they affect absolutely all players and affect the very basis of the game—its gameplay.
Therefore, to fix such an error, it is necessary to accurately analyze player complaints and various reports, find the cause of the problem, and develop and verify a solution. In special cases, it can be necessary to rewrite the game code to fix a bug or optimize it using new technology. This process is called refactoring. Despite the fact that nothing seems to change from the player's point of view, this process takes quite a long time.
In addition, the time frame for correcting errors depends on the ways in which fixes are delivered to players. For example:
- To fix some bugs, it is enough to make small fixes to the server side of the game, which most players will not even notice.
- The situation is more complicated when the error also affects the player's client or it is necessary to carry out server maintenance to fix the error. In this case, a small patch can be prepared and additional work performed on the server. The server is not available to players during this time.
- Fixing some bugs requires so many changes in the code and additional tests that fixes can only be released with the release of the next major game update.
The decision on how to deliver each fix and the priority of bug fixing is determined by numerous factors:
- Bug severity: Inoperable Random Battles should be fixed as soon as possible, but a typo in the ship's performance specifications can wait a bit.
- Scale: Some errors affect all players; other errors occur for one player out of a thousand.
- Risks and necessary fix tests: As mentioned above, sloppy fixes in the game logic can only make things worse.
- Time spent on server maintenance and the update size: To fix an error, we will need to turn off the server for several hours, and players will need to download quite a big patch. In such cases, it is worthwhile to think again how important it is to fix an error immediately because many players will not be able to enter the game for a long time.
Unfortunately, despite all of our desires to fix all possible bugs, the resources of the World of Warships team are finite, so we focus on fixing the most critical and frequently encountered bugs.
[hr][/hr]
[h2]How to Report a Bug[/h2]
You can always inform us about any errors you find via one of the following resources:
To help facilitate quicker and more efficient error fixing, please add the following to your report:
- Error description.
- Screenshot, video, replay, or any other visual confirmation of an error. If you attach a battle replay, make sure to indicate the timing of the error.
- WGCheck file. Detailed information about this file is available in a separate article.
- In today's article, we did our best to tell you about our "invisible front" in the fight against bugs in the simplest possible way. We want to thank you for your bug reports, feedback, participation in our tests, and playing World of Warships! Without your contribution, we would not have been able to quickly find and correct various errors that have occurred in the game.
Start: Fri. 10 Dec. 06:00 CET (UTC+1)
End: Thu. 20 Jan. 06:00 CET (UTC+1)
2 days of Warships Premium Account, festive camouflages, and Coal!
In Update 0.10.11, you will be able to start building new British battleship Marlborough at the Clydebank Dockyard, as well as obtain two other ships—Dreadnought and Repulse!
IX Marlborough is based on a 1947 project with some of the design solutions borrowed from the Vanguard-class and King George V-class ships. Her primary feature is extensive but not particularly accurate artillery consisting of sixteen 356 mm guns housed in four turrets. While the battleship's armor is rather mediocre, this is compensated for by low detectability and numerous main battery guns that fire quite rapidly for her type and have a decent firing range.
1,500 Doubloons to pass through each phase.
[/expand]
5,250,000 Credits as compensation.
2,000 Coal
10,000 Free XP
[hr][/hr]
VI Weser, 
VI Fubuki, and 
VI Izmail, respectively. Each ship comes with the Snow and Stars permanent camouflage and a Commander with 6 skill points. The reward for completing the last combat mission group is 
VI Kijkduin in the Snow and Stars permanent camouflage and Dutch Commander Sinterklaas with 10 skill points.
[/expand]